[XOM-interest] Very high main memory consumption on compression level higher then 0

Martin Schaaf ms at 101tec.com
Wed Jun 18 15:30:06 EDT 2008


Hi,

I use the following code

//serialize
private nu.xom.Document _doc
byte[] binaryDoc = _binaryXMLCodec.serialize(_doc, 0);

//deserialize
_doc = _binaryXMLCodec.deserialize(_xmlRecord.get());

to serialize xml. I want to compress it too, but get very high memory  
usage. This leads in the worst case to an OOM and in other cases it  
brings the system to swap memory out. If I set the compression level  
to 0 I do not get the high memory usage effects. I use version 1.1. It  
does not make a difference ehether I use 1 or 9 or any othe number  
between for compression. All show the same effect. I use it in a map  
reduce job to compress 50.000.000 documents. One document is about  
10kb in size.

Is this known? Do I smething wrong?

Thanks for all answers
martin


More information about the XOM-interest mailing list