[XOM-interest] Very high main memory consumption on compression level higher then 0
Martin Schaaf
ms at 101tec.com
Wed Jun 18 15:30:06 EDT 2008
Hi,
I use the following code
//serialize
private nu.xom.Document _doc
byte[] binaryDoc = _binaryXMLCodec.serialize(_doc, 0);
//deserialize
_doc = _binaryXMLCodec.deserialize(_xmlRecord.get());
to serialize xml. I want to compress it too, but get very high memory
usage. This leads in the worst case to an OOM and in other cases it
brings the system to swap memory out. If I set the compression level
to 0 I do not get the high memory usage effects. I use version 1.1. It
does not make a difference ehether I use 1 or 9 or any othe number
between for compression. All show the same effect. I use it in a map
reduce job to compress 50.000.000 documents. One document is about
10kb in size.
Is this known? Do I smething wrong?
Thanks for all answers
martin
More information about the XOM-interest
mailing list