Skip to Content.
Sympa Menu

xom-interest - [XOM-interest] Very high main memory consumption on compression level higher then 0

xom-interest AT lists.ibiblio.org

Subject: XOM API for Processing XML with Java

List archive

Chronological Thread  
  • From: Martin Schaaf <ms AT 101tec.com>
  • To: xom-interest AT lists.ibiblio.org
  • Subject: [XOM-interest] Very high main memory consumption on compression level higher then 0
  • Date: Wed, 18 Jun 2008 21:30:06 +0200

Hi,

I use the following code

//serialize
private nu.xom.Document _doc
byte[] binaryDoc = _binaryXMLCodec.serialize(_doc, 0);

//deserialize
_doc = _binaryXMLCodec.deserialize(_xmlRecord.get());

to serialize xml. I want to compress it too, but get very high memory usage. This leads in the worst case to an OOM and in other cases it brings the system to swap memory out. If I set the compression level to 0 I do not get the high memory usage effects. I use version 1.1. It does not make a difference ehether I use 1 or 9 or any othe number between for compression. All show the same effect. I use it in a map reduce job to compress 50.000.000 documents. One document is about 10kb in size.

Is this known? Do I smething wrong?

Thanks for all answers
martin




Archive powered by MHonArc 2.6.24.

Top of Page