Skip to Content.
Sympa Menu

xom-interest - Re: [XOM-interest] indexOf O(1) patch?

xom-interest AT lists.ibiblio.org

Subject: XOM API for Processing XML with Java

List archive

Chronological Thread  
  • From: Wolfgang Hoschek <whoschek AT lbl.gov>
  • To: Wolfgang Hoschek <whoschek AT lbl.gov>
  • Cc: Elliotte Harold <elharo AT metalab.unc.edu>, xom-interest AT lists.ibiblio.org, Wolfgang Hoschek <whoschek AT yahoo.com>
  • Subject: Re: [XOM-interest] indexOf O(1) patch?
  • Date: Tue, 1 Feb 2005 21:16:29 -0800

Note that the overall goal of the codec is maximum performance, not compression. Here, data reduction techniques are used where they actively improve performance, and are considered harmful where they degrade performance. To summarize, its selective compactness is a means towards a well-defined end, not a goal in itself.

A main memory tree model can play an interesting role in efficiently processing large numbers of small to medium sized documents, and that's what I'm targetting. A main memory tree model clearly has *no* meaningful role in processing very large individual documents.

On Feb 1, 2005, at 8:55 PM, Wolfgang Hoschek wrote:

On Feb 1, 2005, at 8:46 PM, Elliotte Harold wrote:

Wolfgang Hoschek wrote:


As an aside, if i remember it correctly, the BinaryXMLCodec deserializes in a more compact manner, somewhere around 4 memory bytes per file byte.


Now how does it do that? If it's starting with the same XML document and eventually arriving at the same XOM Document object, albeit with a detour through binary XML, that's a bit surprising. Is it interning strings the standard builder is not interning?

It's interning *all* strings (but not with String.intern(), but rather with a better application-level scheme). A large fraction of the binary data is essentially an array of unique string occurances, stored in compact form. Then there are byte level tokens identifying node types and compact byte-level integer indexes referring to the string table.

Beyond that, it also using the XOM Node copy constructors as much as as possible, to share more memory, faster object construction, and to avoid reverification of things that have already been verified before.

_______________________________________________
XOM-interest mailing list
XOM-interest AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/xom-interest





Archive powered by MHonArc 2.6.24.

Top of Page