Skip to Content.
Sympa Menu

xom-interest - Re: [XOM-interest] Writing large XML documents

xom-interest AT lists.ibiblio.org

Subject: XOM API for Processing XML with Java

List archive

Chronological Thread  
  • From: Peter Corbett <ptc24 AT cam.ac.uk>
  • To: XOM interest <xom-interest AT lists.ibiblio.org>
  • Subject: Re: [XOM-interest] Writing large XML documents
  • Date: Tue, 18 Dec 2007 09:25:19 +0000

Eric Wang wrote:
On Dec 17, 2007 11:22 PM, Tatu Saloranta <cowtowncoder AT yahoo.com> wrote:


That is bit misleading, in a way -- it's more accurate
to say that you can build tree representations for
sub-trees of xml content (and potentially also discard
earlier parts, trim while you go, if you want to get
fancy). Although underlying parsers are all streaming
(XOM is not a parser but uses underlying parsers like
Xerces), result is still a tree that has to completely
fit in memory.


True, but I don't build a tree at all when processing large XML files.
I use a custom NodeFactory subclass and do all my processing in
finishMakingElement().

I've had a lot of joy using XOM to read subtrees from a large XML file, basically the XML file was a long list (tens or hundreds of thousands) of moderately-sized elements (about a hundred to a thousand subnodes or so each, I'd guess), and you could read one into a XOM tree, process the tree in the normal XOMmy ways including running XPaths on it, discard it and move onto the next element, thus making the whole process fit nicely in memory when slurping in the entire tree would just have failed.

My instinct would be to subclass the Serializer; it contains lots of protected methods that look like they could be hooked up to the NodeFactory methods with a bit of coding. You'd need to do something slightly clever to get around the fact that they are protected, but that shouldn't be too tricky.

Peter Corbett





Archive powered by MHonArc 2.6.24.

Top of Page