[XOM-interest] Writing large XML documents
Peter Corbett
ptc24 at cam.ac.uk
Tue Dec 18 04:25:19 EST 2007
Eric Wang wrote:
> On Dec 17, 2007 11:22 PM, Tatu Saloranta <cowtowncoder at yahoo.com> wrote:
>
>
>>That is bit misleading, in a way -- it's more accurate
>>to say that you can build tree representations for
>>sub-trees of xml content (and potentially also discard
>>earlier parts, trim while you go, if you want to get
>>fancy). Although underlying parsers are all streaming
>>(XOM is not a parser but uses underlying parsers like
>>Xerces), result is still a tree that has to completely
>>fit in memory.
>
>
> True, but I don't build a tree at all when processing large XML files.
> I use a custom NodeFactory subclass and do all my processing in
> finishMakingElement().
>
I've had a lot of joy using XOM to read subtrees from a large XML file,
basically the XML file was a long list (tens or hundreds of thousands)
of moderately-sized elements (about a hundred to a thousand subnodes or
so each, I'd guess), and you could read one into a XOM tree, process the
tree in the normal XOMmy ways including running XPaths on it, discard it
and move onto the next element, thus making the whole process fit nicely
in memory when slurping in the entire tree would just have failed.
My instinct would be to subclass the Serializer; it contains lots of
protected methods that look like they could be hooked up to the
NodeFactory methods with a bit of coding. You'd need to do something
slightly clever to get around the fact that they are protected, but that
shouldn't be too tricky.
Peter Corbett
More information about the XOM-interest
mailing list