Skip to Content.
Sympa Menu

xom-interest - Re: [XOM-interest] Text.copy

xom-interest AT lists.ibiblio.org

Subject: XOM API for Processing XML with Java

List archive

Chronological Thread  
  • From: Elliotte Harold <elharo AT metalab.unc.edu>
  • To: Wolfgang Hoschek <whoschek AT lbl.gov>, xom-interest <xom-interest AT lists.ibiblio.org>
  • Cc:
  • Subject: Re: [XOM-interest] Text.copy
  • Date: Thu, 25 Nov 2004 18:54:35 -0500

Wolfgang Hoschek wrote:

Improves CDATASection, looks good.

On a different, but related issue: I get more and more the impression that the UTF-8 encoding trick to safe memory at the expense of CPU isn't quite worth it. When the text is a constant, or a substring, or from an interning source the UTF trick actually increases memory consumption when used through the normal Text(String) constructor. I'd rather store a plain string and perhaps try to eliminate memory overhead elsewhere (e.g. in namespaces).

I have measured it and it does make a big difference in a lot of common use cases. SAX does *not* intern PCDATA, only element names, attribute names, and the like; and I think that most of the time this accounts for the majority of text nodes in large documents in most use cases.

This is a judgment call. There's no one right answer here. It's one of those rare cases where there's a really clear trade-off between size and speed, and I've made the decision that size is more important. In my experience, tree-based APIs like XOM have real problems with footprint that make them unable to handle many common use cases. That comes up a lot more frequently than a library being so slow it can't be used.

I do recognize that not everyone may have these issues. If you're only processing small documents, and you're processing a lot of them, you may prefer the increased speed even at the cost of increased size. I've tried to keep the exact text storage algorithm private to the Text class, so it's straight-forward to change. This is a rare case where I really wish Java had a macro facility so I could set a build time option to compile for space or speed. I could probably do that by maintaining two copies of the Text class, and copying one or the other into the src directory at build time, but, really, yuck. Duplication on that scale is a major code smell. I may get around to doing that one of these days, but it won't be in 1.0; and I suspect the default will always be to optimize for space over speed.

I have experimented with other ways of storing the text, hoping to find something that was fast and small, or a reasonable compromise between the two, but nothing performed adequately and gave close to the compression of plain UTF-8. I've thought about various dynamic algorithms that would use UTF-8 or Strings depending on content, but those would exact an even bigger performance penalty, and seem to require me to carry around extra fields just to track how any given text node is stored.

--
Elliotte Rusty Harold elharo AT metalab.unc.edu
XML in a Nutshell 3rd Edition Just Published!
http://www.cafeconleche.org/books/xian3/
http://www.amazon.com/exec/obidos/ISBN=0596007647/cafeaulaitA/ref=nosim




Archive powered by MHonArc 2.6.24.

Top of Page