Skip to Content.
Sympa Menu

xom-interest - Re: [XOM-interest] More parsing performance patches

xom-interest AT lists.ibiblio.org

Subject: XOM API for Processing XML with Java

List archive

Chronological Thread  
  • From: Wolfgang Hoschek <whoschek AT lbl.gov>
  • To: Wolfgang Hoschek <whoschek AT lbl.gov>
  • Cc: Elliotte Harold <elharo AT metalab.unc.edu>, xom-interest AT lists.ibiblio.org
  • Subject: Re: [XOM-interest] More parsing performance patches
  • Date: Fri, 22 Jul 2005 16:56:40 -0700

And since the whitespace interning isn't really applicable to the Text version with UTF8 storage, it can be skipped via

private static String firstCharacters(char[] text, int start, int length) {
if (Text.IS_FAT && length < SPACES.length) {
int i = start + length;
...
}
return new String(text, start, length);
}

Text.java:
static final boolean IS_FAT = true/false depending on the version


Wolfgang.

On Jul 22, 2005, at 3:16 PM, Wolfgang Hoschek wrote:

The

algorithm is really not that difficult, and there's a reason it is
formulated the way it is.

Wolfgang.


A little cosmetic modularization yields:

public void characters(char[] text, int start, int length) {
/*
* Based on measurements of a wide variety of real-world documents, one
* call to characters() per flushText() is extremely common, a second
* one quite uncommon, and more calls to it are extremely uncommon.
* Hence, unnecessary temporary String and StringBuffer memory copies
* and allocations can be avoided, improving performance, *irrespective* whether
* the "fat" stringified Text.java is used, or the Text.java version with
* the UTF-8 storage trick.
* See endDocument() on how to measure character() per flushText() frequency, uncommenting
* some diagnostic statistics gathering code.
*
* Plus, whitepace separator lines make up a large fraction of data in
* heavily pretty printed documents. So, let's share most of those
* string objects via an extremely low-overhead interning mechanism.
*
* All in all this should make average memory consumption of a XOM
* document with the stringified Text patch comparable or lower than with the
* Text.java UTF-8 trick. Plus, it retains the performance benefits of
* the stringified Text patch, for use cases reading or writing Text a
* lot, for example many XPath/XQuery use cases.
*/
// invariants throughout this class:
// only one of (first, buffer) has content, i.e. at any time there holds:
// first != null implies buffer == null
// buffer != null implies first == null
// if there is content, it is non-empty, i.e at any time there holds:
// first != null implies first.length() > 0
// buffer != null implies buffer.length() > 0

if (length <= 0) return; // nothing to do (important: do not remove; ensures correct algo semantics)

if (first == null && buffer == null) { // first call to characters()
first = firstCharacters(text, start, length);
// first = new String(text, start, length);
} else {
if (buffer == null) { // second call to characters(); infrequently called
buffer = new StringBuffer(first.length() + length); // doesn't waste memory
buffer.append(first);
first = null;
}
// second or higher call to characters(); third or higher call is very rare
buffer.append(text, start, length);
}
if (finishedCDATA) inCDATA = false;
// callCounter++;
}


private static String firstCharacters(char[] text, int start, int length) {
if (length < SPACES.length) { // any possibility for interning whitespace-only?
// scan backwards to beginning of run of blanks or tabs
int i = start + length;
char c = text[--i];
if (c != ' ' && c != '\t') {
c = ' ';
} else {
i++; while (--i >= start && text[i] == c) ;
}

switch (i+1-start) { // is there internable whitespace- only?
case 0: {
return c == ' ' ? SPACES[length] : TABS[length];
}
case 1: {
if (text[start] == '\n')
return c == ' ' ? LF_SP[length-1] : LF_TAB [length-1];
break;
}
case 2: {
if (text[start] == '\r' && text[start+1] == '\n')
return c == ' ' ? CRLF_SP[length-2] : CRLF_TAB[length-2];
break;
}
default : // fall through to non-interned string copy
}
}
// fallback to non-interned string copy
return new String(text, start, length);
}






Archive powered by MHonArc 2.6.24.

Top of Page