XOM API for Processing XML with Java

[Wolfgang Hoschek <whoschek AT lbl.gov>]

And since the whitespace interning isn't really applicable to the  
Text version with UTF8 storage, it can be skipped via

    private static String firstCharacters(char[] text, int start,  
int length) {
        if (Text.IS_FAT && length < SPACES.length) {
            int i = start + length;
            ...
        }
        return new String(text, start, length);
    }

    Text.java:
         static final boolean IS_FAT = true/false depending on the  
version


Wolfgang.

On Jul 22, 2005, at 3:16 PM, Wolfgang Hoschek wrote:

> The
>
> > algorithm is really not that difficult, and there's a reason it is
> > formulated the way it is.
> >
> > Wolfgang.
>
> A little cosmetic modularization yields:
>
>     public void characters(char[] text, int start, int length) {
>         /*
>          * Based on measurements of a wide variety of real-world  
> documents, one
>          * call to characters() per flushText() is extremely  
> common, a second
>          * one quite uncommon, and more calls to it are extremely  
> uncommon.
>          * Hence, unnecessary temporary String and StringBuffer  
> memory copies
>          * and allocations can be avoided, improving performance,  
> *irrespective* whether
>          * the "fat" stringified Text.java is used, or the  
> Text.java version with
>          * the UTF-8 storage trick.
>          * See endDocument() on how to measure character() per  
> flushText() frequency, uncommenting
>          * some diagnostic statistics gathering code.
>          *
>          * Plus, whitepace separator lines make up a large fraction  
> of data in
>          * heavily pretty printed documents. So, let's share most  
> of those
>          * string objects via an extremely low-overhead interning  
> mechanism.
>          *
>          * All in all this should make average memory consumption  
> of a XOM
>          * document with the stringified Text patch comparable or  
> lower than with the
>          * Text.java UTF-8 trick. Plus, it retains the performance  
> benefits of
>          * the stringified Text patch, for use cases reading or  
> writing Text a
>          * lot, for example many XPath/XQuery use cases.
>          */
>         // invariants throughout this class:
>         // only one of (first, buffer) has content, i.e. at any  
> time there holds:
>         // first  != null implies buffer == null
>         // buffer != null implies first == null
>         // if there is content, it is non-empty, i.e at any time  
> there holds:
>         // first  != null implies first.length() > 0
>         // buffer != null implies buffer.length() > 0
>
>         if (length <= 0) return; // nothing to do (important: do  
> not remove; ensures correct algo semantics)
>
>         if (first == null && buffer == null) { // first call to  
> characters()
>             first = firstCharacters(text, start, length);
> //            first = new String(text, start, length);
>         } else {
>             if (buffer == null) { // second call to characters();  
> infrequently called
>                 buffer = new StringBuffer(first.length() +  
> length); // doesn't waste memory
>                 buffer.append(first);
>                 first = null;
>             }
>             // second or higher call to characters(); third or  
> higher call is very rare
>             buffer.append(text, start, length);
>         }
>         if (finishedCDATA) inCDATA = false;
> //        callCounter++;
>     }
>
>
>     private static String firstCharacters(char[] text, int start,  
> int length) {
>         if (length < SPACES.length) { // any possibility for  
> interning whitespace-only?
>             // scan backwards to beginning of run of blanks or tabs
>             int i = start + length;
>             char c = text[--i];
>             if (c != ' ' && c != '\t') {
>                 c = ' ';
>             } else {
>                 i++; while (--i >= start && text[i] == c) ;
>             }
>
>             switch (i+1-start) { // is there internable whitespace- 
> only?
>                 case 0: {
>                     return c == ' ' ? SPACES[length] : TABS[length];
>                 }
>                 case 1: {
>                     if (text[start] == '\n')
>                         return c == ' ' ? LF_SP[length-1] : LF_TAB 
> [length-1];
>                     break;
>                 }
>                 case 2: {
>                     if (text[start] == '\r' && text[start+1] == '\n')
>                         return c == ' ' ? CRLF_SP[length-2] :  
> CRLF_TAB[length-2];
>                     break;
>                 }
>                 default : // fall through to non-interned string copy
>             }
>         }
>         // fallback to non-interned string copy
>         return new String(text, start, length);
>     }