XOM API for Processing XML with Java

["Galip Aydin" <gaydin AT grids.ucs.indiana.edu>]

Wolfgang,
I've been doing some tests to compare BNUX and Fast Infoset performance for 
various document sizes, my findings are;
- Encoding: FI (with SAX) is almost twice as fast as BNUX (with stax) ,
- Decoding: BNUX is 2 to 3 times faster than FI.

So I have no problem with decoding but somehow I can't get good results with 
encoding, and was wondering if I am doing anything wrong. The fastest way I 
could measure for xml2bnux among your previous suggestions is this:

serializer = codec.createStreamingSerializer(out, 0);
redirector = XOMUtil.getRedirectingNodeFactory(serializer);
StaxUtil.createBuilder(null, redirector).build(in);


The sizes of the XML documents are between 500KB to 5MB.
Any ideas what might be happening?
Thanks,
Galip.


----- Original Message ----- 
From: "Wolfgang Hoschek" <wolfgang.hoschek AT mac.com>
To: "Galip Aydin" <gaydin AT grids.ucs.indiana.edu>
Cc: <xom-interest AT lists.ibiblio.org>
Sent: Tuesday, July 18, 2006 6:57 PM
Subject: Re: [XOM-interest] BNUX Question


> On Jul 18, 2006, at 3:42 PM, Wolfgang Hoschek wrote:
>
> > On Jul 18, 2006, at 3:12 PM, Galip Aydin wrote:
> >
> > > Hi All;
> > > What is the fastest way to convert an XML document to bnux and vice-
> > > versa,
> > > and does anybody have any examples?
> > > I see that the last version of bnux is faster than Fast Infoset but
> > > I want
> > > to make sure that I am using the right methods.
> > >
> > > Here is what I use:
> > >      Document doc = new Builder().build(new File(XMLpath));
> > >      BinaryXMLCodec codec = new BinaryXMLCodec();
> > >      byte[] bnuxDoc = codec.serialize(doc, 0);
> >
> > The given code materializes the document as a XOM main memory tree,
> > requiring memory proportional to the size of the input document. It
> > is convenient but can be expensive or even prohibitive for large
> > documents. A more efficient approach is using streaming to convert
> > arbitrarily large XML documents to and from bnux, requiring only
> > memory proportional to the maximum depth of the XML tree, i.e.
> > constant memory for all practical purposes. This is outlined by the
> > example in section "Streaming conversion of standard textual XML to
> > and from binary format"
> > http://dsd.lbl.gov/nux/#Streaming%20conversion%20of%20standard%
> > 20textual%20XML%20to%20and%20from%20binary%20format
> >
> >    // streaming conversion of standard textual XML to bnux binary XML:
> >    InputStream  in  = new FileInputStream("samples/data/weblog.xml");
> >    OutputStream out = new FileOutputStream("/tmp/weblog.xml.bnux");
> >    StreamingSerializerFactory factory = new StreamingSerializerFactory
> > ();
> >    StreamingSerializer serializer = factory.createBinaryXMLSerializer
> > (out, 0);
> >    NodeFactory redirector = XOMUtil.getRedirectingNodeFactory
> > (serializer);
> >
> >    new Builder(redirector).build(in); // performs streaming conversion
> >
> >    in.close();
> >    out.close();
> >
> >
> >
> >    // streaming conversion of bnux binary XML to standard textual XML:
> >    InputStream  in  = new FileInputStream("/tmp/weblog.xml.bnux");
> >    OutputStream out = new FileOutputStream("/tmp/weblog.xml");
> >    StreamingSerializerFactory factory = new StreamingSerializerFactory
> > ();
> >    StreamingSerializer serializer = factory.createXMLSerializer(out,
> > "UTF-8");
> >    NodeFactory redirector = XOMUtil.getRedirectingNodeFactory
> > (serializer);
> >
> >    new BinaryXMLCodec().deserialize(in, redirector); // performs
> > streaming conversion
> >
> >    in.close();
> >    out.close();
> >
> > There's also a small command line test tool that's essentially a
> > glorified version of said streaming example, allowing to convert
> > arbitrarily large XML documents to and from bnux. Example:
> >
> > # bin/fire-bnux < samples/data/teams.xml > /tmp/teams.xml.bnux
> > # bin/fire-bnux < /tmp/teams.xml.bnux > /tmp/teams.xml
> >
> > See class BinaryXMLConverter for the source code. Previously
> > mentioned approach uses SAX. Alternatively, you can also use StAX by
> > replacing the line
> >
> > new Builder(redirector).build(in) // performs streaming conversion
> >
> > with
> >
> > StaxUtil.createBuilder(null, redirector).build(in) // performs
> > streaming conversion
> >
> > Using StAX with woodstox-3.0 is more efficient than SAX with
> > xerces-2.8.0, in particular for very small documents. For detailed
> > numbers, see http://dsd.lbl.gov/~hoschek/tmp/plots-stax2.pdf (test
> > documents are sorted by file size, left to right).
> >
> > For reading or writing large numbers of very small documents (e.g.
> > 200 bytes), it is more efficient to reuse the Builder and
> > BinaryXMLCodec objects, minimizing setup time.
> >
> > Wolfgang.
>
> I should add that when writing standard XML it is more efficient to  use 
> StAX as well (see the plots). In the example above this can be  done by 
> replacing the line
>
> // use XOM serializer:
> StreamingSerializer serializer = factory.createXMLSerializer(out,
> "UTF-8");
>
> with
>
> // use StAX serializer (woodstox-3.0 recommended):
> XMLStreamWriter writer = XMLOutputFactory.newInstance 
> ().createXMLStreamWriter(out, "UTF-8");
> StreamingSerializer serializer = factory.createStaxSerializer(writer);
>
> Again, when using many small documents, reusing the StAX  XMLOutputFactory 
> instance increases efficiency significantly.
>
> Similarly, such object reuse on the XML --> bnux route can be  achieved by 
> replacing
>
> StreamingSerializer serializer = factory.createBinaryXMLSerializer (out, 
> 0);
>
> with
>
> StreamingSerializer serializer = codec.createStreamingSerializer (out, 0);
>
> where "codec" is a BinaryXMLCodec instance that's beeing reused N times.
>
> Wolfgang.