Skip to Content.
Sympa Menu

xom-interest - Re: [XOM-interest] BNUX Question

xom-interest AT lists.ibiblio.org

Subject: XOM API for Processing XML with Java

List archive

Chronological Thread  
  • From: "Galip Aydin" <gaydin AT grids.ucs.indiana.edu>
  • To: "Wolfgang Hoschek" <wolfgang.hoschek AT mac.com>
  • Cc: xom-interest AT lists.ibiblio.org
  • Subject: Re: [XOM-interest] BNUX Question
  • Date: Thu, 20 Jul 2006 18:28:02 -0400

Wolfgang,
I've been doing some tests to compare BNUX and Fast Infoset performance for various document sizes, my findings are;
- Encoding: FI (with SAX) is almost twice as fast as BNUX (with stax) ,
- Decoding: BNUX is 2 to 3 times faster than FI.

So I have no problem with decoding but somehow I can't get good results with encoding, and was wondering if I am doing anything wrong. The fastest way I could measure for xml2bnux among your previous suggestions is this:

serializer = codec.createStreamingSerializer(out, 0);
redirector = XOMUtil.getRedirectingNodeFactory(serializer);
StaxUtil.createBuilder(null, redirector).build(in);


The sizes of the XML documents are between 500KB to 5MB.
Any ideas what might be happening?
Thanks,
Galip.


----- Original Message ----- From: "Wolfgang Hoschek" <wolfgang.hoschek AT mac.com>
To: "Galip Aydin" <gaydin AT grids.ucs.indiana.edu>
Cc: <xom-interest AT lists.ibiblio.org>
Sent: Tuesday, July 18, 2006 6:57 PM
Subject: Re: [XOM-interest] BNUX Question


On Jul 18, 2006, at 3:42 PM, Wolfgang Hoschek wrote:

On Jul 18, 2006, at 3:12 PM, Galip Aydin wrote:

Hi All;
What is the fastest way to convert an XML document to bnux and vice-
versa,
and does anybody have any examples?
I see that the last version of bnux is faster than Fast Infoset but
I want
to make sure that I am using the right methods.

Here is what I use:
Document doc = new Builder().build(new File(XMLpath));
BinaryXMLCodec codec = new BinaryXMLCodec();
byte[] bnuxDoc = codec.serialize(doc, 0);

The given code materializes the document as a XOM main memory tree,
requiring memory proportional to the size of the input document. It
is convenient but can be expensive or even prohibitive for large
documents. A more efficient approach is using streaming to convert
arbitrarily large XML documents to and from bnux, requiring only
memory proportional to the maximum depth of the XML tree, i.e.
constant memory for all practical purposes. This is outlined by the
example in section "Streaming conversion of standard textual XML to
and from binary format"
http://dsd.lbl.gov/nux/#Streaming%20conversion%20of%20standard%
20textual%20XML%20to%20and%20from%20binary%20format

// streaming conversion of standard textual XML to bnux binary XML:
InputStream in = new FileInputStream("samples/data/weblog.xml");
OutputStream out = new FileOutputStream("/tmp/weblog.xml.bnux");
StreamingSerializerFactory factory = new StreamingSerializerFactory
();
StreamingSerializer serializer = factory.createBinaryXMLSerializer
(out, 0);
NodeFactory redirector = XOMUtil.getRedirectingNodeFactory
(serializer);

new Builder(redirector).build(in); // performs streaming conversion

in.close();
out.close();



// streaming conversion of bnux binary XML to standard textual XML:
InputStream in = new FileInputStream("/tmp/weblog.xml.bnux");
OutputStream out = new FileOutputStream("/tmp/weblog.xml");
StreamingSerializerFactory factory = new StreamingSerializerFactory
();
StreamingSerializer serializer = factory.createXMLSerializer(out,
"UTF-8");
NodeFactory redirector = XOMUtil.getRedirectingNodeFactory
(serializer);

new BinaryXMLCodec().deserialize(in, redirector); // performs
streaming conversion

in.close();
out.close();

There's also a small command line test tool that's essentially a
glorified version of said streaming example, allowing to convert
arbitrarily large XML documents to and from bnux. Example:

# bin/fire-bnux < samples/data/teams.xml > /tmp/teams.xml.bnux
# bin/fire-bnux < /tmp/teams.xml.bnux > /tmp/teams.xml

See class BinaryXMLConverter for the source code. Previously
mentioned approach uses SAX. Alternatively, you can also use StAX by
replacing the line

new Builder(redirector).build(in) // performs streaming conversion

with

StaxUtil.createBuilder(null, redirector).build(in) // performs
streaming conversion

Using StAX with woodstox-3.0 is more efficient than SAX with
xerces-2.8.0, in particular for very small documents. For detailed
numbers, see http://dsd.lbl.gov/~hoschek/tmp/plots-stax2.pdf (test
documents are sorted by file size, left to right).

For reading or writing large numbers of very small documents (e.g.
200 bytes), it is more efficient to reuse the Builder and
BinaryXMLCodec objects, minimizing setup time.

Wolfgang.


I should add that when writing standard XML it is more efficient to use StAX as well (see the plots). In the example above this can be done by replacing the line

// use XOM serializer:
StreamingSerializer serializer = factory.createXMLSerializer(out,
"UTF-8");

with

// use StAX serializer (woodstox-3.0 recommended):
XMLStreamWriter writer = XMLOutputFactory.newInstance ().createXMLStreamWriter(out, "UTF-8");
StreamingSerializer serializer = factory.createStaxSerializer(writer);

Again, when using many small documents, reusing the StAX XMLOutputFactory instance increases efficiency significantly.

Similarly, such object reuse on the XML --> bnux route can be achieved by replacing

StreamingSerializer serializer = factory.createBinaryXMLSerializer (out, 0);

with

StreamingSerializer serializer = codec.createStreamingSerializer (out, 0);

where "codec" is a BinaryXMLCodec instance that's beeing reused N times.

Wolfgang.






Archive powered by MHonArc 2.6.24.

Top of Page