XOM API for Processing XML with Java

[Wolfgang Hoschek <wolfgang.hoschek AT mac.com>]

Can't reproduce this here.

Are you sure you're using woodstox? Other stax impls are known to be  
buggy, including the RI. Running

     XMLInputFactory inputFactory = XMLInputFactory.newInstance();
     System.out.println(inputFactory.getClass().getName());

should yield "com.ctc.wstx.stax.WstxInputFactory".

To be sure, I'd recommend getting rid of *all* jars related to STAX  
on the classpath, except wstx-asl-3.0.jar and stax-1.0.1.jar, both of  
which ship with woodstox. In particular, remove jsr173_1.0_ri.jar and  
jsr173_api.jar.

With a buggy XML parser, there's potential for malformed input to be  
handed to a serializer such as BinaryXMLStreamWriter or  
StreamingBinaryXMLSerializer.

StreamingBinaryXMLSerializer is a high level API, performing full  
wellformedness checks. Serialization would throw an exception if  
there were a problem. In constrast, BinaryXMLStreamWriter is a low  
level API, performing only basic wellformedness checks, and those  
don't include checking for malformed NCNames. BinaryXMLStreamWriter  
would let such malformed data pass through into the output file,  
where the problem would later be detected when attempting to reparse,  
throwing a BinaryParsingException.

[As an aside, XMLInputFactory.newInstance() is expensive, so reusing  
the instance N times is often a good idea].

Wolfgang.

On Jul 25, 2006, at 12:10 PM, Galip Aydin wrote:

> Wolfgang,
> I was able to compile and run the prototype code, however when I  
> try to convert the generated bnux documents back to XML I am  
> getting following error, any ideas?
>
> nux.xom.binary.BinaryParsingException: NCNames cannot start with  
> the character a
> at nux.xom.binary.BinaryXMLCodec.deserialize(BinaryXMLCodec.java:627)
> at nux.xom.binary.BinaryXMLCodec.deserialize(BinaryXMLCodec.java:535)
> at nux.xom.binary.TestNux.bnux2xml(TestNux.java:46)
> at nux.xom.binary.TestNux.main(TestNux.java:60
>
>
> Here is methods I use:
>
>  public void xml2bnux(InputStream in, OutputStream out) throws  
> Exception {
>      //XMLStreamReader reader = StaxUtil.createXMLStreamReader(in,  
> "");
>      XMLStreamWriter writer = new BinaryXMLCodec 
> ().createXMLStreamWriter(out,0);
>      XMLInputFactory inputFactory = XMLInputFactory.newInstance();
>      XMLStreamReader reader = inputFactory.createXMLStreamReader(in);
>      new XMLStreamCopier(reader, writer).copy(false); // performs  
> streaming conversion
>      reader.close();
>      writer.close();
>  }
>
>  public void bnux2xml (byte[] bytes)throws Exception{
>      BinaryXMLCodec bnuxCodec = new BinaryXMLCodec();
>      Document d = bnuxCodec.deserialize(bytes);
>      System.out.println(d.toXML());
>  }
>
> Thanks,
> Galip.
>
> ----- Original Message ----- From: "Wolfgang Hoschek"  
> <wolfgang.hoschek AT mac.com>
> To: "Galip Aydin" <gaydin AT grids.ucs.indiana.edu>
> Cc: <xom-interest AT lists.ibiblio.org>
> Sent: Monday, July 24, 2006 12:51 AM
> Subject: Re: [XOM-interest] BNUX Question
>
>
> > Galip,
> >
> > The prototype source code is at
> >
> > http://dsd.lbl.gov/nux-download/snapshots/bnuxwriter-0.1.tar.gz
> >
> > The main class is BinaryXMLStreamWriter, instantiated via
> >
> > XMLStreamWriter writer = BinaryXMLCodec.createXMLStreamWriter(...)
> >
> > To compile the files into a nux.jar file, copy the unpacked files   
> > into nux/src/nux/xom/binary/
> >
> > and run
> >
> > cd nux; ant jar
> >
> > (Quite a few Stax features aren't implemented, but those aren't   
> > required for copying via XMLStreamCopier from an XMLStreamReader  
> > to  an XMLStreamWriter. Of course, turning this class into  
> > production  quality code would take a bit more effort.)
> >
> > Enjoy,
> > Wolfgang.
> >
> > On Jul 23, 2006, at 12:59 PM, Galip Aydin wrote:
> >
> > > Wolfgang,
> > > First, thank you for taking your time this weekend to do the   
> > > profiling.
> > >
> > > [Let me know if you'd like to experiment with the prototype.]
> > >
> > > Yes I would like that, please let me know how.
> > >
> > > [As an aside, some of the larger documents appear to consist of N
> > > concatenations of a base document. If that's indeed the case, if
> > > would be better to use real data throughout, if available. The  
> > > reason
> > > is that binary tokenization schemes can take advantage of such
> > > artificial redundancy in manners that are not representative of real
> > > world usage.]
> > >
> > >
> > > That is indeed the case, I just wanted to have certain number of  
> > > features per doc, so simply did copy/paste. But I will generate  
> > > a  new test set from real data.
> > >
> > > Galip.