XOM API for Processing XML with Java

["Galip Aydin" <gaydin AT grids.ucs.indiana.edu>]

Wolfgang,

I tried to re-compile the snapshot code but got this error message;
"BinaryXMLCodec.java": getAttributeTypeCode(nu.xom.Attribute) in 
nux.xom.binary.Util cannot be applied to (nu.xom.Attribute.Type) at line 
1432, column 37

The line that causes this message:
nodeTokens.add(Util.getAttributeTypeCode(attrType));

I previously removed this line to get it to compile (bad idea!) which may be 
the reason I was getting the error message I mentioned in my previous 
message.

Do you know if anything is wrong with this line?

Thanks
Galip.

----- Original Message ----- 
From: "Wolfgang Hoschek" <wolfgang.hoschek AT mac.com>
To: "Galip Aydin" <gaydin AT grids.ucs.indiana.edu>
Cc: <xom-interest AT lists.ibiblio.org>
Sent: Tuesday, July 25, 2006 8:34 PM
Subject: Re: [XOM-interest] BNUX Question


> Can't reproduce this here.
>
> Are you sure you're using woodstox? Other stax impls are known to be 
> buggy, including the RI. Running
>
>      XMLInputFactory inputFactory = XMLInputFactory.newInstance();
>      System.out.println(inputFactory.getClass().getName());
>
> should yield "com.ctc.wstx.stax.WstxInputFactory".
>
> To be sure, I'd recommend getting rid of *all* jars related to STAX  on 
> the classpath, except wstx-asl-3.0.jar and stax-1.0.1.jar, both of  which 
> ship with woodstox. In particular, remove jsr173_1.0_ri.jar and 
> jsr173_api.jar.
>
> With a buggy XML parser, there's potential for malformed input to be 
> handed to a serializer such as BinaryXMLStreamWriter or 
> StreamingBinaryXMLSerializer.
>
> StreamingBinaryXMLSerializer is a high level API, performing full 
> wellformedness checks. Serialization would throw an exception if  there 
> were a problem. In constrast, BinaryXMLStreamWriter is a low  level API, 
> performing only basic wellformedness checks, and those  don't include 
> checking for malformed NCNames. BinaryXMLStreamWriter  would let such 
> malformed data pass through into the output file,  where the problem would 
> later be detected when attempting to reparse,  throwing a 
> BinaryParsingException.
>
> [As an aside, XMLInputFactory.newInstance() is expensive, so reusing  the 
> instance N times is often a good idea].
>
> Wolfgang.
>
> On Jul 25, 2006, at 12:10 PM, Galip Aydin wrote:
>
> > Wolfgang,
> > I was able to compile and run the prototype code, however when I  try to 
> > convert the generated bnux documents back to XML I am  getting following 
> > error, any ideas?
> >
> > nux.xom.binary.BinaryParsingException: NCNames cannot start with  the 
> > character a
> > at nux.xom.binary.BinaryXMLCodec.deserialize(BinaryXMLCodec.java:627)
> > at nux.xom.binary.BinaryXMLCodec.deserialize(BinaryXMLCodec.java:535)
> > at nux.xom.binary.TestNux.bnux2xml(TestNux.java:46)
> > at nux.xom.binary.TestNux.main(TestNux.java:60
> >
> >
> > Here is methods I use:
> >
> >  public void xml2bnux(InputStream in, OutputStream out) throws  Exception 
> > {
> >      //XMLStreamReader reader = StaxUtil.createXMLStreamReader(in,  "");
> >      XMLStreamWriter writer = new BinaryXMLCodec 
> > ().createXMLStreamWriter(out,0);
> >      XMLInputFactory inputFactory = XMLInputFactory.newInstance();
> >      XMLStreamReader reader = inputFactory.createXMLStreamReader(in);
> >      new XMLStreamCopier(reader, writer).copy(false); // performs 
> > streaming conversion
> >      reader.close();
> >      writer.close();
> >  }
> >
> >  public void bnux2xml (byte[] bytes)throws Exception{
> >      BinaryXMLCodec bnuxCodec = new BinaryXMLCodec();
> >      Document d = bnuxCodec.deserialize(bytes);
> >      System.out.println(d.toXML());
> >  }
> >
> > Thanks,
> > Galip.
> >
> > ----- Original Message ----- From: "Wolfgang Hoschek" 
> > <wolfgang.hoschek AT mac.com>
> > To: "Galip Aydin" <gaydin AT grids.ucs.indiana.edu>
> > Cc: <xom-interest AT lists.ibiblio.org>
> > Sent: Monday, July 24, 2006 12:51 AM
> > Subject: Re: [XOM-interest] BNUX Question
> >
> >
> > > Galip,
> > >
> > > The prototype source code is at
> > >
> > > http://dsd.lbl.gov/nux-download/snapshots/bnuxwriter-0.1.tar.gz
> > >
> > > The main class is BinaryXMLStreamWriter, instantiated via
> > >
> > > XMLStreamWriter writer = BinaryXMLCodec.createXMLStreamWriter(...)
> > >
> > > To compile the files into a nux.jar file, copy the unpacked files   into 
> > > nux/src/nux/xom/binary/
> > >
> > > and run
> > >
> > > cd nux; ant jar
> > >
> > > (Quite a few Stax features aren't implemented, but those aren't 
> > > required for copying via XMLStreamCopier from an XMLStreamReader  to  an 
> > > XMLStreamWriter. Of course, turning this class into  production  quality 
> > > code would take a bit more effort.)
> > >
> > > Enjoy,
> > > Wolfgang.
> > >
> > > On Jul 23, 2006, at 12:59 PM, Galip Aydin wrote:
> > >
> > > > Wolfgang,
> > > > First, thank you for taking your time this weekend to do the 
> > > > profiling.
> > > >
> > > > [Let me know if you'd like to experiment with the prototype.]
> > > >
> > > > Yes I would like that, please let me know how.
> > > >
> > > > [As an aside, some of the larger documents appear to consist of N
> > > > concatenations of a base document. If that's indeed the case, if
> > > > would be better to use real data throughout, if available. The  reason
> > > > is that binary tokenization schemes can take advantage of such
> > > > artificial redundancy in manners that are not representative of real
> > > > world usage.]
> > > >
> > > >
> > > > That is indeed the case, I just wanted to have certain number of 
> > > > features per doc, so simply did copy/paste. But I will generate  a  new 
> > > > test set from real data.
> > > >
> > > > Galip.