XOM API for Processing XML with Java

[Wolfgang Hoschek <wolfgang.hoschek AT mac.com>]

The same OOM happens with java-1.6 if XOM and Nux are excluded from  
the picture completely, by running the XML file through an empty SAX  
handler, even with 100 MB heap. Hm, beyond the SAX parser class name,  
something in the jaxp XMLReader impl must have changed from java-1.5  
to 1.6.


Using SAX parser class:  
com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

    public static void main(String[] args) throws Exception {
        System.setProperty("entityExpansionLimit", String.valueOf 
(Integer.MAX_VALUE));
        XMLReader reader = createXMLReader();
        reader.parse(new InputSource(new FileInputStream(args[0])));
    }
        
    private static XMLReader createXMLReader() throws SAXException,  
ParserConfigurationException {
        SAXParserFactory factory = SAXParserFactory.newInstance();
        factory.setNamespaceAware(true);
        factory.setValidating(false);
        XMLReader reader = factory.newSAXParser().getXMLReader();
        System.out.println("Using SAX parser class: " +  
reader.getClass().getName());
        reader.setFeature("http://xml.org/sax/features/namespaces";,  
true);
        reader.setFeature("http://xml.org/sax/features/namespace- 
prefixes", false);
        return reader;
    }

Wolfgang.

On Nov 19, 2007, at 9:47 PM, Wolfgang Hoschek wrote:

> Thanks. Interestingly this OOM can be reproduced with sun java-1.6
> but not with java-1.5. I'm not sure why yet.
>
> Wolfgang.
>
> On Nov 19, 2007, at 9:01 PM, Cedric Coulon wrote:
>
> > Hi,
> >
> > First, thanks you for your help.
> >
> > I tried with this two options (setProperty entityExpansionLimit and
> > java -Xmx10m or 512m) but the result is the same... I use Eclipse,
> > jdk 1.6.0_03-b05 and nux 1.6 (with xom-1.1 and saxon9 but I also
> > tried with jars included in nux) under Windows XP.
> >
> > Here is the detailed exception:
> >
> > Exception in thread "main" java.lang.OutOfMemoryError: Java heap  
> > space
> >    at com.sun.org.apache.xerces.internal.util.XMLStringBuffer.append
> > (XMLStringBuffer.java:205)
> >    at
> > com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.refres 
> > h
> > (XMLDocumentScannerImpl.java:1493)
> >    at
> > com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.invokeListen 
> > e
> > rs(XMLEntityScanner.java:2070)
> >    at
> > com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.scanQName
> > (XMLEntityScanner.java:820)
> >    at
> > com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImp 
> > l
> > .scanStartElement(XMLDocumentFragmentScannerImpl.java:1277)
> >    at
> > com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImp 
> > l
> > $FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
> >    at
> > com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next
> > (XMLDocumentScannerImpl.java:645)
> >    at
> > com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImp 
> > l
> > .scanDocument(XMLDocumentFragmentScannerImpl.java:508)
> >    at
> > com.sun.org.apache.xerces.internal.parsers.DTDConfiguration.parse
> > (DTDConfiguration.java:496)
> >    at
> > com.sun.org.apache.xerces.internal.parsers.DTDConfiguration.parse
> > (DTDConfiguration.java:552)
> >    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse
> > (XMLParser.java:107)
> >    at
> > com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse
> > (AbstractSAXParser.java:1205)
> >    at nu.xom.Builder.build(Builder.java:1127)
> >    at nu.xom.Builder.build(Builder.java:614)
> >    at nu.xom.Builder.build(Builder.java:990)
> >    at Parser.main(Parser.java:35)
> >
> > 35             builder.build(new File("dblp.xml"));    // http://
> > dblp.uni-trier.de/xml/
> >
> >
> >
> > Wolfgang Hoschek wrote:
> > > This example works fine here with a 10 MB heap (java -Xmx10m) and
> > > the nux-1.6 release. I just downloaded the DBLP data (400 MB)
> > > again and ran your snippet with sun java-1.5. BTW, since the DBLP
> > > data includes a lot of external entities you need to bump up the
> > > entity expansion limit according to http://java.sun.com/
> > > webservices/docs/1.5/jaxp/ReleaseNotes.html via
> > >
> > > System.setProperty("entityExpansionLimit", String.valueOf
> > > (Integer.MAX_VALUE));
> > >
> > > More detailed config and version info for your environment might
> > > help to pinpoint the problem.
> > >
> > > Wolfgang.
> > >
> > > On Nov 19, 2007, at 2:39 AM, Cedric Coulon wrote:
> > >
> > > > Hi,
> > > >
> > > > I want to parse a large xml file and extract the element book.
> > > > The xml
> > > > source file is dblp database (dblp.xml - 411M -
> > > > http://dblp.uni-trier.de/xml/).
> > > > So I use NUX and the StreamingPathFilter class. However I have an
> > > > OutOfMemoryError... I don't know what I'm doing wrong.
> > > >
> > > > Thanks you for your help.
> > > >
> > > > Bye,
> > > > Cedric.
> > > >
> > > > Here is the source of my java program:
> > > >
> > > >     public static void main(String[] args) {
> > > >         try {
> > > >             StreamingTransform myTransform = new
> > > > StreamingTransform() {
> > > >                 public Nodes transform(Element e) {
> > > >                     System.out.println(XOMUtil.toPrettyXML(e));
> > > >                     return new Nodes();
> > > >                 }
> > > >             };
> > > >
> > > >             Builder builder = new Builder(new
> > > > StreamingPathFilter("/dblp/book", null).createNodeFactory(null,
> > > > myTransform));
> > > >             builder.build(new File("dblp.xml"));    //
> > > > http://dblp.uni-trier.de/xml/
> > > >         } catch (Exception e) {
> > > >             e.printStackTrace();
> > > >         }
> > > >     }
> > > >
> > > > _______________________________________________
> > > > XOM-interest mailing list
> > > > XOM-interest AT lists.ibiblio.org
> > > > http://lists.ibiblio.org/mailman/listinfo/xom-interest
>
> _______________________________________________
> XOM-interest mailing list
> XOM-interest AT lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/xom-interest