[XOM-interest] NUX, StreamingPathFilter and OutOfMemoryError
Cedric Coulon
coulon at comp.nus.edu.sg
Tue Nov 20 00:01:48 EST 2007
Hi,
First, thanks you for your help.
I tried with this two options (setProperty entityExpansionLimit and java
-Xmx10m or 512m) but the result is the same... I use Eclipse, jdk
1.6.0_03-b05 and nux 1.6 (with xom-1.1 and saxon9 but I also tried with
jars included in nux) under Windows XP.
Here is the detailed exception:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at
com.sun.org.apache.xerces.internal.util.XMLStringBuffer.append(XMLStringBuffer.java:205)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.refresh(XMLDocumentScannerImpl.java:1493)
at
com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.invokeListeners(XMLEntityScanner.java:2070)
at
com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.scanQName(XMLEntityScanner.java:820)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(XMLDocumentFragmentScannerImpl.java:1277)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:645)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
at
com.sun.org.apache.xerces.internal.parsers.DTDConfiguration.parse(DTDConfiguration.java:496)
at
com.sun.org.apache.xerces.internal.parsers.DTDConfiguration.parse(DTDConfiguration.java:552)
at
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
at
com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
at nu.xom.Builder.build(Builder.java:1127)
at nu.xom.Builder.build(Builder.java:614)
at nu.xom.Builder.build(Builder.java:990)
at Parser.main(Parser.java:35)
35 builder.build(new File("dblp.xml")); //
http://dblp.uni-trier.de/xml/
Wolfgang Hoschek wrote:
> This example works fine here with a 10 MB heap (java -Xmx10m) and the
> nux-1.6 release. I just downloaded the DBLP data (400 MB) again and
> ran your snippet with sun java-1.5. BTW, since the DBLP data includes
> a lot of external entities you need to bump up the entity expansion
> limit according to
> http://java.sun.com/webservices/docs/1.5/jaxp/ReleaseNotes.html via
>
> System.setProperty("entityExpansionLimit",
> String.valueOf(Integer.MAX_VALUE));
>
> More detailed config and version info for your environment might help
> to pinpoint the problem.
>
> Wolfgang.
>
> On Nov 19, 2007, at 2:39 AM, Cedric Coulon wrote:
>
>> Hi,
>>
>> I want to parse a large xml file and extract the element book. The xml
>> source file is dblp database (dblp.xml - 411M -
>> http://dblp.uni-trier.de/xml/).
>> So I use NUX and the StreamingPathFilter class. However I have an
>> OutOfMemoryError... I don't know what I'm doing wrong.
>>
>> Thanks you for your help.
>>
>> Bye,
>> Cedric.
>>
>> Here is the source of my java program:
>>
>> public static void main(String[] args) {
>> try {
>> StreamingTransform myTransform = new StreamingTransform() {
>> public Nodes transform(Element e) {
>> System.out.println(XOMUtil.toPrettyXML(e));
>> return new Nodes();
>> }
>> };
>>
>> Builder builder = new Builder(new
>> StreamingPathFilter("/dblp/book", null).createNodeFactory(null,
>> myTransform));
>> builder.build(new File("dblp.xml")); //
>> http://dblp.uni-trier.de/xml/
>> } catch (Exception e) {
>> e.printStackTrace();
>> }
>> }
>>
>> _______________________________________________
>> XOM-interest mailing list
>> XOM-interest at lists.ibiblio.org
>> http://lists.ibiblio.org/mailman/listinfo/xom-interest
>
More information about the XOM-interest
mailing list