[XOM-interest] NUX, StreamingPathFilter and OutOfMemoryError
Wolfgang Hoschek
wolfgang.hoschek at mac.com
Tue Nov 20 00:47:58 EST 2007
Thanks. Interestingly this OOM can be reproduced with sun java-1.6
but not with java-1.5. I'm not sure why yet.
Wolfgang.
On Nov 19, 2007, at 9:01 PM, Cedric Coulon wrote:
> Hi,
>
> First, thanks you for your help.
>
> I tried with this two options (setProperty entityExpansionLimit and
> java -Xmx10m or 512m) but the result is the same... I use Eclipse,
> jdk 1.6.0_03-b05 and nux 1.6 (with xom-1.1 and saxon9 but I also
> tried with jars included in nux) under Windows XP.
>
> Here is the detailed exception:
>
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
> at com.sun.org.apache.xerces.internal.util.XMLStringBuffer.append
> (XMLStringBuffer.java:205)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.refresh
> (XMLDocumentScannerImpl.java:1493)
> at
> com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.invokeListene
> rs(XMLEntityScanner.java:2070)
> at
> com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.scanQName
> (XMLEntityScanner.java:820)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl
> .scanStartElement(XMLDocumentFragmentScannerImpl.java:1277)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl
> $FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next
> (XMLDocumentScannerImpl.java:645)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl
> .scanDocument(XMLDocumentFragmentScannerImpl.java:508)
> at
> com.sun.org.apache.xerces.internal.parsers.DTDConfiguration.parse
> (DTDConfiguration.java:496)
> at
> com.sun.org.apache.xerces.internal.parsers.DTDConfiguration.parse
> (DTDConfiguration.java:552)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse
> (XMLParser.java:107)
> at
> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse
> (AbstractSAXParser.java:1205)
> at nu.xom.Builder.build(Builder.java:1127)
> at nu.xom.Builder.build(Builder.java:614)
> at nu.xom.Builder.build(Builder.java:990)
> at Parser.main(Parser.java:35)
>
> 35 builder.build(new File("dblp.xml")); // http://
> dblp.uni-trier.de/xml/
>
>
>
> Wolfgang Hoschek wrote:
>> This example works fine here with a 10 MB heap (java -Xmx10m) and
>> the nux-1.6 release. I just downloaded the DBLP data (400 MB)
>> again and ran your snippet with sun java-1.5. BTW, since the DBLP
>> data includes a lot of external entities you need to bump up the
>> entity expansion limit according to http://java.sun.com/
>> webservices/docs/1.5/jaxp/ReleaseNotes.html via
>>
>> System.setProperty("entityExpansionLimit", String.valueOf
>> (Integer.MAX_VALUE));
>>
>> More detailed config and version info for your environment might
>> help to pinpoint the problem.
>>
>> Wolfgang.
>>
>> On Nov 19, 2007, at 2:39 AM, Cedric Coulon wrote:
>>
>>> Hi,
>>>
>>> I want to parse a large xml file and extract the element book.
>>> The xml
>>> source file is dblp database (dblp.xml - 411M -
>>> http://dblp.uni-trier.de/xml/).
>>> So I use NUX and the StreamingPathFilter class. However I have an
>>> OutOfMemoryError... I don't know what I'm doing wrong.
>>>
>>> Thanks you for your help.
>>>
>>> Bye,
>>> Cedric.
>>>
>>> Here is the source of my java program:
>>>
>>> public static void main(String[] args) {
>>> try {
>>> StreamingTransform myTransform = new
>>> StreamingTransform() {
>>> public Nodes transform(Element e) {
>>> System.out.println(XOMUtil.toPrettyXML(e));
>>> return new Nodes();
>>> }
>>> };
>>>
>>> Builder builder = new Builder(new
>>> StreamingPathFilter("/dblp/book", null).createNodeFactory(null,
>>> myTransform));
>>> builder.build(new File("dblp.xml")); //
>>> http://dblp.uni-trier.de/xml/
>>> } catch (Exception e) {
>>> e.printStackTrace();
>>> }
>>> }
>>>
>>> _______________________________________________
>>> XOM-interest mailing list
>>> XOM-interest at lists.ibiblio.org
>>> http://lists.ibiblio.org/mailman/listinfo/xom-interest
>>
>
More information about the XOM-interest
mailing list