[XOM-interest] NUX, StreamingPathFilter and OutOfMemoryError

Wolfgang Hoschek wolfgang.hoschek at mac.com
Tue Nov 20 00:47:58 EST 2007


Thanks. Interestingly this OOM can be reproduced with sun java-1.6  
but not with java-1.5. I'm not sure why yet.

Wolfgang.

On Nov 19, 2007, at 9:01 PM, Cedric Coulon wrote:

> Hi,
>
> First, thanks you for your help.
>
> I tried with this two options (setProperty entityExpansionLimit and  
> java -Xmx10m or 512m) but the result is the same... I use Eclipse,  
> jdk 1.6.0_03-b05 and nux 1.6 (with xom-1.1 and saxon9 but I also  
> tried with jars included in nux) under Windows XP.
>
> Here is the detailed exception:
>
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>    at com.sun.org.apache.xerces.internal.util.XMLStringBuffer.append 
> (XMLStringBuffer.java:205)
>    at  
> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.refresh 
> (XMLDocumentScannerImpl.java:1493)
>    at  
> com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.invokeListene 
> rs(XMLEntityScanner.java:2070)
>    at  
> com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.scanQName 
> (XMLEntityScanner.java:820)
>    at  
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl 
> .scanStartElement(XMLDocumentFragmentScannerImpl.java:1277)
>    at  
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl 
> $FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
>    at  
> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next 
> (XMLDocumentScannerImpl.java:645)
>    at  
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl 
> .scanDocument(XMLDocumentFragmentScannerImpl.java:508)
>    at  
> com.sun.org.apache.xerces.internal.parsers.DTDConfiguration.parse 
> (DTDConfiguration.java:496)
>    at  
> com.sun.org.apache.xerces.internal.parsers.DTDConfiguration.parse 
> (DTDConfiguration.java:552)
>    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse 
> (XMLParser.java:107)
>    at  
> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse 
> (AbstractSAXParser.java:1205)
>    at nu.xom.Builder.build(Builder.java:1127)
>    at nu.xom.Builder.build(Builder.java:614)
>    at nu.xom.Builder.build(Builder.java:990)
>    at Parser.main(Parser.java:35)
>
> 35             builder.build(new File("dblp.xml"));    // http:// 
> dblp.uni-trier.de/xml/
>
>
>
> Wolfgang Hoschek wrote:
>> This example works fine here with a 10 MB heap (java -Xmx10m) and  
>> the nux-1.6 release. I just downloaded the DBLP data (400 MB)  
>> again and ran your snippet with sun java-1.5. BTW, since the DBLP  
>> data includes a lot of external entities you need to bump up the  
>> entity expansion limit according to http://java.sun.com/ 
>> webservices/docs/1.5/jaxp/ReleaseNotes.html via
>>
>> System.setProperty("entityExpansionLimit", String.valueOf 
>> (Integer.MAX_VALUE));
>>
>> More detailed config and version info for your environment might  
>> help to pinpoint the problem.
>>
>> Wolfgang.
>>
>> On Nov 19, 2007, at 2:39 AM, Cedric Coulon wrote:
>>
>>> Hi,
>>>
>>> I want to parse a large xml file and extract the element book.  
>>> The xml
>>> source file is dblp database (dblp.xml - 411M -
>>> http://dblp.uni-trier.de/xml/).
>>> So I use NUX and the StreamingPathFilter class. However I have an
>>> OutOfMemoryError... I don't know what I'm doing wrong.
>>>
>>> Thanks you for your help.
>>>
>>> Bye,
>>> Cedric.
>>>
>>> Here is the source of my java program:
>>>
>>>     public static void main(String[] args) {
>>>         try {
>>>             StreamingTransform myTransform = new  
>>> StreamingTransform() {
>>>                 public Nodes transform(Element e) {
>>>                     System.out.println(XOMUtil.toPrettyXML(e));
>>>                     return new Nodes();
>>>                 }
>>>             };
>>>
>>>             Builder builder = new Builder(new
>>> StreamingPathFilter("/dblp/book", null).createNodeFactory(null,
>>> myTransform));
>>>             builder.build(new File("dblp.xml"));    //
>>> http://dblp.uni-trier.de/xml/
>>>         } catch (Exception e) {
>>>             e.printStackTrace();
>>>         }
>>>     }
>>>
>>> _______________________________________________
>>> XOM-interest mailing list
>>> XOM-interest at lists.ibiblio.org
>>> http://lists.ibiblio.org/mailman/listinfo/xom-interest
>>
>



More information about the XOM-interest mailing list