XOM API for Processing XML with Java

[Michael Kay <mike AT saxonica.com>]

In that case I would suggest starting with an initial path of 
/xbrli:xbrl/xbrli:context rather than /xbrli:xbrl.

Michael Kay
Saxonica

On 30/04/2012 14:16, Geet Gangwar wrote:
> There multiple occurrences of contexts element say upto 10 million.
>
> Validation logic for each object is same. I need to pick one context
> perform the validation logic, pick another perform the same logic.
>
> Regards
>
> Geet
>
> On Mon, Apr 30, 2012 at 6:33 PM, Michael Kay<mike AT saxonica.com>  wrote:
>
> > Follow-up questions then:
> >
> > (1) how big is an xbrli:context object?
> >
> > (2) is the validation of each xbrli:context object independent of the
> > others?
> >
> > (3) what is the logic of the validation that needs to be performed?
> >
> > Michael Kay
> > Saxonica
> >
> > On 30/04/2012 13:37, Geet Gangwar wrote:
> > > Hi Michael,
> > >
> > > Basically I need to perform some validation on the xml files. So my
> > > requirement is to pick particular block from xml and validate that block.
> > > For example say I want to pull all context i.e
> > >
> > > XPATH expression like :  /xbrli:xbrl/xbrli:context
> > >
> > > from the document and validate them.
> > >
> > > Regards
> > >
> > > Geet
> > >
> > > On Mon, Apr 30, 2012 at 5:41 PM, Michael Kay<mike AT saxonica.com>   wrote:
> > >
> > > > NUX implements what one might call "chunked streaming": it selects a
> > > > sequence of subtrees of your document and applies XQuery processing to
> > > > each one in turn. (This is effectively the same as saxon:stream() in
> > > > Saxon.) You need to have enough memory to hold the largest of these
> > > > subtrees. Because your path /xbrli:xbrl effectively selects the whole
> > > > document, your largest subtree is the same as the document size, so you
> > > > aren't really streaming at all.
> > > >
> > > > If you can describe the query/transformation you want to carry out, then
> > > > I might be able to suggest a way of doing it in a streamed manner either
> > > > using NUX, or using native facilities of Saxon (whose streaming
> > > > capabilities have advanced considerably since NUX was last released).
> > > >
> > > > Michael Kay
> > > > Saxonica
> > > >
> > > >
> > > > 2012/4/30 Geet Gangwar<geetgangwar AT gmail.com>
> > > > > > Hi,
> > > > > >
> > > > > > I have a requirement of processing large XML files of about 4GB. I am
> > > > using
> > > > > > the NUX distribution to process the files.
> > > > > > But when I fire a Xquery to this read this file, I get JAVA Heap Space
> > > > > > error.
> > > > > >
> > > > > > I am using 64bit JDK1.6 and also I have increased the JVM size to 5 GB
> > > > by
> > > > > > -Xmx5G.
> > > > > >
> > > > > > Please help me how to solve this problem.
> > > > > >
> > > > > > I am pasting my sample code also.
> > > > > >
> > > > > > String path = "/xbrli:xbrl/";
> > > > > >                Map prefixes = new HashMap();
> > > > > >                prefixes.put("xbrli", "
> > http://www.xbrl.org/2003/instance
> > > > ");
> > > > > >                prefixes.put("link", "http://www.xbrl.org/2003/linkbase
> > ");
> > > > > >                prefixes.put("xlink", "http://www.w3.org/1999/xlink";);
> > > > > >                prefixes.put("xsd", "http://www.w3.org/2001/XMLSchema
> > ");
> > > > > >                StreamingTransform myTransform = new
> > StreamingTransform() {
> > > > > >                    private Nodes NONE = new Nodes();
> > > > > >
> > > > > >                    // execute XQuery against each element matching
> > > > location
> > > > > > path
> > > > > >                    public Nodes transform(Element subtree) {
> > > > > >                        Nodes results = XQueryUtil.xquery(subtree,
> > > > > > "xbrli:context[id = 'c-01']");
> > > > > >
> > > > > >                        for (int i=0; i<    results.size(); i++) {
> > > > > >                            // do something useful with query results;
> > > > here we
> > > > > > just print them
> > > > > >
> > > > > > System.out.println(XOMUtil.toPrettyXML(results.get(i)));
> > > > > >                        }
> > > > > >                        return NONE; // current subtree becomes subject
> > to
> > > > > > garbage collection
> > > > > >                        // returning empty node list removes current
> > > > subtree
> > > > > > from document being build.
> > > > > >                        // returning new Nodes(subtree) retains the
> > current
> > > > > > subtree.
> > > > > >                        // returning new Nodes(some other nodes)
> > replaces
> > > > the
> > > > > > current subtree with
> > > > > >                        // some other nodes.
> > > > > >                        // if you want (SAX) parsing to terminate at
> > this
> > > > > > point, simply throw an exception
> > > > > >                    }
> > > > > >                };
> > > > > >
> > > > > >                // parse document with a filtering Builder
> > > > > >                StreamingPathFilter filter = new
> > StreamingPathFilter(path,
> > > > > > prefixes);
> > > > > >                Builder builder = new
> > > > Builder(filter.createNodeFactory(null,
> > > > > > myTransform));
> > > > > >                Document doc = builder.build(new
> > > > File("F:/sample/cntxt.xml"));
> > > > > >                System.out.println(doc.getRootElement().getValue());
> > > > > >                System.out.println("doc.size()=" +
> > > > > > doc.getRootElement().getChildCount());
> > > > > >                System.out.println(XOMUtil.toPrettyXML(doc));
> > > > > >
> > > > > > Regards
> > > > > >
> > > > > > Geet
> > > > > > _______________________________________________
> > > > > > XOM-interest mailing list
> > > > > > XOM-interest AT lists.ibiblio.org
> > > > > > http://lists.ibiblio.org/mailman/listinfo/xom-interest
> > > > > _______________________________________________
> > > > > XOM-interest mailing list
> > > > > XOM-interest AT lists.ibiblio.org
> > > > > http://lists.ibiblio.org/mailman/listinfo/xom-interest
> > > > _______________________________________________
> > > > XOM-interest mailing list
> > > > XOM-interest AT lists.ibiblio.org
> > > > http://lists.ibiblio.org/mailman/listinfo/xom-interest
> > > _______________________________________________
> > > XOM-interest mailing list
> > > XOM-interest AT lists.ibiblio.org
> > > http://lists.ibiblio.org/mailman/listinfo/xom-interest
> > _______________________________________________
> > XOM-interest mailing list
> > XOM-interest AT lists.ibiblio.org
> > http://lists.ibiblio.org/mailman/listinfo/xom-interest
> _______________________________________________
> XOM-interest mailing list
> XOM-interest AT lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/xom-interest