Skip to Content.
Sympa Menu

xom-interest - Re: [XOM-interest] open HTML entities

xom-interest AT lists.ibiblio.org

Subject: XOM API for Processing XML with Java

List archive

Chronological Thread  
  • From: "Michael Kay" <mike AT saxonica.com>
  • To: "'Diego Campo'" <diego.campo AT properazzi.com>
  • Cc: elharo AT metalab.unc.edu, xom-interest AT lists.ibiblio.org
  • Subject: Re: [XOM-interest] open HTML entities
  • Date: Wed, 16 Jan 2008 15:36:58 -0000

Well, what we've learnt from this is that you are using the TagSoup HTML
parser (why didn't you tell us that before?). This is where you should now
focus your investigations, not on XOM.

The input HTML is invalid ("&" in HTML should be escaped as "&amp;", just as
in XML). TagSoup usually does a very good job of coping with invalid HTML,
but you can't expect it to succeed every time. After all, the only way it can
tell whether "&lang" was supposed to be "&amp;lang" or "&lang;" is by
guessing, and sometimes it will guess wrong.

Michael Kay
http://www.saxonica.com/


> -----Original Message-----
> From: Diego Campo [mailto:diego.campo AT properazzi.com]
> Sent: 16 January 2008 15:20
> To: Michael Kay
> Cc: elharo AT metalab.unc.edu; xom-interest AT lists.ibiblio.org
> Subject: RE: [XOM-interest] open HTML entities
>
> Sure, sorry about that.
>
> I attach the xml code.
> The important areas are like this:
>
> <td><div align="right"><a
> href="http://www.sucasita.es/search.cgi?wherefrom=web&lang=D&j
> ob=detail&ref=PAN-100-A03">Details</a></div></td>
>
> This attribute href comes as
> http://www.sucasita.es/search.cgi?wherefrom=web
> 〈=D&job=detail&ref=PAN-100-A03
>
> because the &lang is understood as its HTML Entity
> conversion, '〈'. The &lang is not followed by a ';', so I
> assumed it wouldn't be converted as its value. I'd like to
> know if this is a bug or an expected behaviour.
>
> There are a few of them on the code as you can see. This is a
> site of real estate properties, and this is the link to the
> page with details of the listing.
>
> The code provided code is the printout of the String I have.
> This String come from a process of conversion from the page's
> charset to UTF8. This is already done so don't worry about that.
>
> Then I feed a StringReader with it.
>
> Then I get the nu.xom.Builder with the method getBuilder() below.
>
> Should you need any more information about the
> TaggingNodeFactory.java, or anything else, please let me know.
>
> Thanks a lot for your time.
>
>
> ================================================================
>
> public final static String[] EMPTY_HTML_ELEMENTS = {
> "marquee","option", "script",
> "select", "style", "title" };
>
> protected Builder getBuilder(){
> try{
> XMLReader xmlreader;
> long start = System.currentTimeMillis();
> xmlreader =
> XMLReaderFactory.createXMLReader("org.ccil.cowan.tagsoup.Parser");
>
> xmlreader.setFeature(org.ccil.cowan.tagsoup.Parser.bogonsEmptyFeature,
> false);
> HTMLSchema theSchema = new HTMLSchema(){
> public String getURI() {
> return "";
> }
> public String getPrefx() {
> return "";
> }
> };
> for(int i=0;i <
> TaggingNodeFactory.EMPTY_HTML_ELEMENTS.length;i++){
> /*
> * there's no way to tell
> tagsoup to ignore certain elements and their content.
> * the best way to do this is
> to pass them as cdata, and ignore their content in the
> * NodeFactory
> */
> ElementType et=
> theSchema.getElementType(TaggingNodeFactory.EMPTY_HTML_ELEMENTS[i]);
> // ignorable elements are
> always type cdata
> if(et == null){
> //add schema definition
> if it doesn't exist
>
> theSchema.elementType(TaggingNodeFactory.EMPTY_HTML_ELEMENTS[i],
> Schema.M_EMPTY, HTMLSchema.M_PCDATA|HTMLSchema.M_INLINE,
> Schema.F_CDATA);
> //elementType("b",
> M_PCDATA|M_INLINE, M_INLINE|M_NOLINK, F_RESTART);
> }
> else{
> et.setFlags(Schema.F_CDATA);
>
> /* some html elements
> like <br> <p> don't require closing tags and have no content
> *
> * html elements that
> require closing tags should be allowed to contain anything.
> *
> */
> if(et.model() != Schema.M_EMPTY)
>
> et.setModel(Schema.M_ANY);
> }
> }
>
> xmlreader.setProperty(org.ccil.cowan.tagsoup.Parser.schemaProperty,
> theSchema);
>
> long before = System.currentTimeMillis();
> Builder builder = new
> Builder(xmlreader, false, new TaggingNodeFactory(this,true));
> return builder;
> }
> catch(Exception e){
>
> _logger.error(EncodingUtils.exceptionToString(e));
> return null;
> }
>
>
> }
>
>
> El mié, 16-01-2008 a las 12:03 +0000, Michael Kay escribió:
> > > > The thing is, I don't expect it to be taken as an entity,
> > > because it's
> > > > not. Since xom has correctness as its first goal, this
> surprises me.
> > > >
> > > > I hope this is clear enough.
> > > >
> > >
> > > I'm sorry but it's not. You need to provide a complete test case
> > > that shows the input XML document (not just the URL), the
> code that
> > > reads it, and the output.
> > >
> >
> > It looks to me as if you are trying to parse the input
> twice. No idea
> > how, but if you want help from the list in debugging your code, you
> > will have to show us your code, and that is what I would be
> looking for.
> >
> > Michael Kay
> > http://www.saxonica.com/
> >
>





Archive powered by MHonArc 2.6.24.

Top of Page