xom-interest AT lists.ibiblio.org

Subject: XOM API for Processing XML with Java

List archive

[XOM-interest] Bug: XOM's rules for system ids are too stringent

From: John Cowan <cowan AT ccil.org>
To: xom-interest AT lists.ibiblio.org
Subject: [XOM-interest] Bug: XOM's rules for system ids are too stringent
Date: Thu, 21 Feb 2008 11:17:22 -0500

Currently, XOM throws an error if the system identifier in a DOCTYPE
declaration is not a well-formed URI. But that's too stringent: the
XML Recommendation says that a system identifier has to be *convertible*
to a well-formed URI by escaping inappropriate characters. The ipsissima
verba are in Section 4.2.2 of XML 1.0:

System identifiers (and other XML strings meant to be used as URI
references) may contain characters that, according to [IETF RFC
3986], must be escaped before a URI can be used to retrieve the
referenced resource. The characters to be escaped are the control
characters #x0 to #x1F and #x7F (most of which cannot appear in
XML), space #x20, the delimiters '<' #x3C, '>' #x3E and '"' #x22,
the unwise characters '{' #x7B, '}' #x7D, '|' #x7C, '\' #x5C,
'^' #x5E and '`' #x60, as well as all characters above #x7F.

This came up from a private bug report to me as TagSoup maintainer
(don't they all): a page with this DOCTYPE declaration:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.
org/TR/1999/REC-html401-19991224/loose.dtd">

caused XOM to throw an "illegal host character" exception because of the
newline,
whereas it should have been treated the same as

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.%0Aorg/TR/1999/REC-html401-19991224/loose.dtd";>

Now it's true that no sane DNS registrar will allow you to register
a hostname with an embedded newline (though that is a social, not a
technical, limitation of DNS) and I will be adding code to the next
TagSoup release to scrub system identifiers, but it seems clear that the
letter of RFC 3986 permits it: see the host, reg-name, and pct-encoded
productions.

--
If you have ever wondered if you are in hell, John Cowan
it has been said, then you are on a well-traveled
http://www.ccil.org/~cowan
road of spiritual inquiry. If you are absolutely cowan AT ccil.org
sure you are in hell, however, then you must be
on the Cross Bronx Expressway. --Alan Feuer, NYTimes, 2002-09-20

[XOM-interest] Bug: XOM's rules for system ids are too stringent, John Cowan, 02/21/2008
- Re: [XOM-interest] Bug: XOM's rules for system ids are too stringent, Elliotte Rusty Harold, 02/21/2008
  - Re: [XOM-interest] Bug: XOM's rules for system ids are too stringent, John Cowan, 02/22/2008
    - Re: [XOM-interest] Bug: XOM's rules for system ids are too stringent, Elliotte Harold, 02/23/2008