[XOM-interest] Bug: XOM's rules for system ids are too stringent

John Cowan cowan at ccil.org
Thu Feb 21 11:17:22 EST 2008


Currently, XOM throws an error if the system identifier in a DOCTYPE
declaration is not a well-formed URI.  But that's too stringent:  the
XML Recommendation says that a system identifier has to be *convertible*
to a well-formed URI by escaping inappropriate characters.  The ipsissima
verba are in Section 4.2.2 of XML 1.0:

	System identifiers (and other XML strings meant to be used as URI
	references) may contain characters that, according to [IETF RFC
	3986], must be escaped before a URI can be used to retrieve the
	referenced resource. The characters to be escaped are the control
	characters #x0 to #x1F and #x7F (most of which cannot appear in
	XML), space #x20, the delimiters '<' #x3C, '>' #x3E and '"' #x22,
	the unwise characters '{' #x7B, '}' #x7D, '|' #x7C, '\' #x5C,
	'^' #x5E and '`' #x60, as well as all characters above #x7F.

This came up from a private bug report to me as TagSoup maintainer
(don't they all): a page with this DOCTYPE declaration:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.
org/TR/1999/REC-html401-19991224/loose.dtd">

caused XOM to throw an "illegal host character" exception because of the newline,
whereas it should have been treated the same as

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.%0Aorg/TR/1999/REC-html401-19991224/loose.dtd">

Now it's true that no sane DNS registrar will allow you to register
a hostname with an embedded newline (though that is a social, not a
technical, limitation of DNS) and I will be adding code to the next
TagSoup release to scrub system identifiers, but it seems clear that the
letter of RFC 3986 permits it: see the host, reg-name, and pct-encoded
productions.

-- 
If you have ever wondered if you are in hell,         John Cowan
it has been said, then you are on a well-traveled     http://www.ccil.org/~cowan
road of spiritual inquiry.  If you are absolutely     cowan at ccil.org
sure you are in hell, however, then you must be
on the Cross Bronx Expressway.          --Alan Feuer, NYTimes, 2002-09-20


More information about the XOM-interest mailing list