[bittorrent] The size of XML (was: DTD for hashed files)

Justin Cormack justin at street-vision.com
Wed Apr 13 04:58:09 EDT 2005


On Tue, 2005-04-12 at 18:26 -0700, Joseph Ashwood wrote:
> ----- Original Message ----- 
> From: "Andrew Brampton" <andrew at bramp.freeserve.co.uk>
> To: <bittorrent at lists.ibiblio.org>
> Sent: Tuesday, April 12, 2005 8:15 AM
> Subject: Re: [bittorrent] The size of XML (was: DTD for hashed files)
> 
> 
> > 2) XML will break the SHA1 hashing of the info dictionary. Its perfectly 
> > valid for XML to have new lines, spaces, different case, different 
> > ordering, etc. All these mean the info block may look different even 
> > though it represents the same data. This would mean the SHA1 torrent hash 
> > would have to be made of something else. Do I hear you shout, hash just 
> > the pieces. Well yes that would work, but then you lose the hashing of the 
> > filenames, filesizes (which could be bad).
> 
> You are looking for Canonical XML (http://www.w3.org/TR/xml-c14n), as 
> specifically used in both XML Encryption 
> (http://www.w3.org/Encryption/2001/) and XML Signature 
> (http://www.w3.org/Signature/) which incidently already provide everything 
> else you're looking for to verify the torrent file and the external data.
> 
> > 3) Beencoding is smaller than XML. Ok, we can live with a little overhead 
> > caused by the file names, file sizes, dates, etc (basically anything at 
> > the top of the .torrent). Its the binary encoding of the pieces that's the 
> > problem. XML as far as I know uses Base64 to encode binary data, now that 
> > adds a 33% overhead instantly [1].
> 
> XOP, aka binary XML reduces this overhead substantially bringing it in line 
> with the overhead of bencoding, or asn.1.

Ok, you have convinced me it is possible.

(actually the other way to fix the canonical problem is just to extract
the content and put it into a particular for before hashing - you could
even convert to the bencoded form for compatibility).

Actually if you could be bothered to write some code that converts XOP
or normal XML to a bencoded info blob (and perhaps vice versa for
testing) I might be interested...
 
j







More information about the BitTorrent mailing list