[bittorrent] The size of XML (was: DTD for hashed files)

Andrew Brampton andrew at bramp.freeserve.co.uk
Tue Apr 12 11:15:31 EDT 2005

I know this wasn't the topic of the original posting, but I thought I'll 
just splinter off to provide dpmott with some "overhead numbers"

Using XML for the torrent file would be bad in my opinion. The reasons being 
are these:

1) Beencoding data can represent everything XML can. It might not be as 
flexible, but it supports all the simple constructs

2) XML will break the SHA1 hashing of the info dictionary. Its perfectly 
valid for XML to have new lines, spaces, different case, different ordering, 
etc. All these mean the info block may look different even though it 
represents the same data. This would mean the SHA1 torrent hash would have 
to be made of something else. Do I hear you shout, hash just the pieces. 
Well yes that would work, but then you lose the hashing of the filenames, 
filesizes (which could be bad).

3) Beencoding is smaller than XML. Ok, we can live with a little overhead 
caused by the file names, file sizes, dates, etc (basically anything at the 
top of the .torrent). Its the binary encoding of the pieces that's the 
problem. XML as far as I know uses Base64 to encode binary data, now that 
adds a 33% overhead instantly [1].

Now taking a look at a .torrent file I have. Its total size is 33509 bytes, 
Roughly 487 of that is before the binary data. So that's 33022 of binary. 
Now add 33% to make 42928 Base64 bytes, plus say 500 bytes of the XML 
header. Total size now is just under 42k, where it was 32k. I don't think 
XML is worth it here in this case.

If anyone can give me a good reason to use XML which would justify gaining 
10k, I'll be glad to here it.


P.S Don't take this as a rant, or a attack, I just thought we needed some 
numbers, so I provided them. Also this list was getting a little dead ;)

[1] http://en.wikipedia.org/wiki/Base64

----- Original Message ----- 
From: "David P. Mott" <dpmott at sep.com>
To: <bittorrent at lists.ibiblio.org>
Sent: Monday, April 11, 2005 5:14 PM
Subject: Re: [bittorrent] DTD for hashed files

> (I personally like the idea of XML for metadata, and maybe even for the 
> wire protocol, but I'd have to see some overhead numbers for an XML 
> implementation (that supported embedded binary data) to really understand 
> the overhead).
> -dpmott

More information about the BitTorrent mailing list