[bittorrent] The size of XML (was: DTD for hashed files)

larytet.8708132 at bloglines.com larytet.8708132 at bloglines.com
Tue Apr 12 13:10:13 EDT 2005


the moment you declare that block size is 4MB instead of 16K you don't need
any binary data in XML. you can keep hashes as strings and size of XML file
(uncompressed) is going to be ~64K for *G file

the first argument - hash
of the torrent file itself, is more interesting. but it is not clear to me
why do we need hash validation of the torrent file ? XML can be parsed and
verified and found kosher. this is a problem of trust. do you trust the source
where you got torrent file from or you don't trust. if you don't trust the
source valid hash of the torrent file is not going to be very helpful. i mean
i do not see use-case where hash of the torrent file is important.


---
Andrew Brampton" <andrew at bramp.freeserve.co.uk wrote:
I know this wasn't
the topic of the original posting, but I thought I'll 
> just splinter off
to provide dpmott with some "overhead numbers"
> 
> Using XML for the torrent
file would be bad in my opinion. The reasons being 
> are these:
> 
> 1)
Beencoding data can represent everything XML can. It might not be as 
> flexible,
but it supports all the simple constructs
> 
> 2) XML will break the SHA1
hashing of the info dictionary. Its perfectly 
> valid for XML to have new
lines, spaces, different case, different ordering, 
> etc. All these mean
the info block may look different even though it 
> represents the same data.
This would mean the SHA1 torrent hash would have 
> to be made of something
else. Do I hear you shout, hash just the pieces. 
> Well yes that would work,
but then you lose the hashing of the filenames, 
> filesizes (which could
be bad).
> 
> 3) Beencoding is smaller than XML. Ok, we can live with a
little overhead 
> caused by the file names, file sizes, dates, etc (basically
anything at the 
> top of the .torrent). Its the binary encoding of the pieces
that's the 
> problem. XML as far as I know uses Base64 to encode binary
data, now that 
> adds a 33% overhead instantly [1].
> 
> Now taking a
look at a .torrent file I have. Its total size is 33509 bytes, 
> Roughly
487 of that is before the binary data. So that's 33022 of binary. 
> Now
add 33% to make 42928 Base64 bytes, plus say 500 bytes of the XML 
> header.
Total size now is just under 42k, where it was 32k. I don't think 
> XML
is worth it here in this case.
> 
> If anyone can give me a good reason
to use XML which would justify gaining 
> 10k, I'll be glad to here it.
> 
> Thanks
> Andrew
> 
> P.S Don't take this as a rant, or a attack,
I just thought we needed some 
> numbers, so I provided them. Also this list
was getting a little dead ;)
> 
> [1] http://en.wikipedia.org/wiki/Base64

> 
> ----- Original Message ----- 
> From: "David P. Mott" <dpmott at sep.com>

> To: <bittorrent at lists.ibiblio.org>
> Sent: Monday, April 11, 2005 5:14
PM
> Subject: Re: [bittorrent] DTD for hashed files
> 
> 
> >
> > (I
personally like the idea of XML for metadata, and maybe even for the 
> >
wire protocol, but I'd have to see some overhead numbers for an XML 
> >
implementation (that supported embedded binary data) to really understand

> > the overhead).
> >
> > -dpmott
> >
> 
> _______________________________________________

> BitTorrent mailing list
> BitTorrent at lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/bittorrent

> 



More information about the BitTorrent mailing list