[bittorrent] How should .torrent files with unsorted info dictionary keys be handled

Harold Feit dwknight at depthstrike.com
Thu Jan 21 06:38:35 EST 2010

Dictionaries are for sorted content normally anyway. Lists are for
unsorted. Having dictionary keys be unsorted, especially now with over 5
years into the game of having them sorted, is quite troublesome.

And honestly, out of the top client list, very few are as lax as
KTorrent about bencoding spec compliance. In fact, it's the only client
I know of that outright ignores key order. Everybody else is at least
semi-strict about it.

And honestly, do we need another THOUSAND torrents all identifying the
same file because of unordered dictionaries? There's enough trouble
dealing with swarm fracturing as it is. This is just going to make
things a LOT worse.

Joris Guisson wrote:
> KTorrent computes the hash of the substring, ignoring any key order problems.
> In fact, we ignore the key order everywhere for any bencoded data we
> come across.
> So I would be in favor of dropping the key order requirement.
> Does anybody actually know the reason why the keys should be sorted ?
> I don't see much reason to do so.
> Joris,
> On Wed, Jan 20, 2010 at 9:34 PM,  <arvid at cs.umu.se> wrote:
>> Quoting Alan McGovern <alan.mcgovern at gmail.com>:
>>> [...]
>>> So pros and cons:
>>> Approach 1:
>>> Pro: This approach implies that if an invalid bencoded dictionary is found
>>> it should be converted into a valid representation and used.
>>> Pro: This should be relatively trivial for most clients to implement.
>>> Con: The letter of the spec [2] says that we should always use a substring
>>> of the .torrent metadata. Strictly speaking this approach goes against the
>>> spec. However, we have to assume that when the spec refers to ".torrent
>>> metadata' it refers to *specification compliant* .torrent metadata. i.e. the
>>> keys in the metadata must be sorted for it to be considered valid .torrent
>>> metadata. Conversely, if the keys are *not* sorted then the data should not
>>> be considered valid .torrent metadata. If that's the case, then why are you
>>> generating a valid infohash from invalid metadata?
>>> Con: You need a non-spec compliant way of decoding bencoded data.
>>> Approach 2:
>>> Pro: You follow the letter of the bittorrent specification but still break
>>> the BEncoding specification.
>>> Con: I'd argue that this is slightly more complex to implement as you may
>>> now need to double parse the .torrent file in order to generate the
>>> infohash. The first time you pass it through your bencoded data decoder to
>>> generate your in-memory representation. Then you have to parse the file
>>> manually a second time to find the start and end of the info dictionary and
>>> extract that substring.
>>> Con: I'd argue that you're breaking the spirit of the specification again as
>>> you're now running under the assumption that invalid bencoded data is valid
>>> .torrent metadata and the spec doesn't explicitly allow this ;)
>>> Approach 3:
>>> Pro: Simplest to implement as you just have to make your bencoded data
>>> decoder spec compliant. Once this happens, the .torrent won't be loadable so
>>> that question of how to generate the infohash never has to be answered.
>>> Con: It will render some torrents unloadable but those should be a very
>>> small percentage.
>> If you're talking about modifying the spec. Why not add:
>> Approach 4:
>> Modify the spec to allow unordered keys in dictionaries.
>> Con: All torrents will work as intended (presumably the unordered keys are
>> probably a mistake, and not a clever trick to make clients generate different
>> info-hashes).
>> Pro: some clients would have to be updated to be less strict on their input.
>> --
>> Arvid Norberg
>> _______________________________________________
>> BitTorrent mailing list
>> BitTorrent at lists.ibiblio.org
>> http://lists.ibiblio.org/mailman/listinfo/bittorrent
> _______________________________________________
> BitTorrent mailing list
> BitTorrent at lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/bittorrent

More information about the BitTorrent mailing list