[bittorrent] How should .torrent files with unsorted info dictionary keys be handled

Alan McGovern alan.mcgovern at gmail.com
Thu Jan 21 06:50:13 EST 2010


By enforcing that the keys are sorted you gain the benefit that multiple
torrent generators have a reasonably good chance of generating torrents with
identical infohashes given the same source (assuming the same piece length).
I think that's a good enough reason to keep the spec as-is and just enforce
it better. Similarly, that's probably a good enough reason to agree that if
we absolutely *have* to load invalid bencoded data then we should generate
the infohash from the sorted keys.

Alan.

On Thu, Jan 21, 2010 at 11:38 AM, Harold Feit <dwknight at depthstrike.com>wrote:

> Dictionaries are for sorted content normally anyway. Lists are for
> unsorted. Having dictionary keys be unsorted, especially now with over 5
> years into the game of having them sorted, is quite troublesome.
>
> And honestly, out of the top client list, very few are as lax as
> KTorrent about bencoding spec compliance. In fact, it's the only client
> I know of that outright ignores key order. Everybody else is at least
> semi-strict about it.
>
> And honestly, do we need another THOUSAND torrents all identifying the
> same file because of unordered dictionaries? There's enough trouble
> dealing with swarm fracturing as it is. This is just going to make
> things a LOT worse.
>
> Joris Guisson wrote:
> > KTorrent computes the hash of the substring, ignoring any key order
> problems.
> >
> > In fact, we ignore the key order everywhere for any bencoded data we
> > come across.
> >
> > So I would be in favor of dropping the key order requirement.
> >
> > Does anybody actually know the reason why the keys should be sorted ?
> > I don't see much reason to do so.
> >
> > Joris,
> >
> > On Wed, Jan 20, 2010 at 9:34 PM,  <arvid at cs.umu.se> wrote:
> >> Quoting Alan McGovern <alan.mcgovern at gmail.com>:
> >>
> >>> [...]
> >>> So pros and cons:
> >>> Approach 1:
> >>> Pro: This approach implies that if an invalid bencoded dictionary is
> found
> >>> it should be converted into a valid representation and used.
> >>>
> >>> Pro: This should be relatively trivial for most clients to implement.
> >>>
> >>> Con: The letter of the spec [2] says that we should always use a
> substring
> >>> of the .torrent metadata. Strictly speaking this approach goes against
> the
> >>> spec. However, we have to assume that when the spec refers to ".torrent
> >>> metadata' it refers to *specification compliant* .torrent metadata.
> i.e. the
> >>> keys in the metadata must be sorted for it to be considered valid
> .torrent
> >>> metadata. Conversely, if the keys are *not* sorted then the data should
> not
> >>> be considered valid .torrent metadata. If that's the case, then why are
> you
> >>> generating a valid infohash from invalid metadata?
> >>>
> >>> Con: You need a non-spec compliant way of decoding bencoded data.
> >>>
> >>> Approach 2:
> >>> Pro: You follow the letter of the bittorrent specification but still
> break
> >>> the BEncoding specification.
> >>>
> >>> Con: I'd argue that this is slightly more complex to implement as you
> may
> >>> now need to double parse the .torrent file in order to generate the
> >>> infohash. The first time you pass it through your bencoded data decoder
> to
> >>> generate your in-memory representation. Then you have to parse the file
> >>> manually a second time to find the start and end of the info dictionary
> and
> >>> extract that substring.
> >>>
> >>> Con: I'd argue that you're breaking the spirit of the specification
> again as
> >>> you're now running under the assumption that invalid bencoded data is
> valid
> >>> .torrent metadata and the spec doesn't explicitly allow this ;)
> >>>
> >>> Approach 3:
> >>>
> >>> Pro: Simplest to implement as you just have to make your bencoded data
> >>> decoder spec compliant. Once this happens, the .torrent won't be
> loadable so
> >>> that question of how to generate the infohash never has to be answered.
> >>>
> >>> Con: It will render some torrents unloadable but those should be a very
> >>> small percentage.
> >> If you're talking about modifying the spec. Why not add:
> >>
> >> Approach 4:
> >>
> >> Modify the spec to allow unordered keys in dictionaries.
> >>
> >> Con: All torrents will work as intended (presumably the unordered keys
> are
> >> probably a mistake, and not a clever trick to make clients generate
> different
> >> info-hashes).
> >>
> >> Pro: some clients would have to be updated to be less strict on their
> input.
> >>
> >> --
> >> Arvid Norberg
> >> _______________________________________________
> >> BitTorrent mailing list
> >> BitTorrent at lists.ibiblio.org
> >> http://lists.ibiblio.org/mailman/listinfo/bittorrent
> >>
> > _______________________________________________
> > BitTorrent mailing list
> > BitTorrent at lists.ibiblio.org
> > http://lists.ibiblio.org/mailman/listinfo/bittorrent
> >
> _______________________________________________
> BitTorrent mailing list
> BitTorrent at lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/bittorrent
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.ibiblio.org/pipermail/bittorrent/attachments/20100121/5becf826/attachment.html 


More information about the BitTorrent mailing list