[BitTorrent] Tracker scrape extension proposal
arachnid at notdot.net
Wed Jan 12 16:48:46 EST 2005
Is this documented? I didn't see it mentioned on the unofficial wiki
that more than one value could be provided.
The fallback behaviour here seems wrong, too. Many systems, such as PHP,
won't support multiple GET values with the same key (in PHP's case,
unless the key is appended with ), so they won't support it, and many
other trackers probably won't either. If a tracker doesn't support it,
it'll only return one key, which is entirely the wrong behaviour if
you're trying to retrieve information on 100 different torrents.
Harold Feit - Depthstrike.com Administrator wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>The current scrape system allows multiple info_hash= values to be
>passed to a tracker. It is up to individual tracker package
>developers to support this.
>Your proposal does allow for more torrents to be scraped in a given
>pass, but it's probably not 100% necessary.
>- -----Original Message-----
>From: Nick Johnson [mailto:arachnid at notdot.net]
>Sent: Wednesday, January 12, 2005 1:11 AM
>To: BitTorrent at yahoogroups.com
>Subject: [BitTorrent] Tracker scrape extension proposal
>A project I'm working on at the moment requires fetching scrape data
>from trackers. Often, I have multiple torrents on the same tracker
>I want scrape data for, so fetching scrape data for each file
>individually is probably very inefficient. On the other hand, the
>tracker's full scrape might be huge (multiple megs!) depending on how
>many torrents are on the tracker that I don't care about.
>With this in mind, I have a couple of proposals for how to improve
>- -Add a GET parameter 'hashes', which is a string that's a multiple of
>bytes. Each 20 bytes corresponds to one info_hash that you want the
>tracker to return. If the tracker recognises the extension, it
>only the requested keys, otherwise it returns all keys (since this
>be the default behaviour of trackers getting unknown get parameters
>anyway - hopefully!)
>Due to limitations on GET string length, this is probably limited to
>about 100 torrents before problems start to crop up. URLs like this
>quite likely to result in messes in log files, too. :/
>- -Somewhat more complicated: The client supplies a Bloom Filter
>(http://en.wikipedia.org/wiki/Bloom_filter) to the tracker (using a
>defined set of hash functions), with the appropriate entry in the
>set for each torrent it wants. The nature of the Bloom Filter
>all the requested entries will be returned, but extra entries may be
>returned as well. This way, an unlimited number of hashes can be
>requested with a static request size, just with an increasing
>Other suggestions are welcomed. It just seems to me that, at the
>for fetching 100 torrents from a tracker that's tracking 1000, either
>option is rather inefficient.
>- -Nick Johnson
>No virus found in this outgoing message.
>Checked by AVG Anti-Virus.
>Version: 7.0.300 / Virus Database: 265.6.10 - Release Date: 1/10/2005
>-----BEGIN PGP SIGNATURE-----
>Version: PGP 8.0.3
>-----END PGP SIGNATURE-----
>Yahoo! Groups Links
Yahoo! Groups Links
<*> To visit your group on the web, go to:
<*> To unsubscribe from this group, send an email to:
BitTorrent-unsubscribe at yahoogroups.com
<*> Your use of Yahoo! Groups is subject to:
More information about the BitTorrent