[BitTorrent] Tracker scrape extension proposal

Harold Feit - Depthstrike.com Administrator dwknight at depthstrike.com
Thu Jan 13 00:16:26 EST 2005

Hash: SHA1

The client that scrapes and wants five is more common than you think.
Azureus' scrape code allows it to scan the torrents (both active and
not) in the list and scrape all the common-tracker torrents in one
request (when the tracker supports it). It's not uncommon to have a
tracker with 4000 torrents listed and have one client request scrapes
for batches of 5-10 at a time.

How is requesting the scrape information for several dozen (of
several thousand) torrents and getting the scrape information for
several thousand more efficient? It sounds quite wasteful if you ask
me, especially considering the wide support for connection
keep-alives in both clients and trackers.

Because of limitations in the HTTP protocols, there's a limit to the
number of torrents you can request the scrape information for before
the protocol breaks the request, and average users won't reach that
limit very often.

As for your "wasted request" theory, the clients that support
gathering multiple scrapes in one connection KNOW when they request
mulitple scrapes, and KNOW that in the case of them only getting one
back that the tracker doesn't support multiscrape. After finding out
one way or another if multiscrape is supported, they either wait for
the next cycle (in the case of it being supporteD) or request the
remaining scrapes in separate, individual requests.

Two mechanisms is just more code to break trackers AND clients right
now. I've started work on adding support for the current multiscrape
handling in the trackers I help program.

- -----Original Message-----
From: Nick Johnson [mailto:arachnid at notdot.net] 
Sent: Wednesday, January 12, 2005 11:23 PM
To: BitTorrent at yahoogroups.com
Subject: Re: [BitTorrent] Tracker scrape extension proposal

Harold Feit - Depthstrike.com Administrator wrote:

>Hash: SHA1
>Taking a look at the way get values are generated in cases of
>multiple paramaters of the same key typically overwrites the value
>in default configurations, preventing other values from being
>available. Other configurations only read the first value, not
>allowing access
>to those after it.
>Clients that don't support asking for multiple scrapes in the same
>request will only request 1 at a time, consistent with the way they
>have done in the past.
>For tracker-side development, I feel your suggestion of returning
>all torrents when processing multiple info_hash values isn't
>supported is quite damaging to the tracker (pointing back to your
>"thousands of
>torrents" tracker example). If you requested the scrape data for 5
>torrents and got 5000, you would be doing more damage to the tracker
>than getting 5 separate requests.
Then perhaps two mechanisms is better. At the moment, I'm requesting
full scrape for each tracker, because individual requests are usually
inefficient. If I could specify a list of desired keys and have it
me them or everything, this would be an improvement in efficiency.
one that only wants 5 (but really, how often will that happen in 
reality?) can use the current mechanism.
As it stands, if I try and use the current system to request more
one hash, I'm quite likely to get a response with only one, wasting a

- -Nick Johnson

- -- 
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.300 / Virus Database: 265.6.10 - Release Date: 1/10/2005

Version: PGP 8.0.3


Yahoo! Groups Links

<*> To visit your group on the web, go to:

<*> To unsubscribe from this group, send an email to:
    BitTorrent-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:

More information about the BitTorrent mailing list