[bittorrent] Fwd: Load questions

Joseph Ashwood ashwood at msn.com
Tue Dec 5 07:30:43 EST 2006


----- Original Message ----- 
From: "Olaf van der Spek" <olafvdspek at gmail.com>
Subject: Re: [bittorrent] Load questions


> On 12/2/06, Joseph Ashwood <ashwood at msn.com> wrote:
>> > XBT Tracker (C++) can
>> > handle up to at least 800k peers.
>>
>> What class of process/memory? It's one thing to say that about a 
>> BlueGene,
>> another to say that about a PII, I'm thinking that's about the level of a 
>> P4
>
> I don't run the trackers myself and I don't have the details, but I'm
> thinking about a decent K8/P4 at least.

So then for a safety margin I'll just assume it is at least 600K on the 
Xeons and Opterons I'll be using. Shouldn't be an issue, I suspect each of 
those systems could handle in the neighborhood of 1M.

>
>> @ 3GHz. Also is XBTT written in such a way that it can be clustered, and
>> have multiple copies running locally, if so what are the limitations 
>> (e.g.
>> shared torrents?), or would I have use virtualization for that?
>
> What do you want to achieve?

Scalability and reliability. If a server goes does for whatever reason, 
having a cluster of some kind (even multi-tracker torrents) provides 
fail-over and the system stays running. Multiple trackers per system also 
allows for software fault recovery in many cases. Virtual machines are a 
truly blessed thing for huge scalability.

> There's no special support for clustering, but you could run two
> independent instances on two servers and have 50% of the torrents
> tracked by one and the others by the other.

I'll be doing that too, but the multi-tracker torrents present something 
very attractive to me as well. Since most of the torrents will basically 
result in HTTP seeding only it shouldn't be too much of an issue to have a 
large quantity of torrents on a system, or a large quantity of trackers each 
with a few torrents virtualized onto the same system (more likely the 
former).

>> > RAM is no issue for XBTT and the
>> > first bottleneck most operators encounter is network bandwidth.
>>
>> I already know bandwidth is going to be the choking point on this 
>> (primarily
>> because of the seeding of rare files), and that is why the final
>
> But I mean, even for just tracker traffic bandwidth is usually the 
> bottleneck.

I know, actually Alan's numbers look to be pretty decent on that front, at 
the very least they have reasonable arguments behind them.

>> implementation for this will likely preference peer-of-peer referrals to
>> tracker referrals. RAM may not normally be an issue, but when you're 
>> trying
>> to make something as dense as possible it becomes one, what kind of RAM
>> usage is expected, and what is a rough upper bound for XBTT?
>
> To be honest I really don't know. I think 640k is enough. :)
>
>
>
> Just kidding. 640m may be enough though. I'll have to ask one of the 
> admins.

I'll plan on 1G, RAM is so cheap it's not worth bickering over if 640M is 
the right ballpark. I have some old numbers for a 100K user site hitting 
256MB, but I don't know what tracker.
                    Joe
                    (hopefully soon speccing out a massive setup) 





More information about the BitTorrent mailing list