[BitTorrent] Bittorrent and filesystem API / integration

Justin Cormack justin at street-vision.com
Tue Jan 11 12:42:55 EST 2005


> 
> 
> 
> Hi,
> 
> Checking the integrity of a file before sharing it is logical
> behaviour of BitTorrent. But, on the other hand, it is also kind of
> annoying, having to wait until the file is fully scanned. It also
> costs a lot of disk bandwidth, a device that has to deal already with
> quite a lot of 'contention' for seeking opeations on it.

Technically a BT client doesnt have to do this, it could treat startup
like having an empty file, and then as it checks blocks it could send
have messages to the peers. There is a cost as the have messages are
substantially larger than a bitmap however.

Another way to save time would be some way to detect sparse files quickly
(one way is to check if a block is all zeros and if the expected hash
is not the hash of a zero block, which can be precomputed. This would
save some CPU time when scanning files.
 
> So, I was thinking: would it be cool to find a filesystem and API,
> that is more advanced than the basic open/read/write/append/close
> behaviour we seen unchanged since the 70's ? 
> 
> Things like:
> - Integrity checks on chunks of a file, signatures of an application 
>   that has written part of a file, and kernel guarantees that only you 
>   have access to a file, until you 'unlock' the file for usage by other 
>   applications.

This is very dubious, as disks are unreliable and guaranteeing this in
any error situation is very difficult.

> - Transactional file modifications in chunks, queing up transactions,
>   prioritizing them, optimizing a filesystem for the anticipated size
>   of a file before even a single byte is written.

Some file systems do support preallocation for files (forget offhand
which ones). Asynchronous io gives you queued transactions.

> - Other things that make development of other applications that are
>   easier to build with proper locking. Think of circular disk buffers,
>   editing video, copying (chunks of) files without having to move 
>   data around on the filesystem, etc.

Sharing parts of files (eg copy on write semantics) has been discussed,
and is implemented by some snapshot type mechanisms. You can do a lot by
just using file offsets though, it is not clear you really need much
filesystem support.

> - Maybe this is more for a filesystem mailinglist than bittorrent :)
> 
> Just some raw idea's...
> 
> Mark-Jan
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>  
> Yahoo! Groups Links
> 
> 
> 
>  
> 
> 
> 



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/BitTorrent/

<*> To unsubscribe from this group, send an email to:
    BitTorrent-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 





More information about the BitTorrent mailing list