Jari Sundell | 3 Oct 2006 13:54
Picon
Gravatar

Fast Resume Extension, Draft1

Every client around probably do their own thing when it comes to
saving fast resume data, some better implemented than others. As
people have been asking me for support for skipping hash check on
newly created torrents that are added to my client, I thought I'd
throw out a proposal for how this could be supported in a safe way.

This could also be considered my idea of how to implement fast resume
in a safe way. I'm just throwing this out for comments, so the grammar
and logic is probably a bit borked.

http://rakshasa.no/downloads/fast_resume_extension.txt

Rakshasa

---

Fast Resume Extension - Draft 1

This extension seeks to provide specifications for storage and usage
of fast resume data in BitTorrent meta-info files. The client should
be strict about the input accepted, and ignore all the fast resume
data if any inconsistencies are encountered.

Extension of the meta-info file

The root bencoded dictionary may contain the key "fast_resume" mapping
a dictionary, which holds data necessary to support resuming a torrent
in a fast and safe manner. This dictionary will be referred to as the
fast resume dictionary.

(Continue reading)

David P. Mott | 3 Oct 2006 18:49
Favicon

Re: Fast Resume Extension, Draft1


Comments inline below.

> Date: Tue, 3 Oct 2006 20:54:36 +0900
> From: "Jari Sundell" <sundell.software <at> gmail.com>
> Subject: [bittorrent] Fast Resume Extension, Draft1
>
> Every client around probably do their own thing when it comes to
> saving fast resume data, some better implemented than others. As
> people have been asking me for support for skipping hash check on
> newly created torrents that are added to my client, I thought I'd
> throw out a proposal for how this could be supported in a safe way.
>
> This could also be considered my idea of how to implement fast resume
> in a safe way. I'm just throwing this out for comments, so the grammar
> and logic is probably a bit borked.
>
> http://rakshasa.no/downloads/fast_resume_extension.txt
>
> Rakshasa
>
> ---
>
> Fast Resume Extension - Draft 1
>
> This extension seeks to provide specifications for storage and usage
> of fast resume data in BitTorrent meta-info files. The client should
> be strict about the input accepted, and ignore all the fast resume
> data if any inconsistencies are encountered.
>
(Continue reading)

Alan McGovern | 3 Oct 2006 19:03
Picon
Gravatar

Re: Fast Resume Extension, Draft1

Personally, i'd go for a definate no vote.

Firstly, fast resume is not portable between different users, so putting it in the .torrent file is a *bad* idea. It's a complete waste of bytes.

Secondly, fast resume can be implemented in 2 easy steps.

1) When a torrent stops running, you serialise the BitField which represents what pieces you have and don't have into a file (possibly called whatever.torrent.fresume).

2) When you start/load a torrent (i.e . c:\mytorrent.torrent) check for c:\mytorrent.torrent.fresume, if it exists, then load it and don't run a hashcheck.

Two caveats: When a torrent reaches 100%, you should force a hashcheck. You should also store the modification date/time of the file when you finish using it ( i.e. when you generate the .fresume file) and use that as an additional check to see if a hashcheck is needed when reloading the fresume data.

Alan.


On 10/3/06, Jari Sundell < sundell.software <at> gmail.com> wrote:
Every client around probably do their own thing when it comes to
saving fast resume data, some better implemented than others. As
people have been asking me for support for skipping hash check on
newly created torrents that are added to my client, I thought I'd
throw out a proposal for how this could be supported in a safe way.

This could also be considered my idea of how to implement fast resume
in a safe way. I'm just throwing this out for comments, so the grammar
and logic is probably a bit borked.

http://rakshasa.no/downloads/fast_resume_extension.txt

Rakshasa

---

Fast Resume Extension - Draft 1

This extension seeks to provide specifications for storage and usage
of fast resume data in BitTorrent meta-info files. The client should
be strict about the input accepted, and ignore all the fast resume
data if any inconsistencies are encountered.


Extension of the meta-info file

The root bencoded dictionary may contain the key "fast_resume" mapping
a dictionary, which holds data necessary to support resuming a torrent
in a fast and safe manner. This dictionary will be referred to as the
fast resume dictionary.

The fast resume dictionary contains the following keys:

"bitfield"

  A string containing the bitfield as defined in the main
  protocol. The client shall ensure the string length matches the
  bitfield length, and clear the unused bits at the end. Alternatively
  it may be an integer containing the number of set bits, where only
  an empty or full bitfield is valid.

  If the file does not exist on disk or does not have the expected
  size, then the bitfield of that file shall be invalidated. The
  client invalidates the bitfield of a file by clearing all bits
  corresponding to chunks containing the file.

"files"

  A list where each entry is a dictionary associated with the
  corresponding filename in info_hash's "path" list, or one entry in
  the case of a single-file torrent. This list will be referred to as
  the fast resume files list and each entry as fast resume file entry.

The fast resume file entry dictionary contains the following keys:

"mtime"

  An integer containing the time of last modification of the file in
  unix time, which is seconds since 00:00:00 UTC, 1970/01/01,
  excluding leap seconds. If the time does not match the modification
  time of the corresponding file on disk, then the client shall
  invalidate the bitfield of that file.

The fast resume file entry dictionary may be extended to include
other keys, f.ex "priority" and "path".
_______________________________________________
BitTorrent mailing list
BitTorrent <at> lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/bittorrent

_______________________________________________
BitTorrent mailing list
BitTorrent <at> lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/bittorrent
Jari Sundell | 4 Oct 2006 08:01
Picon
Gravatar

Re: Fast Resume Extension, Draft1

On 10/4/06, David P. Mott <dpmott <at> sep.com> wrote:
> My question is this: why are we wanting to extend the metadata?
>
> The metadata (eventually) gets uploaded to a site where potentially
> thousands of people can download it.  Presumably, all of the information
> in that file must be useful to those thousands of people, otherwise we've
> wasted potentially tens of thousands of bytes of bandwidth for the server
> (and multiply that by the number of torrents with this extension...)
>
> Okay, so let's say that *I* make a metadata file with this extended
> information, and I upload it.  If someone chooses a different save path,
> then all of the entries in "files" is invalid.  If they don't already have
> the files on their drive with the exact same timestamp, then all entries
> in "mtime" will be discarded.
>
> This strikes me as information that is a lot more useful for the local
> user.  On MSWindows, for instance, this is something that could be stored
> in the "C:\Documents and Settings\<user>\Application Data\<bittorrent
> client>" folder.  On *NIX systems, it could go in "/var/lib/≤bittorrent
> client>" folder (for a global setting), or in the user's home directory as
> a dot file.
>
> My concern is that this extension will "leak" out in a released torrent.
> I *already* see lots of metdata files that have lots of data under
> nonstandard dictionary keys (on the order of hundreds of kilobytes of
> binary garbage).  Presumably this is someone else that has had the same
> idea as you, put the fast resume data into the metadata file, and didn't
> "sanitize" it or prevent the user from copying the file from the filesytem
> and uploading it to a server somewhere.

It is actually meant as information only useful for the local user,
specifically to allow different locally used tools to work together.
It is a problem that you cannot safely use different tools to create
the torrent, then seed it.

It is both ugly and rather messy in my opinion to put stuff in a
static location rather than allowing the information to reside with
the torrent. A torrent maker would probably make two different torrent
files, one for public distribution and another that gets put in f.ex a
directory monitored by the seeding client.

If there is a concern about leaking this meta-data, then some
recommendations could be added that would avoid that. The client could
even load the resume data from a seperate bencoded file.

> It seems to me that the better way to store transient information about
> local files is to keep it close to the local files (perhaps even *in* the
> local files, if that interests you).  I would discourage anyone from
> putting the information into a file with a '.torrent' extension,
> especially a valid-looking metadata file that might be uploaded by an
> end-user for general consumption.

If by local files you mean the downloaded data, then I do not agree.
To keep meta-data in/with the data being downloaded/uploaded means you
may need to clean it up later.

> Let me know if I went off in the wrong direction here, or misunderstood
> your intent or presentation.

Well, I'd say not that far off, but probably missed the indented use case.

Rakshasa
TCUAG | 5 Oct 2006 10:20
Picon
Favicon

We hire python and c++ professionals

Python:
- Peer to peer software (XP)
- Migration to Linux

C++
- Active X, Firefox Plugins,
- GUI

Please provide
- fulltime or freelancer
- telecommuting or Koblenz
- your skills
- your salary €/h


Regards
Petra Bauersachs , CEO TC Unterhaltungselektronik AG
56073 Koblenz, Germany
Tel: 0049 0172 65 84 139,
telecontrol <at> t-online.de, www.telecontrol.de
Frankfurt Stock Exchange: 745420



P.S. We still hire
Python:
- Peer to peer software (XP)
- Migration to Linux

C++
- Active X, Firefox Plugins, GUI




Click the following link if you do not want to receive these mails:
Abbestellen
_______________________________________________
BitTorrent mailing list
BitTorrent <at> lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/bittorrent
Harold Feit | 5 Oct 2006 13:48
Favicon
Gravatar

Re: We hire python and c++ professionals


This is spam, pure and simple.

Stop this unsolicited message campaign please.

Harold Feit
Network Security
Depthstrike Entertainment.
dwknight <at> depthstrike.com

TCUAG wrote:
> *Python*:
> - Peer to peer software (XP)
> - Migration to Linux
> 
> *C++*
> - Active X, Firefox Plugins,
> - GUI
> 
> Please provide
> - fulltime or freelancer
> - telecommuting or Koblenz
> - your skills
> - your salary ?/h
> 
> 
> Regards
> Petra Bauersachs , CEO
> ------------------------------------------------------------------------
> TC Unterhaltungselektronik AG
> 56073 Koblenz, Germany
> Tel: 0049 0172 65 84 139,
> telecontrol <at> t-online.de, www.telecontrol.de
> Frankfurt Stock Exchange: 745420
> 
> 
> 
> P.S. We still hire
> ------------------------------------------------------------------------
> *Python:*
> - Peer to peer software (XP)
> - Migration to Linux
> 
> *C++*
> - Active X, Firefox Plugins, GUI
> 
> 
> 
> 
> Click the following link if you do not want to receive these mails:
> Abbestellen
> <http://kunden.tivion.de/edit.php?email=bittorrent <at> lists.ibiblio.org&key=3be671b391db37beed68618cbb9529f1>
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> BitTorrent mailing list
> BitTorrent <at> lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/bittorrent
TCUAG | 6 Oct 2006 10:39
Picon
Favicon

We hire Flash-Professionals or Java Professionals

We hire for a new project (GUI + videoplayer) Flash-Professionals or Java Professionals

Please provide
- fulltime or freelancer
- telecommuting or Koblenz
- your skills
- your salary €/h

Regards
Petra Bauersachs , CEO
---------------------------------------------------------

TC Unterhaltungselektronik AG
56073 Koblenz, Germany
Tel: 0049 0172 65 84 139,
telecontrol <at> t-online.de, www.telecontrol.de
Frankfurt Stock Exchange: 745420


P.S. We still hire
---------------------------------------------------------
Python
- Peer to peer software (XP)
- Migration to Linux

C++
- Active X, Firefox Plugins, GUI



Click the following link if you do not want to receive these mails:
Abbestellen

_______________________________________________
BitTorrent mailing list
BitTorrent <at> lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/bittorrent
Harold Feit | 6 Oct 2006 12:40
Favicon
Gravatar

Re: We hire Flash-Professionals or Java Professionals


This is now harassment.

Harold Feit
Network Security
Depthstrike Entertainment
dwknight <at> depthstrike.com

TCUAG wrote:
> We hire for a new project (GUI + videoplayer) *Flash-Professionals or
> Java Professionals*
> 
> Please provide
> - fulltime or freelancer
> - telecommuting or Koblenz
> - your skills
> - your salary ?/h
> 
> Regards
> Petra Bauersachs , CEO
> ---------------------------------------------------------
> 
> *TC Unterhaltungselektronik AG*
> 56073 Koblenz, Germany
> Tel: 0049 0172 65 84 139,
> telecontrol <at> t-online.de, www.telecontrol.de
> Frankfurt Stock Exchange: 745420
> 
> 
> *P.S. We still hire*
> ---------------------------------------------------------
> *Python*
> - Peer to peer software (XP)
> - Migration to Linux
> 
> *C++*
> - Active X, Firefox Plugins, GUI
> 
> 
> 
> Click the following link if you do not want to receive these mails:
> Abbestellen
> <http://kunden.tivion.de/edit.php?email=bittorrent <at> lists.ibiblio.org&key=3be671b391db37beed68618cbb9529f1>
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> BitTorrent mailing list
> BitTorrent <at> lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/bittorrent
Demetres Antoniades | 11 Oct 2006 16:24
Picon

question about bittorent traffic

Hello,

I am trying to understand when a bittorrent client will initiate a
connection to another
client in order to upload a chunk of the file. I would expect that the
initiator of the
connection would be the interested leecher but in a packet trace that
I have I see
the uploader initiating the connection sending the handshake message
"19BitTorrent Protocol" and after he gets a response it uploads a
chunk of the file.
Does anyone know if this is the default way of sharing chunks or if
this happens under certain circumstances. I would expect this to
happen when the uploader is behind a NAT and so the downloader can not
access him. But in my experiment this is not the case.

Thanks,
Demetres
David P. Mott | 11 Oct 2006 19:21
Favicon

Re: question about bittorent traffic


It doesn't matter who initiates the connection, or for what reason.  Once 
a connection is established, either side can request a packet.

You should double-check your packet trace and determine if a request was 
sent (since you did not note that in your description).  If clients are 
connecting and then sending *unsolicited* chunks, then that's bad and 
should be seen as an attempted denial of service attack (it wastes the 
peer's bandwidth).

Note: I couldn't quote you on the rules that seeders use to connect to the 
swarm; they are well within their rights to sit back and only accept 
connections, or they could actively connect to leechers.  As I said above, 
the reason doesn't matter -- requests can flow either way once a 
connection is established.

-dpmott

> From: "Demetres Antoniades" <deanton <at> gmail.com>
>
> Hello,
>
> I am trying to understand when a bittorrent client will initiate a 
> connection to another client in order to upload a chunk of the file. I 
> would expect that the initiator of the connection would be the 
> interested leecher but in a packet trace that I have I see the uploader 
> initiating the connection sending the handshake message "19BitTorrent 
> Protocol" and after he gets a response it uploads a chunk of the file. 
> Does anyone know if this is the default way of sharing chunks or if this 
> happens under certain circumstances. I would expect this to happen when 
> the uploader is behind a NAT and so the downloader can not access him. 
> But in my experiment this is not the case.
>
> Thanks,
> Demetres

Gmane