buildsystem | 3 May 09:46 2015

Private torrents and duplicates

I recently raised an issue with the developers of qBittorrent, a client 
based on libtorrent. The client automatically merges trackers and HTTP 
seed URLs of torrents that are already in the session. There is at least 
one case where this is undesirable: private torrents. I was directed to 
this mailing list in the resulting discussion. I'd like to request 
feedback regarding the solution I offered as well as make a suggestion. 
Here's the context of the issue:

Private torrents are often tracked by sites which maintain user accounts 
with associated upload and download statistics; not necessarily private, 
invitation-only trackers. They do this in order to create a sharing 
economy where the users are encouraged to seed back as much as they have 
leeched in order to maintain an acceptable share ratio. Thus, these 
trackers depend on the statistics reported by the torrent client in 
order to function; clients which cannot be trusted to report accurate 
statistics are banned from use.

It is often the case that the same files are tracked by multiple sites. 
The users, having already acquired the data from one site, would do well 
to seed it on all trackers. However, that is currently not possible.

Libtorrent looks for duplicates in the session when the `add_torrent` 
method is called. qBittorrent, anticipating this, does its own duplicate 
check and merges the trackers of torrents should a match be found. This 
is problematic because it will cause the misreporting of statistics, 
which will put user accounts and the standing of the torrent client in 
jeopardy.

Suppose there was a completed private torrent in the session. The user 
then downloaded the exact same torrent from two other trackers and they 
(Continue reading)

Bedeho Mender | 30 Apr 16:22 2015
Picon

Posting costume alerts from plugins

I am able to post costume alerts from torrent level plugins by
using torrent::alerts().post_alert(). However, when I try to post a costume
alert from plugin (that is not torrent or peer level, but top level), I get
into trouble.

In the on_tick(), I use the aux::session_impl pointer received in the
added() callback, and call ->m_alerts.post_alert(). This routine attempts
to send the alert back all extensions by iterating m_ses_extensions, which
is a list of shared pointers to plugins. For some reason, dereferencing
these pointers causes the shared pointer dereferencing routine to assert
that it is a null pointer.

Why would the shared pointer to the plugin be a null pointer after the
added() callback has been made? Is this the proper way to post plugin
alerts?

Best
 Bedeho
------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud 
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
Bedeho Mender | 28 Apr 20:06 2015
Picon

Maximum packet size

It seems that libtorrent has a hardcoded limit of 1MB for any message
received, anything beyond this leads to the peer being disconnected.

1) Is there a reason for this particular hard limit?
2) I have implemented an extended message for sending full pieces, and
consequently this limit will be exceeded on some torrents.

What is the best way to deal with this?

Would bulding a modified version of libtorrent where this hard limit is
altered break something else in libtorrent?

Bedeho
------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud 
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
Bedeho Mender | 25 Apr 12:05 2015
Picon

peer_plugin::on_extended meaning

While parsing extended messages arriving on peer through

on_extended(int length, int msg, libtorrent::buffer::const_interval body)

I have experienced undeterministic behaviour where the length of the body
field is frequently not as long as the length variable. I took a close look
at some at the example plugins and realized that the they were indeed
testing for body being the correct length, albeit in different ways.

Does this mean that this callback does not try to buffer a full extended
message, even though it knows the full length, but instead makes a callback
whenever some data comes in?

If so

1) What is the proper way to test if the full message has arrived?

ut_metadata.cpp uses peer_plugin::packet_finished(), while ut_pex.cpp
uses body.left() < length.

2) Why why doesnt libtorrent just make the callback one time at the end,
given that it has the length field?

3) This would be potentially be worth nothing in the www docs, or on the
declaration of the callback in the peer_plugin type.

Bedeho
------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud 
Widest out-of-the-box monitoring support with 50+ applications
(Continue reading)

Jeff Waller | 25 Apr 06:14 2015
Picon

Scenario: failure to find peers quickly leads to peer rate miscalculation or possible buffer misallocation?

I believe the title tells the proximate cause, but am unsure, and so this message.

This is a transfer between 2 peers discovered using the DHT — hash only, no magnet,
no tracker.

There may be multiple way to reach this state, but the following is 100% repeatable.

The connection mechanism might be obfuscated by Resume data — can get rid of this
possibility if need be.

Peer1 and Peer2 are geographically distant (one way delay ~= 70 ms), but the bandwidth
is high (500 Mbit) — maybe even 1 Gbit?  Thus the bandwidth delay product is at least
8.75MB (70Mbits) which requires large buffers.  Thus the kernel parameters have
been tweaked along with libtorrent read and send buffers to arbitrarily large values (64MB
currently).  Under normal circumstances, this leads to a throughput of approximately
300 Mbits, but it has been seen as high as 500 Mbits on occasion.  There’s variability
and the network is not under our direct control.

However the following scenario leads to a maximum throughput of approximately 500 KB/sec
rather than the normal 25-40 MB/sec (50 to 80 times slower).

Steps to reproduce

1) Start with Peer1 having a 1 GB torrent with hash X and nothing else in session.
2) Start with Peer2 having nothing in session.
3) Pause torrent in Peer1.
4) Add infoHash X in Peer2.  

=====
Since Peer 1 has announced, but is currently paused, DHT search will will result in
(Continue reading)

Bedeho Mender | 22 Apr 18:56 2015
Picon

too many connections error on peer_plugin

After installing a torrent plugin on a torrent, an external peer connects
to us, and a peer plugin is installed on this connection.

Some times it happens that that peer_plugin::on_disconnect() is immediately
called with the error "too many connections", and the connection is
dropped. Other times not.

Why would this be? Why would this be an error for only one or two
connections, when docs say for session_settings::connections_limit:

"sets a global limit on the number of connections opened. The number of
connections is set to a hard minimum of at least two per torrent, so if you
set a too low connections limit, and open too many torrents, the limit will
not be met."

Bedeho
------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
Angel Leon | 19 Apr 06:05 2015
Picon

Any luck building libtorrent on windows with the new Boost 1.58?

I can't fully build, I get weird compiler output on some places
<http://i.imgur.com/Zx9YVrF.png>, not sure if this has to do with Functors
and something that may have changed about them with Boost 1.58, at the end
of my builds I can't get a libtorrent.lib anymore. Reverted my
%BOOST_SOURCE% to 1.55 and all good.

*I'm interested in building with 1.58 for windows, because they seem to
have fixed a networking related race condition that would cause our app to
freeze when using libtorrent. The issue manifests itself if you build
libtorrent with boost 1.56 and 1.57.*

http://twitter.com/gubatron
------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
yannick inizan | 18 Apr 02:38 2015
Picon

C / Vala bindings

Hi all :)

I work on a BitTorrent manager in Vala, so I chose libtorrent for this.
Because I haven't been able to find recent C bindings,  I started writting
a wrapper for GLib, for generate instrospection file for glib-based object
language (js, vala, ...)

the application : https://github.com/inizan-yannick/transfert/
and my wrapper here :
https://github.com/inizan-yannick/transfert/tree/master/torrent-glib

regards, Yannick Inizan.
------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
Bedeho Mender | 16 Apr 16:24 2015
Picon

Reading blocks from downloaded torrent

I am trying to use piece_manager::async_read(), where I get the
piece_manager via torrent::filesystem(), to read individual blocks from a
downloaded torrent.

However:

1) it seems that async_read tries to queue the asynchronous read job by
calling

int disk_io_thread::add_job(disk_io_job const& j
, boost::function<void(int, disk_io_job const&)> const& f)

which again calls

int disk_io_thread::add_job(disk_io_job const& j
, mutex::scoped_lock& l
, boost::function<void(int, disk_io_job const&)> const& f)

but this latter routine only supports write actions (disk_io_job::write),
the other job types have been commented out (disk_io_thread.cpp)?

I am using libtorrent-rasterbar-1.0.3.

2) The callback passed to async_read must have signature

(int A, const libtorrent::disk_io_job & job)

Is A the number of bytes read? If so, what would cause A != job.buffer_size?

Presumably, the block data is in the char * job.buffer? if so, who owns
(Continue reading)

Ricky Huang | 15 Apr 19:49 2015
Picon

Single peer download using ipfilter

Hello all,

This is something I am experimenting with, not sure if anyone have tried and can provide their guess.

If I were to set the ipfilter of a libtorrent client from 1.1.1.1 - remote_server_ip-1 to
remote_server_ip+1 - 255.255.255.255, I can effectively turn bittorrent/libtorrent into a direct
download client.  How likely is the client going to find the remote server assuming PEX is enabled (but DHT off)?

What I am seeing is that after 1 hour of running the client, I still have not see the remote server (that is seeding).

Thanks.

------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
Ruslan Fedoseenko | 14 Apr 15:21 2015
Picon

A problem related to file_priorities

I ran into an issue with latest versions of libtorrent (> 1.0.2).
File priorities a not actualy working.
And resume file contains a list 'file_priorities' filled with zeros.
File priorities are specifed through add_torrent_params::file_priorities. I
digg into sources(session_impl.cpp session_impl::add_torrent) and  se that
there is no usage of this value.
--

-- 
Федосеенко Руслан.
---------------------------------------------
Fedoseyenko Ruslan
------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
Libtorrent-discuss mailing list
Libtorrent-discuss <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libtorrent-discuss

Gmane