Synchronizing on the problem statement

Folks: One of the action items in the Minneapolis meeting was
to ensure that all working group participants have a common
understanding of the problem we have been chartered to solve.

Clearly, those following the proceedings from the May 2008
IETF/MIT workshop may have an advantage over others in this
respect.  To remedy that situation, I will like to direct the
attention of the participants to the following pair of documents,
possibly read in this order, as the first draft provides context
on the discussion we had in the MIT workshop back in May 2008:

1) The P2Pi workshop report (please see
   http://tools.ietf.org/html/draft-p2pi-cooper-workshop-report-00)

2) The problem statement draft (please see
   http://tools.ietf.org/html/draft-marocco-alto-problem-statement-03)

To move forward in a concrete manner, I would like to entreat
those who will be reading the above two documents for the first
time: could you kindly ensure that the problem is sufficiently
identified and documented in the problem-statement draft such
that a neophyte reader is able to quickly understand and appreciate
it.

Among those that read the problem-statement draft or have been
following this work since the workshop, the bulk of the discussion
in the Minneapolis IETF was on the fate of the Solutions
Consideration section of the draft.  To wit, there were two ways
to handle this section:

Paper on "Pushing BitTorrent Locality to the Limit"

I thought this paper is relevant and did not see it in the ALTO or  
P2PI archives

http://arxiv.org/abs/0812.0581

 From the abstract :

  In this paper, we perform extensive experiments on a controlled  
environment with up to 10 000 peers to evaluate the impact of locality  
on inter-ISP links traffic and peers download completion time. In  
particular, we show that high locality values enable up to two orders  
of magnitude saving on inter-ISP links without any significant impact  
on peers download completion time.

------

This would be a very encouraging reduction of inter-ISP traffic if it  
could be approached in practice by what we are doing here.

This is not a simulation, but an experiment run on the Grid’5000  
network with 1700 nodes. Since the lead author is subscribed to the  
ALTO list, I am going to wonder publicly if it would be
possible (and sensible) to test any ALTO solution with this  
experimental setup.

Regards
Marshall

Re: Paper on "Pushing BitTorrent Locality to the Limit"

On Dec 3, 2008, at 1:10 PM, Marshall Eubanks wrote:
>
> This would be a very encouraging reduction of inter-ISP traffic if  
> it could be approached in practice by what we are doing here.
>
> This is not a simulation, but an experiment run on the Grid’5000  
> network with 1700 nodes. Since the lead author is subscribed to the  
> ALTO list, I am going to wonder publicly if it would be
> possible (and sensible) to test any ALTO solution with this  
> experimental setup.

The only thing that is different in this approach from the standard  
simulation is that a regular bittorrent client is used. The ISPs  
topology, seeders to leechers ratio, upload speeds and so on are  
purely artificial. The main problem with bittorrent simulations is not  
the inaccuracy of the simulated software but wrong assumptions about  
how the network really looks like. With respect to that, the  
abovementioned experiment is not much different from "simulating  
bittorrent with parameters that have unknown relation to real-world  
values".  Since much of the bittorrent behavior characteristics remain  
unknown (although many great measurement papers have been published)  
it is very hard to do credible simulations of protocol performance in  
changed conditions.

AFAIK the only locality-based p2p solution, that was deployed and  
tested in the real-world, with real bittorrent client, users, network  
etc. is Ono. The download speed turned out to be even a bit slower  
than for an unmodified client. (see figure 6a of the Ono SIGCOMM paper
http://www.aqualab.cs.northwestern.edu/publications/DChoffnes08Sigcomm.pdf) 
. Locality, as measured by latency, is dropping down, but it does not  

Re: Paper on "Pushing BitTorrent Locality to the Limit"

Maciej Wojciechowski wrote:
> AFAIK the only locality-based p2p solution, that was deployed and  
> tested in the real-world, with real bittorrent client, users, network  
> etc. is Ono.

The P4P solution has been studied in the real world as well, with real
users and a proprietary client running a variant of the bittorrent protocol.

Results collected on the Comcast's network are discussed in
http://tools.ietf.org/id/draft-livingood-woundy-p4p-experiences.

--

-- 
Ciao,
Enrico

_______________________________________________
alto mailing list
alto <at> ietf.org
https://www.ietf.org/mailman/listinfo/alto

Re: Paper on "Pushing BitTorrent Locality to the Limit"

Enrico,

On Dec 3, 2008, at 2:12 PM, Enrico Marocco wrote:
> The P4P solution has been studied in the real world as well, with real
> users and a proprietary client running a variant of the bittorrent  
> protocol.
>
> Results collected on the Comcast's network are discussed in
> http://tools.ietf.org/id/draft-livingood-woundy-p4p-experiences.

Thanks for the link - I was unaware of this trial. Unfortunately, the  
draft discusses mostly the results and does not give much technical  
information on how the trial was exactly set up. One of the things  
that I'm missing is: how did the users actually find the content that  
was distributed in the trial?

Could you provide a link to a more in-depth explanation of the actual  
experiment? Based on the draft itself it is very hard for me to  
analyze applicability of the presented results.

Regards,
  Maciek Wojciechowski

Re: Paper on "Pushing BitTorrent Locality to the Limit"

On Dec 3, 2008, at 5:44 AM, Maciej Wojciechowski wrote:

> Enrico,
>
> On Dec 3, 2008, at 2:12 PM, Enrico Marocco wrote:
>> The P4P solution has been studied in the real world as well, with  
>> real
>> users and a proprietary client running a variant of the bittorrent  
>> protocol.
>>
>> Results collected on the Comcast's network are discussed in
>> http://tools.ietf.org/id/draft-livingood-woundy-p4p-experiences.
>
> Thanks for the link - I was unaware of this trial. Unfortunately,  
> the draft discusses mostly the results and does not give much  
> technical information on how the trial was exactly set up. One of  
> the things that I'm missing is: how did the users actually find the  
> content that was distributed in the trial?

That was discussed in the talk itself.  The file was a ~20MB (so  
unfortunatly small) mandatory download IIRC, presumably an  
advertisement or similar as part of the Pando p2p system.

Re: Paper on "Pushing BitTorrent Locality to the Limit"

Nicholas Weaver wrote:
>>> The P4P solution has been studied in the real world as well, with  
>>> real
>>> users and a proprietary client running a variant of the bittorrent  
>>> protocol.
>>>
>>> Results collected on the Comcast's network are discussed in
>>> http://tools.ietf.org/id/draft-livingood-woundy-p4p-experiences.
>> Thanks for the link - I was unaware of this trial. Unfortunately,  
>> the draft discusses mostly the results and does not give much  
>> technical information on how the trial was exactly set up. One of  
>> the things that I'm missing is: how did the users actually find the  
>> content that was distributed in the trial?
> 
> That was discussed in the talk itself.  The file was a ~20MB (so  
> unfortunatly small) mandatory download IIRC, presumably an  
> advertisement or similar as part of the Pando p2p system.

Slides, minutes and an audio recording of that talk are available at
http://trac.tools.ietf.org/wg/alto/trac/wiki (#6). I'm sure that if
other material about the experiment is publicly available, it will be
pointed out on this list shortly.

--

-- 
Ciao,
Enrico

_______________________________________________

Re: Paper on "Pushing BitTorrent Locality to the Limit"

Hi Maciej,

> Could you provide a link to a more in-depth explanation of the actual
> experiment? Based on the draft itself it is very hard for me to
> analyze applicability of the presented results.

There was some discussion on the previous p2pi list that includes many of the 
details.  See:
  http://www.ietf.org/mail-archive/web/p2pi/current/msg00877.html
and the followup posts.

--

-- 
Richard Alimi
Department of Computer Science
Yale University 

Re: Paper on "Pushing BitTorrent Locality to the Limit"

Hi Marshall,

Marshall Eubanks wrote: 
> This is not a simulation, but an experiment run on the Grid’5000 
> network with 1700 nodes. Since the lead author is subscribed to the 
> ALTO list, I am going to wonder publicly if it would be
> possible (and sensible) to test any ALTO solution with this 
> experimental setup.

I am not sure what you mean by "test an ALTO solution", but for sure we 
can run (and are running) additional experiments
with real data.
Can you give a bit more details on which kind of experiments you would 
like to perform?

Regards,
Arnaud

Re: Paper on "Pushing BitTorrent Locality to the Limit"

Hi,

Maciej Wojciechowski wrote:
> The only thing that is different in this approach from the standard 
> simulation is that a regular bittorrent client is used. The ISPs 
> topology, seeders to leechers ratio, upload speeds and so on are 
> purely artificial. The main problem with bittorrent simulations is not 
> the inaccuracy of the simulated software but wrong assumptions about 
> how the network really looks like. With respect to that, the 
> abovementioned experiment is not much different from "simulating 
> bittorrent with parameters that have unknown relation to real-world 
> values".  Since much of the bittorrent behavior characteristics remain 
> unknown (although many great measurement papers have been published) 
> it is very hard to do credible simulations of protocol performance in 
> changed conditions.

You are right that you cannot obtain the best of each world.

However, it is plain wrong to claim that our results are equivalent to 
what would have been obtained
with simulations, or that our results do not bring any new significant 
insight compared to previous works.
I hope that a detailed reading of the paper will convince you. If this 
is not the case, I would be pleased to
discuss specific concerns.

As we explain in section 3.2, the results we obtained would have been 
hard, if not impossible, to obtain with simulations.
We show that the dynamics of the packets and of BitTorrent algorithms 
have a major impact on the inter-ISP traffic savings.