headers
ghrt | 2 Nov 00:37
Picon

fragmentation

i have noticed that valknut (and earlier versions, too) seems to 
unnecessary fragmentate a file which is downloading.
for example, in a file there are 3 blocks and 2 running downloads.
when the 3rd download is started, it doesn't begin with the third 
block, but at a new location in a middle of the others too.

this makes the downloading lasts longer, because at the end of a small 
block is stops and tries to connect again, and the others may be busy 
as well.

why can't begin at an "unused" start of an existing block? (those 
three blocks were quite large, each aprox.  200megs)

thanks
ghrt
Permalink | Reply | 4 comments |
headers
Munteanu Eduard Gabriel | 2 Nov 13:00
Picon

Re: fragmentation

ghrt wrote:

>i have noticed that valknut (and earlier versions, too) seems to 
>unnecessary fragmentate a file which is downloading.
>for example, in a file there are 3 blocks and 2 running downloads.
>when the 3rd download is started, it doesn't begin with the third 
>block, but at a new location in a middle of the others too.
>
>this makes the downloading lasts longer, because at the end of a small 
>block is stops and tries to connect again, and the others may be busy 
>as well.
>
>why can't begin at an "unused" start of an existing block? (those 
>three blocks were quite large, each aprox.  200megs)
>
>thanks
>ghrt
>_______________________________________________
>Dcgui-common mailing list
>Dcgui-common <at> lists.berlios.de
>http://lists.berlios.de/mailman/listinfo/dcgui-common
>
>  
>
Are you sure your filesystem isn't fragmented? If the file you are 
downloading is big enough, there a lot of chances it will not be in one 
piece.
Permalink | Reply | 1 comment |
headers
justice8 | 2 Nov 12:00
Picon

Re: fragmentation

Hi,

Munteanu Eduard Gabriel wrote:

> Are you sure your filesystem isn't fragmented? If the file you are 
> downloading is big enough, there a lot of chances it will not be in 
> one piece.

I don't understand why filesystem fragmentation could impact on valknut 
behavior, theorically, valknu just want <n> MB and filesystem manage 
itself file fragmentation.

Regards,

   J8.
Permalink | Reply | 0 comments |
headers
Andrew Greig | 2 Nov 12:49

Re: fragmentation

ghrt wrote:
> i have noticed that valknut (and earlier versions, too) seems to 
> unnecessary fragmentate a file which is downloading.
> for example, in a file there are 3 blocks and 2 running downloads.
> when the 3rd download is started, it doesn't begin with the third 
> block, but at a new location in a middle of the others too.
> 
> this makes the downloading lasts longer, because at the end of a small 
> block is stops and tries to connect again, and the others may be busy 
> as well.
> 
> why can't begin at an "unused" start of an existing block? (those 
> three blocks were quite large, each aprox.  200megs)
> 
> thanks
> ghrt

My observation is that when starting additional downloads, Valknut finds the biggest incomplete 
block and starts in the middle of it.  This gives the download as large chunk to get before it needs 
to reconnect for another block.  However its probably not optimal when there is another download 
working on that block, as it 'cuts in front' of the existing download and fragments what was a 
contiguous piece.

Beginning a new download at the start of an existing fragment is probably a good idea, as this would 
tend to 'clean up' smaller fragments first and tackle big chunks later.  This would be the opposite 
of the current method, where the big chunks get started first, leaving many small fragments for the 
'end game' which is the problematic last few percent of the file.  How this would actually effect 
overall download efficiency is hard to judge.

Regards,
(Continue reading)

Permalink | Reply | 1 comment |
headers
ghrt | 2 Nov 13:28
Picon

Re: fragmentation

Pe data de Mar 02 Noi 2004 13:49, a-ţi scris:
> My observation is that when starting additional downloads, Valknut
> finds the biggest incomplete block and starts in the middle of it. 
> This gives the download as large chunk to get before it needs to
> reconnect for another block.  However its probably not optimal when
> there is another download working on that block, as it 'cuts in
> front' of the existing download and fragments what was a contiguous
> piece.

whether is optimal or not depends on the speed of the other download; 
if are of equal speeds middle is the answer, if it's a lower speed 
tail is best, and so on.

i don't know how to pre-determine the speed of the future download 
(the speed setting in client may not be accurate), so it will always 
be a guess. i thing the middle is ok.

what is bad is that, even when exists another "idle" fragment, (little 
or big), this isn't choosed as a starting point.
>
> Beginning a new download at the start of an existing fragment is
> probably a good idea, as this would tend to 'clean up' smaller
> fragments first and tackle big chunks later.  
No tackle big chunks later, what i say is that if there is a chunk 
which is not being downloaded, it could start at its beginning, 
regardless of its size (it could be the biggest cunk left as well).

> This would be the 
> opposite of the current method, where the big chunks get started
> first, leaving many small fragments for the 'end game' which is the
(Continue reading)

Permalink | Reply | 0 comments |
headers
Tommy Thorsen | 4 Nov 21:00
Picon
Favicon

DC++ incompatibility

The background for this mail is the problem described in (at least) two 
threads in the forum, namely:
http://dcgui.berlios.de/forum/viewtopic.php?t=2363
and
http://dcgui.berlios.de/forum/viewtopic.php?t=2538

I've posted using the name SuperTommy.

The function CDownloadManager::DM_TransferCallBack() in the file 
"cdownloadmanager.cpp" (line 3785 onwards, from "case DC_MESSAGE_KEY:") 
contains some code where the hierarchy of if's looks like this pseudo-code:

01: if ( Transfer->GetDstDirection() == Transfer->GetSrcDirection() )
02: {
03:     if ( Transfer->GetDstLevel() < Transfer->GetSrcLevel() )
04:     {
05:         if ( Transfer->GetSrcDirection() == edDOWNLOAD )
06:         {
07:             <change DstDirection to edUPLOAD>
08:         }
09:         else if ( Transfer->GetSrcDirection() == edUPLOAD )
10:         {
11:             <change DstDirection to edDOWNLOAD>
12:         }
13:     }
14:     else if ( Transfer->GetDstLevel() > Transfer->GetSrcLevel() )
15:     {
16:         if ( Transfer->GetSrcDirection() == edDOWNLOAD )
17:         {
18:             <change SrcDirection to edUPLOAD>
(Continue reading)

Permalink | Reply | 8 comments |
headers
Robin Hill | 4 Nov 22:08
Picon

Re: DC++ incompatibility

On Thu Nov 04, 2004 at 09:00:11PM +0100, Tommy Thorsen wrote:

> The background for this mail is the problem described in (at least) two 
> threads in the forum, namely:
> http://dcgui.berlios.de/forum/viewtopic.php?t=2363
> and
> http://dcgui.berlios.de/forum/viewtopic.php?t=2538
> 
> I've posted using the name SuperTommy.
> 
> The function CDownloadManager::DM_TransferCallBack() in the file 
> "cdownloadmanager.cpp" (line 3785 onwards, from "case DC_MESSAGE_KEY:") 
> contains some code where the hierarchy of if's looks like this pseudo-code:
> 
> 01: if ( Transfer->GetDstDirection() == Transfer->GetSrcDirection() )
> 02: {
> 03:     if ( Transfer->GetDstLevel() < Transfer->GetSrcLevel() )
> 04:     {
> 05:         if ( Transfer->GetSrcDirection() == edDOWNLOAD )
> 06:         {
> 07:             <change DstDirection to edUPLOAD>
> 08:         }
> 09:         else if ( Transfer->GetSrcDirection() == edUPLOAD )
> 10:         {
> 11:             <change DstDirection to edDOWNLOAD>
> 12:         }
> 13:     }
> 14:     else if ( Transfer->GetDstLevel() > Transfer->GetSrcLevel() )
> 15:     {
> 16:         if ( Transfer->GetSrcDirection() == edDOWNLOAD )
(Continue reading)

Permalink | Reply | 7 comments |
headers
Tommy Thorsen | 4 Nov 22:48
Picon
Favicon

Re: Re: DC++ incompatibility

Thanks for the quick response Robin :-)

>>Question 2: This is really my most important question; Why do we not try 
>>to change SrcDirection in line 22?
>>
>>    
>>
>Because it doesn't make sense here.  Both parties are trying to upload a
>file. but the other party has priority in this case, so they should
>switch direction or retry (if they're not wanting to download).
>  
>
I'm reaching this code when I try to download files from certain DC++ 
users. I haven't explicitly tried to upload anything. Are files (like 
filelists) automatically and secretly uploaded to the other party when I 
try to download from them?

If the other party has priority, shouldn't we yield and switch 
direction? Or do I misunderstand the nature of this priority?

>>Question 4: What does SrcDirection and DstDirection do anyway? It looks 
>>to me like they need do be different, right? One side of the 
>>communication should upload while the other downloads, is that it?
>>
>>    
>>
>Yes, usually they'll be different.  If everyone was active then theere
>wouldn't be a problem as you'd always be uploading to incoming
>connections and downloading from outgoing connections.  With passive
>users you just don't know whether someone connecting to you is
(Continue reading)

Permalink | Reply | 6 comments |
headers
Robin Hill | 5 Nov 10:02
Picon

Re: DC++ incompatibility

On Thu Nov 04, 2004 at 10:48:06PM +0100, Tommy Thorsen wrote:

> Thanks for the quick response Robin :-)
> 
> >>Question 2: This is really my most important question; Why do we not try 
> >>to change SrcDirection in line 22?
> >>
> >>   
> >>
> >Because it doesn't make sense here.  Both parties are trying to upload a
> >file. but the other party has priority in this case, so they should
> >switch direction or retry (if they're not wanting to download).
> > 
> >
> I'm reaching this code when I try to download files from certain DC++ 
> users. I haven't explicitly tried to upload anything. Are files (like 
> filelists) automatically and secretly uploaded to the other party when I 
> try to download from them?
> 
No, but it may be that they have a download queued from you which is not
running either beacuse you have no slots free or because they're
limiting the number of concurrent downloads.

> If the other party has priority, shouldn't we yield and switch 
> direction? Or do I misunderstand the nature of this priority?
> 
I'm not sure on this one.  I'd've thought we should switch direction if
we have a download waiting, though in this case we'd presumably have
started the transaction in download mode, not upload.
(Continue reading)

Permalink | Reply | 5 comments |
headers
Ion-Mihai Tetcu | 5 Nov 12:08
Picon
Favicon

Re: Re: DC++ incompatibility

On Fri, 5 Nov 2004 09:02:49 +0000
Robin Hill <robin <at> robinhill.me.uk> wrote:

>  > Is that right? And how does my passiveness affect the scenario where I 
>  > want to download something from someone?
>  > 
>  That's right, yes.  When you want to download from someone then they'll
>  connect to you.  Your client _should_ then be in download mode (as it
>  knows it's wanting to download from this user) and the remote client is
>  either in upload mode (in which case the download starts) or also in
>  download mode (in which case the priority is looked at and you either
>  switch to upload or the remote switches to upload).

How are these priorities defined/computed ?

-- 
IOnut
Unregistered ;) FreeBSD "user"

--

-- 
IOnut
Unregistered ;) FreeBSD "user"
Permalink | Reply | 0 comments |

Gmane