Bug with searching for files by TTH

While trying to add support for ADCGet and TTHF to dclib/valknut 0.3.7,
I have discovered a serious bug which causes files to not be found when
searched for by their TTH.

I have confirmed the bug occurrs when you rename a file that has
already been hashed and is in your share, but there may be other cases
which trigger the bug.

When a file is searched for by TTH, but not found as it should be, I
have discovered that the following happens:

CSearchIndex::SearchHash is called
CSearchIndex::FindHash is then called, and finds a value
CSearchIndex::HashBaseIndexFromHashIndex converts this value into
another (often the same)
CSearchIndex::BaseIndexFromHashBaseIndex then fails

But I'm thinking the problem lies in the code for adding the values to
the various indexes (*.bin files in ~/.dc/).

So far, I have only managed to add extra debugging statements to dclib,
I haven't been able to make any progress in fixing this problem.
A patch for the extra debugging output is available here:
http://www.dcs.warwick.ac.uk/~csucda/extra-debug-dclib.diff

We need to fix this bug because DC++ >= 0.696 will require support for
ADCGet and TTHF, so we need to fix this bug for valknut to have any
future.

--

Re: Bug with searching for files by TTH

On Sat, 2006-10-14 at 14:38 +0100, Edward J. Sheldrake wrote:
> While trying to add support for ADCGet and TTHF to dclib/valknut 0.3.7,
> I have discovered a serious bug which causes files to not be found when
> searched for by their TTH.

Do you have the fix for the known problem in csearchindex applied?
http://dcgui.berlios.de/forum/viewtopic.php?t=2934

I'm not saying it will work, but it might help.

Sincerely
Henning

_______________________________________________
Dcgui-common mailing list
Dcgui-common <at> lists.berlios.de
https://lists.berlios.de/mailman/listinfo/dcgui-common

Re: Bug with searching for files by TTH (FIXED!)

OK, I think I have finally found and fixed the problem:

diff -Naur dclib-0.3.7.orig/dclib/csearchindex.cpp
dclib-0.3.7.new/dclib/csearchindex.cpp
--- dclib-0.3.7.orig/dclib/csearchindex.cpp	2006-10-14
16:51:43.000000000 +0100
+++ dclib-0.3.7.new/dclib/csearchindex.cpp	2006-10-14
20:09:23.000000000 +0100
 <at>  <at>  -164,7 +164,7  <at>  <at> 
 				sl->Add( *ps, ps );
 			}
 		}
-		hi++;
+		hi+=TIGERSIZE;
 	}

 	return sl;

Sorry to have bothered you, but this bug has been driving me nuts for a
few days now!

Send instant messages to your online friends http://uk.messenger.yahoo.com 

Auto-Search

I'm running Valknut v20061017 by ejs and i was
wondering about the Auto-Search function in
the configs...

What does it actually do? The most obvious ('Auto-
Search for new sources') don't seem to work/do anything
(visable).
--

-- 
spy Khaddafi pits security congress Ortega iodine ammonium Albanian
$400 million in gold bullion cracking explosion Saddam Hussein jihad
nuclear
[See http://www.aclu.org/echelonwatch/index.html for more about this]
[Or http://www.europarl.eu.int/tempcom/echelon/pdf/rapport_echelon_en.pdf]
If neither of these works, try http://www.aclu.org and search for echelon.
Note. This is a real, not fiction.
http://www.theregister.co.uk/2001/09/06/eu_releases_echelon_spying_report/
http://www.aclu.org/safefree/nsaspying/23989res20060131.html#echelon

Re: Auto-Search

--- Turbo Fredriksson <turbo <at> bayour.com> wrote:

> I'm running Valknut v20061017 by ejs and i was
> wondering about the Auto-Search function in
> the configs...
> 
> What does it actually do? The most obvious ('Auto-
> Search for new sources') don't seem to work/do anything
> (visable).
The Auto-Search for new sources only works when all three of the
following conditions are true:
- the transfer is for a file
- the transfer is in multi-download mode
- the start of the file has already been hashed (not TTH, but valknut's
own hash for checking the file is the same)

It doesn't open the search dialog or anything, but you should see the
search in the search spy of another client. You should see "init
search" printed on standard output.

When in single download mode, you'll see the lines "init search" and
then "failed" printed out on standard output.

Unfortunately, the testing I did to check this seems to have uncovered
a bug in the ADCGet work, when in multi download mode. When the remote
client is valknut, the chunked downloading (fetching 1MiB at a time)
seems to work fine, but when the remote client is DC++, the transfer
freezes after the end of one chunk, and sometimes we even get "File Not
Available" errors from DC++. 

Re: Auto-Search

Quoting "Edward J. Sheldrake" <ejs1920 <at> yahoo.co.uk>:

>> How do one tell if it's a multi-download? Is this specified globaly,
>> or localy for each download?
>> 
>> Sometimes I got the question and selected multi. Can that be changed?
>
> File > Quick options > Download mode.
> It is set when the download starts, from the option specified under
> quick options (or from which button you press if it's set to "Ask")
> It cannot be changed after the download has started.

'after the download has started'. For this session, or at all?
If I change the setting and restart Valknut. Will those particular
downloads still be multi?

> A bug with TTHF download was recently fixed, (version 20061029), that
> should fix the "File Not Available" errors.

I'll try that version. In the mean time I've been running my own version,
with SOME of the latest patches.

The only patches I have NOT applied are the following. My version works
perfectly. In two hours running, it downloaded 22 files (roughly a gig)
with more than 14 concurrent inbout (i.e. me downloading) connections
where as 20061017ejs didn't download ANY.

----- s n i p -----
dclib-0.3.7-autoconf-crap.patch
dclib-0.3.7-change-ugetblock-handling.patch

Re: Auto-Search

--- Turbo Fredriksson <turbo <at> bayour.com> wrote:

> Quoting "Edward J. Sheldrake" <ejs1920 <at> yahoo.co.uk>:
> 
> >> How do one tell if it's a multi-download? Is this specified
> globaly,
> >> or localy for each download?
> >> 
> >> Sometimes I got the question and selected multi. Can that be
> changed?
> >
> > File > Quick options > Download mode.
> > It is set when the download starts, from the option specified under
> > quick options (or from which button you press if it's set to "Ask")
> > It cannot be changed after the download has started.
> 
> 'after the download has started'. For this session, or at all?
> If I change the setting and restart Valknut. Will those particular
> downloads still be multi?
Multi or single download mode is saved in the download queue, so it
will be for all sessions. And the good news is I fixed the bug with
multi download mode in the latest version of dclib (20061030).

> 
> > A bug with TTHF download was recently fixed, (version 20061029),
> that
> > should fix the "File Not Available" errors.
> 
> I'll try that version. In the mean time I've been running my own
> version,

cvs or stable?

Hello,
I'd like to ask a couple of questions. 

It seems that the latest stable relase 0.3.7 is a few years old. Does
that mean that the development stopped altogether? Does the CVS version
differ?

The second question concerns using valknut: When Valknut successfully
downloads a file, it seems impossible to do more downloads. I can search
the hub, view the results, but when I choose to download some other
file, it complains: "Not connected to required hub!" When I click
"connect", the Valknut will try to connect to the hub, but it will get
immediately disconnected by the hub, because it says that the there is
already a user connected with the same nick.
Am i doing something wrong, or is this a bug?
Thanks for your work,
petr

Re: cvs or stable?

--- Petr Danecek <danecek <at> ucl.cas.cz> wrote:

> Hello,
> I'd like to ask a couple of questions. 
> 
> It seems that the latest stable relase 0.3.7 is a few years old. Does
> that mean that the development stopped altogether? Does the CVS
> version
> differ?
The CVS head version is currently unusable as far as I know. Mathias
started an extensize set of changes, but unfortunately didn't have time
to finish them. More recently, I have done some work on 0.3.7, which is
summarised here http://www.dcs.warwick.ac.uk/~csucda/valknut.html . If
this ever gets into the cvs repository, it will be branched off from
0.3.7.

> 
> The second question concerns using valknut: When Valknut successfully
> downloads a file, it seems impossible to do more downloads. I can
> search
> the hub, view the results, but when I choose to download some other
> file, it complains: "Not connected to required hub!" When I click
> "connect", the Valknut will try to connect to the hub, but it will
> get
> immediately disconnected by the hub, because it says that the there
> is
> already a user connected with the same nick.
> Am i doing something wrong, or is this a bug?
> Thanks for your work,
> petr

Re: Auto-Search

Quoting "Edward J. Sheldrake" <ejs1920 <at> yahoo.co.uk>:

>> 'after the download has started'. For this session, or at all?
>> If I change the setting and restart Valknut. Will those particular
>> downloads still be multi?
>
> Multi or single download mode is saved in the download queue, so it
> will be for all sessions. And the good news is I fixed the bug with
> multi download mode in the latest version of dclib (20061030).

That version works perfect, thanx! 

>> ----- s n i p -----
>> dclib-0.3.7-autoconf-crap.patch
>> dclib-0.3.7-change-ugetblock-handling.patch
>> dclib-0.3.7-fix-downloading-files-from-search.patch
> The above patch may be very important for you, since it removes the
> "TTH:" prefix from TTH search results so they can be used properly in
> TTHF requests. Otherwise, valknut will send an invalid TTH and get a
> "File Not Available" error.

Ah, ok. But now I'm running your version, and I STILL get the 'TTH:'
prefix!

I'm now running

dclib: 20061030ejs
valknut: 20061017ejs

and when doing a 'Search for Clones by TTH' it prefixes the search