lftp-3.5.5 new bug found: mget -c: Segmentation fault

Bug:

Only occured once so far:

1) cd /directory
2) mget -c file.txt
3) at the end of the trasnfer, it barfs

Here is what it looks like from the lftp/client side:

<--- 200 Type set to I.
---> MDTM /pub/file.txt
<--- 213 20021007100044
---> SIZE /pub/file.txt
<--- 213 384452  
---> PASV
<--- 227 Entering Passive Mode (1.2.3.4,123,456)     
---- Connecting data socket to (1.2.3.4) port 1234
---- Data connection established
---> RETR /pub/file.txt
<--- 150 Opening BINARY mode data connection for /pub/file.txt (384452 bytes).
<--- 226 Transfer complete.
---- Got EOF on data connection
---- Closing data socket
Segmentation fault (core dumped)
user <at> host:/home/user

Here is the backtrace, I always compile --with-debug now :)

(gdb) bt

Re: lftp-3.5.5 new bug found: mget -c: Segmentation fault

Mirroring directories seems OK but get/mget seems to cause a segmentation 
fault every time!

(gdb) bt
#0  0x61000001 in ?? ()
#1  0x08065449 in GetJob::Do (this=0x83c8d58) at GetJob.cc:47
#2  0x08061945 in mgetJob::Do (this=0x83c8d58) at mgetJob.cc:159
#3  0x0807d90c in SMTask::Schedule () at SMTask.cc:241
#4  0x08052671 in Job::WaitDone (this=0x810fe68) at Job.cc:557
#5  0x0804e1ad in main (argc=2, argv=0xbf9b9344) at lftp.cc:489

The second time it crashed.

On Sat, 7 Oct 2006, Justin Piszcz wrote:

> Bug:
> 
> Only occured once so far:
> 
> 1) cd /directory
> 2) mget -c file.txt
> 3) at the end of the trasnfer, it barfs
> 
> Here is what it looks like from the lftp/client side:
> 
> <--- 200 Type set to I.
> ---> MDTM /pub/file.txt
> <--- 213 20021007100044
> ---> SIZE /pub/file.txt
> <--- 213 384452  

Re: lftp-3.5.5 new bug found: mget -c: Segmentation fault

When one put/mputs a file, the same bug occurs.

#0  0x00000000 in ?? ()
(gdb) bt
#0  0x00000000 in ?? ()
#1  0x08065449 in GetJob::Do (this=0x83ab278) at GetJob.cc:47
#2  0x08061945 in mgetJob::Do (this=0x83ab278) at mgetJob.cc:159
#3  0x0807d90c in SMTask::Schedule () at SMTask.cc:241
#4  0x08052671 in Job::WaitDone (this=0x810fe68) at Job.cc:557
#5  0x0804e1ad in main (argc=1, argv=0xbf8f0284) at lftp.cc:489

On Sat, 7 Oct 2006, Justin Piszcz wrote:

> Mirroring directories seems OK but get/mget seems to cause a segmentation 
> fault every time!
> 
> (gdb) bt
> #0  0x61000001 in ?? ()
> #1  0x08065449 in GetJob::Do (this=0x83c8d58) at GetJob.cc:47
> #2  0x08061945 in mgetJob::Do (this=0x83c8d58) at mgetJob.cc:159
> #3  0x0807d90c in SMTask::Schedule () at SMTask.cc:241
> #4  0x08052671 in Job::WaitDone (this=0x810fe68) at Job.cc:557
> #5  0x0804e1ad in main (argc=2, argv=0xbf9b9344) at lftp.cc:489
> 
> The second time it crashed.
> 
> On Sat, 7 Oct 2006, Justin Piszcz wrote:
> 
> > Bug:
> > 

remote mirroring issue with sh

Hi there,

I'm currently trying to write a simple bash shell script that
recursively loops through a directory and for each directory entry
will mirror the contents to a remote backup server.  In principal
everything works fine when i step through the commands step by step.

One big issue I am having however is integrating the commands into a
shell script.

One of the limits of shell scripting, is that by placing single quotes
( ' ) within a shell script, variable expansion doesn't occur between
the single quotes ..

#!/bin/sh
backupDirectory="/tmp/somedir/for/backups"
for backupDir in `ls -1 ${backupDirectory}/`;
     msg="attempting backup on ${backupDir}: -- [ "
     cmdString="mirror -R ${backupDirectory}/${backupDir} backups/${backupDir}"
     # no keyword expansion
     ${ftpProgram} -c 'open -e \"${cmdString}\" ${siteUrl}'

     # no errors , return code of 0 but doesn't transfer files remotely
     # ${ftpProgram} -c \'open -e \"${cmdString}\" ${siteUrl}\'

     cmdStatus=${?}
      if [ ${cmdStatus} -gt 0 ]; then
             msg="${msg} failed ] --"
      else
              msg="${msg} success ] --"

Re: remote mirroring issue with sh

Hi Sasha,  

 > I'm currently trying to write a simple bash shell script that
 > recursively loops through a directory and for each directory entry
 > will mirror the contents to a remote backup server.  

Why do you want to loop it in a shell script?
If you need to mirror _all_ directories of a certain directory, then just
mirror that whole directory with one lftp's command.

--

-- 
Michael

Re: remote mirroring issue with sh

On Monday 09 October 2006 10:04, Sasha Dolgy wrote:
> I'm currently trying to write a simple bash shell script that
> recursively loops through a directory and for each directory entry
> will mirror the contents to a remote backup server.  In principal
> everything works fine when i step through the commands step by step.

Which shell are you using? (echo $SHELL)

> One big issue I am having however is integrating the commands into a
> shell script.
>
> One of the limits of shell scripting, is that by placing single quotes
> ( ' ) within a shell script, variable expansion doesn't occur between
> the single quotes ..

This is not a limitation, it is a feature. Use double quotes if you want 
variable expansion.

> #!/bin/sh
> backupDirectory="/tmp/somedir/for/backups"

Here you don't need double quotes.

> for backupDir in `ls -1 ${backupDirectory}/`;
>      msg="attempting backup on ${backupDir}: -- [ "
>      cmdString="mirror -R ${backupDirectory}/${backupDir}
> backups/${backupDir}" # no keyword expansion

Yes, you have variable expansion here..

Re: lftp-3.5.5 new bug found: mget -c: Segmentation fault

On Sat, Oct 07, 2006 at 06:29:52AM -0400, Justin Piszcz wrote:
> #1  0x08065449 in GetJob::Do (this=0x8360800) at GetJob.cc:47

Please test this patch.

--
   Alexander.

Index: CopyJob.cc
===================================================================
RCS file: /home/lav/cvsroot/lftp/src/CopyJob.cc,v
retrieving revision 1.34
diff -u -p -r1.34 CopyJob.cc
--- CopyJob.cc	27 Sep 2006 07:24:04 -0000	1.34
+++ CopyJob.cc	9 Oct 2006 11:56:08 -0000
 <at>  <at>  -244,6 +244,10  <at>  <at>  int CopyJobEnv::Do()
    bytes+=j->GetBytesCount();
    time_spent+=j->GetTimeSpent();
    Delete(j);
+   if(cp==j)
+      cp=0;
+   if(waiting_num>0 && cp==0)
+      cp=(CopyJob*)waiting[0];
    return MOVED;
 }
 void CopyJobEnv::AddCopier(FileCopy *c,const char *n)

Re: lftp-3.5.5 new bug found: mget -c: Segmentation fault

Giving it a try now.

On Mon, 9 Oct 2006, Alexander V. Lukyanov wrote:

> On Sat, Oct 07, 2006 at 06:29:52AM -0400, Justin Piszcz wrote:
> > #1  0x08065449 in GetJob::Do (this=0x8360800) at GetJob.cc:47
> 
> Please test this patch.
> 
> --
>    Alexander.
> 

Re: lftp-3.5.5 new bug found: mget -c: Segmentation fault

That appears to have fixed the problem, I can get and mput, no 
segmentation fault, by the way, what is the ALLO 753 (filesize)? I assume 
this is something that was introduced recently to certain FTP daemons?

Thanks!

Justin.

On Mon, 9 Oct 2006, Justin Piszcz wrote:

> Giving it a try now.
> 
> On Mon, 9 Oct 2006, Alexander V. Lukyanov wrote:
> 
> > On Sat, Oct 07, 2006 at 06:29:52AM -0400, Justin Piszcz wrote:
> > > #1  0x08065449 in GetJob::Do (this=0x8360800) at GetJob.cc:47
> > 
> > Please test this patch.
> > 
> > --
> >    Alexander.
> > 
> 

ALLO command (was: Re: lftp-3.5.5 new bug found: mget -c: Segmentation fault)

On Mon, Oct 09, 2006 at 01:17:26PM -0400, Justin Piszcz wrote:
> by the way, what is the ALLO 753 (filesize)? I assume 
> this is something that was introduced recently to certain FTP daemons?

Actually it is a very old command which indicates the file size before
upload. Lftp started to use it recently. Most ftp servers ignore it,
though. To disable it, use `set ftp:use-allo off'.

--

-- 
   Alexander..