general discussions and major version announcements for the lftp ftp client

headers Radhakrishna Benne | 1 Dec 2011 08:35

_______________________________________________
lftp mailing list
lftp <at> uniyar.ac.ru
http://univ.uniyar.ac.ru/mailman/listinfo/lftp

_______________________________________________
lftp mailing list
lftp <at> uniyar.ac.ru
http://univ.uniyar.ac.ru/mailman/listinfo/lftp

_______________________________________________
lftp mailing list
lftp <at> uniyar.ac.ru
http://univ.uniyar.ac.ru/mailman/listinfo/lftp

Hi,
I would to add one feature request in github for LFTP but it's disabled: 
https://github.com/lavv17/lftp/pulls
I think it would be a good idea to enable it.
Regards.
_______________________________________________
lftp mailing list
lftp <at> uniyar.ac.ru
http://univ.uniyar.ac.ru/mailman/listinfo/lftp

Hi lftp list,

I'm using the mirror function of lftp to regularly update a huge amount 
of data. Because of space issues, I need to split the data on several 
volumes, and I use symbolic links (symlinks) to simulate the original 
directory structure. However, in such a situtation, lftp, at mirroring 
time, destroys the existing symlink, recreates the directory and 
downloads its whole content again :

Example, in the below structure, the mirror command destroys symlinks b 
and c (FTP server contains

/mnt/vol1/data
     a
     b -> /mnt/vol2/data/b
     c -> /mnt/vol2/data/c

Transferring file `a/file.ext'
Removing old local file `b'
Making directory `b'
Transferring file `b/file.ext'

Isn't there a way to tell lftp not to delete local symlinks if existing 
and consider them as directory (the normal behaviour when using the 
shell) ? I tried the -L and mirror:dereference options, but I think they 
are only for remote symlinks, not local ones.

For now, to avoid the problem, I have to run the script below, having 
separates commands for any directory represented by a symlink, but I 
don't like it as if I need to split more, I will every time need to 
modify the script:

------
mirror --exclude=b/.* --exclude=c/.*

lcd b #this command follows correctly the local symlink
mirror b .
lcd -

lcd c
mirror c.
lcd -
---------

If it is not yet possible, this would be feature request (I could try to 
contribute, if you think it would require few changes, but I didn't work 
with c/c++ for a long time)

Thank you anyway for this wonderful tool,

Franck
_______________________________________________
lftp mailing list
lftp <at> uniyar.ac.ru
http://univ.uniyar.ac.ru/mailman/listinfo/lftp

On 05/12/2011 13:47, franck.eyraud <at> jrc.ec.europa.eu wrote:
> (I could try to contribute, if you think it would require few changes, 
> but I didn't work with c/c++ for a long time)
I anyway had a curious look at the source code; I noticed at line 349 of 
MirrorJob.cc the following code :

          if(target_is_local && !script_only)
          {
             if(lstat(target_name,&st)!=-1)
             {
                if(S_ISDIR(st.st_mode))
                {
                   chmod(target_name,st.st_mode|0700);
                   create_target_dir=false;
                }
                else
                {
                   Report(_("Removing old local file `%s'"),
                            dir_file(target_relative_dir,file->name));
                   if(remove(target_name)==-1)
                   {
                      eprintf("mirror: remove(%s): 
%s\n",target_name,strerror(errno));
                      goto skip;
                   }
                   create_target_dir=true;
                }
             }
          }

The S_ISDIR at line 353 test doesn't take into acount the case where the 
target file is a link and the links points to a directory, which would 
suits more my case. In fact, since the whole code above is executed only 
when !script_only, I realized that running mirror --script=script.lft 
and then lftp -c script.lftp, it mirrors correctly (but I'm loosing the 
possibility of parallel downloads). So lftp is able to follow the links, 
it is just that is decides a priori to delete them.
However, I'm not quite sure yet how to handle the "target is link and 
link points to a directory" test in C++, and don't have time now.

How that helps,

Franck
_______________________________________________
lftp mailing list
lftp <at> uniyar.ac.ru
http://univ.uniyar.ac.ru/mailman/listinfo/lftp

On Mon, Dec 05, 2011 at 03:16:05PM +0100, franck.eyraud <at> jrc.ec.europa.eu wrote:
> On 05/12/2011 13:47, franck.eyraud <at> jrc.ec.europa.eu wrote:
> >(I could try to contribute, if you think it would require few
> >changes, but I didn't work with c/c++ for a long time)
> I anyway had a curious look at the source code; I noticed at line
> 349 of MirrorJob.cc the following code :
>
>          if(target_is_local && !script_only)
>          {
>             if(lstat(target_name,&st)!=-1)

Please try this patch and report if it works well for you.
(use -L option).

--
   Alexander.

diff --git a/src/MirrorJob.cc b/src/MirrorJob.cc
index 4b945d6..af27f39 100644
--- a/src/MirrorJob.cc
+++ b/src/MirrorJob.cc
 <at>  <at>  -348,7 +348,7  <at>  <at>  void  MirrorJob::HandleFile(FileInfo *file)
 	 }
 	 if(target_is_local && !script_only)
 	 {
-	    if(lstat(target_name,&st)!=-1)
+	    if((flags&RETR_SYMLINKS?stat:lstat)(target_name,&st)!=-1)
 	    {
 	       if(S_ISDIR(st.st_mode))
 	       {
 <at>  <at>  -693,7 +693,7  <at>  <at>  int   MirrorJob::Do()
       if(target_is_local)
       {
 	 struct stat st;
-	 if(lstat(target_dir,&st)!=-1)
+	 if((flags&RETR_SYMLINKS?stat:lstat)(target_dir,&st)!=-1)
 	 {
 	    if(S_ISDIR(st.st_mode))
 	    {

_______________________________________________
lftp mailing list
lftp <at> uniyar.ac.ru
http://univ.uniyar.ac.ru/mailman/listinfo/lftp

On 06/12/2011 07:01, Alexander V. Lukyanov wrote:
> On Mon, Dec 05, 2011 at 03:16:05PM +0100, franck.eyraud <at> jrc.ec.europa.eu wrote:
>
> Please try this patch and report if it works well for you.
> (use -L option).
Yes it works fine on one of the test case I recreated. I will try to 
compile and run in the environment where I really need it, but I think 
it should be fine.

Thank you for your quick answer !

Franck
_______________________________________________
lftp mailing list
lftp <at> uniyar.ac.ru
http://univ.uniyar.ac.ru/mailman/listinfo/lftp

On 06/12/2011 10:01, franck.eyraud <at> jrc.ec.europa.eu wrote:
> On 06/12/2011 07:01, Alexander V. Lukyanov wrote:
>> On Mon, Dec 05, 2011 at 03:16:05PM +0100, 
>> franck.eyraud <at> jrc.ec.europa.eu wrote:
>>
>> Please try this patch and report if it works well for you.
>> (use -L option).
> Yes it works fine on one of the test case I recreated. I will try to 
> compile and run in the environment where I really need it, but I think 
> it should be fine.
I confirm that it works fine also in the real case, thank you for that.

Do you think this patch will be included in next releases ?

Franck

_______________________________________________
lftp mailing list
lftp <at> uniyar.ac.ru
http://univ.uniyar.ac.ru/mailman/listinfo/lftp

On Tue, Dec 06, 2011 at 11:51:51AM +0100, franck.eyraud <at> jrc.ec.europa.eu wrote:
> Do you think this patch will be included in next releases ?

Yes, it will.

--

-- 
   Alexander.
_______________________________________________
lftp mailing list
lftp <at> uniyar.ac.ru
http://univ.uniyar.ac.ru/mailman/listinfo/lftp

Mon	Tue	Wed	Thu	Fri	Sat	Sun

			1	2	3	4
5	6	7	8	9	10	11
12	13	14	15	16	17	18
19	20	21	22	23	24	25
26	27	28	29	30	31

14	May 2013
19	April 2013
27	March 2013
50	February 2013
46	January 2013
18	December 2012
11	November 2012
13	October 2012
12	September 2012
29	August 2012
32	July 2012
20	June 2012
17	May 2012
9	April 2012
21	March 2012
8	February 2012
20	January 2012
30	December 2011
52	November 2011
33	October 2011
46	September 2011
27	August 2011
15	July 2011
40	June 2011
15	May 2011
29	April 2011
13	March 2011
16	February 2011
7	January 2011
4	December 2010
9	November 2010
14	October 2010
5	September 2010
11	August 2010
17	July 2010
6	June 2010
4	May 2010
48	April 2010
25	March 2010
25	February 2010
16	January 2010
23	December 2009
26	November 2009
29	October 2009
27	September 2009
17	August 2009
21	July 2009
38	June 2009
34	May 2009
22	April 2009
15	March 2009
28	February 2009
8	January 2009
11	December 2008
38	November 2008
15	October 2008
15	September 2008
16	August 2008
25	July 2008
27	June 2008
41	May 2008
20	April 2008
15	March 2008
27	February 2008
24	January 2008
15	December 2007
36	November 2007
53	October 2007
30	September 2007
68	August 2007
25	July 2007
32	June 2007
14	May 2007
27	April 2007
16	March 2007
13	February 2007
20	January 2007
29	December 2006
17	November 2006
38	October 2006
21	September 2006
61	August 2006
17	July 2006
14	June 2006
23	May 2006
41	April 2006
26	March 2006
22	February 2006
19	January 2006
59	December 2005
38	November 2005
2	October 2005
1	June 2005
1	April 2004
1	February 2004
2	December 2003
2	November 2003
3	August 2003
3	May 2003
1	April 2003
2	March 2003
3	February 2003

ftps

different function of ls and [re]cls

[Feature request] "cmd:at-finish" or "cmd:at-queue-complete"

Feature requests in github?

Follow local symlinks when mirroring

Re: Follow local symlinks when mirroring

Re: Follow local symlinks when mirroring

Re: Follow local symlinks when mirroring

Re: Follow local symlinks when mirroring

Re: Follow local symlinks when mirroring