Alexander V. Lukyanov | 4 Dec 09:09 2006
Picon

Re: little question about lftp and RFC2228

On Fri, Dec 01, 2006 at 04:59:27PM -0500, Abu-Rashid, Ebrahim wrote:
> I would like to be able to use lftp with CCC; is there a way to do that.

set ftp:use-ccc yes

--

-- 
   Alexander.

stefan.schwenkenbecher | 4 Dec 10:25 2006
Picon

Re: Problem with net:timeout on SunOS 5.8 servers

Hello again,
is there anybody who can tell me something about this issue? Is it a bug or a feature?

Thanx and best regards
Stefan.

On 11/1/06, stefan.schwenkenbecher <at> gmail.com <stefan.schwenkenbecher <at> gmail.com> wrote:
Hello,
I'm using lftp (old version 3.2.1) for a longer time without problems, and now I have updated to version 3.5.6 but found a problem (bug?) regarding the net:timeout setting:
When trying to upload a file which transfer time takes more than the value set in net:timeout, the transfer restarts at the end because of supposed timeout, although the upload has been active all the time :-(

Example:
- File upload with about 200 MB takes approximately 25 seconds
- set net:timeout 10s
- set net:idle 3m
=> After _complete_ transfer the client seems to wait for a put confirmation (FileCopy(139ea0) enters state CONFIRM_WAIT), but in same moment recognizes the timeout and starts reconnecting - and everything starts again from the beginning, because the server does not allow continuation of transfer.

Is this a bug? Or is there another setting I may forgot (I don't want to set the timeout to a too high value because than other real timeouts wouldn't be caught)...


Here is my debug log:

lftp <***> <at> fscad62:/media> put -c testFile.cpio
2006-11-01 13:03:34 FileCopy(139ea0) enters state INITIAL
2006-11-01 13:03:34 FileCopy(139ea0) enters state GET_INFO_WAIT
2006-11-01 13:03:34 FileCopy(139ea0) enters state INITIAL
2006-11-01 13:03:34 FileCopy(139ea0) enters state PUT_WAIT
2006-11-01 13:03:34 FileCopy(139ea0) enters state DO_COPY
2006-11-01 13:03:34 ---> PORT 147,54,132,248,247,224
2006-11-01 13:03:34 dns cache hit
2006-11-01 13:04:04 ---- Connecting to fscad62 (MailScanner warning: numerical links are often malicious: 192.168.138.82) port 21
2006-11-01 13:04:04 <--- 220 fscad62 FTP server (SunOS 5.8) ready.
2006-11-01 13:04:04 ---> FEAT
2006-11-01 13:04:04 <--- 500 'FEAT': command not understood.
2006-11-01 13:04:04 ---> USER <***>
2006-11-01 13:04:04 <--- 331 Password required for <***>.
2006-11-01 13:04:04 ---> PASS <***>
2006-11-01 13:04:04 <--- 230 User <***> logged in.
2006-11-01 13:04:04 ---> CWD /media
2006-11-01 13:04:04 <--- 250 CWD command successful.
2006-11-01 13:04:04 ---> TYPE I
2006-11-01 13:04:04 <--- 200 Type set to I.
2006-11-01 13:04:04 ---> SIZE testFile.cpio
2006-11-01 13:04:04 <--- 500 'SIZE testFile.cpio': command not understood.
2006-11-01 13:04:04 copy: get hit eof
2006-11-01 13:04:04 copy: waiting for put confirmation
2006-11-01 13:04:04 FileCopy(139ea0) enters state CONFIRM_WAIT
2006-11-01 13:04:04 ---> PORT 147,54,132,248,247,243
2006-11-01 13:04:04 FileCopy(139ea0) enters state DO_COPY
2006-11-01 13:04:04 copy: put rolled back to 0, seeking get accordingly
2006-11-01 13:04:04 copy: get position was 200359936
2006-11-01 13:04:04 <--- 200 PORT command successful.
2006-11-01 13:04:04 ---> ALLO 200359936
2006-11-01 13:04:04 <--- 202 ALLO command ignored.
2006-11-01 13:04:04 ---> STOR testFile.cpio
2006-11-01 13:04:04 <--- 150 Binary data connection for testFile.cpio (MailScanner warning: numerical links are often malicious: 147.54.132.248,63475).
2006-11-01 13:04:23 copy: get hit eof
2006-11-01 13:04:23 copy: waiting for put confirmation
2006-11-01 13:04:23 FileCopy(139ea0) enters state CONFIRM_WAIT
2006-11-01 13:04:23 ---- Closing data socket
2006-11-01 13:04:23 **** Timeout - reconnecting
2006-11-01 13:04:23 ---- Closing control socket
2006-11-01 13:04:23 try_time=1162382614, retries=0
2006-11-01 13:04:23 FileCopy(139ea0) enters state DO_COPY
2006-11-01 13:04:23 dns cache hit
2006-11-01 13:04:23 ---- Connecting to fscad62 (MailScanner warning: numerical links are often malicious: 192.168.138.82) port 21
2006-11-01 13:04:23 <--- 220 fscad62 FTP server (SunOS 5.8) ready.
2006-11-01 13:04:23 ---> FEAT
2006-11-01 13:04:23 <--- 500 'FEAT': command not understood.
2006-11-01 13:04:23 ---> USER <***>
2006-11-01 13:04:23 <--- 331 Password required for <***>.
2006-11-01 13:04:23 ---> PASS <***>
2006-11-01 13:04:23 <--- 230 User <***> logged in.
2006-11-01 13:04:23 ---> CWD /media
2006-11-01 13:04:23 <--- 250 CWD command successful.
2006-11-01 13:04:23 ---> TYPE I
2006-11-01 13:04:23 <--- 200 Type set to I.
2006-11-01 13:04:23 ---> SIZE testFile.cpio
2006-11-01 13:04:23 <--- 500 'SIZE testFile.cpio': command not understood.
2006-11-01 13:04:23 copy: get hit eof
2006-11-01 13:04:23 copy: waiting for put confirmation
2006-11-01 13:04:23 FileCopy(139ea0) enters state CONFIRM_WAIT
2006-11-01 13:04:23 ---> PORT 147,54,132,248,247,245
2006-11-01 13:04:23 FileCopy(139ea0) enters state DO_COPY
2006-11-01 13:04:23 copy: put rolled back to 0, seeking get accordingly
2006-11-01 13:04:23 copy: get position was 200359936
2006-11-01 13:04:23 <--- 200 PORT command successful.
2006-11-01 13:04:23 ---> ALLO 200359936
2006-11-01 13:04:23 <--- 202 ALLO command ignored.
2006-11-01 13:04:23 ---> STOR testFile.cpio
2006-11-01 13:04:23 <--- 150 Binary data connection for testFile.cpio (MailScanner warning: numerical links are often malicious: 147.54.132.248 ,63477).
...

Thank you and best regards
Stefan.

stefan.schwenkenbecher | 4 Dec 13:39 2006
Picon

Re: Problem with net:timeout on SunOS 5.8 servers

Hey, thanks for your response.
But I do think that there must have been changed something in lftp, as older version 3.2.1 works fine with the same server...
Furthermore, as I described, the problem does only occure when transfer time is longer than net:timeout value, so there seems to be any interconnection between them :-( In fact, the lftp client doesn't even wait for a confirmation, moreover, it catches a timeout immediately, so there is no chance for the server to confirm successful transfer.

2006-11-01 13:04:23 copy: waiting for put confirmation
2006-11-01 13:04:23 FileCopy(139ea0) enters state CONFIRM_WAIT
2006-11-01 13:04:23 ---- Closing data socket
2006-11-01 13:04:23 **** Timeout - reconnecting
2006-11-01 13:04:23 ---- Closing control socket

Best regards
Stefan.

On 12/4/06, Alexander V. Lukyanov < lav <at> netis.ru> wrote:
On Mon, Dec 04, 2006 at 10:25:27AM +0100, stefan.schwenkenbecher <at> gmail.com wrote:
> is there anybody who can tell me something about this issue? Is it a bug or
> a feature?

It is certainly a bug, but probably not in lftp.

lftp waits for server reply which confirms successful transfer, but does not
receive it for some reason.

--
   Alexander.

Alexander V. Lukyanov | 4 Dec 13:29 2006
Picon

Re: Problem with net:timeout on SunOS 5.8 servers

On Mon, Dec 04, 2006 at 10:25:27AM +0100, stefan.schwenkenbecher <at> gmail.com wrote:
> is there anybody who can tell me something about this issue? Is it a bug or
> a feature?

It is certainly a bug, but probably not in lftp.

lftp waits for server reply which confirms successful transfer, but does not
receive it for some reason.

--
   Alexander.

Alexander V. Lukyanov | 5 Dec 07:52 2006
Picon

Re: Problem with net:timeout on SunOS 5.8 servers

On Mon, Dec 04, 2006 at 01:39:53PM +0100, stefan.schwenkenbecher <at> gmail.com wrote:
> between them :-( In fact, the lftp client doesn't even wait for a
> confirmation, moreover, it catches a timeout immediately, so there is no
> chance for the server to confirm successful transfer.

Ok, it changes the matter. I'll check and see what could be the problem.

--

-- 
   Alexander..

Alexander V. Lukyanov | 5 Dec 15:22 2006
Picon

Re: Problem with net:timeout on SunOS 5.8 servers

On Mon, Dec 04, 2006 at 01:39:53PM +0100, stefan.schwenkenbecher <at> gmail.com wrote:
> between them :-( In fact, the lftp client doesn't even wait for a
> confirmation, moreover, it catches a timeout immediately, so there is no
> chance for the server to confirm successful transfer.

Please try attached patch.

--
   Alexander.
Index: ftpclass.cc
===================================================================
RCS file: /home/lav/cvsroot/lftp/src/ftpclass.cc,v
retrieving revision 1.396
diff -u -p -r1.396 ftpclass.cc
--- ftpclass.cc	30 Nov 2006 11:59:08 -0000	1.396
+++ ftpclass.cc	5 Dec 2006 14:01:20 -0000
 <at>  <at>  -2210,6 +2210,7  <at>  <at>  int   Ftp::Do()
 	    // prevent infinite NOOP's
 	    if(nop_offset==pos && timeout_timer.GetLastSetting()<nop_count*nop_interval)
 	    {
+	       DebugPrint("**** ","NOOP timeout",1);
 	       HandleTimeout();
 	       return MOVED;
 	    }
 <at>  <at>  -2262,7 +2263,7  <at>  <at>  int   Ftp::Do()
 	 if(conn->data_iobuf->Size()>=rate_limit->BytesAllowedToGet())
 	 {
 	    conn->data_iobuf->Suspend();
-	    Timeout(1000);
+	    TimeoutS(1);
 	 }
 	 else if(conn->data_iobuf->Size()>=max_buf)
 	 {
 <at>  <at>  -2321,19 +2322,23  <at>  <at>  int   Ftp::Do()
 	 return MOVED;
       }

-      if(expect->IsEmpty() && conn->data_sock==-1 && conn->data_iobuf && !conn->data_iobuf->Eof())
+      if(conn->data_iobuf)
       {
-	 conn->data_iobuf->PutEOF();
-	 m=MOVED;
-      }
-      if(conn->data_iobuf && conn->data_iobuf->Eof() && conn->data_iobuf->Size()==0)
-      {
-	 state=EOF_STATE;
-	 DataAbort();
-	 DataClose();
-	 idle_timer.Reset();
-	 eof=true;
-	 return MOVED;
+	 if(expect->IsEmpty() && conn->data_sock==-1 && !conn->data_iobuf->Eof())
+	 {
+	    conn->data_iobuf->PutEOF();
+	    m=MOVED;
+	 }
+	 timeout_timer.Reset(conn->data_iobuf->EventTime());
+	 if(conn->data_iobuf->Eof() && conn->data_iobuf->Size()==0)
+	 {
+	    state=EOF_STATE;
+	    DataAbort();
+	    DataClose();
+	    idle_timer.Reset();
+	    eof=true;
+	    return MOVED;
+	 }
       }

       if(copy_mode==COPY_DEST && !copy_allow_store)
 <at>  <at>  -3057,7 +3062,6  <at>  <at>  int  Ftp::FlushSendQueue(bool all)
    if(!conn || !conn->control_send)
       return m;

-   timeout_timer.Reset(conn->control_send->EventTime());
    if(conn->control_send->Error())
    {
       DebugPrint("**** ",conn->control_send->ErrorText(),0);
 <at>  <at>  -3072,6 +3076,13  <at>  <at>  int  Ftp::FlushSendQueue(bool all)
       return MOVED;
    }

+   if(!conn->send_cmd_buffer || conn->send_cmd_buffer->Size()==0)
+      return m;
+
+   // prevent timeout after some idle time
+   if(conn->control_send->Size()==0)
+      timeout_timer.Reset();
+
    while(conn->sync_wait<=0 || all || !(flags&SYNC_MODE))
    {
       int res=conn->FlushSendQueueOneCmd();
 <at>  <at>  -3082,6 +3093,7  <at>  <at>  int  Ftp::FlushSendQueue(bool all)

    if(m==MOVED)
       Roll(conn->control_send);
+   timeout_timer.Reset(conn->control_send->EventTime());

    return m;
 }
 <at>  <at>  -3405,7 +3417,7  <at>  <at>  int   Ftp::Write(const void *buf,int siz
       return DO_AGAIN;

    IOBuffer *iobuf=conn->data_iobuf;
-   if(!buf)
+   if(!iobuf)
       return DO_AGAIN;

    {
Index: Timer.cc
===================================================================
RCS file: /home/lav/cvsroot/lftp/src/Timer.cc,v
retrieving revision 1.16
diff -u -p -r1.16 Timer.cc
--- Timer.cc	4 Aug 2006 07:11:49 -0000	1.16
+++ Timer.cc	5 Dec 2006 13:57:27 -0000
 <at>  <at>  -60,6 +60,8  <at>  <at>  void Timer::Set(const TimeInterval &i)
 }
 void Timer::Reset(const Time &t)
 {
+   if(start>=t)
+      return;
    start=t;
    stop=t;
    stop+=last_setting;
stefan.schwenkenbecher | 6 Dec 16:36 2006
Picon

Re: Problem with net:timeout on SunOS 5.8 servers

Hi,
this patch seems to work! :-)

Unfortunately I can't say anything about potential side effects, but for now everything seems to be fine.
Thanks a lot and best regards
Stefan.

PS: That's the debug output now
lftp 6 <***> <at> mys/media> put testFile.cpio
FileCopy(12d0f8) enters state INITIAL
FileCopy(12d0f8) enters state DO_COPY
---> TYPE I
<--- 200 Type set to I.
---> PORT 147,54,132,241,145,28
<--- 200 PORT command successful.
---> ALLO 36118528
<--- 202 ALLO command ignored.
---> STOR testFile.cpio
<--- 150 Binary data connection for testFile.cpio (MailScanner warning: numerical links are often malicious: 147.54.132.241,37148).
copy: get hit eof
copy: waiting for put confirmation
FileCopy(12d0f8) enters state CONFIRM_WAIT
---- Closing data socket
<--- 226 Transfer complete.
copy: put confirmed store
FileCopy(12d0f8) enters state GET_DONE_WAIT
copy: get is finished - all done
FileCopy(12d0f8) enters state ALL_DONE
36118528 bytes transferred in 35 seconds (998.8K/s)


On 12/5/06, Alexander V. Lukyanov <lav <at> netis.ru> wrote:
On Mon, Dec 04, 2006 at 01:39:53PM +0100, stefan.schwenkenbecher <at> gmail.com wrote:
> between them :-( In fact, the lftp client doesn't even wait for a
> confirmation, moreover, it catches a timeout immediately, so there is no
> chance for the server to confirm successful transfer.

Please try attached patch.

--
   Alexander.



nectar | 7 Dec 03:08 2006
Picon

I can't used 'ls' command on FreeBSD when connect a Pure-FTPd(TLS)

Sir:

I am a newbie, but enjoy lftp which is a power and easy ftp client to used. But I met a problem when connect to a Pure-FTPd(TLS) on my FreeBSD.

1. Install lftp via FreeBSD Ports, there is Lftp Version 3.5.6 with Libraries used: Readline 5.1, Expat 2.0.0, OpenSSL 0.9.8d 28 Sep 2006, libiconv 1.9.
2. Connect to
ProFTPd out of our NAT device, works. But failed when connect to a Pure-FTPd(TLS), looks lftp session death at 'ls' command(but I can do 'cd' to sub-directory, for example 'cd public_html'), no return information.

I searched, find there are similar situation, like 1. Problems with lftp and PureFTPd server (gnutls bug?)  links is http://www.mail-archive.com/lftp <at> uniyar.ac.ru/msg02610.html , 2. FreeBSD SSL/TLS problem, links is  http://www.mail-archive.com/lftp <at> uniyar.ac.ru/msg01518.html .

Here is part of leftp debug out put information:
%lftp -d enchorg <at> 70.98.54.43
Password:
lftp enchorg <at> 70.98.54.43:~> ls
---- Connecting to MailScanner warning: numerical links are often malicious: 70.98.54.43 (MailScanner warning: numerical links are often malicious: 70.98.54.43) port 21
<--- 220---------- Welcome to Pure-FTPd [TLS] ----------
<--- 220-You are user number 14 of 50 allowed.
<--- 220-Local time is now 19:34. Server port: 21.
<--- 220-IPv6 connections are also welcome on this server.
<--- 220 You will be disconnected after 15 minutes of inactivity.
---> FEAT
<--- 211-Extensions supported:
<--- EPRT
<--- IDLE
<--- MDTM
<--- SIZE
<--- REST STREAM
<--- MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;
<--- MLSD
<--- ESTP
<--- PASV
<--- EPSV
<--- SPSV
<--- ESTA
<--- AUTH TLS
<--- PBSZ
<--- PROT
<--- 211 End.
---> AUTH TLS
<--- 234 AUTH TLS OK.
---> OPTS MLST type;size;modify;UNIX.mode;UNIX.uid;UNIX.gid;
Certificate depth: 0; subject: /C=US/ST=Unknown/L=Unknown/O=Unknown/OU=Unknown/CN=host43.hostmonster.com/emailAddress=ssl <at> cpanel.net; issuer: /C=US/ST=Unknown/L=Unknown/O=Unknown/OU=Unknown/CN=host43.hostmonster.com/emailAddress=ssl <at> cpanel.net
WARNING: Certificate verification: self signed certificate
<--- 530 You aren't logged in
---> USER enchorg
<--- 331 User enchorg OK. Password required
---> PASS XXXX
<--- 230-Your bandwidth usage is restricted
<--- 230-User enchorg has group access to: enchorg
<--- 230 OK. Current restricted directory is /
---> PWD
<--- 257 "/" is your current location
---> PBSZ 0
<--- 200 PBSZ=0
---> PROT P
<--- 534 Fallback to [C]
---> PASV
<--- 227 Entering Passive Mode (70,98,54,43,174,240)
---- Connecting data socket to (MailScanner warning: numerical links are often malicious: 70.98.54.43) port 44784
`ls' at 0 [Making data connection...]

Thanks.


bruno | 7 Dec 16:59 2006
Picon
Picon

TLS client certificate

hello,

Is it possible to use TLS client certificate authentification with lftp ?

regards

Bruno

Alexander V. Lukyanov | 8 Dec 15:28 2006
Picon

Re: TLS client certificate

On Thu, Dec 07, 2006 at 04:59:04PM +0100, bruno wrote:
> Is it possible to use TLS client certificate authentification with lftp ?

Yes, see ssl:key-file and ssl:cert-file settings.

--

-- 
   Alexander.


Gmane