Use STAT instead of LIST

lftp hangs on data read

Hi Alexander,

I hope it's ok to send you and email to this address.
I wanted to ask your opinion about a problem I'm having.

I'm trying to connect to an ftp site and the ls command get stuck.
Not sure if this is related or not, but it seems that there's also a problem with the site certificate (not sure why it's checked as I've marked set ssl:verify-certificate no).

I'm running lftp 3.5.13 on cygwin.

Help would be greatly appreciated...

#
# Here is what happens on debug mode:
#
[23:49:37][ *mickey* <at> baluga1:~]$ lftp baluga-backup -d
---- Resolving host address...
---- 1 address found
---- Connecting to ftp.drivehq.com (66.220.9.50) port 21
<--- 220 Welcome to the most popular FTP hosting service! Save on hardware, software, hosting and admin. Share files/folders with read-write permission. Visit http://www.drivehq.com/ftp/
---> FEAT
<--- 530 Please login with USER and PASS.
---> AUTH TLS
<--- 234 AUTH Command OK. Initializing TLS connection.
---> USER yadayada
Certificate depth: 0; subject: /C=US/O=www.drivehq.com/OU=GT22613917/OU=See www.rapidssl.com/resources/cps (c)07/OU=Domain Control Validated - RapidSSL(R)/CN=www.drivehq.com; issuer:/C=US/O=Equifax Secure Inc./CN=Equifax Secure Global eBusiness CA-1
WARNING: Certificate verification: unable to get local issuer certificate
WARNING: Certificate verification: certificate not trusted
WARNING: Certificate verification: unable to verify the first certificate
<--- 331 User name ok, need password.
---> PASS XXXX
<--- 230 User successfully logged on.
---> FEAT
<--- 211-Extensions supported:
<---  SIZE
<---  MLSD
<---  MLST
<---  NLST
<---  MDTM
<---  REST STREAM
<---  UTF8
<--- 211 END
---> PWD
<--- 257 "/" is current directory.
---> PBSZ 0
<--- 200 PBSZ Command OK. Protection buffer size set to 0.
---> OPTS UTF8 ON
<--- 200 Enable UTF8 mode.
cd ok, cwd=/
lftp yadayada <at> ftp.drivehq.com:/> cd work
---> CWD /work
<--- 250 CWD command successful. "/work" is current directory.
cd ok, cwd=/work
lftp yadayada <at> ftp.drivehq.com:/work> ls
---> PASV
<--- 227 Entering Passive Mode (66,220,9,50,16,0).
getpeername(control_sock): Transport endpoint is not connected
---- Connecting data socket to (66.220.9.50) port 4096
---- Data connection established
---> LIST
<--- 150 Connection accepted
Interrupt
---> ABOR
---- Closing aborted data socket
<--- 226 ABOR command successful.
<--- 226 Transfer complete
lftp yadayada <at> ftp.drivehq.com:/work> exit
---> QUIT
---- Closing control socket


#
# Just so you'd see the ssl abort certificate check
#
[23:57:48][ *mickey* <at> baluga1:~]$ cat /etc/lftp.conf | grep ssl
set ssl:verify-certificate no


#
#  Version
#
[23:57:58][ *mickey* <at> baluga1:~]$ lftp --version
LFTP | Version 3.5.13 | Copyright (c) 1996-2006 Alexander V. Lukyanov

LFTP is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
There is absolutely no warranty for LFTP.  See COPYING for details.

Send bug reports and questions to <lftp <at> uniyar.ac.ru>.

Libraries used: Readline 5.2, Expat 1.95.8, OpenSSL 0.9.8e 23 Feb 2007, libiconv 1.11

Re: Use STAT instead of LIST

On Mon, Oct 01, 2007 at 09:59:23AM -0400, Jeffrey Litvin wrote:
> My ftpd administrator recently disabled the use of the LIST command and
> wants STAT used instead.

You can try lftp 3.6.0 pre-release. It uses STAT by default and falls back
to LIST if needed.

--

-- 
   Alexander.

Re: lftp hangs on data read

> I'm trying to connect to an ftp site and the ls command get stuck.
> Not sure if this is related or not, but it seems that there's also a problem with the site certificate (not
sure why it's checked as I've marked set ssl:verify-certificate no).

I see that Alexander has fixed an SSL-related bug in version 3.5.14.  I'll try
to release that for Cygwin within the next day or two, then you can try again.
Meanwhile, if you want to compile version 3.5.14 for Cygwin and try it
yourself, please let us know if that solves the problem.

Re: lftp hangs on data read

Re: lftp hangs on data read

>>>>> Regarding 'Re: lftp hangs on data read'; Andrew Schulman adds:

  > I see that Alexander has fixed an SSL-related bug in version 3.5.14.

Upgrading to this version solved my (drivehq.com) problem too, thanks.

--

-- 
Richard

set -a list

lftp :~> set -a
set bmk:auto-sync yes
set bmk:save-passwords no
set cache:cache-empty-listings no
set cache:enable yes
set cache:expire 60m
set cache:expire-negative 1m
set cache:size 1048576
set cmd:at-exit ""
set cmd:cls-completion-default -FB
set cmd:cls-default -F
set cmd:csh-history off
set cmd:default-protocol ftp
set cmd:default-title "lftp \\h:\\w"
set cmd:fail-exit no
set cmd:interactive no
set cmd:long-running 30
set cmd:ls-default ""
set cmd:move-background yes
set cmd:prompt "lftp \\S\\? \\u\\ <at> \\h:\\w> "
set cmd:remote-completion on
set cmd:save-cwd-history yes
set cmd:save-rl-history yes
set cmd:set-term-status no
set cmd:status-interval 0.8s
set cmd:term-status ""
set cmd:term-status/*rxvt* "\\e[11;0]\\e]2;\\T\\007\\e[11]"
set cmd:term-status/*screen* \\e_\\T\\e\\
set cmd:term-status/*xterm* "\\e[11;0]\\e]2;\\T\\007\\e[11]"
set cmd:time-style "%b %e  %Y|%b %e %H:%M"
set cmd:trace no
set cmd:verbose no
set cmd:verify-host yes
set cmd:verify-path yes
set cmd:verify-path-cached no
set color:dir-colors "no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=00;31:*.arj=00;31:*.taz=00;31:*.lzh=00;31:*.zip=00;31:*.z=00;31:*.Z=00;31:*.gz=00;31:*.bz2=00;31:*.bz=00;31:*.tz=00;31:*.rpm=00;31:*.cpio=00;31:*.jpg=00;35:*.gif=00;35:*.bmp=00;35:*.xbm=00;35:*.xpm=00;35:*.png=00;35:*.tif=00;35:"
set color:use-color auto
set dns:SRV-query no
set dns:cache-enable yes
set dns:cache-expire 1h
set dns:cache-size 256
set dns:fatal-timeout 7d
set dns:max-retries 1000
set dns:order "inet6 inet"
set dns:use-fork yes
set file:charset UTF-8
set fish:charset ""
set fish:connect-program "ssh -a -x"
set fish:shell /bin/sh
set ftp:acct ""
set ftp:anon-pass lftp <at> 
set ftp:anon-user anonymous
set ftp:auto-passive-mode yes
set ftp:auto-sync-mode "icrosoft FTP Service|MadGoat"
set ftp:bind-data-socket yes
set ftp:charset ""
set ftp:client lftp/3.4.7
set ftp:device-prefix no
set ftp:fix-pasv-address yes
set ftp:fxp-force no
set ftp:fxp-passive-source no
set ftp:fxp-passive-sscn yes
set ftp:home ""
set ftp:ignore-pasv-address no
set ftp:lang ""
set ftp:list-empty-ok no
set ftp:list-options ""
set ftp:nop-interval 120
set ftp:passive-mode on
set ftp:port-ipv4 ""
set ftp:port-range full
set ftp:proxy ""
set ftp:proxy-auth-joined no
set ftp:rest-list no
set ftp:rest-stor yes
set ftp:retry-530 "too many|overloaded|try (again |back )?later|is restricted to|maximum
number|number of connect|only.*session.*allowed|more connection"
set ftp:retry-530-anonymous "Login incorrect"
set ftp:site-group ""
set ftp:skey-allow yes
set ftp:skey-force no
set ftp:ssl-allow yes
set ftp:ssl-allow-anonymous no
set ftp:ssl-auth TLS
set ftp:ssl-force no
set ftp:ssl-protect-data no
set ftp:ssl-protect-fxp no
set ftp:ssl-protect-list yes
set ftp:ssl-use-ccc no
set ftp:stat-interval 1
set ftp:sync-mode on
set ftp:sync-mode/ftp.idsoftware.com on
set ftp:sync-mode/ftp.microsoft.com on
set ftp:sync-mode/sunsolve.sun.com on
set ftp:timezone GMT
set ftp:use-abor yes
set ftp:use-feat yes
set ftp:use-fxp yes
set ftp:use-hftp yes
set ftp:use-mdtm yes
set ftp:use-mdtm-overloaded no
set ftp:use-mlsd no
set ftp:use-pret yes
set ftp:use-quit yes
set ftp:use-site-chmod yes
set ftp:use-site-idle no
set ftp:use-site-utime no
set ftp:use-size yes
set ftp:use-stat yes
set ftp:use-telnet-iac yes
set ftp:verify-address no
set ftp:verify-port no
set ftp:web-mode off
set ftps:initial-prot ""
set hftp:cache yes
set hftp:cache-control ""
set hftp:proxy ""
set hftp:use-authorization yes
set hftp:use-head yes
set hftp:use-mkcol no
set hftp:use-propfind no
set hftp:use-type yes
set http:accept */*
set http:accept-charset ""
set http:accept-language ""
set http:authorization ""
set http:cache yes
set http:cache-control ""
set http:cookie ""
set http:post-content-type application/x-www-form-urlencoded
set http:proxy ""
set http:put-content-type ""
set http:put-method PUT
set http:referer ""
set http:set-cookies no
set http:use-mkcol yes
set http:use-propfind no
set http:user-agent lftp/3.4.7
set https:proxy ""
set mirror:dereference no
set mirror:exclude-regex "(^|/)(\\.in\\.|\\.nfs)"
set mirror:order "*.sfv *.sig *.md5* *.sum * */"
set mirror:parallel-directories yes
set mirror:parallel-transfer-count 1
set mirror:set-permissions yes
set mirror:use-pget-n 1
set module:path /usr/lib/lftp/3.4.7:/usr/lib/lftp
set net:connection-limit 0
set net:connection-takeover yes
set net:idle 180
set net:limit-max 0
set net:limit-rate 0:0
set net:limit-total-max 0
set net:limit-total-rate 0:0
set net:max-retries 4096
set net:no-proxy ""
set net:persist-retries 0
set net:reconnect-interval-base 30
set net:reconnect-interval-max 600
set net:reconnect-interval-multiplier 1.5
set net:socket-bind-ipv4 ""
set net:socket-bind-ipv6 ""
set net:socket-buffer 0
set net:socket-maxseg 0
set net:timeout 5m
set sftp:charset ""
set sftp:connect-program "ssh -a -x"
set sftp:max-packets-in-flight 16
set sftp:protocol-version 4
set sftp:server-program sftp
set sftp:size-read 0x8000
set sftp:size-write 0x8000
set ssl:ca-file ""
set ssl:ca-path ""
set ssl:cert-file /etc/pki/tls/emmaCert.pem
set ssl:crl-file ""
set ssl:crl-path ""
set ssl:key-file /etc/pki/tls/emmaKey.pem
set ssl:verify-certificate no
set xfer:clobber yes
set xfer:destination-directory ""
set xfer:disk-full-fatal no
set xfer:eta-period 120
set xfer:eta-terse yes
set xfer:make-backup yes
set xfer:max-redirections 10
set xfer:rate-period 15
set xfer:verify no
set xfer:verify-command /usr/share/lftp/verify-file

Tom Winegar
STARS Administrator
Subaru Telescope
Hilo, Hawaii USA
808-934-5961

lftp error: 'SSL_connect: unknown protocol'

dear alexander:

the error message from lftp 3.4.7 is 'SSL_connect: unknown protocol'

the error message from proftpd 3.4.2 TLS Log:
'unable to accept TLS connection
SSL3_GET_RECORD:wrong version number'

no matter my settings/ports in proftpd or lftp.

1. lftp debug on 'ls' says:
connect OK
-->FEAT
<--211 Features list OK
<---211 End
---> AUTH TLS
<--- 234 AUTH TLS successful
---> USER twin
**** SSL_connect: unknown protocol
----Closing control socket 

2. my proftpd.conf TLS sections is:
<IfModule mod_tls.c>
 TLSEngine on
 TLSRequired			off
 TLSLog /var/log/tls.log
# TLSProtocol SSLv3
 TLSProtocol TLSv1
 TLSVerifyClient		off
 TLSRenegotiate required off
 TLSOptions NoCertRequest

 # Server's Certificate
 TLSRSACertificateFile /etc/pki/tls/sheilaCert.pem
 TLSRSACertificateKeyFile /etc/pki/tls/sheilaKey.pem

</IfModule>

3. both computers are FC5.

4. I have SSL certificates on both sides which seem OK.

any ideas?

thanks,
tom

Tom Winegar
STARS Administrator
Subaru Telescope
Hilo, Hawaii USA
808-934-5961

- Version 2.4.9 does NOT hang at [Making data connection...]

With an old Debian distribution, lftp version 2.4.9 works with a .lftprc
startup file. But with the same startup file copied to a fully up-to-date
Debian distribution, lftp hangs at [Making data connection...]

step 1: $ lftp
step 2: Password: foobar\n
step 3: lftp ... :~> ls
`ls' at 0 [Making data connection...]

It sits at making data connection for minutes and minutes. With version 2.4.9
it works immediately. The software is the latest Debian software.  Here is my
/etc/apt/sources.list:

deb http://mirrors.usc.edu/pub/linux/distributions/debian/ etch main
deb-src http://mirrors.usc.edu/pub/linux/distributions/debian/ etch main

deb http://security.debian.org/ etch/updates main
deb-src http://security.debian.org/ etch/updates main

Help. Robert Cymbala

Re: - Version 2.4.9 does NOT hang at [Making data connection...]

> step 1: $ lftp
> step 2: Password: foobar\n
> step 3: lftp ... :~> ls
> `ls' at 0 [Making data connection...]

This looks like what happens when you're trying to use ftp over ssl, but your
firewall blocks it.  E.g. if your firewall allows ftp control connections, and
uses nf_conntrack_ftp to detect and allow related ftp data connections, then
data connections will fail with ftp over ssl because nf_conntrack_ftp can't
observe the traffic to see which port it needs to open.

Try running

set ftp:ssl-allow 0

and try your connection again.

Good luck,
Andrew.