Niall OConnor | 1 May 18:48
Picon

FTPOSError

I get this error with ftputil.  I thought it was because the version ( ftp2.6) was too great for python 2.5.2 so I installed ftp2.5 .  I still had no luck.  Can you tell me waht might be wrong here?

Python 2.5.2 (r252:60911, Jan 24 2010, 14:53:14) 
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ftputil
>>> pem_user = {'PEM_password': xxx', 'PEM_account_id': u'xx', 'domain_name': xxx, 'FTP_login': xxx', 'FTP_password': xxx', 'FTP_host_name': xxx, 'PEM_password_confirm': xxx', 'PEM_subscription_id': xxx', 'PEM_login_id': xxx', 'domain_id': 1}
>>> host = ftputil.FTPHost(pem_user['FTP_host_name'], pem_user['FTP_login'], pem_user['FTP_password'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "build/bdist.linux-i686/egg/ftputil/ftputil.py", line 104, in __init__
  File "build/bdist.linux-i686/egg/ftputil/ftputil.py", line 158, in _make_session
  File "build/bdist.linux-i686/egg/ftputil/ftp_error.py", line 149, in _try_with_oserror
ftputil.ftp_error.FTPOSError: -2
Debugging info: ftputil 2.6, Python 2.5.2 (linux2)
>>> 
KeyboardInterrupt
>>> 
vivien:/home/www/sitebuilder.blacknight.ie/web# pip uninstall ftputil
Uninstalling ftputil:
  /usr/lib/python2.5/site-packages/ftputil-2.6-py2.5.egg
Proceed (y/n)? y
  Successfully uninstalled ftputil
vivien:/home/www/sitebuilder.blacknight.ie/web# python
Python 2.5.2 (r252:60911, Jan 24 2010, 14:53:14) 
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ftputil
>>> pem_user = {'PEM_password': xxx', 'PEM_account_id': u'xx', 'domain_name': xxx, 'FTP_login': xxx', 'FTP_password': xxx', 'FTP_host_name': xxx, 'PEM_password_confirm': xxx', 'PEM_subscription_id': xxx', 'PEM_login_id': xxx', 'domain_id': 1}
>>> host = ftputil.FTPHost(pem_user['FTP_host_name'], pem_user['FTP_login'], pem_user['FTP_password'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/site-packages/ftputil/ftputil.py", line 103, in __init__
    self._session = self._make_session()
  File "/usr/lib/python2.5/site-packages/ftputil/ftputil.py", line 157, in _make_session
    return ftp_error._try_with_oserror(factory, *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/ftputil/ftp_error.py", line 149, in _try_with_oserror
    raise FTPOSError(*exc.args)
ftputil.ftp_error.FTPOSError: -2
Debugging info: ftputil 2.5, Python 2.5.2 (linux2)
>>> 

_______________________________________________
ftputil mailing list
ftputil@...
http://codespeak.net/mailman/listinfo/ftputil
Stefan Schwarzer | 7 Apr 20:51

Fwd: Re: throttled upload with ftputil?

Hello,

Here's an application of the callback parameter for
uploads and downloads which I didn't think of when
I added callbacks.

Forwarded with consent from Alec. :)

Stefan

-------- Original Message --------
Subject: Re: throttled upload with ftputil?
Date: Sun, 27 Mar 2011 11:28:56 +0200
From: Stefan Schwarzer <sschwarzer@...>
To: Alec Bennett <wrybread@...>

Hello Alec,

Thanks for your feedback!

On 2011-03-27 06:53, Alec Bennett wrote:
> I was wondering if its possible to throttle the upload speed when uploading
> a file through ftputil?
> 
> I'm currently uploading like this, if its a factor:
> 
> ftp = ftputil.FTPHost(hostname, username, password)
> ftp.upload(fname, target, mode = "b")

ftputil doesn't have a parameter to specifically manipulate
the upload or download speed. Usually such limiting is done
on the operating system level.

That said, I guess you could achieve something useful by
using a callback parameter [1] in the `upload` method.
(Callbacks were added in ftputil 2.5.) I think you could
just use

  import time

  import ftputil

  def throttle_callback(chunk):
      """Insert some sleep time to decrease the transfer speed."""
      # Use whatever interval you find appropriate.
      time.sleep(1)

  ftp = ftputil.FTPHost(hostname, username, password)
  ftp.upload(fname, target, mode='b', callback=throttle_callback)

If you need more control, e. g. want to throttle down to a
specific transfer rate, you could count the bytes
transferred so far (the sum of the lengths of the chunks)
and the elapsed time (`time.time()` minus start time of
transfer) and insert corresponding sleep intervals if the
rate is too high. If you need more concrete help, let me
know.

> And thanks for the spectacular module, by the way.

Thanks! You're welcome. :)

Although you wrote me in private, I think the question and
the approach to it are interesting. I had never had such an
application in mind when I added the callback support. I'd
like to forward this e-mail to the ftputil mailing list.
Would that be ok for you?

[1] http://ftputil.sschwarzer.net/trac/wiki/Documentation#uploading-and-downloading-files

Stefan

ftp download issue

Hi All,

I am having an issue trying to download files from our remote FTP  
server using ftputils-2.2.3.

  In local development, we can connect to the remote FTP server and  
download the files to our local box, no problem. When we deploy the  
codebase to our development webserver and try, the code just hangs,  
throwing no exception. Our webhosting company, Webfaction, says it  
doesn't look like an issue from their end.

Basically the user selects the required files from the page via  
checkboxes ('download' parameter).

Code that downloads the files:

c = Context({})
     if request.method == 'POST' and request.is_ajax():

         cloud_user = get_object_or_404(User,username=username)
         user_profile = UserProfile.objects.get(user=cloud_user)
         data = {}
         client =  
FTPHost 
(settings 
.CLOUD_SERVER,user_profile.cloud_username,user_profile.cloud_password)
         adir = os.path.dirname(user_profile.download_dir)
         if not os.path.isabs(adir):
             adir = ('%s%s' % ("/",adir))

         if os.path.isdir(adir):

             downloads = request.POST.getlist('download')
             for download in downloads:

                 filename = os.path.split(download)[1]
                 localfile = os.path.join(adir,filename)
                 client.download(download, localfile)

             data['outcome'] = True
             data['results_text'] = ('%s%s' % ("All files ",  
"successfully downloaded."))

             client.close()

         else:

             data['outcome'] = False
             data['results_text'] = ('%s%s%s' % ("The download  
directory ",user_profile.download_dir, " does not exist on your local  
system. No files downloaded."))

         return _json_response(data)

     return render_to_response('cloud/directory.html', c,  
context_instance=RequestContext(request))

Any help much appreciated!

Cheers
Rick
Stefan Schwarzer | 15 Mar 19:38

Mailing lists migration

Hello,

For several years, the ftputil mailing lists (i. e. the
general discussion list and the ticket mailing list) have
thankfully been hosted by Codespeak. However, as nowadays
there are lots of hosting possibilities for open source
projects, Codespeak is going to shut down its hosting. So
we/I have to migrate the ftputil mailing lists to a new
provider. (I'm not really eager to host the lists myself.)

I opened a ticket [1] on this and outlined what's to do. I'd
like to know if I forgot something. Also, I'm very
interested in suggestions of appropriate mailing list
providers. I'd like to finish the migration by the end of
April but have nothing against finishing it earlier. :)

As usual, I'm looking forward to your feedback. :-)

[1] http://ftputil.sschwarzer.net/trac/ticket/57

Stefan
Stefan Schwarzer | 12 Mar 19:32

[ANN] ftputil 2.6 released

ftputil 2.6 is now available from
http://ftputil.sschwarzer.net/download .

Changes since version 2.5
-------------------------

- The stat caching has been improved. There's now an "auto-grow"
  feature for `FTPHost.listdir` which in turn applies to
  `FTPHost.walk`. Moreover, there are several performance
  optimizations.

- A few bugs were fixed [1-3].

What is ftputil?
----------------

ftputil is a high-level FTP client library for the Python programming
language. ftputil implements a virtual file system for accessing FTP
servers, that is, it can generate file-like objects for remote files.
The library supports many functions similar to those in the os,
os.path and shutil modules. ftputil has convenience functions for
conditional uploads and downloads, and handles FTP clients and servers
in different timezones.

Read the documentation at
http://ftputil.sschwarzer.net/documentation .

License
-------

ftputil is Open Source software, released under the revised BSD
license (see http://www.opensource.org/licenses/bsd-license.php ).

[1] http://ftputil.sschwarzer.net/trac/ticket/53
[2] http://ftputil.sschwarzer.net/trac/ticket/55
[3] http://ftputil.sschwarzer.net/trac/ticket/56

Stefan
Stefan Schwarzer | 19 Feb 16:05

[ANN] ftputil 2.6b released

ftputil 2.6b is now available from
http://ftputil.sschwarzer.net/download .

Changes since version 2.5
-------------------------

- The stat caching has been improved. I've added an "auto-grow"
  feature for `FTPHost.listdir` which in turn applies to
  `FTPHost.walk`. Moreover, there were several performance
  optimizations.

- A few bugs were fixed [1-3].

What is ftputil?
----------------

ftputil is a high-level FTP client library for the Python programming
language. ftputil implements a virtual file system for accessing FTP
servers, that is, it can generate file-like objects for remote files.
The library supports many functions similar to those in the os,
os.path and shutil modules. ftputil has convenience functions for
conditional uploads and downloads, and handles FTP clients and servers
in different timezones.

Read the documentation at
http://ftputil.sschwarzer.net/documentation .

License
-------

ftputil is Open Source software, released under the revised BSD
license (see http://www.opensource.org/licenses/bsd-license.php ).

[1] http://ftputil.sschwarzer.net/trac/ticket/53
[2] http://ftputil.sschwarzer.net/trac/ticket/55
[3] http://ftputil.sschwarzer.net/trac/ticket/56

Stefan
Volodymyr Kotulskyi | 11 Feb 14:35
Picon

ftputil and py2exe error

Here is the steps to reproduce this bug

Python version 2.6.6
ftputil-2.5-py2.6
py2exe-0.6.9.win32-py2.6.exe

ftp_example.py

import re
import ftputil

pattern = re.compile('^.*$', re.I)
print pattern
connection = ftputil.FTPHost('127.0.0.1', 'anonymous', '')
print connection

setup.py

from distutils.core import setup
import py2exe

setup(
      console=['ftp_example.py'],
      options={"py2exe": { "includes" : "ftputil"}})

Here is the output

D:\PROJECTS\IKISPFU\_DB\IKIS_Queue\ASOPDExchange\src>python setup.py py2exe
C:\DevTools\Python26\lib\site-packages\py2exe\build_exe.py:16:
DeprecationWarning: the sets module is deprecated
  import sets
running py2exe
*** searching for required modules ***
Traceback (most recent call last):
  File "setup.py", line 6, in <module>
    options={"py2exe": { "includes" : "ftputil"}})
  File "C:\DevTools\Python26\lib\distutils\core.py", line 152, in setup
    dist.run_commands()
  File "C:\DevTools\Python26\lib\distutils\dist.py", line 975, in run_commands
    self.run_command(cmd)
  File "C:\DevTools\Python26\lib\distutils\dist.py", line 995, in run_command
    cmd_obj.run()
  File "C:\DevTools\Python26\lib\site-packages\py2exe\build_exe.py",
line 243, in run
    self._run()
  File "C:\DevTools\Python26\lib\site-packages\py2exe\build_exe.py",
line 296, in _run
    self.find_needed_modules(mf, required_files, required_modules)
  File "C:\DevTools\Python26\lib\site-packages\py2exe\build_exe.py",
line 1297, in find_needed_modules
    mf.import_hook(mod)
  File "C:\DevTools\Python26\lib\site-packages\py2exe\mf.py", line
719, in import_hook
    return Base.import_hook(self,name,caller,fromlist,level)
  File "C:\DevTools\Python26\lib\site-packages\py2exe\mf.py", line
136, in import_hook
    q, tail = self.find_head_package(parent, name)
  File "C:\DevTools\Python26\lib\site-packages\py2exe\mf.py", line
204, in find_head_package
    raise ImportError, "No module named " + qname
ImportError: No module named ftputil

--

-- 
С уважением, Владимир.
_______________________________________________
ftputil mailing list
ftputil <at> codespeak.net
http://codespeak.net/mailman/listinfo/ftputil
Stefan Schwarzer | 24 Oct 14:14

[ANN] ftputil 2.5 released

ftputil 2.5 is now available from
http://ftputil.sschwarzer.net/download .

Changes since version 2.4.2
---------------------------

- As announced over a year ago [1], the `xreadlines` method for
  FTP file objects has been removed, and exceptions can no longer be
  accessed via the `ftputil` namespace. Only use `ftp_error` to access
  the exceptions.

  The distribution contains a small tool `find_deprecated_code.py` to
  scan a directory tree for the deprecated uses. Invoke the program
  with the `--help` option to see a description.

- Upload and download methods now accept a `callback` argument to do
  things during a transfer. Modification time comparisons in
  `upload_if_newer` and `download_if_newer` now consider the timestamp
  precision of the remote file which may lead to some unnecessary
  transfers. These can be avoided by waiting at least a minute between
  calls of `upload_if_newer` (or `download_if_newer`) for the same
  file. See the documentation for details [2].

- The `FTPHost` class got a `keep_alive` method. It should be used
  carefully though, not routinely. Please read the description [3] in
  the documentation.

- Several bugs were fixed [4-7].

- The source code was restructured. The tests are now in a `test`
  subdirectory and are no longer part of the release archive. You can
  still get them via the source repository. Licensing matters have
  been moved to a common `LICENSE` file.

What is ftputil?
----------------

ftputil is a high-level FTP client library for the Python programming
language. ftputil implements a virtual file system for accessing FTP
servers, that is, it can generate file-like objects for remote files.
The library supports many functions similar to those in the os,
os.path and shutil modules. ftputil has convenience functions for
conditional uploads and downloads, and handles FTP clients and servers
in different timezones.

Read the documentation at
http://ftputil.sschwarzer.net/documentation .

License
-------

ftputil is Open Source software, released under the revised BSD
license (see http://www.opensource.org/licenses/bsd-license.php ).

[1] http://codespeak.net/pipermail/ftputil/2009q1/000256.html
[2] http://ftputil.sschwarzer.net/trac/wiki/Documentation#uploading-and-downloading-files
[3] http://ftputil.sschwarzer.net/trac/wiki/Documentation#keep-alive
[4] http://ftputil.sschwarzer.net/trac/ticket/44
[5] http://ftputil.sschwarzer.net/trac/ticket/46
[6] http://ftputil.sschwarzer.net/trac/ticket/47
[7] http://ftputil.sschwarzer.net/trac/ticket/51

Stefan
Matthieu Bizien | 22 Oct 16:55
Picon

Patch to walk

Hi everyone, I discovered ftputil because I wanted to index ftp servers. So I have to list directories with walk. But the current code make too many requests (1 per file). In a directory with thousands of files it is unusable.

To this is a patch which use retrlines("LIST"). It makes only one request per dir.

    def walk(self, top, topdown=True, onerror=None):
        """
        Iterate over directory tree and return a tuple (dirpath,
        dirnames, filenames) on each iteration, like the `os.walk`
        function (see http://docs.python.org/lib/os-file-dir.html ).
        """
        # Hack with retrlines("LIST"). 
       
        # The following code is copied from `os.walk` in Python 2.4
        #  and adapted to ftputil.
        dirs, nondirs = [], []
        def callback(line):
            #example of a line :
            #drwxrwxrwx   2 ftp      nogroup    159744 Oct 22 04:48 Upload
            #their neither is . and ..
            name = line[55:] #Ugly hack, must check that is correct. It work ith proftpd.
            if line.startswith("d"):
                dirs.append(name)
            elif not line.startswith("l"): #No links
                nondirs.append(name)
        try:
            raw = self._session.retrlines("LIST "+ top, callback)
        except ftp_error.FTPOSError:
            if onerror is not None:
                onerror(err)
            return
        if topdown:
            yield top, dirs, nondirs
        for name in dirs:
            path = self.path.join(top, name)
            for item in self.walk(path, topdown, onerror):
                yield item
        if not topdown:
            yield top, dirs, nondirs

Matthieu Bizien

_______________________________________________
ftputil mailing list
ftputil@...
http://codespeak.net/mailman/listinfo/ftputil
Henning Hraban Ramm | 13 Oct 15:51
Favicon
Gravatar

UnicodeDecodeError with walk in current directory

Hello Stefan and all the silent folks,

I run into a strange problem:

* If the current dir is the root of my FTP account
* and I start ftputil's walk (host.walk or host.path.walk)
* with a root of '.',
my script hangs _or_ breaks at the point where it encounters files or  
directories whose names contain non-ASCII chars.
I still don't understand, when I'll get a traceback and when not.

The traceback I get with ftputil 2.4.2 is:

   File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ 
python2.6/site-packages/ftputil/ftp_stat.py", line 549, in  
__call_with_parser_retry
     result = method(*args, **kwargs)
   File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ 
python2.6/site-packages/ftputil/ftp_stat.py", line 421, in _real_listdir
     loop_path = self._path.join(path, stat_result._st_name)
   File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ 
python2.6/posixpath.py", line 70, in join
     path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xdf in position  
41: ordinal not in range(128)

In ftputil 2.5b it is:

   File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ 
python2.6/site-packages/ftputil/ftp_stat.py", line 565, in  
__call_with_parser_retry
     result = method(*args, **kwargs)
   File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ 
python2.6/site-packages/ftputil/ftp_stat.py", line 437, in _real_listdir
     loop_path = self._path.join(path, stat_result._st_name)
   File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ 
python2.6/posixpath.py", line 70, in join
     path += '/' + b

(In this case some file names contain 'ß'.)

With a root of '/', './' or every other absolute or relative path I  
tried, it works (I get everything as byte strings, regardless what  
their encoding might once have been, and that's expected FTP behaviour).

Since the real problem seems to be in posixpath, I don't know if it's  
possible to work around this in ftputil.

Somewhat confused greetlings from Lake Constance!
Hraban
---
http://www.fiee.net
https://www.cacert.org (I'm an assurer)
Stefan Schwarzer | 19 Sep 13:09

[ANN] ftputil 2.5b released

ftputil 2.5b is now available from
http://ftputil.sschwarzer.net/download .

Changes since version 2.4.2
---------------------------

- As announced over a year ago [1], the `xreadlines` method for
  FTP file objects has been removed, and exceptions can no longer be
  accessed via the `ftputil` namespace. Only use `ftp_error` to access
  the exceptions.

  The distribution contains a small tool `find_deprecated_code.py` to
  scan a directory tree for the deprecated uses. Invoke the program
  with the `--help` option to see a description.

- Upload and download methods now accept a `callback` argument to do
  things during a transfer. Modification time comparisons in
  `upload_if_newer` and `download_if_newer` now consider the timestamp
  precision of the remote file which may lead to some unnecessary
  transfers. These can be avoided by waiting at least a minute between
  calls of `upload_if_newer` (or `download_if_newer`) for the same
  file. See the documentation for details [2].

- The `FTPHost` class got a `keep_alive` method. It should be used
  carefully though, not routinely. Please read the description [3] in
  the documentation.

- Several bugs were fixed [4-7].

- The source code was restructured. The tests are now in a `test`
  subdirectory and are no longer part of the release archive. You can
  still get them via the source repository. Licensing matters have
  been moved to a common `LICENSE` file.

What is ftputil?
----------------

ftputil is a high-level FTP client library for the Python programming
language. ftputil implements a virtual file system for accessing FTP
servers, that is, it can generate file-like objects for remote files.
The library supports many functions similar to those in the os,
os.path and shutil modules. ftputil has convenience functions for
conditional uploads and downloads, and handles FTP clients and servers
in different timezones.

Read the documentation at
http://ftputil.sschwarzer.net/documentation .

License
-------

ftputil is Open Source software, released under the revised BSD
license (see http://www.opensource.org/licenses/bsd-license.php ).

[1] http://codespeak.net/pipermail/ftputil/2009q1/000256.html
[2] http://ftputil.sschwarzer.net/trac/wiki/Documentation#uploading-and-downloading-files
[3] http://ftputil.sschwarzer.net/trac/wiki/Documentation#keep-alive
[4] http://ftputil.sschwarzer.net/trac/ticket/44
[5] http://ftputil.sschwarzer.net/trac/ticket/46
[6] http://ftputil.sschwarzer.net/trac/ticket/47
[7] http://ftputil.sschwarzer.net/trac/ticket/51

Stefan

Gmane