Yannick MOLINET | 27 Nov 14:25 2014

ncFTP issue

Hi all,

 

I’m new with duplicity and I try to backup my directory to one of my own ftp server (Pure-FTP).

During upload ncftp abort upload with a timeout error ; but my ftp server work as fine with other ftp client.

After googling this error, I found some thread about issue between ncftp et pure-ftpd.

I see that ncftpd do not have received any update from 2011.

Is it possible to used another ftp client instead of ncftp.

 

Thanks,

Yannick

_______________________________________________
Duplicity-talk mailing list
Duplicity-talk <at> nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Karl Forner - Quartz Bio | 27 Nov 13:23 2014

get rid of empty incremental changes

Hello,

When performing an incremental backup, if there is no change, duplicity produces 3 files (the manifest, the difftar and the sigtar) that (as far as I understand) are useless.
I mean that if you delete them it does not change a thing, or I missed something.

Is there an easy way not to produce these empty backups, or to remove them ?

Thanks,

Karl Forner
_______________________________________________
Duplicity-talk mailing list
Duplicity-talk <at> nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Fiorenza Meini | 27 Nov 11:31 2014
Picon

Webdav backend

Hi there,
I'm trying to have duplicity working with webdav backend.

Has anyone an example how should be the right syntax ?

I read man page, but I cannot understand.

Regards

Fiorenza Meini
Remy van Elst | 21 Nov 08:02 2014
Picon

par2 + encryption

Hello,

When I have a gpg encrypted backup and want to add par2 to the backend
using Duplicity 7, when is the par2 done? Before or after the encryption?
Attachment (0x1B7F88DC.asc): application/pgp-keys, 2452 bytes
_______________________________________________
Duplicity-talk mailing list
Duplicity-talk <at> nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Cláudio Gil | 20 Nov 23:55 2014
Picon

Finding why changes are detected

Hi,

I'm trying to setup a backup from a locally CIFS mount into S3 but I'm struggling with a strange behavior. After the first full backup, the next incremental backup detects several deleted files. But the files have not changed and every "deleted" file is still there. The original situation where I detected something strange was happening was when, after a full-backup of 250GB, an incremental backup uploaded ~100GB, again without changes in the files.

I've tried "-v debug" and all it shows are the "Comparing None and <path>" and paths of the deleted items. How can I pinpoint the cause? Is the an hidden "-v tell-me-what-i-need-to-know"?

It does not seem a bug (as in "Google does not show me that it is a known issue"), so is there something strange with with my setup? My theory is that this happens for full backups that are interrupted (mount becomes inaccessible) and restarted later. Because when I try with smaller quick-to-backup directories the incremental always works as expected.

All this his happened with version 0.6.18. I've upgraded to version 0.6.24 and see the same behavior, although I've reused the full-backup that was made by the previous version. All this with python 2.7.3 on a raspbian (debian wheezy).

Cheers,
Cláudio

_______________________________________________
Duplicity-talk mailing list
Duplicity-talk <at> nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Fiorenza Meini | 18 Nov 10:14 2014
Picon

SSL Error

Hi there,
I'm trying to configuring duplicity to save backup to a swift container, 
but I'm experiencing some problem with certificates.

This is the output:
Connection failed: SSLError [Errno 1] _ssl.c:510: error:14090086:SSL 
routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

The program's version is 0.6.23.
The environment variables are these one:

export SWIFT_USERNAME="****:****"
export SWIFT_PASSWORD="*******"
export SWIFT_AUTHURL="http://arwen.bi.esseweb.intra/auth/v1.0"
export SWIFT_AUTHVERSION="1"

Any suggestion, please ?

Regards

Fiorenza Meini
Spazio Web srl
Marc Evans | 17 Nov 13:11 2014

best practices for handling hard links

Hello,

I am trying to determine what the current state of hard link handling is
for duplicity, and then assuming that my belief is correct, that hard
links are not preserved by duplicity, I would like to understand what
current best practices are?

Background: I have about 26TB of raw data that I am backing up to the
cloud via duplicity. Once it is encrypted, etc, it consumes about 48TB
in the cloud, which includes 1 full backup plus daily incrementals
spanning a 1 month period. With the data is considerable files that are
highly compressed as well as thousands of hard-link files.
Experimentation is finding that the hard-link files are getting stored
multiple times, and further when restored the hard links are not preserved.

Based on my reading of the mailing list archives my observations seem to
be confirmed, though that is in years-old threads. I see in the code
various pieces that are hard link oriented though. I also see discussion
of special casing hard link handling at duplicity invocation, such that
excludes are used to insure that only one copy is actually backed up and
a hard link manifest is generated that can be used by scripts to restore
hard links.

Given the above, what is the state of hard link handling and what are
current best practices for dealing with them?

Thanks in advance - Marc
Thiago Coutinho | 14 Nov 19:19 2014
Picon

Recover backup after using PhotoRec

Hi all.

I've partly formatted a disc where my duplicity files was stored. I have used PhotoRec to recover the files but now they're all unsorted and without the correct filename. I have the files separated by extension. Is there a way to discover the duplicity files and recover my backup?

--
Thiago Coutinho

"O povo não deveria temer o governo. O governo é quem deveria temer o povo."
V de Vingança
_______________________________________________
Duplicity-talk mailing list
Duplicity-talk <at> nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Yves Goergen | 1 Nov 23:24 2014
Picon

FTP upload failed

Hello,

Today I got the error message that the backend FTP upload has failed for 
25 times and it was giving up. Do I have to do something now or will it 
upload any missing pieces later? How is an upload failure handled by 
duplicity?

The file names look like it's been an incremental backup run. I could 
now run a full backup if it's necessary, but I don't know whether it is.

Here's a part of the output:

> Lost data connection to remote host: Broken pipe.
> ncftpput backup-all/duplicity-inc.20141101T082808Z.to.20141101T162809Z.vol2.difftar.gpg:
socket write error.
> Running 'ncftpput -f /backup/tmp/duplicity-__ksg6-tempdir/mkstemp-RoqnL4-1 -F -t 30 -o
useCLNT=0,useHELP_SITE=0  -m -V -C '/backup/tmp/duplicity-__ksg6-tempdir/mktemp-QDNgMO-5'
'backup-all/duplicity-inc.20141101T082808Z.to.20141101T162809Z.vol2.difftar.gpg'' failed
(attempt #1)
> Lost data connection to remote host: Broken pipe.
> ncftpput backup-all/duplicity-inc.20141101T082808Z.to.20141101T162809Z.vol2.difftar.gpg:
socket write error.
> Running 'ncftpput -f /backup/tmp/duplicity-__ksg6-tempdir/mkstemp-RoqnL4-1 -F -t 30 -o
useCLNT=0,useHELP_SITE=0  -m -V -C '/backup/tmp/duplicity-__ksg6-tempdir/mktemp-QDNgMO-5'
'backup-all/duplicity-inc.20141101T082808Z.to.20141101T162809Z.vol2.difftar.gpg'' failed
(attempt #2)
> (...)
> Lost data connection to remote host: Broken pipe.
> ncftpput backup-all/duplicity-inc.20141101T082808Z.to.20141101T162809Z.vol2.difftar.gpg:
socket write error.
> Running 'ncftpput -f /backup/tmp/duplicity-__ksg6-tempdir/mkstemp-RoqnL4-1 -F -t 30 -o
useCLNT=0,useHELP_SITE=0  -m -V -C '/backup/tmp/duplicity-__ksg6-tempdir/mktemp-QDNgMO-5'
'backup-all/duplicity-inc.20141101T082808Z.to.20141101T162809Z.vol2.difftar.gpg'' failed
(attempt #25)
> Giving up trying to execute 'ncftpput -f /backup/tmp/duplicity-__ksg6-tempdir/mkstemp-RoqnL4-1 -F
-t 30 -o useCLNT=0,useHELP_SITE=0  -m -V -C
'/backup/tmp/duplicity-__ksg6-tempdir/mktemp-QDNgMO-5'
'backup-all/duplicity-inc.20141101T082808Z.to.20141101T162809Z.vol2.difftar.gpg'' after 25 attempts
> BackendException: Error running 'ncftpput -f
/backup/tmp/duplicity-__ksg6-tempdir/mkstemp-RoqnL4-1 -F -t 30 -o useCLNT=0,useHELP_SITE=0  -m -V
-C '/backup/tmp/duplicity-__ksg6-tempdir/mktemp-QDNgMO-5' 'backup-all/duplicity-inc.20141101T082808Z.to.20141101T162809Z.vol2.difftar.gpg''

My duplicity version is still old. Did that behaviour change in recent 
versions?

--

-- 
Yves Goergen
http://unclassified.de
http://dev.unclassified.de
edgar.soldin | 30 Oct 19:11 2014
Picon

duplicity 0.7 and prefixes / cloudfiles, s3 testing

hi All,

just uploaded a branch starting to simplify how backends are selected.
 https://code.launchpad.net/~ed.so/duplicity/lftp.ncftp.and.prefixes/+merge/240149

Changelog:
- retire --ssh-backend, --use-scp parameters
- introduce scheme prefixes for alternative backend selection e.g. ncftp+ftp://, see manpage
- scp is now selected via scheme e.g. scp://
- added lftp fish, webdav(s), sftp support

as we have some more backends that use private switches (--s3-use-multiprocessing, --cf-backend ...)
that i unfortunately can't test i hereby ask you to step forward and volunteer to beta test my upcoming
changes for these backends.
alternatively i would be grateful for hint's on how to test these backends.

Mike: will there be tgz snapshots for people not on Ubuntu/Debian or something similar that does not
require testers to check out via bazaar?

thanks.. ede/duply.net
Russell Clemings | 30 Oct 00:51 2014
Picon

Upgrade/rollback difficulties

I've been using duplicity-0.6.08b (Python 2.4.3, boto-1.9b, GnuPG 1.4.5) successfully for quite some time, but decided it was time to upgrade. (Famous last words.)

I installed duplicity-0.6.25 and boto-2.1.1 and after resolving some new dependencies, I got the following when I tried to run a backup:

---------------------------------------------

root <at> server1 [/path/to/scripts/s3backup]# ./backup.sh
Import of duplicity.backends.cloudfilesbackend Failed: the scheme cf+http already has a backend associated with it
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1239, in ?
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1232, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1118, in main
    action = commandline.ProcessCommandLine(sys.argv[1:])
  File "/usr/lib/python2.4/site-packages/duplicity/commandline.py", line 1062, in ProcessCommandLine
    backup, local_pathname = set_backend(args[0], args[1])
  File "/usr/lib/python2.4/site-packages/duplicity/commandline.py", line 955, in set_backend
    globals.backend = backend.get_backend(bend)
  File "/usr/lib/python2.4/site-packages/duplicity/backend.py", line 163, in get_backend
    return _backends[pu.scheme](pu)
  File "/usr/lib/python2.4/site-packages/duplicity/backends/_boto_single.py", line 163, in __init__
    self.resetConnection()
  File "/usr/lib/python2.4/site-packages/duplicity/backends/_boto_single.py", line 185, in resetConnection
    self.conn = get_connection(self.scheme, self.parsed_url, self.storage_uri)
  File "/usr/lib/python2.4/site-packages/duplicity/backends/_boto_single.py", line 38, in get_connection
    from boto.s3.connection import S3Connection
  File "/usr/lib/python2.4/site-packages/boto-2.1.1-py2.4.egg/boto/s3/connection.py", line 292
    delimiter = '?' if '?' not in auth_path else '&'
                     ^
SyntaxError: invalid syntax

---------------------------------------------

So I cleaned out all of the new duplicity and boto files and reinstalled the old versions.

It started up just fine but then quit as follows:

---------------------------------------------

root <at> server1 [/path/to/scripts/s3backup]# ./backup.sh
Synchronizing remote metadata to local cache...
Copying duplicity-full-signatures.20141025T050704Z.sigtar to local cache.
Copying duplicity-full.20140928T050703Z.manifest to local cache.
Copying duplicity-full.20141025T050704Z.manifest to local cache.
Copying duplicity-inc.20140928T050703Z.to.20140929T040702Z.manifest to local cache.

(and so on ... snipped a long list of files)

Copying duplicity-inc.20141027T040702Z.to.20141028T040704Z.manifest to local cache.
Copying duplicity-new-signatures.20141025T050704Z.to.20141026T040703Z.sigtar to local cache.
Copying duplicity-new-signatures.20141026T040703Z.to.20141027T040702Z.sigtar to local cache.
Copying duplicity-new-signatures.20141027T040702Z.to.20141028T040704Z.sigtar to local cache.
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1239, in ?
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1232, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1137, in main
    globals.archive_dir).set_values()
  File "/usr/lib/python2.4/site-packages/duplicity/collections.py", line 679, in set_values
    (backup_chains, self.orphaned_backup_sets,
  File "/usr/lib/python2.4/site-packages/duplicity/collections.py", line 804, in get_backup_chains
    map(add_to_sets, filename_list)
  File "/usr/lib/python2.4/site-packages/duplicity/collections.py", line 794, in add_to_sets
    if set.add_filename(filename):
  File "/usr/lib/python2.4/site-packages/duplicity/collections.py", line 89, in add_filename
    self.set_manifest(filename)
  File "/usr/lib/python2.4/site-packages/duplicity/collections.py", line 117, in set_manifest
    assert not self.remote_manifest_name, (self.remote_manifest_name,
AssertionError: ('duplicity-full.20140928T050703Z.manifest.gpg', 'duplicity-full.20140928T050703Z.manifest.gpg')

---------------------------------------------

I'd appreciate some guidance on resolving the second error so that I can get my backups going again.

After that, I may try the upgrade again, so any advice on the first error would be useful as well.

rac


_______________________________________________
Duplicity-talk mailing list
Duplicity-talk <at> nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk

Gmane