Jelle de Jong | 19 May 2013 15:35
Picon

volume with different sign-keys


Hello everyone,

I'm trying to figure out how I can use two used signed keys in one
volume, I encrypted with key CD88F0BC and signed one increment with
E6A3BB95 and signed an other increment with key CD88F0BC.

When I try to restore the full volume I get either "Volume was signed
by key CD88F0BC, not E6A3BB95" or "Volume was signed by key E6A3BB95,
not CD88F0BC"

I'm using duplicity 0.6.20-3 (from testing) under Debian 7.0 (stable).

Kind regards,

Jelle de Jong
Laurent Lavaud | 16 May 2013 13:24
Picon

failed to remove old backup chain

Hello,

I have two backup chain and i want to keep only the last one, i try this command:

duplicity -v 9 remove-all-but-n-full 1 --force ssh://xxx <at> xxx/backup/documents

but it failed with the following errors:

Deleting complete backup chain Fri May  3 01:00:08 2013
sftp rm duplicity-inc.20130501T230007Z.to.20130502T230008Z.manifest.gpg failed: [Errno 2] No such
file (Try 2 of 5) Will retry in 10 seconds.
sftp rm duplicity-inc.20130501T230007Z.to.20130502T230008Z.manifest.gpg failed: [Errno 2] No such
file (Try 3 of 5) Will retry in 10 seconds.
sftp rm duplicity-inc.20130501T230007Z.to.20130502T230008Z.manifest.gpg failed: [Errno 2] No such
file (Try 4 of 5) Will retry in 10 seconds.
sftp rm duplicity-inc.20130501T230007Z.to.20130502T230008Z.manifest.gpg failed: [Errno 2] No such file
Using temporary directory /tmp/duplicity-iGw2GL-tempdir

i have done a cleanup and a collections-status and everything seems ok

i have attached the full log.

thanks in advance for your help
Attachment (duplicity.log): text/x-log, 68 KiB
_______________________________________________
Duplicity-talk mailing list
Duplicity-talk <at> nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
(Continue reading)

Zbynek Michl | 14 May 2013 19:42
Picon

Warning, found signatures but no corresponding backup files

Hi there,

when I make a full backup over sftp, it pass fine. But I have an issue with first incremental backup -- it seems that duplicity can't find the full backed up data.

Here is the full backup output:

$ duplicity -v 9 --ssh-askpass --tempdir /volume1/ <at> tmp /volume1/temp/test/ sftp://admin <at> 192.168.6.98//share/michlz/_synced        
Using archive dir: /volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394
Using backup name: 84b9906c005abdfee5882a72553fa394
Import of duplicity.backends.webdavbackend Succeeded
Import of duplicity.backends.imapbackend Succeeded
Import of duplicity.backends.botobackend Succeeded
Import of duplicity.backends.localbackend Succeeded
Import of duplicity.backends.giobackend Failed: No module named gio
Import of duplicity.backends.ftpbackend Succeeded
Import of duplicity.backends.ftpsbackend Succeeded
Import of duplicity.backends.rsyncbackend Succeeded
Import of duplicity.backends.sshbackend Succeeded
Import of duplicity.backends.cloudfilesbackend Succeeded
Import of duplicity.backends.tahoebackend Succeeded
Import of duplicity.backends.hsibackend Succeeded
Import of duplicity.backends.u1backend Succeeded
Password for 'admin <at> 192.168.6.98':
Main action: inc
================================================================================
duplicity 0.6.14 (June 18, 2011)
Args: /opt/bin/duplicity -v 9 --ssh-askpass --tempdir /volume1/ <at> tmp /volume1/temp/test/ sftp://admin <at> 192.168.6.98//share/michlz/_synced
Linux dspr 2.6.32.12 #2661 Mon Nov 12 23:09:58 CST 2012 armv5tel
/opt/bin/python2.6 2.6.8 (unknown, Apr 12 2012, 13:02:25)
[GCC 4.2.3]
================================================================================
Using temporary directory /volume1/ <at> tmp/duplicity-FfHAD4-tempdir
Registering (mkstemp) temporary file /volume1/ <at> tmp/duplicity-FfHAD4-tempdir/mkstemp-Jde6ZL-1
Temp has 532733779968 available, backup will use approx 34078720.
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=2 admin <at> 192.168.6.98' (attempt #1)
State = sftp, Before = 'admin <at> 192.168.6.98's'
State = sftp, Before = 'Connected to 192.168.6.98.'
sftp command: 'mkdir ""'
State = sftp, Before = 'mkdir ""
Couldn't create directory: Failure'
sftp command: 'cd ""'
State = sftp, Before = 'cd ""'
sftp command: 'mkdir "share"'
State = sftp, Before = 'mkdir "share"
Couldn't create directory: Failure'
sftp command: 'cd "share"'
State = sftp, Before = 'cd "share"'
sftp command: 'mkdir "michlz"'
State = sftp, Before = 'mkdir "michlz"
Couldn't create directory: Failure'
sftp command: 'cd "michlz"'
State = sftp, Before = 'cd "michlz"'
sftp command: 'mkdir "_synced"'
State = sftp, Before = 'mkdir "_synced"
Couldn't create directory: Failure'
sftp command: 'cd "_synced"'
State = sftp, Before = 'cd "_synced"'
sftp command: 'ls -1'
State = sftp, Before = 'ls -1'
State = sftp, Before = 'quit'
Local and Remote metadata are synchronized, no sync needed.
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=2 admin <at> 192.168.6.98' (attempt #1)
State = sftp, Before = 'admin <at> 192.168.6.98's'
State = sftp, Before = 'Connected to 192.168.6.98.'
sftp command: 'mkdir ""'
State = sftp, Before = 'mkdir ""
Couldn't create directory: Failure'
sftp command: 'cd ""'
State = sftp, Before = 'cd ""'
sftp command: 'mkdir "share"'
State = sftp, Before = 'mkdir "share"
Couldn't create directory: Failure'
sftp command: 'cd "share"'
State = sftp, Before = 'cd "share"'
sftp command: 'mkdir "michlz"'
State = sftp, Before = 'mkdir "michlz"
Couldn't create directory: Failure'
sftp command: 'cd "michlz"'
State = sftp, Before = 'cd "michlz"'
sftp command: 'mkdir "_synced"'
State = sftp, Before = 'mkdir "_synced"
Couldn't create directory: Failure'
sftp command: 'cd "_synced"'
State = sftp, Before = 'cd "_synced"'
sftp command: 'ls -1'
State = sftp, Before = 'ls -1'
State = sftp, Before = 'quit'
0 files exist on backend
0 files exist in cache
Extracting backup chains from list of files: []
Last full backup date: none
Collection Status
-----------------
Connecting with backend: SSHBackend
Archive dir: /volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394

Found 0 secondary backup chains.
No backup chains with active signatures found
No orphaned or incomplete backup sets found.
PASSPHRASE variable not set, asking user.
GnuPG passphrase:
PASSPHRASE variable not set, asking user.
Retype passphrase to confirm:
No signatures found, switching to full backup.
Using temporary directory /volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394/duplicity-9yCAmI-tempdir
Registering (mktemp) temporary file /volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394/duplicity-9yCAmI-tempdir/mktemp-kiYjvd-1
Using temporary directory /volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394/duplicity-vjPgcH-tempdir
Registering (mktemp) temporary file /volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394/duplicity-vjPgcH-tempdir/mktemp-86rY8a-1
AsyncScheduler: instantiating at concurrency 0
Registering (mktemp) temporary file /volume1/ <at> tmp/duplicity-FfHAD4-tempdir/mktemp-kjaV0M-2
Selecting /volume1/temp/test
Comparing () and None
Getting delta of (() /volume1/temp/test dir) and None
A .
Selecting /volume1/temp/test/aaa.txt
Comparing ('aaa.txt',) and None
Getting delta of (('aaa.txt',) /volume1/temp/test/aaa.txt reg) and None
A aaa.txt
Removing still remembered temporary file /volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394/duplicity-9yCAmI-tempdir/mktemp-kiYjvd-1
Cleanup of temporary file /volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394/duplicity-9yCAmI-tempdir/mktemp-kiYjvd-1 failed
Removing still remembered temporary file /volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394/duplicity-vjPgcH-tempdir/mktemp-86rY8a-1
Cleanup of temporary file /volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394/duplicity-vjPgcH-tempdir/mktemp-86rY8a-1 failed
AsyncScheduler: running task synchronously (asynchronicity disabled)
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=2 admin <at> 192.168.6.98' (attempt #1)
State = sftp, Before = 'admin <at> 192.168.6.98's'
State = sftp, Before = 'Connected to 192.168.6.98.'
sftp command: 'put "/volume1/ <at> tmp/duplicity-FfHAD4-tempdir/mktemp-kjaV0M-2" "/share/michlz/_synced/.duplicity-full.20130513T205250Z.vol1.difftar.gpg.part"'
State = sftp, Before = 'put "/volume1/ <at> tmp/duplicity-FfHAD4-tempdir/mktemp-kjaV0M-2" "/share/michlz/_synced/.duplicity-full.20130513T205250Z.vol1.difftar.gpg.part"
Uploading /volume1/ <at> tmp/duplicity-FfHAD4-tempdir/mktemp-kjaV0M-2 to /share/michlz/_synced/.duplicity-full.20130513T205250Z.vol1.difftar.gpg.part
/volume1/ <at> tmp/duplicity-FfHAD4-temp 100%  226     0.2KB/s   0.2KB/s   00:00'ETA
sftp command: 'rename "/share/michlz/_synced/.duplicity-full.20130513T205250Z.vol1.difftar.gpg.part" "/share/michlz/_synced/duplicity-full.20130513T205250Z.vol1.difftar.gpg"'
State = sftp, Before = 'rename "/share/michlz/_synced/.duplicity-full.20130513T205250Z.vol1.difftar.gpg.part" "/share/michlz/_synced/duplicity-full.20130513T205250Z.vol1.difftar.gpg"'
State = sftp, Before = 'quit'
Deleting /volume1/ <at> tmp/duplicity-FfHAD4-tempdir/mktemp-kjaV0M-2
Forgetting temporary file /volume1/ <at> tmp/duplicity-FfHAD4-tempdir/mktemp-kjaV0M-2
AsyncScheduler: task completed successfully
Processed volume 1
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=2 admin <at> 192.168.6.98' (attempt #1)
State = sftp, Before = 'admin <at> 192.168.6.98's'
State = sftp, Before = 'Connected to 192.168.6.98.'
sftp command: 'put "/volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394/duplicity-full-signatures.20130513T205250Z.sigtar.gpg" "/share/michlz/_synced/.duplicity-full-signatures.20130513T205250Z.sigtar.gpg.part"'
State = sftp, Before = 'put "/volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394/duplicity-full-signatures.20130513T205250Z.sigtar.gpg" "/share/michlz/_synced/.duplicity-full-signatures.20130513T205250Z.sigtar.gpg.part"
Uploading /volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394/duplicity-full-signatures.20130513T205250Z.sigtar.gpg to /share/michlz/_synced/.duplicity-full-signatures.20130513T205250Z.sigtar.gpg.part
/volume1/homes/admin/.cache/duplici 100%  252     0.3KB/s   0.3KB/s   00:00'ETA
sftp command: 'rename "/share/michlz/_synced/.duplicity-full-signatures.20130513T205250Z.sigtar.gpg.part" "/share/michlz/_synced/duplicity-full-signatures.20130513T205250Z.sigtar.gpg"'
State = sftp, Before = 'rename "/share/michlz/_synced/.duplicity-full-signatures.20130513T205250Z.sigtar.gpg.part" "/share/michlz/_synced/duplicity-full-signatures.20130513T205250Z.sigtar.gpg"'
State = sftp, Before = 'quit'
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=2 admin <at> 192.168.6.98' (attempt #1)
State = sftp, Before = 'admin <at> 192.168.6.98's'
State = sftp, Before = 'Connected to 192.168.6.98.'
sftp command: 'put "/volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394/duplicity-full.20130513T205250Z.manifest.gpg" "/share/michlz/_synced/.duplicity-full.20130513T205250Z.manifest.gpg.part"'
State = sftp, Before = 'put "/volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394/duplicity-full.20130513T205250Z.manifest.gpg" "/share/michlz/_synced/.duplicity-full.20130513T205250Z.manifest.gpg.part"
Uploading /volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394/duplicity-full.20130513T205250Z.manifest.gpg to /share/michlz/_synced/.duplicity-full.20130513T205250Z.manifest.gpg.part
/volume1/homes/admin/.cache/duplici 100%  197     0.2KB/s   0.2KB/s   00:00'ETA
sftp command: 'rename "/share/michlz/_synced/.duplicity-full.20130513T205250Z.manifest.gpg.part" "/share/michlz/_synced/duplicity-full.20130513T205250Z.manifest.gpg"'
State = sftp, Before = 'rename "/share/michlz/_synced/.duplicity-full.20130513T205250Z.manifest.gpg.part" "/share/michlz/_synced/duplicity-full.20130513T205250Z.manifest.gpg"'
State = sftp, Before = 'quit'
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=2 admin <at> 192.168.6.98' (attempt #1)
State = sftp, Before = 'admin <at> 192.168.6.98's'
State = sftp, Before = 'Connected to 192.168.6.98.'
sftp command: 'mkdir ""'
State = sftp, Before = 'mkdir ""
Couldn't create directory: Failure'
sftp command: 'cd ""'
State = sftp, Before = 'cd ""'
sftp command: 'mkdir "share"'
State = sftp, Before = 'mkdir "share"
Couldn't create directory: Failure'
sftp command: 'cd "share"'
State = sftp, Before = 'cd "share"'
sftp command: 'mkdir "michlz"'
State = sftp, Before = 'mkdir "michlz"
Couldn't create directory: Failure'
sftp command: 'cd "michlz"'
State = sftp, Before = 'cd "michlz"'
sftp command: 'mkdir "_synced"'
State = sftp, Before = 'mkdir "_synced"
Couldn't create directory: Failure'
sftp command: 'cd "_synced"'
State = sftp, Before = 'cd "_synced"'
sftp command: 'ls -1'
State = sftp, Before = 'ls -1'
State = sftp, Before = 'quit'
0 files exist on backend
2 files exist in cache
Extracting backup chains from list of files: []
--------------[ Backup Statistics ]--------------
StartTime 1368478382.33 (Mon May 13 22:53:02 2013)
EndTime 1368478382.42 (Mon May 13 22:53:02 2013)
ElapsedTime 0.09 (0.09 seconds)
SourceFiles 2
SourceFileSize 4100 (4.00 KB)
NewFiles 2
NewFileSize 4100 (4.00 KB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 2
RawDeltaSize 4 (4 bytes)
TotalDestinationSizeChange 226 (226 bytes)
Errors 0
-------------------------------------------------

Removing still remembered temporary file /volume1/ <at> tmp/duplicity-FfHAD4-tempdir/mkstemp-Jde6ZL-1




Then I can see on destination machine:

$ ls -la /share/michlz/_synced/
drwxr-xr-x    2 admin    administ     4096 May 13 22:48 ./
drwxrwxrwx    5 admin    administ     4096 Mar 22 11:24 ../
-rw-------    1 admin    administ      252 May 13 22:48 duplicity-full-signatures.20130513T205250Z.sigtar.gpg
-rw-------    1 admin    administ      197 May 13 22:48 duplicity-full.20130513T205250Z.manifest.gpg
-rw-------    1 admin    administ      226 May 13 22:48 duplicity-full.20130513T205250Z.vol1.difftar.gpg



And the first incremental backup:

$ duplicity -v 9 --ssh-askpass --tempdir /volume1/ <at> tmp /volume1/temp/test/ sftp://admin <at> 192.168.6.98//share/michlz/_synced
Using archive dir: /volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394
Using backup name: 84b9906c005abdfee5882a72553fa394
Import of duplicity.backends.webdavbackend Succeeded
Import of duplicity.backends.imapbackend Succeeded
Import of duplicity.backends.botobackend Succeeded
Import of duplicity.backends.localbackend Succeeded
Import of duplicity.backends.giobackend Failed: No module named gio
Import of duplicity.backends.ftpbackend Succeeded
Import of duplicity.backends.ftpsbackend Succeeded
Import of duplicity.backends.rsyncbackend Succeeded
Import of duplicity.backends.sshbackend Succeeded
Import of duplicity.backends.cloudfilesbackend Succeeded
Import of duplicity.backends.tahoebackend Succeeded
Import of duplicity.backends.hsibackend Succeeded
Import of duplicity.backends.u1backend Succeeded
Password for 'admin <at> 192.168.6.98':
Main action: inc
================================================================================
duplicity 0.6.14 (June 18, 2011)
Args: /opt/bin/duplicity -v 9 --ssh-askpass --tempdir /volume1/ <at> tmp /volume1/temp/test/ sftp://admin <at> 192.168.6.98//share/michlz/_synced
Linux dspr 2.6.32.12 #2661 Mon Nov 12 23:09:58 CST 2012 armv5tel
/opt/bin/python2.6 2.6.8 (unknown, Apr 12 2012, 13:02:25)
[GCC 4.2.3]
================================================================================
Using temporary directory /volume1/ <at> tmp/duplicity-Hz70qz-tempdir
Registering (mkstemp) temporary file /volume1/ <at> tmp/duplicity-Hz70qz-tempdir/mkstemp-2FULjM-1
Temp has 532733771776 available, backup will use approx 34078720.
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=2 admin <at> 192.168.6.98' (attempt #1)
State = sftp, Before = 'admin <at> 192.168.6.98's'
State = sftp, Before = 'Connected to 192.168.6.98.'
sftp command: 'mkdir ""'
State = sftp, Before = 'mkdir ""
Couldn't create directory: Failure'
sftp command: 'cd ""'
State = sftp, Before = 'cd ""'
sftp command: 'mkdir "share"'
State = sftp, Before = 'mkdir "share"
Couldn't create directory: Failure'
sftp command: 'cd "share"'
State = sftp, Before = 'cd "share"'
sftp command: 'mkdir "michlz"'
State = sftp, Before = 'mkdir "michlz"
Couldn't create directory: Failure'
sftp command: 'cd "michlz"'
State = sftp, Before = 'cd "michlz"'
sftp command: 'mkdir "_synced"'
State = sftp, Before = 'mkdir "_synced"
Couldn't create directory: Failure'
sftp command: 'cd "_synced"'
State = sftp, Before = 'cd "_synced"'
sftp command: 'ls -1'
State = sftp, Before = 'ls -1'
State = sftp, Before = 'quit'
Synchronizing remote metadata to local cache...
Deleting local /volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394/duplicity-full-signatures.20130513T205250Z.sigtar.gz (not authoritative at backend).
Deleting local /volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394/duplicity-full.20130513T205250Z.manifest (not authoritative at backend).
Running 'sftp  -oServerAliveInterval=15 -oServerAliveCountMax=2 admin <at> 192.168.6.98' (attempt #1)
State = sftp, Before = 'admin <at> 192.168.6.98's'
State = sftp, Before = 'Connected to 192.168.6.98.'
sftp command: 'mkdir ""'
State = sftp, Before = 'mkdir ""
Couldn't create directory: Failure'
sftp command: 'cd ""'
State = sftp, Before = 'cd ""'
sftp command: 'mkdir "share"'
State = sftp, Before = 'mkdir "share"
Couldn't create directory: Failure'
sftp command: 'cd "share"'
State = sftp, Before = 'cd "share"'
sftp command: 'mkdir "michlz"'
State = sftp, Before = 'mkdir "michlz"
Couldn't create directory: Failure'
sftp command: 'cd "michlz"'
State = sftp, Before = 'cd "michlz"'
sftp command: 'mkdir "_synced"'
State = sftp, Before = 'mkdir "_synced"
Couldn't create directory: Failure'
sftp command: 'cd "_synced"'
State = sftp, Before = 'cd "_synced"'
sftp command: 'ls -1'
State = sftp, Before = 'ls -1'
State = sftp, Before = 'quit'
0 files exist on backend
0 files exist in cache
Extracting backup chains from list of files: []
Last full backup date: none
Collection Status
-----------------
Connecting with backend: SSHBackend
Archive dir: /volume1/homes/admin/.cache/duplicity/84b9906c005abdfee5882a72553fa394

Found 0 secondary backup chains.
No backup chains with active signatures found
No orphaned or incomplete backup sets found.
PASSPHRASE variable not set, asking user.
GnuPG passphrase:



What am I doing wrong?

Thanks,
Zbynek


_______________________________________________
Duplicity-talk mailing list
Duplicity-talk <at> nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Joseph D. Wagner | 14 May 2013 01:32

Excessive Swapping Under Minimal Load

I am trying to use duplicity to backup my /home to an external hard 
drive
(usb 2 interface).  After running for a few minutes, my io slows to a 
crawl.
My system reports using about 300-400MB of swap, even though only about
600MB / 4GB is in use.

I am trying to isolate the cause.  It may be duplicity in conjunction 
with
the particulars of my configuration (see below).  I get good 
performance
under normal circumstances.  I am using fedora 18, which has /tmp 
mounted
using tmpfs, but duplicity never uses more than 1.0GB - 1.1GB.

First, is this the correct forum for asking for this kind of help?

Second, what information

The particulars of my configuration:
sda1 -> win7 hidden boot
sda2 -> win7 regular
sda3 -> grub2 boot
sda5 -> luks swap
sda6 -> luks ext4 root
sda7 -> regular ext4 data
sdb1 -> luks userdata1 -> md raid 0 with userdata1 & userdata2-> ext4 
home
sdc1 -> luks userdata2 -> md raid 0 with userdata1 & userdata2-> ext4 
home

My thinking in putting md raid on top of luks was that I would be able 
to
better use multithreading (2 luks, 1 per physical drive), whereas 
putting
luks on top of md raid might create a choke point (1 luks for 1 virtual
drive).  I'll admit that this might be the problem.  However, it works 
fine
under normal circumstances.

I was hoping someone could help me either 1) isolate what is causing 
the
swapping, or 2) tell me what is wrong with my configuration based upon 
their
knowledge of how the kernel works.

Thanks in advance.

Joseph D. Wagner
Jelle de Jong | 13 May 2013 16:55
Picon

ssh port backupninja configuration with duplicty paramiko

Hello everyone,

Where should I configure my ssh port 2227 in the backup ninja config
files in Debian 7.0?

I used to use the /etc/ssh_config but paramiko doesn’t seem to use this.

With the duplicty command i can add :2227 behind the fqdn and it works,
but this doesn’t work for the desthost in backupninja 1.0.1-1

Kind regards,

Jelle de Jong
Davide Setti | 29 Apr 2013 23:43
Picon
Gravatar

lftp doesn't mkdir

Hi all,
i'm using duplicity version 4.3.8 from debian sid, and i noticed that,
using lftp, duplicity is not able to connect to the FTP server to list
files.

My FTP server is accessible using both FTP and FTPS. If i use ftp (and
ncftp) everything works fine, but with FTPS (and lftp) duplicity can't
access the file list because the "ls" command retries forever until:

<--- 450 backup_dir: No such file or directory
ls: Fatal error: max-retries exceeded

I suppose this happens because of the fault tolerancy of lftp: given a
command lftp tries to repeat it until it succeed. In this case the 405
return code is interpreted like a failure.

To replicate this enable debug in lftp.

--

Davide Setti
code: http://github.com/vad
dcottingham00 | 17 Apr 2013 15:44
Picon
Gravatar

suggested feature to help Amazon Glacier users

Here's a feature I'd like to request to help out Amazon Glacier users, that I thought I'd first post here and see if anyone has comments before I put it on launchpad. This is related to my earlier thread, since this feature would provide an easy workaround to that bug.

When using Amazon S3 with a lifecycle transition rule to Glacier, there is really no good reason for transitioning the manifests and sigtars -- they are a negligible fraction of the storage. But the way S3's rules work, and the way duplicity names the files, there is no way to do that. The rules work by a string prefix match, not a regexp match. And duplicity's different file types cannot be distinguished this way.

One way of making this possible would be to insert the file type in the filename before the timestamps. For example, one could (somewhat redundantly) insert "manifest" or "sigtar" or "difftar" right after the "duplicity".

Another approach would be to permit the user to specify one folder for difftars and another for the manifests and sigtars. This is because S3's prefix match is applied to the whole pathname relative to the bucket.

Any comments and suggestions appreciated.

 - Dave Cottingham

_______________________________________________
Duplicity-talk mailing list
Duplicity-talk <at> nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Elvar | 16 Apr 2013 18:05

long term incrementals and scalability question


I am currently using Duplicity to make backups of a fast growing email 
archive solution. I have Duplicity backing the data up via FTP to an 
offsite server. I performed the initial full backup and have been doing 
incrementals since. I'm using 250M volumes to try and cut down on the 
number of files on the remote server. The question I have is, is this a 
viable long term method I'm using? Performing semi routine full backups 
is not an option due to how long they take and the amount of data that 
has to be transferred.

Kind regards,
Elvar
dcottingham00 | 8 Apr 2013 02:58
Picon
Gravatar

Sometimes reads manifest from backup location even though it's in cache -- why?

I see that duplicity will sometimes decide to retrieve a manifest from the remote backup location even though there's a copy in the local cache. Can anyone clarify what circumstances cause this to happen, and why?

Here's why I care. I am backing up to Amazon S3 and using a lifecycle rule to transition everything to Glacier. So, of course, nothing can be retrieved without prior manual intervention. I expected this to work since the local cache has everything duplicity needs, and in fact this worked with nightly backups for over two weeks. But now duplicity fails because it is trying to retrieve the last manifest, even though that file is in the cache.

Thanks for any help,
Dave Cottingham

_______________________________________________
Duplicity-talk mailing list
Duplicity-talk <at> nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Roman | 6 Apr 2013 14:22
Picon

Backup to pool with different hostkeys

Hi everyone,

i was just revisiting duplicity for my server-backups.

I am running a cluster of backup-servers under the same dns 
"fs.hasnoname.de" with different IPs/Hostnames and Hostkeys.

Via ssh/scp on the console this is not a problem with the following 
~/.ssh/config:

Host	fs.hasnoname.de
	user	xyz
	CheckHostIP no

But running duplicity as the same user:

~> duplicity /etc scp://fs.hasnoname.de:22/backup

Produces this error about the different Hostkey. Shouldn't duplicity 
take the settings from ~/.ssh/config? Or where else can i change this 
behaviour of duplicity?

Benutze temporäres Verzeichnis /tmp/duplicity-Gd8Pr_-tempdir
Backend Fehler Detail: Traceback (most recent call last):
   File "/usr/bin/duplicity", line 1411, in <module>
     with_tempdir(main)
   File "/usr/bin/duplicity", line 1404, in with_tempdir
     fn()
   File "/usr/bin/duplicity", line 1255, in main
     action = commandline.ProcessCommandLine(sys.argv[1:])
   File "/usr/lib/python2.7/dist-packages/duplicity/commandline.py", 
line 1007, in ProcessCommandLine
     backup, local_pathname = set_backend(args[0], args[1])
   File "/usr/lib/python2.7/dist-packages/duplicity/commandline.py", 
line 900, in set_backend
     globals.backend = backend.get_backend(bend)
   File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 
158, in get_backend
     return _backends[pu.scheme](pu)
   File 
"/usr/lib/python2.7/dist-packages/duplicity/backends/sshbackend.py", 
line 115, in __init__
     raise BackendException("ssh connection to %s:%d failed: %s" % 
(parsed_url.hostname,portnumber,e))
BackendException: ssh connection to fs.hasnoname.de:22 failed: Host key 
for server fs.hasnoname.de does not match!

BackendException: ssh connection to fs.hasnoname:22 failed: Host key for 
server fs.domain.de does not match!

thx in advance!

greetings,
Roman
-- 
Arnd Grossmann | 4 Apr 2013 18:11

Newbie, trying to synchronise encrypted data

Hello All,

I am a system administrator in Berlin, Germany and 
working in the educational sector.

I am trying to use duplicity to synchronise files 
between different computers and a central server all 
running linux.

This is not actually what I understand duplicity was 
made for but it is the only way I could figure out to 
keep my files encrypted on transport and on the 
central server.

My problem is, that duplicity will not restore the 
files to an existing folder. If I try to use the 
force-option it will restore but will not do an 
incremental restore.

So I can incementally backup my files to the server from 
one or more machines but cannot restore to any of 
these without downloading all the files again.

Do you know a way to make duplicity restore 
incrementally to an existing folder?

Thank you for any advice. I read the man page and I will 
read it again an apologise a hundred times if it is in 
there somewhere.

Beste Regards
Arnd

I use the following commend:

duplicity --allow-source-mismatch --tempdir /media/some_disk --encrypt-key MYKEY MYDIR scp://USER:PASSWORD <at> MY.SERVER.de/MYDIR

btw: At the moment I am just gpg-encrypting a gzipped 
tarball which I have to transmit as a whole everytime 
something changed.
This is worse for sure. So duplicity does make things 
better even if it works only one way.
--

-- 

Gmane