E.B. | 19 Dec 00:49 2014
Picon

Duplicity hanging after errors

Hi,
I'm running duplicity from a cron script on two ubuntu 12.04 servers
(installed from apt-get, so duplicity version 0.6.25) and they both
stopped working lately. Not sure if the problems are related or just
coincidence.

Big problem that the error causes duplicity to hang and not exit.
Wondering why that is? If I add "--progress" then it will die (the
trace goes through dummy_backup first where it dies, but
write_multivol just hangs). Is my only solution here to add
"--progress" to my cron script?

But the errors themselves I don't understand and would appreciate
feedback. Is the only fix to wipe the archive completely and start
with new full backup (so I lose any backup histories)?

Error trace below with control-c to cancel at the end (run from cli
but same trace showing in my cron log). Both machines errors appear
during normal diff processing of files on the server (A /some/path...
A /some/path M /some/path.... M /some/path... etc).

REALLY appreciate the software and the help.

First machine:

Traceback (most recent call last):
  File "/usr/local/bin/duplicity", line 1509, in <module>
    with_tempdir(main)
  File "/usr/local/bin/duplicity", line 1503, in with_tempdir
    fn()
(Continue reading)

Philip Jocks | 11 Dec 11:38 2014

GnuPG 2.1

Hi,

we're running duply on a bunch of FreeBSD machines and run into trouble after upgrading GnuPG to 2.1. This is
what "duply thismachine status" looks like:

Start duply v1.7.4, time is 2014-12-11 11:33:53.
Using profile '/root/.duply/thismachine'.
Using installed duplicity version 0.6.24, python 2.7.8, gpg 2.1.0 (Home: ~/.gnupg), awk 'version
20121220 (FreeBSD)', bash '4.3.30(1)-release (amd64-portbld-freebsd10.0)'.
Autoset found secret key of first GPG_KEY entry 'XXXXXXXX' for signing.
Test - Encrypt to XXXXXXXX,YYYYYYYY & Sign with XXXXXXXX (FAILED)

Sorry. A fatal ERROR occured:

Encryption failed (Code 2).
[GNUPG:] BEGIN_SIGNING H2
[GNUPG:] PINENTRY_LAUNCHED 38244
gpg: signing failed: Operation cancelled
[GNUPG:] BEGIN_ENCRYPTION 2 9
gpg: /usr/local/bin/duply: sign+encrypt failed: Operation cancelled

Hint:
  This error means that gpg is probably misconfigured or not working 
  correctly. The error message above should help to solve the problem.
  However, if for some reason duply should misinterpret the situation you 
  can define GPG_TEST='disabled' in the conf file to bypass the test.
  Please do not forget to report the bug in order to resolve the problem
  in future versions of duply.

Before that, it complained about "gpg: WARNING: forcing compression algorithm BZIP2 (3) violates
(Continue reading)

edgar.soldin | 10 Dec 14:25 2014
Picon

Re: Removed .cache dir requires GnuPG passphrase

Andrew,

if possible update to 0.6.25 to make sure this is not something fixed in between.

to simply workaround it. just start a new chain into a new remote target folder and duplicity will have
nothing to sync anymore.

..ede/duply.net

On 10.12.2014 14:03, Andrew Langhorn wrote:
> From what I can see, it has unencrypted *.manifest and *.sigtar.gz files in
> two of the directories under ~/.cache/duplicity, but nothing in the
> remaining two; here's a Gist of the tree:
> https://gist.github.com/ajlanghorn/1f3c38e362a5d72990af
> 
> We're running 0.6.18, which does appear to be a few behind given that the
> current is 0.6.25. Nothing else, that we're aware of, was deleted with the
> ~/.cache directory, but at this stage, I suppose it's difficult to tell
> either way.
> 
> Thanks Ken!
> 
> 
> 
> On 10 December 2014 at 12:55, Kenneth Loafman <kenneth <at> loafman.com> wrote:
> 
>> Duplicity needs the *.manifest and *.sigtar.gz files to be unencrypted in
>> the local cache.  When syncing the remote to the local, the key is needed
>> in order to decrypt those files back into the cache.  When you look in the
>> cache you should see only those kinds of files.
(Continue reading)

Kenneth Loafman | 10 Dec 14:23 2014

Re: Removed .cache dir requires GnuPG passphrase

My first suggestion is an upgrade to 0.6.25.  There have been a lot of fixes since then.

As to getting in sync, you could place the private key on the machine for a short while, let things settle, then remove it, or you could manually decrypt the files from the remote and put them in the cache.

...Ken


On Wed, Dec 10, 2014 at 7:03 AM, Andrew Langhorn <andrew.langhorn <at> digital.cabinet-office.gov.uk> wrote:
From what I can see, it has unencrypted *.manifest and *.sigtar.gz files in two of the directories under ~/.cache/duplicity, but nothing in the remaining two; here's a Gist of the tree: https://gist.github.com/ajlanghorn/1f3c38e362a5d72990af

We're running 0.6.18, which does appear to be a few behind given that the current is 0.6.25. Nothing else, that we're aware of, was deleted with the ~/.cache directory, but at this stage, I suppose it's difficult to tell either way.

Thanks Ken!



On 10 December 2014 at 12:55, Kenneth Loafman <kenneth <at> loafman.com> wrote:
Duplicity needs the *.manifest and *.sigtar.gz files to be unencrypted in the local cache.  When syncing the remote to the local, the key is needed in order to decrypt those files back into the cache.  When you look in the cache you should see only those kinds of files.

It sounds like collection-status may be wrong about the sync of the data.  What version of duplicity are you running?  What else was deleted along with the .cache?  Maybe it was something that supplied the password to gpg (has nothing to do with private key).

...Ken


On Wed, Dec 10, 2014 at 6:10 AM, Andrew Langhorn <andrew.langhorn <at> digital.cabinet-office.gov.uk> wrote:
Hi,

Recently, the contents of the ~/.cache directory of the user which some Duplicity jobs run under were removed. Since then, Duplicity backup jobs have failed to run because the GnuPG passphrase is now required. We don't store the private key on this machine, as we only need it to encrypt with the public key - decryption (if required) takes place elsewhere.

From running 'collection-status', there appear to be some orphaned/incomplete backups on the remote, which I think is a new thing - these don't appear to have been there from before we lost `~/.cache`. 'collection-status' reckons that the local cache has been fully synced from the remote metadata.

I don't want to pass the private key in to Duplicity to get this working, but I don't know how to get over this hurdle.

Help?!

Thanks! :)

--
Andrew Langhorn
Web Operations
Government Digital Service

a: 6th Floor, Aviation House, 125 Kingsway, London, WC2B 6NH

_______________________________________________
Duplicity-talk mailing list
Duplicity-talk <at> nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk



_______________________________________________
Duplicity-talk mailing list
Duplicity-talk <at> nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk




--
Andrew Langhorn
Web Operations
Government Digital Service

a: 6th Floor, Aviation House, 125 Kingsway, London, WC2B 6NH

_______________________________________________
Duplicity-talk mailing list
Duplicity-talk <at> nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk


_______________________________________________
Duplicity-talk mailing list
Duplicity-talk <at> nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Andrew Langhorn | 10 Dec 13:10 2014
Picon

Removed .cache dir requires GnuPG passphrase

Hi,

Recently, the contents of the ~/.cache directory of the user which some Duplicity jobs run under were removed. Since then, Duplicity backup jobs have failed to run because the GnuPG passphrase is now required. We don't store the private key on this machine, as we only need it to encrypt with the public key - decryption (if required) takes place elsewhere.

From running 'collection-status', there appear to be some orphaned/incomplete backups on the remote, which I think is a new thing - these don't appear to have been there from before we lost `~/.cache`. 'collection-status' reckons that the local cache has been fully synced from the remote metadata.

I don't want to pass the private key in to Duplicity to get this working, but I don't know how to get over this hurdle.

Help?!

Thanks! :)

--
Andrew Langhorn
Web Operations
Government Digital Service

t: +44 (0)7810 737375
a: 6th Floor, Aviation House, 125 Kingsway, London, WC2B 6NH
_______________________________________________
Duplicity-talk mailing list
Duplicity-talk <at> nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Yves Goergen | 9 Dec 22:03 2014
Picon

SSH backend and shell escaping

I have a problem with shell escaping now. There's too many quotes 
somewhere and I don't know who doesn't do it right. Here's the essence 
of my backup script. Some of these values come from a shared 
configuration file somewhere else, so they need to be structured like this.

> SFTP_HOST=xxxx.de
> SFTP_USER=xxxx
> SFTP_KEYFILE=/root/.ssh/id_backup
> BACKUP_NAME=test
> KEY_ID=ABCDEF12
>
> COMMON_ARGS="--archive-dir /backup --ssh-options=\"-oIdentityFile=$SFTP_KEYFILE\""
> COMMON_BACKUP_ARGS="$COMMON_ARGS --name $BACKUP_NAME --encrypt-sign-key $KEY_ID"
>
> SOURCE=/tmp
> DEST=sftp://$SFTP_USER <at> $SFTP_HOST/backup-$BACKUP_NAME
>
> export PASSPHRASE=
>
> duplicity $COMMON_BACKUP_ARGS $FULL_EVERY_DAYS "$SOURCE" "$DEST"

The result is this:

BackendException: ssh connection to xxxx <at> xxxx.de:22 failed: [Errno 2] No 
such file or directory: '/root/.ssh/id_backup"'

Note the extra " at the end of the SSH key file name. It's not supposed 
to be there. Is duplicity not parsing shell arguments right? If I 
specify the --ssh-options argument directly with the duplicity call, not 
inside a variable, it works fine. So my setup is basically correct. This 
change where I want that parameter for all different duplicity calls 
just can't be parsed.

I know that in this case I might just leave out the nested quotes. But 
as soon as I need a second SSH option, it's mandatory.

My shell is bash on Ubuntu 14.04. Duplicity 0.6.23.

--

-- 
Yves Goergen
http://unclassified.de
http://dev.unclassified.de
Yves Goergen | 8 Dec 18:08 2014
Picon

Encryption password selection

For duplicity 0.6.23 (Ubuntu 14.04), what are the recommendations for 
selecting an encryption password? I can't find any information about how 
it's even used and what requirements (minimum/maximum length, allowed 
characters) there are.

Also, I've never seen a system encrypting and signing data with a 
password only (no key file involved), and now I've read that duplicity 
also signs the backup volumes and can detect changes to it. Is that true 
or have I misunderstood something? I only know GnuPG with keys, not with 
passwords only.

--

-- 
Yves Goergen
http://unclassified.de
http://dev.unclassified.de
Duplicity Mailing List | 28 Nov 22:38 2014
Picon

RSync over SSH using ssh config?

Using `rsync://server_name_from_ssh_config/` (To backup to the
quote-on-quote 'home' (Chrooted) dir) resutls in:-

`Attempt 1 failed. BackendException: Error running 'rsync -e 'ssh
-oBatchMode=yes' /server_name_from_ssh_config/': returned 23, with output:`.

The correct command would be:-

`rsync -e 'ssh -oBatchMode=yes' server_name_from_ssh_config:`.

I'm almost positive this was working back in 0.6.X, 0.7 is now erroring
out. How am I meant to do it in 0.7? I tried replacing the trailing
forward slash with a colon and removing one of the protocol slashes
(`rsync://` becoming `rsync:/`), but, it ran a slightly modified command:-

`rsync -e 'ssh -oBatchMode=yes' /server_name_from_ssh_config:/`

I then tried to remove the other protocol slash and I got:-

`InvalidBackendURL: missing // - relative paths not supported for scheme
rsync: rsync:server_name_from_ssh_config:`.

Any help? What should I make my destination be?

P.S. a working rsync command, directly from the command line, would be:-

`rsync ${FILE} server_name_from_ssh_config:`
Jon Beyer | 28 Nov 14:46 2014
Picon

S3 costs for PUT requests

Hi there, I'm interested in using Duplicity, but I have a server with
a lot of small files.  Does each file map to a PUT request, or are
multiple files streamlined into a single PUT request?

Thanks in advance!

Cheers,
Jon
Yannick MOLINET | 27 Nov 14:25 2014

ncFTP issue

Hi all,

 

I’m new with duplicity and I try to backup my directory to one of my own ftp server (Pure-FTP).

During upload ncftp abort upload with a timeout error ; but my ftp server work as fine with other ftp client.

After googling this error, I found some thread about issue between ncftp et pure-ftpd.

I see that ncftpd do not have received any update from 2011.

Is it possible to used another ftp client instead of ncftp.

 

Thanks,

Yannick

_______________________________________________
Duplicity-talk mailing list
Duplicity-talk <at> nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
Karl Forner - Quartz Bio | 27 Nov 13:23 2014

get rid of empty incremental changes

Hello,

When performing an incremental backup, if there is no change, duplicity produces 3 files (the manifest, the difftar and the sigtar) that (as far as I understand) are useless.
I mean that if you delete them it does not change a thing, or I missed something.

Is there an easy way not to produce these empty backups, or to remove them ?

Thanks,

Karl Forner
_______________________________________________
Duplicity-talk mailing list
Duplicity-talk <at> nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk

Gmane