Fabian Cenedese | 25 May 12:00 2016
Picon

Combine exclude files

Hi

I'd like to have a list of excluded items that I can always use
and combine them with job-specific excludes (or generally filters).

Is it possible to use --exclude-from more than once? Do I have
to combine the exclude files into one and give this to rsync?
Is there another possibility (that also works on Windows)?

Thanks

bye  Fabi

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

rsync@contenidosonline.xyz | 15 May 15:04 2016

--inplace option seems sending whole file


Hi

I'm having issues sendig a lot of tar.gz backup files to a ZFS remote filesystem server.

This files are compressed with the --rsyncable option.

Sending without --inplace option rsync works well and send only the differences, but to create a temporary file and rewrite the file destination, zfs snapshots contain the full size of the backup, not only differences block.

I've tried with the --inplace option but seems not workig, send whole file, not deltas.

I'm missing something ?

This in my rsync setup:


/usr/local/bin/rsync -avzrh --inplace  --delete -e 'ssh -p xxxx' /backup/ backup <at> xxx.xxxx.xx:/home/backup/server

Thanks in advance


roberto

 
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Steve Schow | 11 May 17:56 2016

iconv problem with rsync

I am having a problem using rsync 3.1.2 to backup a remote linux machine to local OSX machine.  It works fine, but one large directory full of files has a lot of files with foreign characters in the filenames and this blows up rsync.  

I tried to use:

    —iconv=utf-8-mac,utf-8  

But still it gets to certain files…and I can’t even see which file is causing the problem, because rsync blows up before reporting the filename…but I get the dreaded

error in rsync protocol data stream (code 12) at io.c(226)

does anyone have any suggestions for how I can get rsync to backup all files from remote linux to local mac….  Its possible that there is something else funky about the unknown file causing the problem…I would think utf-8 on the remote host ought to cover it…but maybe not?  I have the locale set to en_us.UTF-8 on the remote linux box.  

When I exclude the large dir full of files…  500GB or so…then everything else syncs fine…

If I don’t use —iconv, then it ends up getting stuck in a strange loop where it complains about files vanishing and keeps retrying in an endless loop.

Any suggestions for how I can fix this would be appreciated.  Here is the complete rsync command line I am using:

/usr/local/bin/rsync -av --delete --numeric-ids --relative \
    --delete-excluded --iconv=utf-8-mac,utf-8 \
    --rsync-path=/usr/bin/rsync \
    --rsh=/usr/bin/ssh -i /etc/rsnapshot/rsnapshot-key root <at> 192.168.1.80:/ \
    /Volumes/backup/raid_snapshots/daily.0/myhost/ 



--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Fabian Cenedese | 11 May 08:30 2016
Picon

Synch group with different gid

Hello

I have a problem while synching between two NAS boxes with Linux.
First NAS: group staff=50 (3.0.3pre1)
Second NAS: groups ftp=50, staff=500 (rsync 3.1.1)

I synched with -rptgo, without --numeric-ids. The expected result
was that the items on the second group would be staff as well
even with a different gid. However it looks like the 50 was taken
literally and now all staff items are ftp items.

1. How could this happen? Is this expected, did I do something wrong?
2. How can I now synch the ownership correctly?

I tried with -a, with --super, but so far the ownership is not synched.

Thanks

bye  Fabi

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

samba-bugs | 8 May 04:10 2016
Picon

[Bug 11909] New: Doc patch for INCLUDE/EXCLUDE PATTERN RULES

https://bugzilla.samba.org/show_bug.cgi?id=11909

            Bug ID: 11909
           Summary: Doc patch for INCLUDE/EXCLUDE PATTERN RULES
           Product: rsync
           Version: 3.1.2
          Hardware: All
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P5
         Component: core
          Assignee: wayned <at> samba.org
          Reporter: kop <at> meme.com
        QA Contact: rsync-qa <at> samba.org

Created attachment 12087
  --> https://bugzilla.samba.org/attachment.cgi?id=12087&action=edit
Patch to rsync.yo

Doc patch (v2) as sent to mailing list vis INCLUDE/EXCLUDE PATTERN RULES
and recursive (-r) copy.

Seems like a good idea to put it in bugzilla so it does not get lost.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

samba-bugs | 4 May 11:04 2016
Picon

[Bug 11893] New: rsync should check local file access permission before connecting to remote end

https://bugzilla.samba.org/show_bug.cgi?id=11893

            Bug ID: 11893
           Summary: rsync should check local file access permission before
                    connecting to remote end
           Product: rsync
           Version: 3.1.2
          Hardware: All
                OS: All
            Status: NEW
          Severity: minor
          Priority: P5
         Component: core
          Assignee: wayned <at> samba.org
          Reporter: rami.lehti <at> gmail.com
        QA Contact: rsync-qa <at> samba.org

I'm using rsync to transfer large OS image files around and sometimes the local
permissions change so that the sender does not have access to the source file
any more.

Having previously transferred the file there is already a file of matching size
on the target host. Now attempting the transfer again rsync connects to the
remote end and the remote end begins to do its checks before the transfer
proper. Because the image file is large this will take a long time (10
minutes). 

After this rsync will report the "Permission denied" error for the local file. 

It would be more user-friendly to check the local file access permissions
before doing anything time consuming and exit with an error as soon as
possible.

Please consider changing this behaviour.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Christoph Biedl | 3 May 22:07 2016
Picon
Picon

Yet another filter question

Hello,

Since the very first day I've been using rsync - some 15 years ago -
the filtering rules caused great grieve. Their behaviour is just not
the way I'd expect it be be and as I read the manpage. Usually I end
up with some hand-written recipes, carefully documented,y including all
the gotchas.

This time however I failed and I see no other way than to ask for
advice.

Given the following structure

        project/
            .rsync-filter
            project.git/
                .git/

Now, the following command (rsync 3.1.1)

    rsync -av --del --deleted-excluded -F \
        /path/to/project/ /path/to/backup/project/

should transfer

- sync: project/project.git/.git/
- skip: Everything else in project/project.git/
- sync: Everything else in project/

In other words: Don't transfer the git repo except for .git/ itself.
Yes, this means files not checked in are lost, that's by intention.

Now, how is .rsync-filter supposed to look? The first of many failing
attempts was

    + *.git/.git/*
    - *.git/

This however kills the entire project.git/ directory, in violation of
"the first matching pattern is acted on". Given previous bad
experiences I've tried (using a script) all 128 combination of

- line order
- '+' or '-' at the start of any line
- '*' appended to any line, or not, with without trailing slash

but no avail.

Any clue?

    Christoph

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

samba-bugs | 28 Apr 17:52 2016
Picon

[Bug 11879] New: escape rrsync restricted folder

https://bugzilla.samba.org/show_bug.cgi?id=11879

            Bug ID: 11879
           Summary: escape rrsync restricted folder
           Product: rsync
           Version: 3.1.2
          Hardware: All
                OS: All
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
          Assignee: wayned <at> samba.org
          Reporter: fb102email-sambabugzilla <at> yahoo.fr
        QA Contact: rsync-qa <at> samba.org

It is possible to escape rrsync restricted folder by syncing (using rsync -a
...) a symbolic link to the parent folder and then syncing with this symbolic
link.

Concretely, we could do:

ln -s .. parent
rsync -acrvz . login <at> server:

and then we can rsync with login <at> server:parent to read/write files in the
parent folder of the restricted folder.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

samba-bugs | 21 Apr 08:54 2016
Picon

[Bug 11866] New: rsync fails (failed to re-stat) when using double fuzzy + link-dest on renamed files

https://bugzilla.samba.org/show_bug.cgi?id=11866

            Bug ID: 11866
           Summary: rsync fails (failed to re-stat) when using double
                    fuzzy + link-dest on renamed files
           Product: rsync
           Version: 3.1.1
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
          Assignee: wayned <at> samba.org
          Reporter: mj <at> doze.net
        QA Contact: rsync-qa <at> samba.org

When using rsync with --fuzzy --fuzzy (double fuzzy) and --link-dest, rsync
fails if a file was renamed to a new name on the source that does not exist in
the link-dest.  As an example:

mj <at> backup-server:~/foo$ rm -rf .sync && mkdir .sync && rsync -azSHAXxrsyy
--ignore-existing --fake-super --link-dest=/home/mj/foo/current
--files-from=:/home/mj/files-to-backup root <at> backup-client:/ .sync
rsync: failed to re-stat "/home/mj/foo/.sync/etc/motd.old": No such file or
directory (2)
rsync: failed to re-stat "/home/mj/foo/.sync/var/log/alternatives.log.1": No
such file or directory (2)
rsync: failed to re-stat "/home/mj/foo/.sync/var/log/auth.log.3.gz": No such
file or directory (2)
rsync: failed to re-stat "/home/mj/foo/.sync/var/log/daemon.log.3.gz": No such
file or directory (2)
rsync: failed to re-stat "/home/mj/foo/.sync/var/log/dpkg.log.1": No such file
or directory (2)
rsync: failed to re-stat "/home/mj/foo/.sync/var/log/messages.3.gz": No such
file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code
23) at main.c(1655) [generator=3.1.1]

In each case above, the files which rsync failed to re-stat were ones that
existed in the link-dest (/home/mj/foo/current) but under a different name. 
For instance:

mj <at> backup-server:~/foo$ ls -al /home/mj/foo/current/etc/motd*
-rw-r--r-- 7 mj mj 286 Dec 24  2014 /home/mj/foo/current/etc/motd

root <at> backup-client:/etc# ls -al motd*
-rw-r--r-- 1 root root 314 Apr 21 06:32 motd
-rw-r--r-- 1 root root 286 Apr 21 06:33 motd.bak
-rw-r--r-- 1 root root 286 Dec 24  2014 motd.old

Note that, in this case, rsync didn't fail on motd.bak, just motd.old.  Both
contain the same content, but only motd.old has a matching timestamp.

I can completely reproduce this problem on multiple machine pairs and with
different file sets.  The problem is resolved by removing the double-fuzzy
configuration but, of course, that impairs performance.

All rsync binaries involved are version 3.1.1 (debian 3.1.1-3_amd64).

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Jethro Tull | 16 Apr 14:29 2016
Picon

addind imap support to rsync

when seeking for syncing two imap maildir folders one has the choice
among many programs such as imapsync, isync, offlineimap ... But all of
these provide individually only a few options.

For example syncing a folder by only appending unexisting files, i.e.
without deleting anything seem to be supported by no one real imap
syncing software stable release. Here "real syncing" means a process
that first collects the data it needs about the files in the destination
folder and proceeds with syncing with the requested options.

For example isync does indeed provide an option for only appending
unexisting files in the destination folder but despite its name it does
not seem to care about the content of the destination folder. The first
time it runs it copies all the emails from the source folder and writes
in a text file what it did to avoid duplicates for the next times it
will run. For syncing two folders that initially contain common stuff
this would eventually result in duplicate emails in the destination
folder.


Moreover each of these programs have their own configuration style that
takes a lot of time to learn. Regarding the many features rsync provides
for syncing files and its ease of use, adding to it the ability to sync
imap maildir folders will probably make of it a definitive choice for
this purpose.

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Chris Deigan | 14 Apr 07:30 2016
Picon
Gravatar

"Total File Size" Statistic counts each instance of hard linked files

Hi,

This is a question is seeking clarification of intended behaviour. Right
now, rsync reports a statistic of "Total file size". This represents "the
total sum of all file sizes in the transfer" (as described in the man page).

A case I've hit in using this statistic is that it counts each instance of a
file even when it has multiple hard links. We are using --hard-links to
preserve hard links on the destination.

As a result we get a statistic of, for instance, 2TB when the actual sum on
disk (counted with du, using the default behaviour of counting hard linked
files only once) is only around 80GB.

I'm using the statistic for generating backup disk usage numbers that
eventually become billing data, so this has generated a few surprise cases.

There are a few alternatives for my use-case, but I was wondering if
counting hard links multiple times is actually correct behaviour?

My feeling is no, but this consideration isn't apparent in the source or
docs that I've read. Appreciate any comments, particularly from the project
maintainers.

Thanks,
Chris

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Gmane