Ian Skinner | 1 Oct 01:42 2010

Re: Excluding most and including some problems continue.

>>> Steven Levine <steve53 <at> earthlink.net> Thursday, September 30, 2010 3:22 PM >>>
> It's close, but you need to augment is a bit.  Try

> + das/htdocs/docs/*
> + em/htdocs/docs/*
> etc.
> + */
> - *

Thanks for the suggestion, but that did not seem to produce the desired results.  I did not look into why in
detail, but a dry run produced files from directories I wanted to exclude and apparently not all the files I
wanted to include.

After a day of trail and error and internet searching I now have this that is really close.  It copies all the
directories I earlier identifies that where being falsely excluded.  There are still two or three
individual files that are not copying for some reason.  I am looking into those now.

+ das
+ em
+ enf
+ internal
+ itb
+ medtox
+ pml
+ psb
+ reg
+ whs

+ htdocs
+ docs
(Continue reading)

Kulkarni Shantanu | 1 Oct 02:58 2010

Re: -r required when -a already specified in --files-from?

* Carlos Carvalho <carlos <at> fisica.ufpr.br> [101001 06:16]:
> The man page entry for -a is pretty clear: "The only exception to the
> above equivalence is when --files-from is specified, in which case -r
> is not implied".

Thanks a lot Carlos. I am still not sure how I missed that in the
manual. 

Thanks.
Shantanu
-- 
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Steven Levine | 1 Oct 02:28 2010
Picon
Picon

Re: Excluding most and including some problems continue.

In <4CA4BE08.2858.00AF.0 <at> cdpr.ca.gov>, on 09/30/10
   at 04:42 PM, "Ian Skinner" <iskinner <at> cdpr.ca.gov> said:

Hi,

>> + das/htdocs/docs/*
>> + em/htdocs/docs/*
>> etc.
>> + */
>> - *

>Thanks for the suggestion, but that did not seem to produce the desired
>results.  I did not look into why in detail, but a dry run produced files
>from directories I wanted to exclude and apparently not all the files I
>wanted to include.

Did you add --prune-empty-directorys to the command line?  This filter
setup along with --prune-empty-directories will copy only the files in the
named directores, which is my understanding of what you want.

>After a day of trail and error and internet searching I now have this
>that is really close.

Looks overly complex to me.  Taking your example layout and using this
filter set

+ das/htdocs/docs/*
+ em/htdocs/docs/*
+ enf/htdocs/docs/*
+ internal/htdocs/docs/*
(Continue reading)

Henri Shustak | 1 Oct 12:53 2010
Picon

Re: How to mirror only specified directories

>> I'm still struggling to get just the directory(ies) that I want...
> 
> You may find this <http://tinyurl.com/rsync-exclude-all-include-some> post to the LBackup mailing
list helpful. 
> 
> The example listed (link above) revolves around specifying the root directory as the source and then
specifying a selection of sub-directories to include in the sync and then finally a rule which will
exclude everything else. 
> 
> Please note that each directory within the path must specifically be included for this approach to work.
IN addition, this approach utilizes only an excludes file (no include list via the command line).

Following up with the approach listed above, the excludes file quoted below should provide the results you
are after if modified. Some minor modifications should be all that is required.

The example below (extract from URL above) is going to copy the directories /private and /home. It will
transfer everything recursively within /private/etc/, /private/var/ and /home/

> + private     
> + private/etc/***
> + private/var/***
> + home/***
> - *

I understand you wanted to only transfer the /das/htdocs/docs/ directory and any contents within this
directory recursively with rsync. Provided you are syncing the root directory then the following
excludes file example listed below should make this happen : 

+ das
+ das/htdocs
(Continue reading)

Carney Mimms | 1 Oct 15:20 2010

rsync on AS400-PASE and AIX rsync binary installation

We are currently replicating several of our servers to a remote colocation using rsync 3.0.6 on Mac OS X
10.4, 10.5 and 10.6.  We also have an AS400 system and would like to replicate some of its files, contained in
an IFS file system, to the Mac server running rsync at the colocation facility. The AS400 system is in the
hands of specialists within our organization who are well versed in configuring and maintaining the system.
My online searches have turned up references to a PASE add-on that allows AIX binaries to be installed and
run on an AS400 system. One reference I have found says that rsync needs to be run in daemon mode on the AS400
end but does not provide any details. This reference is from 2002 and I have not found anything more recent.

Can anyone point me to more complete information on installing and running rsync on AS400, including the
use of PASE and the AIX rsync binary?

--Carney Mimms
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Wayne Davison | 1 Oct 16:54 2010
Picon

Re: Excluding most and including some problems continue.

On Thu, Sep 30, 2010 at 8:27 AM, Ian Skinner <iskinner <at> cdpr.ca.gov> wrote:
Unfortunately there are some subdirectories in some of these selected */htdocs/docs* directories that are unintentionally being excluded by these rules.  I.E. */export/home/enf/htdocs/docs/county/internal/*.

It is usually best to anchor your matching terms, unless you want a term to float and match anywhere.  In an .rsync-filter file, terms that start with a slash are anchored in that file's directory.  You can also use a wildcard for all the subdir excusions.  For example:
 
+ /das
+ /em
+ /enf
+ /internal
+ /itb
+ /medtox
+ /pml
+ /psb
+ /reg
+ /whs
- /*

+ /*/htdocs
- /*/*

+ /*/htdocs/docs
- /*/htdocs/*

Another alternative is to sprinkle .rsync-filter files throughout your hierarchy with localized rules for that part of the hierarchy, but the above should do what you want.

..wayne..
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
samba-bugs | 1 Oct 18:27 2010
Picon

DO NOT REPLY [Bug 3653] Reduce the need for the "vanished files" warning

https://bugzilla.samba.org/show_bug.cgi?id=3653

jwz <at> jwz.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jwz <at> jwz.org

------- Comment #17 from jwz <at> jwz.org  2010-10-01 11:27 CST -------
"Me too."

Doing nightly backups from cron is an *extremely* common use-case of rsync, and
telling each and every user of rsync that they should either A) write their own
shell script to filter and discard error messages, or B) play a constant game
of whack-a-mole chasing down new --exclude options to add, is just silly.  It
pushes work downstream that could be done upstream much more efficiently.  Why
make each user write the same code over and over when the developer could write
it just once, and get it right?

Option A is bad because everyone gets to introduce differently-subtle bugs in
their script (wrong error code? accidentally ignoring too many error messages?
who knows. It's fragile.)

Option B is bad because I don't *want* to have a bunch of --exclude options for
my backup: disk and bandwidth are cheap, so I want my backup disk to be an
exact mirror.  That way, when the disk holding / dies, I can just drop the
backup in and boot it, being confident that everything is exactly the same.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Benjamin R. Haskell | 1 Oct 22:28 2010

Re: Excluding most and including some problems continue.

On Fri, 1 Oct 2010, Wayne Davison wrote:

> On Thu, Sep 30, 2010 at 8:27 AM, Ian Skinner wrote:
>
>> Unfortunately there are some subdirectories in some of these selected 
>> */htdocs/docs* directories that are unintentionally being excluded by 
>> these rules.  I.E. */export/home/enf/htdocs/docs/county/internal/*.
>
>
> It is usually best to anchor your matching terms, unless you want a 
> term to float and match anywhere.  In an .rsync-filter file, terms 
> that start with a slash are anchored in that file's directory.  You 
> can also use a wildcard for all the subdir excusions.  For example:
>
>
> + /das
> + /em
> + /enf
> + /internal
> + /itb
> + /medtox
> + /pml
> + /psb
> + /reg
> + /whs
> - /*
>
> + /*/htdocs
> - /*/*
>
> + /*/htdocs/docs
> - /*/htdocs/*
>
> Another alternative is to sprinkle .rsync-filter files throughout your 
> hierarchy with localized rules for that part of the hierarchy, but the 
> above should do what you want.
>

I think I sent a variant of the attached Perl script last time someone 
was asking something similar.  What Wayne suggested is better right now 
(that is: while your patterns are very simple -- just a few root-level 
directories, each of which should include the /htdocs/docs/ subdir). 
But if you start adding more, it could get more annoying to have to 
manually fiddle with the rules.

The attached script takes as input lines of the form:
/rooted/path/to/include

and produces what should work as a filter file. It supports 
'{one,other}' brace-style expansions (but I think that may be 
OS-dependent -- I think it works if your system's 'glob()' function 
supports them).  So, for example, I produced a working filter file for 
your situation from:

{==> input.rsync.rules <==}

/{das,em,enf,internal,itb,medtox,pml,psb,reg,whs}/htdocs/docs

{=========================}

$ perl ./rsync-filter-generate.pl input.rsync.rules
[produces rule file] [1]

$ perl ./rsync-filter-generate.pl input.rsync.rules | rsync --include-from=- /path/to/root/
[shows what would be transferred]

$ perl ./rsync-filter-generate.pl input.rsync.rules | rsync --include-from=- /path/to/root/ /path/to/dest/
[does it]

The general strategy:
$ echo /abc/def/ghi | perl ./rsync-filter-generate.pl
+ /abc               -- first include each path component
+ /abc/def           -- one-at-a-time, for each thing to include
+ /abc/def/ghi
- /abc/def/*         -- then exclude everything else at each
- /abc/*             -- level of the hierarchy
- /*

-- 
Best,
Ben

[1] output for your case:

+ /das
+ /das/htdocs
+ /das/htdocs/docs
+ /das/htdocs/other
+ /em
+ /em/htdocs
+ /em/htdocs/docs
+ /enf
+ /enf/htdocs
+ /enf/htdocs/docs
+ /internal
+ /internal/htdocs
+ /internal/htdocs/docs
+ /itb
+ /itb/htdocs
+ /itb/htdocs/docs
+ /medtox
+ /medtox/htdocs
+ /medtox/htdocs/docs
+ /pml
+ /pml/htdocs
+ /pml/htdocs/docs
+ /psb
+ /psb/htdocs
+ /psb/htdocs/docs
+ /reg
+ /reg/htdocs
+ /reg/htdocs/docs
+ /whs
+ /whs/htdocs
+ /whs/htdocs/docs
- /whs/htdocs/*
- /whs/*
- /reg/htdocs/*
- /reg/*
- /psb/htdocs/*
- /psb/*
- /pml/htdocs/*
- /pml/*
- /medtox/htdocs/*
- /medtox/*
- /itb/htdocs/*
- /itb/*
- /internal/htdocs/*
- /internal/*
- /enf/htdocs/*
- /enf/*
- /em/htdocs/*
- /em/*
- /das/htdocs/*
- /das/*
- /*
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Matt McCutchen | 1 Oct 22:38 2010
Picon

Re: Excluding most and including some problems continue.

On Fri, 2010-10-01 at 16:28 -0400, Benjamin R. Haskell wrote:
> I think I sent a variant of the attached Perl script last time someone 
> was asking something similar.  What Wayne suggested is better right now 
> (that is: while your patterns are very simple -- just a few root-level 
> directories, each of which should include the /htdocs/docs/ subdir). 
> But if you start adding more, it could get more annoying to have to 
> manually fiddle with the rules.
> 
> The attached script takes as input lines of the form:
> /rooted/path/to/include
> 
> and produces what should work as a filter file.

A similar script is distributed with rsync: support/files-to-excludes .

-- 
Matt

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Ian Skinner | 2 Oct 00:12 2010

Re: Excluding most and including some problems continue.

>>> On Friday, October 01, 2010 at 7:54 AM, in message
>> + /das
>> + /em
>> + /enf
>> + /internal
>> + /itb
>> + /medtox
>> + /pml
>> + /psb
>> + /reg
>> + /whs
>> - /*
>>
> + /*/htdocs
> - /*/*
> 
> + /*/htdocs/docs
>> - /*/htdocs/*

Thanks Wayne, this worked well and seems simpler and lazier then my original version.  

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Gmane