request: add TCP buffer options to rsync CLI?
Lawrence D. Dunn <ldunn <at> cisco.com>
2005-11-01 18:03:39 GMT
Dear rsync folks,
I'd like to request/suggest that cli options to set TCP send/receive buffers
be added to rsync client-side.
Summary:
I'm aware that a daemon's config-file can set socket options for
the server side
(e.g. SO_SNDBUF, SO_RCVBUF). That is useful.
But when trying to get high-throughput rsync over
long paths (i.e. large bandwidth*delay product), since the client-side cannot
also set TCP buffers, throughput will be limited.
There are some OS's that are starting to do receive-side buffer-autotuning
(latest Linux, and probably Vista).
But for the rest, I think the most straightforward way to enable
high throughput
would be to also let the client-side make TCP buffer requests.
Request -in-a-nutshell: something like --tcp_sndbuf and --tcp_rcvbuf options
that result in the same setsockopt calls as in the rsync socket.c code
available to rsyncd.conf.
If I've totally missed something, and such functionality is already
available, my apologies (but I'd appreciate a pointer!)
******
More detail:
I was helping resolve a throughput issue between a research network
in France (Renater), and FermiLab in the US (Tridge- Dan Yokum says "hi!").
FermiLab distributes data from the Sloane Digital Sky Survey (SDSS)
using rsync. (cf. www.sdss.org, and
http://www.sdss.org/dr4/access/index.html
for the rsync reference).
Renater was using rsync to pull large amounts of data from FermiLab
across a fast,
long link, and was getting poor throughput (~20mbits/sec).
The core issues turned out to be
1. Too-small TCP send-buffer on the FermiLab server side
(which Dan Y. repaired via rsyncd.conf, and which was later assisted
by installing a recent Linux that allowed send-buffer-autotuning),
*and*
2. Too-small TCP receive-buffer on the client-side.
I couldn't see how to enlarge TCP buffers from the rsync client-side.
By using a web100-enabled client machine (web100.org),
we managed to upsize the TCP receive buffers (and went from 20Mbps
to ~400Mbps).
This by grabbing the running rsync process with a web100 tool,
and manually changing the buffers on-the-fly.
But the process would be a *lot* simpler if the rsync tool could request
TCP buffer allocation from the rsync command line.
The sys-admins would still have to configure large max_tcp_buffers,
but at least the clients could then request large-buffers when needed.
Hopefully someday "all" OS's will incorporate TCP receive-buffer-autotuning.
But in the mean time, I think this would be a significant,
practical addition.
******
Related side-material:
1. For '-e ssh' users, i.e. people that want to get high throughput from
rsync, and encrypt their material, folks should be aware that there
is currently a fixed-limit on SSL-layer buffers. So we get TCP buffers
sized correctly, only to be limited by small-windowing-behavior of
an "upper layer" (SSL).
Chris Rapier (PSC) has addressed this, and is working to get his
fix incorporated
into the SSL libs. In the mean time, patch is available via:
http://www.psc.edu/networking/projects/hpn-ssh/
2. I happen to use rsyncX at home(backups), to handle Apple's HFS+ metadata;
I appreciate all your work on rsync over the years!
Thanks for considering this,
Larry Dunn
ldunn <at> cisco.com
(Manager, Advanced Architecture, Cisco)
--
--
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RSS Feed