Implementation of xattr in sshfs and openssh

Hi,

in the attachment you will find a patch that implements xattr
functionality in sshfs. A corresponding patch to the openssh sftp-server
is filed to the openssh bugzilla, see
https://bugzilla.mindrot.org/show_bug.cgi?id=1953 .

I hope that you are interested in my patch.

Thanks, Eberhard

PS: I hope that the mail attachment can be handled by the mailing list
system. If not, please tell me how to send the patch.

------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------

Re: Fwd: Bug in ID mapping if permissions shall be preserved?

On Fri, Nov 25, 2011 at 07:21:55PM +0100, Miklos Szeredi wrote:
> On Thu, Nov 17, 2011 at 11:42 AM, Vivenzio Pagliari
> <vivenzio.pagliari <at> gmx.de> wrote:
> >
> > IMO, the "preserve ownership" shall preserve the ownership in a
> > "logic" way: As remote files of ruser are seen with uid of luser,
> > uid (and gid) shall be mapped also when trying to "preserve
> > ownership".
> 
> Does the attached patch help?
> 

Hi,

yes, the patch does things as I expect. Thanks a lot!

regards,
Vivenzio

> diff --git a/sshfs.c b/sshfs.c
> index e14aa71..323ab6c 100644
> --- a/sshfs.c
> +++ b/sshfs.c
>  <at>  <at>  -2150,6 +2150,10  <at>  <at>  static int sshfs_chown(const char *path, uid_t uid, gid_t gid)
>  {
>  	int err;
>  	struct buffer buf;
> +
> +	if (sshfs.remote_uid_detected && uid == sshfs.local_uid)
> +		uid = sshfs.remote_uid;

Re: Implementation of xattr in sshfs and openssh

Hi,

I tested my patch with valgrind and found a memory leak, grrr.

Here is the corrected patch (only one more line):

diff -rupN sshfs-fuse-2.3/cache.c sshfs-fuse-2.3_mod_xattr/cache.c
--- sshfs-fuse-2.3/cache.c    2010-06-24 11:32:56.000000000 +0200
+++ sshfs-fuse-2.3_mod_xattr/cache.c    2011-12-01 14:50:11.029508130 +0100
 <at>  <at>  -19,23 +19,20  <at>  <at> 
  #define MIN_CACHE_CLEAN_INTERVAL 5
  #define CACHE_CLEAN_INTERVAL 60

-struct cache {
-    int on;
-    unsigned stat_timeout;
-    unsigned dir_timeout;
-    unsigned link_timeout;
-    struct fuse_cache_operations *next_oper;
-    GHashTable *table;
-    pthread_mutex_t lock;
-    time_t last_cleaned;
-    uint64_t write_ctr;
-};
+#ifndef ENOATTR
+#define ENOATTR ENODATA
+#endif
+
+#define DEBUG(format, args...)    do { if (0) fprintf(stderr, format,
args); } while(0)

Re: Implementation of xattr in sshfs and openssh

Hi,

Features in sshfs will always be driven by SFTP server features.  So
if a popular SFTP server supports the xattr extension than I will add
it to sshfs.  I don't believe it makes sense to do it the other way
round (i.e. adding the extension to sshfs and waiting for the servers
to be updated).

Thanks,
Miklos

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure 
contains a definitive record of customers, application performance, 
security threats, fraudulent activity, and more. Splunk takes this 
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d

Re: Fwd: Bug in ID mapping if permissions shall be preserved?

On Thu, Dec 1, 2011 at 9:36 PM, Vivenzio Pagliari
<vivenzio.pagliari <at> gmx.de> wrote:
> On Fri, Nov 25, 2011 at 07:21:55PM +0100, Miklos Szeredi wrote:
>> On Thu, Nov 17, 2011 at 11:42 AM, Vivenzio Pagliari
>> <vivenzio.pagliari <at> gmx.de> wrote:
>> >
>> > IMO, the "preserve ownership" shall preserve the ownership in a
>> > "logic" way: As remote files of ruser are seen with uid of luser,
>> > uid (and gid) shall be mapped also when trying to "preserve
>> > ownership".
>>
>> Does the attached patch help?

>
> yes, the patch does things as I expect. Thanks a lot!

Thanks for testing.

Pushed fix to git tree.

Thanks,
Miklos

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure 
contains a definitive record of customers, application performance, 
security threats, fraudulent activity, and more. Splunk takes this 
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d

Re: Implementation of xattr in sshfs and openssh

Hi,

sure, you are right!

As I wrote in my first mail, I also filed a corresponding patch for
openssh: https://bugzilla.mindrot.org/show_bug.cgi?id=1953

Unfortunately, Damien Miller from the openssh project answered that he
is in doubt whether the additional complexity would be good for openssh.
If the sshfs developers would appreciate xattr functionality in sshfs,
it would be great to support my proposal in the openssh community!

Thanks,
   Eberhard

Am 05.12.2011 13:09, schrieb Miklos Szeredi:
> Hi,
>
> Features in sshfs will always be driven by SFTP server features.  So
> if a popular SFTP server supports the xattr extension than I will add
> it to sshfs.  I don't believe it makes sense to do it the other way
> round (i.e. adding the extension to sshfs and waiting for the servers
> to be updated).
>
> Thanks,
> Miklos

------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH

Re: Implementation of xattr in sshfs and openssh

On Mon, Dec 5, 2011 at 1:38 PM, E. Kuemmerle <E.Kuemmerle <at> fz-juelich.de> wrote:
> Hi,
>
> sure, you are right!
>
> As I wrote in my first mail, I also filed a corresponding patch for
> openssh: https://bugzilla.mindrot.org/show_bug.cgi?id=1953
>
> Unfortunately, Damien Miller from the openssh project answered that he
> is in doubt whether the additional complexity would be good for openssh.
> If the sshfs developers would appreciate xattr functionality in sshfs,
> it would be great to support my proposal in the openssh community!

I'm natural towards xattr support.

But any ssh users out there who would like this to work, yes, please
show your support for Eberhard's request to the openSSH developers.

Thanks,
Miklos

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure 
contains a definitive record of customers, application performance, 
security threats, fraudulent activity, and more. Splunk takes this 
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d

Proposal for support for 'idmap=auto'

Hi,

I'd like to propose support for an extra mode for '-o idmap', "auto".
When "auto" mode is set, the UID/GID of each file gets translated to a
local UID/GID corresponding to the owner's name and group's name of the
file on the remote server.

I have a situation where I want to mount, via sshfs, a remote file
system from a server which uses different username/groupname to uid/gid
mappings than the local server, but which will have some overlap in user
names & group names. I want to be able to mount this remote file system,
and have all my stat() calls, etc show the correctly remapped UID on
this local system.

Using '-o idmap=none' won't work because the remote and local UID/GIDs
don't always match. Using '-o idmap=user' won't work because some
mounted directories contain files with multiple owners, in an
unpredictable fashion (e.g. the unprivileged Apache user).

I can think of several ways to handle this:

1) Use SFTP protocol version 4 or later, which only refers to
owners/groups by name, instead of by uid/gid. However, this likely would
require a large overhaul of OpenSSH and SSHFS, and it seems unlikely
that either party would want to do this, since they'd still probably
need to support protocol version 3 indefinitely.

2) Extend the SFTP protocol, using one of the existing mechanisms
defined for this purpose.

Re: Proposal for support for 'idmap=auto'

Mike Kelly <mike <at> pair.com> writes:

> Hi,
>
> I'd like to propose support for an extra mode for '-o idmap', "auto".
> When "auto" mode is set, the UID/GID of each file gets translated to a
> local UID/GID corresponding to the owner's name and group's name of the
> file on the remote server.
>
> I have a situation where I want to mount, via sshfs, a remote file
> system from a server which uses different username/groupname to uid/gid
> mappings than the local server, but which will have some overlap in user
> names & group names. I want to be able to mount this remote file system,
> and have all my stat() calls, etc show the correctly remapped UID on
> this local system.
>
> Using '-o idmap=none' won't work because the remote and local UID/GIDs
> don't always match. Using '-o idmap=user' won't work because some
> mounted directories contain files with multiple owners, in an
> unpredictable fashion (e.g. the unprivileged Apache user).
>
> I can think of several ways to handle this:
>
> 1) Use SFTP protocol version 4 or later, which only refers to
> owners/groups by name, instead of by uid/gid. However, this likely would
> require a large overhaul of OpenSSH and SSHFS, and it seems unlikely
> that either party would want to do this, since they'd still probably
> need to support protocol version 3 indefinitely.
>
> 2) Extend the SFTP protocol, using one of the existing mechanisms

Re: Proposal for support for 'idmap=auto'

On 12/13/2011 03:39 AM, Miklos Szeredi wrote:
> Mike Kelly <mike <at> pair.com> writes:
> 
>> Hi,
>>
>> I'd like to propose support for an extra mode for '-o idmap', "auto".
>> When "auto" mode is set, the UID/GID of each file gets translated to a
>> local UID/GID corresponding to the owner's name and group's name of the
>> file on the remote server.
>> [...]
> 
> The non-extensions are the least problematic.  I like both #3 and #4.
> 
> If you want to go for a protocol extension, please consult with the
> openSSH guys first.

I posted to the <openssh-unix-dev <at> mindrot.org> list last week looking
for feedback on the protocol extension approach, but I haven't received
any response yet:

  http://thread.gmane.org/gmane.network.openssh.devel/18363

--

-- 
Mike Kelly

------------------------------------------------------------------------------
Cloud Computing - Latest Buzzword or a Glimpse of the Future?
This paper surveys cloud computing today: What are the benefits? 
Why are businesses embracing it? What are its payoffs and pitfalls?
http://www.accelacomm.com/jaw/sdnl/114/51425149/