headers
James Lentini | 2 Jun 2009 16:45
Picon

Re: [FedFS] Open LDAP and Windows AD performance


On Thu, 28 May 2009, Paul Lemahieu wrote:

> Per our FedFS dicussion on LDAP performance and scaling.
> 
> Some performance numbers for Windows 2003 Active Directory with a 100K user
> entry database:
> 
>    * 2K kerberos logon ops/sec
>    * 22K "search base 1 attr" ops/sec
>    * 2 (yes, 2) "search nonindexed 1 attr" ops/sec
> 
> The full report ishere: http://www.microsoft.com/downloads/thankyou.aspx?familyId=52e7c3bd-57
> 0a-475c-96e0-316dc821e3e7&displayLang=en
> 
> The 2 ops/sec for non-indexed searches is pretty dramatic.
> 
> Some performance numbers for a 1M user entry database:
> 
>    * OpenLDAP: 23K authentication ops/sec
>    * Windows AD: 5K authentication ops/sec
>    
> Here's a link to that information:
> 
>    http://www.connexitor.com/blog/pivot/entry.php?id=185
> 
> For our ballpark calculations, we should comfortably expect many thousands
> of lookups per second.
> 
> --Paul
(Continue reading)

Permalink | Reply | 0 comments |
headers
James Lentini | 2 Jun 2009 17:24
Picon

Re: [FedFS] caching LDAP lookups


On Fri, 29 May 2009, Paul Lemahieu wrote:

> I spoke briefly with Mario today about the LDAP caching issue, and 
> he echoed our comments about how LDAP directories already see peak 
> load situations first thing in the morning, etc, and he would expect 
> the LDAP infrastructure to handle FedFS lookups.

The performance results you cited in your previous email support his 
expectations. Given the strong performance results, long-term caching 
of FSN-to-FSL mappings (anything more than a few minutes) should not 
be necessary. 

<snip> 
> Mario felt caching was a good idea, and that some basic 
> recommendations and defaults should be made in the FedFS standard to 
> guide those who wish to cache LDAP lookups.

Did he mentioned if there were standard, supported mechanisms to 
assist LDAP clients in caching directory data?

I've discovered RFC 3928 "Lightweight Directory Access Protocol (LDAP) 
Client Update Protocol (LCUP)":

 http://www.ietf.org/rfc/rfc3928.txt

which provides a mechanism for LDAP clients to be notified when a 
directory's contents change. This seems like it would be an ideal way 
to learn when a FSN-to-FSL mapping was updated. Unfortunately, LCUP 
appears to have been abandoned:
(Continue reading)

Permalink | Reply | 1 comment |
headers
wurzl_mario | 3 Jun 2009 01:05

Re: [FedFS] caching LDAP lookups


>> -----Original Message-----
>> From: nfsv4-bounces <at> ietf.org [mailto:nfsv4-bounces <at> ietf.org] 
>> On Behalf Of James Lentini
>> Sent: Tuesday, June 02, 2009 11:25 AM
>> To: LeMahieu, Paul
>> Cc: nfsv4 <at> ietf.org
>> Subject: Re: [nfsv4] [FedFS] caching LDAP lookups
>> 
>> 
>> 
>> On Fri, 29 May 2009, Paul Lemahieu wrote:
>> 
>> > I spoke briefly with Mario today about the LDAP caching issue, and 
>> > he echoed our comments about how LDAP directories already see peak 
>> > load situations first thing in the morning, etc, and he 
>> would expect 
>> > the LDAP infrastructure to handle FedFS lookups.
>> 
>> The performance results you cited in your previous email support his 
>> expectations. Given the strong performance results, 
>> long-term caching 
>> of FSN-to-FSL mappings (anything more than a few minutes) should not 
>> be necessary. 
>> 
>> <snip> 
>> > Mario felt caching was a good idea, and that some basic 
>> > recommendations and defaults should be made in the FedFS 
>> standard to 
>> > guide those who wish to cache LDAP lookups.
(Continue reading)

Permalink | Reply | 0 comments |
headers
James Lentini | 4 Jun 2009 15:13
Picon

[FedFS] meeting agenda (6/4)


Note that we will meet at the usual time this week. We'll discuss a 
time change on the call.

    Time: Thursdays 1:30-2:30 PM EST/10:30-11:30 AM PST
 Dial-in: 1-888-765-3653 (us)
          303-218-2659 (international)
      Id: 2354843

Proposed Agenda
---------------

+ IETF Note Well Agreement

  This is a reminder that our discussions are governed by the 
  IETF Note Well Agreement. See:

    http://www.ietf.org/NOTEWELL.html

  We will start each week's meeting with this announcement.

+ Review Action Items

  - James Lentini will poll for times over email.

    Complete. See below.

  - Paul Lemahieu will research LDAP trigger functionality, 
    including standardization, scalability, and configurability 
    (conditions that can cause the trigger to fire).
(Continue reading)

Permalink | Reply | 0 comments |
headers
James Lentini | 4 Jun 2009 22:13
Picon

[FedFS] Meeting Minutes, 06/04/2009


FedFS Meeting Minutes, 06/04/2009
---------------------------------

Attendees
---------

Craig Everhart (NetApp)
James Lentini (NetApp)
Paul Lemahieu (EMC)
Pavan Mettu (Sun)
Renu Tewari (IBM)
Rob Thurlow (Sun)

Minutes
-------

+ IETF Note Well Agreement

  This is a reminder that our discussions are governed by the 
  IETF Note Well Agreement. See:

    http://www.ietf.org/NOTEWELL.html

  We will start each week's meeting with this announcement.

+ Review Action Items

  - James Lentini will poll for times over email.
(Continue reading)

Permalink | Reply | 0 comments |
headers
J. Bruce Fields | 5 Jun 2009 17:31

Re: What is field 'attrset' in struct OPEN4resok designed for ?

On Fri, Jun 05, 2009 at 04:23:45PM +0200, DENIEL Philippe wrote:
> at the page 171 of RFC3530, there is the definition for the structure  
> OPEN4resok. It looks like this:
>
>          struct OPEN4resok {
>               stateid4        stateid;        /* Stateid for open */
>               change_info4    cinfo;          /* Directory Change Info */
>               uint32_t        rflags;         /* Result flags */
>               _*bitmap4         attrset;   *_     /* attributes on
>    create */
>               open_delegation4 delegation;    /* Info on any open
>                                                                    
> delegation */
>       };
>
> There is a 'attrset' field in this structure. When doing a  
> 'OPEN4_NOCREATE' the server replies with OPEN4resok::attrset != 0, which  
> means there is a useful information located here (I do not understand  
> why since they are described as 'attributes on create' in the comments).  
> What is this field made for ? How should it be used ? What is the  
> client's behavior regarding this value ?

I think the only legimate use of the create attributes in the NO_CREATE
case is to implement O_TRUNC: so the size attribute can be specified and
set to 0, but other attributes should be ignored.

But that isn't spelled out in the spec.  The closest I see is:

	When an UNCHECKED create encounters an existing file, the
	attributes specified by createattrs are not used, except that
(Continue reading)

Permalink | Reply | 24 comments |
headers
Noveck, Dave | 8 Jun 2009 17:39
Picon

Re: What is field 'attrset' in struct OPEN4resok designed for ?

> I think the only legimate use of the create attributes in the
NO_CREATE
> case is to implement O_TRUNC: so the size attribute can be specified
and
> set to 0, but other attributes should be ignored.
> 
> But that isn't spelled out in the spec.  The closest I see is:
>
> 	When an UNCHECKED create encounters an existing file, the
>	attributes specified by createattrs are not used, except that
>	when an size of zero is specified, the existing file is
>	truncated.

It doesn't get a 10 on a 0-10 how-clear-is-it scale, but if does imply
what you specified above.  It says the attributes are not used which
sounds like "ignored" to me except that specifying a zero for size
causes a truncate.

Clearly attributes other than size are ignored.

And if the attibute size is specified and is not zero, it says the
attributes are not used.

I'll give it a 6.

-----Original Message-----
From: J. Bruce Fields [mailto:bfields <at> fieldses.org] 
Sent: Friday, June 05, 2009 11:32 AM
To: DENIEL Philippe
Cc: nfsv4 <at> linux-nfs.org; nfsv4 <at> ietf.org
(Continue reading)

Permalink | Reply | 22 comments |
headers
J. Bruce Fields | 8 Jun 2009 22:45

Re: What is field 'attrset' in struct OPEN4resok designed for ?

On Mon, Jun 08, 2009 at 11:39:16AM -0400, Noveck, Dave wrote:
> > I think the only legimate use of the create attributes in the
> NO_CREATE
> > case is to implement O_TRUNC: so the size attribute can be specified
> and
> > set to 0, but other attributes should be ignored.
> > 
> > But that isn't spelled out in the spec.  The closest I see is:
> >
> > 	When an UNCHECKED create encounters an existing file, the
> >	attributes specified by createattrs are not used, except that
> >	when an size of zero is specified, the existing file is
> >	truncated.
> 
> It doesn't get a 10 on a 0-10 how-clear-is-it scale, but if does imply
> what you specified above.  It says the attributes are not used which
> sounds like "ignored" to me except that specifying a zero for size
> causes a truncate.
> 
> Clearly attributes other than size are ignored.
> 
> And if the attibute size is specified and is not zero, it says the
> attributes are not used.
> 
> I'll give it a 6.

OK!  So I guess the right thing for a server to do on an non-creating
open would be to clear all but the SIZE bit in the returned attribute
bitmap?
(Continue reading)

Permalink | Reply | 20 comments |
headers
Noveck, Dave | 8 Jun 2009 23:03
Picon

RE: What is field 'attrset' in struct OPEN4resok designedfor ?

Now I see the ambiguity which I didn't see before.  I interpreted this
from the server's point of view.  No big surprise.  So that the
attributes not being used meant that the server didn't look at them. So,
NFS4ERR_INVAL wouldn't be right in this situation since it depends on
looking at attributes it isn't supposed to.

If you look at it from the client's point of view, the attributes other
than size not being set, means that the client should not set them,
which everybody agrees with, but it then implies that if the client sets
them, it is doing something wrong, and so INVAL makes sense to return in
this case.

> I'll give it a 6.

I think that should go down to a three.  There is ambiguity about what
the server should do but it is clear that the client should not mess
with fields other than size and the size if set should be set to zero.
And if the client does obey that, he isn't hurt by ambiguity about what
the server should do in other cases. 

I think there are probably other cases like this where we describe how
the client and server work together to do the right thing.  That often
tends to lead one to omit various cases where one or the other don't
work together as they should.

-----Original Message-----
From: J. Bruce Fields [mailto:bfields <at> fieldses.org] 
Sent: Monday, June 08, 2009 4:46 PM
To: Noveck, Dave
Cc: nfsv4 <at> linux-nfs.org; nfsv4 <at> ietf.org; DENIEL Philippe
(Continue reading)

Permalink | Reply | 18 comments |
headers
J. Bruce Fields | 9 Jun 2009 20:24

Re: What is field 'attrset' in struct OPEN4resok designedfor ?

On Mon, Jun 08, 2009 at 05:03:53PM -0400, Noveck, Dave wrote:
> I think that should go down to a three.  There is ambiguity about what
> the server should do but it is clear that the client should not mess
> with fields other than size and the size if set should be set to zero.

Sounds fine, but I don't *think* that's explicitly stated anywhere.

--b.

> And if the client does obey that, he isn't hurt by ambiguity about what
> the server should do in other cases. 
> 
> I think there are probably other cases like this where we describe how
> the client and server work together to do the right thing.  That often
> tends to lead one to omit various cases where one or the other don't
> work together as they should.
> 
> -----Original Message-----
> From: J. Bruce Fields [mailto:bfields <at> fieldses.org] 
> Sent: Monday, June 08, 2009 4:46 PM
> To: Noveck, Dave
> Cc: nfsv4 <at> linux-nfs.org; nfsv4 <at> ietf.org; DENIEL Philippe
> Subject: Re: [nfsv4] What is field 'attrset' in struct OPEN4resok
> designedfor ?
> 
> 
> On Mon, Jun 08, 2009 at 11:39:16AM -0400, Noveck, Dave wrote:
> > > I think the only legimate use of the create attributes in the
> > NO_CREATE
> > > case is to implement O_TRUNC: so the size attribute can be specified
(Continue reading)

Permalink | Reply | 16 comments |

Gmane