Re: too many open files

On Friday, 30 April 2010 11:16:09 Klemens Kittan wrote:
> Hello,
> 
> don't want to discourage you, but my slapd (2.4.11 on Debian Lenny)
> still has problems with more than 1024 files.
> 
> I have
> -- ulimit -n 4096 in /etc/default/slapd
> -- checked the output of /proc/`pidof slapd`/limits
> -- downloaded the source package from Debian (version 2.4.11),
>    did a grep for FD_SETSIZE and found that it is set to 4096
> -- built a new package from these sources with configure options:
>    --disable-tcpwrapper
> 
> In a loop from a python script, I then connect, bind, search the ldap
> server and keep all the connections open, like so:
> 
> ...
> def ldap_connection_func():
>   global n
>   l = ldap.initialize("ldaps://ldap...")
>   l.simple_bind_s(username, password)
>   l.search_s('dc=haiti,...', ldap.SCOPE_SUBTREE, 'objectclass=*')
>   n.append(l)
> 
> for x in range(0,1100):
>   ldap_connection_func()
> ...

What was the ulimit -n in the shell from which you ran this python script? If 

Re: too many open files

Hello,

> > Hello,
> > 
> > don't want to discourage you, but my slapd (2.4.11 on Debian Lenny)
> > still has problems with more than 1024 files.
> > 
> > I have
> > -- ulimit -n 4096 in /etc/default/slapd
> > -- checked the output of /proc/`pidof slapd`/limits
> > -- downloaded the source package from Debian (version 2.4.11),
> >    did a grep for FD_SETSIZE and found that it is set to 4096
> > -- built a new package from these sources with configure options:
> >    --disable-tcpwrapper
> > 
> > In a loop from a python script, I then connect, bind, search the ldap
> > server and keep all the connections open, like so:
> > 
> > ...
> > def ldap_connection_func():
> >   global n
> >   l = ldap.initialize("ldaps://ldap...")
> >   l.simple_bind_s(username, password)
> >   l.search_s('dc=haiti,...', ldap.SCOPE_SUBTREE, 'objectclass=*')
> >   n.append(l)
> > 
> > for x in range(0,1100):
> >   ldap_connection_func()
> > ...
> 

Consumer bdb log explosion

Re: Consumer bdb log explosion

--On Tuesday, May 04, 2010 9:48 AM -0700 Mark.Hendricks <at> humboldt.edu wrote:

>
>
>
>
> We are running slapd 2.3.43 (Using bdb db4-4.3.29-10.el5) with one
> provider and two consumers and syncrpl (kerberos/SASL based).
> Occasionally when heavy changes to the provider are made, the consumer
> bdb logs go crazy and fill up the bdb volume.
> When this occurs we see approximately 1 10MB file created per second
> until the 14GB of free space in the volume has been exhausted, often in
> around an hour.
> These packages are from the Buchan Milne repo.
>
> Any ideas would be greatly appreciated.

I would suggest you add the auto remove flag, so that unused logs are 
deleted.

--Quanah

--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration

Re: Consumer bdb log explosion

On Tue, 4 May 2010, Mark.Hendricks <at> humboldt.edu wrote:

> We are running slapd 2.3.43 (Using bdb db4-4.3.29-10.el5) with one provider and two consumers and
syncrpl (kerberos/SASL based).
> Occasionally when heavy changes to the provider are made, the consumer bdb logs go crazy and fill up the bdb volume.

Remove unnecessary log files prior to volume fill, hopefully in 
conjunction with a sane backup procedure (I don't know what 
ldap-hot-backup is, but hopefully it eventually calls slapcat(8) and that 
output gets put somewhere safe). I assume "10MB file per second" refers to 
transaction log file size, which sounds like sufficient granularity to 
remove in a timely fashion in most installations.

http://www.openldap.org/faq/data/cache/738.html

Re: Consumer bdb log explosion

Interesting delta-syncrepl failure

Platform: Red Hat Enterprise Linux 5, update 5, 64-bit
OpenLDAP: 2.4.22 (locally compiled), configured with delta-syncrepl.

The following modify ldif successfully applies to the master:

dn: uid=fcswasey,ou=People,dc=uvm,dc=edu
changetype: modify
replace: sn
sn: Swasey
-
replace: sn
sn: Swasey
-

In OpenLDAP 2.3 -- this modify ldif deck failed because the "sn"
attribute is presented twice.  In OpenLDAP 2.4 -- it works, but the
delta-syncrepl replica pukes on it with this error:

syncrepl_message_to_op: rid=100 mods check (sn: value #0 provided more
than once)

If the OpenLDAP 2.4.22 server accepts it, processes it, and feeds it out
-- shouldn't the OpenLDAP 2.4.22 delta-syncrepl consumer eat it without
complaining?

--

-- 
Frank Swasey                    | http://www.uvm.edu/~fcs
Sr Systems Administrator        | Always remember: You are UNIQUE,
University of Vermont           |    just like everyone else.
  "I am not young enough to know everything." - Oscar Wilde (1854-1900)

Re: Interesting delta-syncrepl failure

--On Wednesday, May 05, 2010 10:31 AM -0400 Francis Swasey 
<Frank.Swasey <at> uvm.edu> wrote:

> In OpenLDAP 2.3 -- this modify ldif deck failed because the "sn"
> attribute is presented twice.  In OpenLDAP 2.4 -- it works, but the
> delta-syncrepl replica pukes on it with this error:
>
> syncrepl_message_to_op: rid=100 mods check (sn: value #0 provided more
> than once)
>
> If the OpenLDAP 2.4.22 server accepts it, processes it, and feeds it out
> -- shouldn't the OpenLDAP 2.4.22 delta-syncrepl consumer eat it without
> complaining?

There's definitely something wrong here, yes.  I suggest you file an ITS. 
Either 2.4 shouldn't have taken the change, or the accesslog should have 
written it in a way the replica would accept.

--Quanah

--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration

what happens in between?

Hi,

on a (a little old) openldap 2.3 server there are 3 meta backends and 
local bdb backends glued together under one suffix ou=foo.

meta: ou=allgv,ou=foo
conn-ttl 30
idle-timeout 1m30s

bdb: ou=humans,ou=foo

When I ask the server for data exactly under one or the other suffix, 
the answer is there immediately.

May  6 15:52:25 rzhs199 local4:debug slapd[4505742]: conn=321210 fd=73 
ACCEPT from IP=127.0.0.1:43816 (IP=0.0.0.0:389)
May  6 15:52:25 rzhs199 local4:debug slapd[4505742]: conn=321210 op=0 
BIND dn="" method=128
May  6 15:52:25 rzhs199 local4:debug slapd[4505742]: conn=321210 op=0 
RESULT tag=97 err=0 text=
May  6 15:52:25 rzhs199 local4:debug slapd[4505742]: conn=321210 op=1 
SRCH base="ou=allgv,ou=foo" scope=2 deref=0 
filter="(&(&(objectClass=person)(|(mail=*@ofd-st.foo)(mail=*@ofd-z.foo)(mail=*@ofd-lbv.foo)(mail=*@ofd-bl.foo)))(|(mail=*franz*)(cn=*franz*)(givenName=*franz*)(sn=*franz*)))"
May  6 15:52:25 rzhs199 local4:debug slapd[4505742]: conn=321210 op=1 
ENTRY 
dn="cn=franziska.afffl,cn=nlbv,cn=migstand,cn=recipients,ou=inf,ou=allgv,ou=foo"
May  6 15:52:25 rzhs199 local4:debug slapd[4505742]: conn=321210 op=1 
ENTRY 
dn="cn=franziska.pfffr,cn=ofd-lba,cn=migstand,cn=recipients,ou=inf,ou=allgv,ou=foo"
May  6 15:52:25 rzhs199 local4:debug slapd[4505742]: conn=321210 op=1 
ENTRY 
dn="cn=fwfffand,cn=nlbv,cn=migstand,cn=recipients,ou=inf,ou=allgv,ou=foo"
May  6 15:52:25 rzhs199 local4:debug slapd[4505742]: conn=321210 op=1 
ENTRY 
dn="cn=cffffen.franz,cn=nlbv,cn=migstand,cn=recipients,ou=inf,ou=allgv,ou=foo"
May  6 15:52:25 rzhs199 local4:debug slapd[4505742]: conn=321210 op=1 
ENTRY 
dn="cn=jefffffer.franzen,cn=nlbv,cn=migstand,cn=recipients,ou=inf,ou=allgv,ou=foo"
May  6 15:52:25 rzhs199 local4:debug slapd[4505742]: conn=321210 op=1 
SEARCH RESULT tag=101 err=0 nentries=5 text=
May  6 15:52:25 rzhs199 local4:debug slapd[4505742]: conn=321210 op=2 UNBIND
May  6 15:52:25 rzhs199 local4:debug slapd[4505742]: conn=321210 fd=73 
closed

May  6 15:54:49 rzhs199 local4:debug slapd[4505742]: conn=321234 fd=17 
ACCEPT from IP=127.0.0.1:43886 (IP=0.0.0.0:389)
May  6 15:54:49 rzhs199 local4:debug slapd[4505742]: conn=321234 op=0 
BIND dn="" method=128
May  6 15:54:49 rzhs199 local4:debug slapd[4505742]: conn=321234 op=0 
RESULT tag=97 err=0 text=
May  6 15:54:49 rzhs199 local4:debug slapd[4505742]: conn=321234 op=1 
SRCH base="ou=humans,ou=foo" scope=2 deref=0 
filter="(&(&(objectClass=person)(|(mail=*@ofd-st.foo)(mail=*@ofd-z.foo)(mail=*@ofd-lbv.foo)(mail=*@ofd-bl.foo)))(|(mail=*franz*)(cn=*franz*)(givenName=*franz*)(sn=*franz*)))"
May  6 15:54:49 rzhs199 local4:debug slapd[4505742]: conn=321234 op=1 
ENTRY dn="employeeNumber=10110,ou=humans,ou=foo"
May  6 15:54:49 rzhs199 local4:debug slapd[4505742]: conn=321234 op=1 
SEARCH RESULT tag=101 err=0 nentries=1 text=
May  6 15:54:49 rzhs199 local4:debug slapd[4505742]: conn=321234 op=2 UNBIND
May  6 15:54:49 rzhs199 local4:debug slapd[4505742]: conn=321234 fd=17 
closed

When I ask with base ou=foo, there is an exact 2:30 min break in between.

May  6 15:27:54 rzhs199 local4:debug slapd[4505742]: conn=320932 fd=28 
ACCEPT from IP=127.0.0.1:43052 (IP=0.0.0.0:389)
May  6 15:27:54 rzhs199 local4:debug slapd[4505742]: conn=320932 op=0 
BIND dn="" method=128
May  6 15:27:54 rzhs199 local4:debug slapd[4505742]: conn=320932 op=0 
RESULT tag=97 err=0 text=
May  6 15:27:54 rzhs199 local4:debug slapd[4505742]: conn=320932 op=1 
SRCH base="ou=foo" scope=2 deref=0 
filter="(&(&(objectClass=person)(|(mail=*@ofd-st.foo)(mail=*@ofd-z.foo)(mail=*@ofd-lbv.foo)(mail=*@ofd-bl.foo)))(|(mail=*franz*)(cn=*franz*)(givenName=*franz*)(sn=*franz*)))"
May  6 15:27:54 rzhs199 local4:debug slapd[4505742]: conn=320932 op=1 
ENTRY 
dn="cn=franziska.axxxl,cn=nlbv,cn=migstand,cn=recipients,ou=inf,ou=allgv,ou=foo"
May  6 15:27:54 rzhs199 local4:debug slapd[4505742]: conn=320932 op=1 
ENTRY 
dn="cn=franziska.pxxxer,cn=ofd-lba,cn=migstand,cn=recipients,ou=inf,ou=allgv,ou=foo"
May  6 15:27:54 rzhs199 local4:debug slapd[4505742]: conn=320932 op=1 
ENTRY 
dn="cn=fwexxxnd,cn=nlbv,cn=migstand,cn=recipients,ou=inf,ou=allgv,ou=foo"
May  6 15:27:54 rzhs199 local4:debug slapd[4505742]: conn=320932 op=1 
ENTRY 
dn="cn=cxxx.franz,cn=nlbv,cn=migstand,cn=recipients,ou=inf,ou=allgv,ou=foo"
May  6 15:27:54 rzhs199 local4:debug slapd[4505742]: conn=320932 op=1 
ENTRY 
dn="cn=jexxxxer.franzen,cn=nlbv,cn=migstand,cn=recipients,ou=inf,ou=allgv,ou=foo"
May  6 15:30:24 rzhs199 local4:debug slapd[4505742]: conn=320932 op=1 
ENTRY dn="employeeNumber=10110,ou=humans,ou=foo"
May  6 15:30:24 rzhs199 local4:debug slapd[4505742]: conn=320932 op=1 
SEARCH RESULT tag=101 err=0 nentries=6 text=
May  6 15:30:24 rzhs199 local4:debug slapd[4505742]: conn=320932 op=2 UNBIND
May  6 15:30:24 rzhs199 local4:debug slapd[4505742]: conn=320932 fd=28 
closed

What could cause this?

Marc

do_bind: invalid dn

Hi, I'm a newbie, downloaded a Windows compilation of OpenLDAP v2.4.18 
from http://www.userbooster.de/en/download/openldap-for-windows.aspx

Am having some problems trying to get a client to bind to my setup, the 
error is: do_bind: invalid dn

Hope to receive some pointers.   Thanks.  : )

My slapd.conf (i basically used the default, only suffix, rootdn and 
rootpw is changed):
********************************
database    bdb
suffix        "dc=click,dc=com"
rootdn        "cn=wael,dc=click,dc=com"
# Cleartext passwords, especially for the rootdn, should
# be avoid.  See slappasswd(8) and slapd.conf(5) for details.
# Use of strong authentication encouraged.
rootpw        password
# The database directory MUST exist prior to running slapd AND
# should only be accessible by the slapd and slap tools.
# Mode 700 recommended.
directory ./data
dirtyread
searchstack 20
# Indices to maintain
index mail pres,eq
index objectclass pres
index default eq,sub
index sn eq,sub,subinitial
index telephonenumber
index cn
********************

openldap.log:
********************
OpenLDAP 2.4.18 Standalone LDAP Server (slapd)daemon_init: listen on 
ldap://JOSHUA:389
daemon_init: listen on ldaps://JOSHUA:636
daemon_init: 2 listeners to open...
ldap_url_parse_ext(ldap://JOSHUA:389)
daemon: listener initialized ldap://JOSHUA:389
ldap_url_parse_ext(ldaps://JOSHUA:636)
daemon: listener initialized ldaps://JOSHUA:636
daemon_init: 6 listeners opened
ldap_create
slapd init: initiated server.
slap_sasl_init: initialized!
bdb_back_initialize: initialize BDB backend
bdb_back_initialize: Berkeley DB 4.6.21: (September 27, 2007)
==>sql_back_initialize()
<==sql_back_initialize()
bdb_db_init: Initializing BDB database
 >>> dnPrettyNormal: <dc=click,dc=com>
<<< dnPrettyNormal: <dc=click,dc=com>, <dc=click,dc=com>
 >>> dnPrettyNormal: <cn=wael,dc=click,dc=com>
<<< dnPrettyNormal: <cn=wael,dc=click,dc=com>, <cn=wael,dc=click,dc=com>
 >>> dnNormalize: <cn=Subschema>
<<< dnNormalize: <cn=subschema>
matching_rule_use_init
................
slapd startup: initiated.
backend_startup_one: starting "cn=config"
config_back_db_open
config_build_entry: "cn=config"
config_build_entry: "cn=schema"
config_build_entry: "cn={0}core"
config_build_entry: "cn={1}cosine"
config_build_entry: "cn={2}nis"
config_build_entry: "cn={3}inetorgperson"
config_build_entry: "cn={4}openldap"
config_build_entry: "cn={5}dyngroup"
config_build_entry: "olcDatabase={-1}frontend"
config_build_entry: "olcDatabase={0}config"
config_build_entry: "olcDatabase={1}bdb"
backend_startup_one: starting "dc=click,dc=com"
bdb_db_open: warning - no DB_CONFIG file found in directory ./data: (2).
Expect poor performance for suffix "dc=click,dc=com".
bdb_db_open: database "dc=click,dc=com": dbenv_open(./data).
bdb_monitor_db_open: monitoring disabled; configure monitor database to 
enable
slapd starting
slap_listener_activate(2):
 >>> slap_listener(ldap://JOSHUA:389)
connection_get(8): got connid=0
connection_read(8): checking for input on id=0
ber_get_next
ber_get_next: tag 0x30 len 20 contents:
op tag 0x60, time 1273085183
ber_get_next
conn=0 op=0 do_bind
ber_scanf fmt ({imt) ber:
ber_scanf fmt (m}) ber:
 >>> dnPrettyNormal: <wael>
conn=0 op=0 do_bind: invalid dn (wael)
send_ldap_result: conn=0 op=0 p=3
send_ldap_response: msgid=1 tag=97 err=34
ber_flush2: 32 bytes to sd 2124
connection_get(8): got connid=0
connection_read(8): checking for input on id=0
ber_get_next
ber_get_next on fd 8 failed errno=0 (unknown WSA error)
connection_close: conn=0 sd=8
********************