Getting back to use openldap...

While trying to get ldap torture back in shape, I had to learn again how to get slapd up and running with a reasonable configs. Here's a few things I had long forgotten and I have learned this morning:

1. The order of the statements in slapd.conf is relevant. Don't be naive, even though the config looks like a normal key value store, some keys can be repeated multiple times (like backend, or database), and can only appear before / after other statements.
2. My good old example slapd.conf file, no longer worked with slapd. Some of it is because the setup is just different, some of it because I probably had a few errors to being of, some of it is because a few statements moved around or are no longer valid. See the changes I had to make.
3. Recent versions of slapd support having configs in the database itself, or at least represented in ldiff format and within the tree. Many distros ship slapd with the new format. To convert from the old format to the new one, you can use:
     slapd -f slapd.conf -F /etc/ldap/slapd.d
4. I had long forgotten how quiet slapd can be, even when things go wrong. Looking in /var/log/syslog might often not be enough. In facts, my database was invalid, configs had error, and there was very little indication of the fact that when I started slapd, it was sitting there idle because it couldn't really start. To debug errors, I ended up running it with:
     slapd -d Any -f slapd.conf
5. slapd will not create the initial database by itself. To do so, I had to use:
     /usr/sbin/slapcat -f slapd.conf < base.ldiffwith base.ldiff being something like this.
6. Even if you set no password, ldapsearch with SASL authentication will likely ask you to confirm. It's easy to fix, though: just pass the -x parameter to go back to simple authentication, like with:
     ldapsearch -x -H "ldap://127.0.0.1:9009/" -b dc=test,dc=it
   Note that I had slapd run on a non standard port for experimentation purposes.
7. Let's say you use -h instead of -H for ldapsearch because your memory is flaky, but you specify the parameter like -H would expect:
     ldapsearch -x -h "ldap://127.0.0.1:9009/" -b dc=test,dc=it
   The command will silently fail. Eg, it will accept -h as "valid" parameter, but still report "unable to connect". Really, -h takes a simple hostname, like 127.0.0.1, but will not fail in a case like above. Took me a few minutes to realize the mistake.

Let's see what the next roadblocks will be ...

LDAP & slapd

Back in 2004 I was playing a lot with OpenLDAP. Getting to run reliably turned out more challenging than I had originally planned for:

• BerkeleyDB performance was terrible if the proper tunings were not provided. Nowhere in the docs was mentioned that this was necessary. The way to do it was to drop a DB_CONFIG file in the top level directory of the database. Not a feature of openldap, rather a feature of BerkeleyDB.
• ... not only performance would be terrible, but even the latest BerkeleyDB versions at the time had a bug (feature?) by which with the indexes used by openldap the would deadlock if certain parts of the index did not fit in memory. I don't remember the details of the problem, it's been too long, but I do remember it was painful, and ended up submitting changes to the openldap package in debian to make sure this was mentioned in the documentation, and that a reasonable default would be provided.
• At the time, OpenLDAP supported two kind of backends: BDB, and HDB, both based on BerkeleyDB. The first, older, did not support operations like 'movedn', which had been standardized in the LDAP protocol for a while, and a few other features that HDB had. HDB though, was marked as experimental. During our use, we found several bugs.

I ended up writing a tool, ldap-torture, to stress test LDAP. You can find it here: https://github.com/ccontavalli/ldap-torture

It allowed us to find a few more bugs, and get them fixed. I hadn't used that tool until yesterday, when I decided to put it on github and try to get it running again. Let's see if I succeed :)

A quick tip if you are getting started with openldap on debian: READ THE DOCUMENTATION! Start from /usr/share/doc/slapd/, the README.Debian.gz is the first file you want to read, followed by README.DB_CONFIG.gz.