In RHEL 6.2, at least, SSSD doesn’t always clear its cache for the Kerberos provider. I found this out when I decided to change the group name on our LDAP server. Computers that used straight up LDAP dutifully reflected the change nearly immediately (we use nslcd, so restarting the nslcd service provided a nearly instantaneous update). However, our Kerberos machines were more troublesome, and they refused to reflect the changes, even after restarting nslcd and sssd. Rebooting didn’t help either. Finally, I uncovered the location of the SSSD cache files. To get SSSD to recreate its cache, stop the service and delete the cache files in /var/lib/sss/db/. I deleted all three (cache_default.ldb, cache_kerberos.ldb, and cache_LDAP.ldb). Then all I had to do was start the SSSD service back up again and the problem was fixed. Now id reports the correct group name and directory listings work too.

Mac OS X 10.7 switched from MIT Kerberos to Heimdal. In order to get this to work, I had to make a few changes to our setup (which is far from mature, unfortunately). If you’ve read my other posts, then you would know that our KDC is running MIT Kerberos on a RHEL 6 server. The same server is also running OpenLDAP. Our goal was to provide Kerberos tickets at login that could be used to SSH into other workstations and protect our NFS shared home directories.

The first thing I had to was to open up the right UDP ports on our KDC (88, 464, and 749). It’s possible to force Heimdal to use TCP instead, but there were a few mixed reports that suggested that it didn’t always work. It seemed easier and more straightforward to let Heimdal use its default settings rather than risk that Apple had made some undocumented tweaks that broke it.

The second thing I had to do was to add authAuthority to our list of mapped LDAP attributes. authAuthority can be found in /etc/openldap/schema/apple.schema on any 10.7 install. The schema file should be added to your OpenLDAP server and the authAuthority attribute type should be uncommented. Using phpldapadmin, I first added authAuthorityObject to my user’s objectClass. Then I added a new attribute, authAuthority and set it to read ;Kerberosv5;;;REALM; (the semicolons are important), replacing REALM with our realm name. Then, using Directory Utility (the easiest way to access it is through the Users & Groups System Preferences pane, but you can find it in /System/Library/CoreServices), I navigated to our LDAP server’s “Search & Mappings” tab, drilled down under Users, and set AuthenticationAuthority on the left side to map to authAuthority on the right side. If AuthenticationAuthority isn’t listed, you can add it. Click on Users (or add it as a Record Type), click add, select Attribute Types, and find AuthenticationAuthority in the list.

A quick note: If you don’t have authAuthority stored in OpenLDAP, but you have /etc/krb5.conf already in place, it’s possible that your network accounts will be unable to log in or use sudo until you do. Deleting /etc/krb5.conf will restore logins for you if you get stuck partway through this process, but it’s better to use a local login until Kerberos is completely set up.

The third thing to do is to add the krb5.conf file. In previous versions of Mac OS X, you had to name your config edu.mit.Kerberos and hide it in /Library/Preferences/. Lion will still look there for the sake of backwards compatibility, but I didn’t like the idea of naming my Heimdal configuration file by another name. Happily, /etc/krb5.conf is supported (and probably the preferred location going forward). I was able to use my existing krb5.conf file from my RHEL 6 server without any changes. You can read man krb5.conf to make sure your configuration file won’t give you any surprises.

The fourth thing I had to do was to make sure to edit /etc/pam.d/authorization. This is what the login window uses to fetch your Kerberos tickets. If you open the file, you’ll see that the first line actually calls pam_krb5.so. This replaces some of /etc/authorization‘s functionality. You only need to edit this file if you want to use uids as login names instead of the fully qualified uid@REALM format, but if you append default_principal to the end of the pam_krb5.so line, you will be able to use short names to login, and it will automatically append your default REALM to the end of your login. Here’s an example:

auth       optional       pam_krb5.so use_first_pass use_kcminit default_principal

At this point, if you’re feeling brave, run killall opendirectoryd if you haven’t already, log out, and log back in as a network user account. If you have already correctly configured network user accounts with OpenLDAP, everything should just work, and when you open Terminal and run klist or open Ticket Viewer (buried in /System/Library/CoreServices), you should see a ticket for your user account with a valid expiration date!

As a final step, if you need host principals for your workstations, I had some interoperability problems between Heimdal and my MIT Kerberos KDC. The way I worked around it was to create a keytab with kadmin on a RHEL 6 workstation, copy it to the new computer, and use ktutil on the destination Mac to append it to the default /etc/krb5.keytab file that comes with Mac OS X. (I managed to really break a Lion install a while back by deleting its keytab file, and rather than risk doing that again, I decided to let the existing principals remain. I will eventually try wiping them again in the future to find out what I did wrong.) Here’s essentially what I do now for new Mac OS X computers:

1. I log into a RHEL 6 workstation running MIT Kerberos, and log into kadmin with my admin principal:

   # kadmin
   kadmin: ktadd -k computername.keytab host/computername.domainname
   kadmin: ktadd -k computername.keytab nfs/computername.domainname
   

   This takes care of SSH and NFS. If your network needs other Kerberos principals on workstations, add them to the keytab at this time.

2. I copy the file (in my case, using SSH) to the newly installed Mac. Then I run ktutil with sudo to merge the two keytab files together.

   # ktutil copy computername.keytab /etc/krb5.keytab
   

3. Now that the host principals have been safely merged, I can delete the temporary keytab I’ve created.

Lion introduced OpenDirectory as a replacement for DirectoryService. The configuration looks practically identical in the GUI, but the output plist files are different enough that you have to recreate them. I use a custom LDAP mapping, so to help me remember my settings, I had my old DSLDAPv3PlugInConfig.plist open in a text editor while I went through the tedious process of recreating it. Not everyone has to make these mappings and I know a lot of people can get away with using the standard RFC mapping. Lucky me.

If you have to create a custom mapping, but haven’t done it before, it’s straightforward enough, but easy to leave things out by mistake. In the Search & Mappings tab, you are mapping LDAP attributes to OpenDirectory attributes. On the left side of the screen, you must add high level Record Types first, then populate them with Attribute Types. Then, on the right side of the screen, you add the LDAP names, like uid or userPassword. If you have an ldif file of a user on your LDAP server as an example, or use a transparent LDAP editor like phpldapadmin, the LDAP names will be easy to find. Then it just becomes a matter of discovering what Mac OS X calls each thing.

In earlier versions of Mac OS X, I copied DSLDAPv3PlugInConfig.plist and SearchNodeConfig.plist to /Library/Preferences/DirectoryService and told puppet to restart the com.apple.DirectoryServices service. Super straightforward! This left me totally unprepared for a truly horrible snafu when trying to script our LDAP settings in Mac OS X Lion the same way.

So in Lion, I copied the correct files (Search.plist, Contacts.plist, and ldap.domainname.plist) into the correct places, but when I told puppet to restart the com.apple.opendirectoryd service, the service fell over and refused to start back up. Worse, now that OpenDirectory was dead and no longer mapped my user name to my user id, suddenly I could’t use sudo anymore or run anything with administrator privileges. Even single user mode couldn’t help me, and deleting the entire contents of /Library/Preferences/OpenDirectory didn’t seem to help. Next I tried reinstalling, but apparently that didn’t overwrite the correct files either. That’s right, I got stuck having to FORMAT my drive just to get a working copy of Lion back.

So where did I go wrong? As it turns out, after reading some online documentation more closely, you are SUPPOSED to run killall opendirectoryd and avoid ever touching its launchctl file at all. I was skeptical, but I tried it and suddenly, it WORKED. From there, it was trivial to script this in puppet:

exec {
  "com.apple.opendirectoryd":
    command     => "killall opendirectoryd",
    refreshonly => true;
}

file {
  "/Library/Preferences/OpenDirectory/Configurations/LDAPv3":
    owner  => root,
    group  => wheel,
    mode   => 750,
    ensure => directory;
  "/Library/Preferences/OpenDirectory/Configurations/Search.plist":
    owner => root,
    group => wheel,
    mode  => 600,
    ensure => present,
    notify => Exec["com.apple.opendirectoryd"],
    source => "puppet:///modules/ldap/Search.plist";
  "/Library/Preferences/OpenDirectory/Configurations/Contacts.plist":
    owner => root,
    group => wheel,
    mode  => 600,
    ensure => present,
    notify => Exec["com.apple.opendirectoryd"],
    source => "puppet:///modules/ldap/Contacts.plist";
  "/Library/Preferences/OpenDirectory/Configurations/LDAPv3/ldap.domainname.plist":
    owner  => root,
    group  => wheel,
    mode   => 600,
    ensure => present,
    notify => Exec["com.apple.opendirectoryd"],
    source => "puppet:///modules/ldap/ldap.domainname.plist"
}

At work, I applied the RHEL 5 Draft STIG to some of our systems in an effort to increase our security. (STIGs are security checklists, and they’re available for a multitude of operating systems and devices. Unfortunately, they are frequently out of date, hence why I’m applying a RHEL 5 STIG to a RHEL 6 network.) A week later, and a routine kernel update brought our network to its knees. The guilty party? /usr/bin/ldd‘s permissions, which the draft STIG required be set to 000.

Apparently, when you update a kernel package in RHEL (and this may be a shortcoming of RPM packages in general, since root should be able to get around this problem), it tries to run /usr/bin/ldd. If you remove all access to ldd by setting its permissions to 000, the kernel does not install correctly (even though yum reports no errors) and you will get a Kernel Panic screen on your next boot. I got a few variations of the same message (one said “VFS: Unable to mount root fs on unknown-block(0,0)” and another one said “Kernel panic - not syncing: No init found. Try passing init= option to kernel.”). When I first encountered the problem, it was a week after I’d applied the STIG, so my gut reaction initially was to say it was RHEL’s fault. I didn’t really believe that, but I couldn’t see what else it could be.

It wasn’t until many hours later that I tried to run mkinitrd manually out of desperation. It started spitting out error messages about ldd. Suddenly it dawned on me what had happened. I fixed the permissions and tried rebooting. Still nothing. Then it occurred to me that ldd was probably only getting run during the initial kernel INSTALL. This time, I booted off the old kernel, ran yum reinstall kernel and rebooted. SUCCESS!

Unfortunately, there isn’t a good way to detect that a kernel update is about to be applied, so it would be hard to script resetting the permissions before an install. I’m going to recommend that administrators skip this STIG setting for now. Bricking your system is too high a price just to be compliant, and I’m unconvinced that changing ldd‘s permissions provides any security benefit whatsoever.

We’re stuck with an old version of puppet on our Macs because our puppet server is running RHEL 6 (surprise) which is stuck at 2.6.14. My previous attempts to bridge major version differences have failed miserably. Now I just keep them in sync and sigh at all the awesome features I don’t get to play with just yet.

I don’t know how we didn’t run into this problem in the past (perhaps it was a change on puppet’s part), but during my Lion testing, I noticed that puppet was trying to load launchctl jobs without the -w flag. In particular, Cups and SSH were refusing to load. After some Googling, I discovered that puppet didn’t know that new versions of Mac OS X no longer stored the Disable key inside the launchctl plist file in /System/Library/LaunchDaemons. Instead, Disable keys are stored in /var/db/launchd.db/com.apple.launchd/overrides.plist.

It looks like newer versions of puppet have a fix, but unless that fix gets back ported, I’m stuck finding my own solution. Enter PlistBuddy, a simple (and somewhat stupid) tool for editing plist files. Here’s an example of how to get the Disable key settings for Cups and SSH (I apologize for my web design laziness; if your window isn’t wide enough, the text will run off the screen):

# /usr/libexec/PlistBuddy -c "Print :org.cups.cupsd:Disabled" /var/db/launchd.db/com.apple.launchd/overrides.plist
# /usr/libexec/PlistBuddy -c "Print :com.openssh.sshd:Disabled" /var/db/launchd.db/com.apple.launchd/overrides.plist

Please note the colons (:) in the command we send to PlistBuddy. If you open the /var/db/launchd.db/com.apple.launchd/overrides.plist file, you’ll see that it’s in XML and that it’s a dictionary with many keys. Each key in that dictionary is the name of a launchctl job. At the highest level, you can run something like:

# /usr/libexec/PlistBuddy -c "Print :" /var/db/launchd.db/com.apple.launchd/overrides.plist

This prints the dictionary keys and their values (which, as you might have noticed, are also dictionaries). The following will let you see the value of just one dictionary key, “com.openssh.sshd”:

# /usr/libexec/PlistBuddy -c "Print :com.openssh.sshd" /var/db/launchd.db/com.apple.launchd/overrides.plist

This prints out the dictionary stored inside the com.openssh.sshd key. And lastly, the original command I gave you will give you the value of the key inside that dictionary:

# /usr/libexec/PlistBuddy -c "Print :com.openssh.sshd:Disabled" /var/db/launchd.db/com.apple.launchd/overrides.plist

A value of true means the service will not start, and a value of false means that the service can start. PlistBuddy can also set values. The syntax is the same, but instead of using Print, you pass it the Set command. For something simple, like setting something to true or false, this is very straightforward:

# /usr/libexec/PlistBuddy -c "Set :com.openssh.sshd:Disabled false" /var/db/launchd.db/com.apple.launchd/overrides.plist

OK, now that I’ve walked you through how PlistBuddy works (you can read about other supported commands in man PlistBuddy), I’ll show you how to set the value in puppet, using the exec type. Here’s an example for SSH:

exec {
  "Turn off Disabled key for SSH":
    command => "/usr/libexec/PlistBuddy -c 'Set :com.openssh.sshd:Disabled false' /var/db/launchd.db/com.apple.launchd/overrides.plist",
    onlyif  => "/usr/libexec/PlistBuddy -c 'Print :com.openssh.sshd:Disabled' /var/db/launchd.db/com.apple.launchd/overrides.plist | grep -q true";
}

Here, the command simply sets the Disabled flag to false, provided that it’s currently set to true. Obviously, this would lend itself very well to scripting. You could replace “:com.openssh.sshd:Disabled” with a puppet variable and call it for each service you wanted to enable. And if you wanted to be sure that the Service resource loaded correctly the first time, you could use a require line to tell puppet to try running the exec first:

Service {
  "com.openssh.sshd":
    ensure  => true,
    enable  => true,
    hasstatus => true,
    require => Exec["Turn off Disabled key for SSH"];
}

I hope this helps someone else. I wasted too much time on this trying to be clever. I’m looking forward to using a version of puppet that has a patch for this built into it.

Work’s been super busy, so I had to put it off longer than I liked, but I finally got to test setting up a Lion OpenLDAP client. Success! OS X 10.7.3 finally fixes our problems. 10.7 and 10.7.1 were of course vulnerable to the widely publicized issue where it would accept any password after binding to an LDAP server. 10.7.2 fixed this problem, but at the expense of breaking our ability to log in at all.

The only downside was that the switch from the old Directory Service to the new Open Directory meant that I had to reenter all our custom mappings through the GUI tool. (The files were soooo close, and yet, not quite the same.) I did notice one oddity in the process. In previous versions of Mac OS X, we didn’t both specifying a record name for our Groups. If you don’t do that in Lion and try to look at NFS file permissions or run the id command from Terminal, it prints out No Record Name in place of your group id. There might be other similar oddities that I haven’t uncovered yet. Happily, they’re easily fixed.

I wasted too much time this afternoon trying to figure out why rdiff-backup failed on ONE backup job in the wee hours of Sunday morning. All the others had completed successfully as expected. I finally Googled it. Guess what? It’s a known problem with daylight savings time. The full explanation is here: http://wiki.rdiff-backup.org/wiki/index.php/NoMetaData.

Here’s the error message that showed up in our logs:

Previous backup seems to have failed, regressing destination now.
Warning, could not find mirror_metadata file.
Metadata will be read from filesystem instead.
Fatal Error: No metadata for time Sun Mar 11 03:00:19 2012 (1331449219) found,
cannot regress
Fatal Error: Lost connection to the remote system

In theory, removing the older of the two current_mirror.* files will fix the problem and rerunning rdiff-backup. I think I managed to botch it and eventually ended up removing most of the older backups in the process. *sigh* At least we only took out the mail backup and most of our users download their mail to their home directories anyway. Everything’s running again now.

This took a little doing, and most of what I found on the Internet was very slightly off. Here’s what I came up with (and what works on RHEL 6):

augeas {
  "Add MD5 password to Grub":
    context => "/files/boot/grub/menu.lst",
    changes => [
      "ins password after timeout",
      "clear password/md5",
      "set password \$1\$KeSTX0\$giM/W8SGhE4tbBTSiaguu.",
    ],
    onlyif  => "match password size == 0";
}

The password here, by the way, is ‘password’ encrypted with the tool grub-md5-crypt.

I found out today that SSSD doesn’t support multiple domains in the way I thought it did. I thought I could tell SSSD to try Kerberos first, and if it failed, try LDAP. What isn’t documented ANYWHERE, but is instead buried in a mailing list post, is that if you use the same LDAP server/id_provider for both domains, SSSD won’t perform the lookup a second time for the LDAP domain. It just dies horribly and silently (I haven’t found SSSD error messages terribly enlightening). In the same mailing list post, it was suggested that different search bases for Kerberos and LDAP users be used, or that usernames be fully qualified. In the first case, different search bases would allow me to use Kerberos as my primary domain, but fully qualified usernames would only work if Kerberos was the second domain. I can see I have more testing to do. In any case, you can read the relevant post on Fedora Hosted, where the poster explains the reason for this decision. In any case, I’m glad I found the post, because man sssd.conf mentions no such limitations.

SSSD supports LDAP and Kerberos with ease. Once I’d ironed out the wrinkle with multiple domains and shelved that problem for another day, I made much better progress on my testbed machine. Here’s an example sssd.conf that will support a Kerberos domain:

[sssd]
config_file_version = 2
services = nss, pam
domains = kerberos

[nss]
filter_groups = root
filter_users = root
reconnection_retries = 3

[pam]
reconnection_retries = 3

[domain/kerberos]
id_provider = ldap
ldap_uri = ldap://ldapserver.domain.com
ldap_search_base = dc=domain,dc=com
ldap_id_use_start_tls = true

auth_provider = krb5
chpass_provider = krb5
krb5_server = kerberos.domain.com
krb5_realm = DOMAIN.COM
krb5_ccname_template = FILE:%d/krb5cc_%U

The [nss] and [pam] sections are straight out of the default config file. The important line to change right at the top of the config file is domains, which should be set to whatever you’ll call your Kerberos section. I named mine [domain/kerberos] because I’m boring that way, so I used domains = kerberos. You could also have named it [domain/awesome] and it would still work as long as your domains line read domains = awesome. The domain/ prefix is the important part.

Once we’ve pointed domains to look at [domain/kerberos], we’ll set up a few options. First, we’ll set up the id_provider to use ldap. SSSD is smart enough to get some of its information from /etc/openldap/openldap.conf, so although we use TLS, I don’t have to tell SSSD where to find our certificate files. Instead, I give it our URI, the search base, and I set ldap_id_use_start_tls to true to make sure it uses TLS. Now we’ll move on to the Kerberos specific part. First, we set auth_provider and chpass_provider to krb5 to make sure it talks to Kerberos instead of falling back to LDAP for authentication or storing passwords. Then we set krb5_server to our Kerberos server (it can be a comma separated list, if you have backup KDCs on your network). krb5_realm should be self explanatory. I’ve modified krb5_ccname_template to use the simpler form /tmp/krb5cc_someuserid. The default is to create a file with a unique suffix, which isn’t what I wanted.

Once you’re done editing /etc/sssd/sssd.conf, start (or restart) the service and ensure it runs at boot:

# /sbin/service sssd start
# /sbin/chkconfig sssd on

SSSD assumes that you’ve configured pam to use its pam_sss module to make everything possible. If you’re using a default RHEL6 install and haven’t touched /etc/pam.d/password-auth-ac or /etc/pam.d/system-auth-ac, everything should be configured correctly. Check both files to make sure you have an auth, an account, a password, and a session line for pam_sss.so in each.

You should now be able to log into your computer and automatically get a Kerberos ticket, either from the command line or from GDM. You can test this by wiping out your tickets with kdestroy, dropping down to a command line, and logging in. You should be able to run klist and see a ticket.

I’ve already discussed how to set up a KDC in my earlier post on NFSv4 (I also explained how to Kerberize NFS). Let’s talk about setting up other services, such as SSH, LDAP, and Postfix (using Dovecot SASL for authentication). RHEL6 makes this dead drop simple, and many services have GSSAPI support built right in. GSSAPI, in turn, talks Kerberos. You create a few host principals, create a few new keytabs, and add a few lines here and there to configuration files. With a working KDC, most services can be configured in a few minutes at most. Rolling it out to every user and computer on a network will take a little more time, but most of that work can be scripted.

Kerberizing SSH with Host Principals

SSH will need a host principal in the form host/computer.domain.com on every computer running SSHD. As an example, assume I have a computer named Server and a computer named Client. I’ll create a host principal on Server called host/Server.domain.com. This will allow a user logged into Client to SSH into Server. It’ll also allow a user on Client1, Client2, and Server2 to do the same. However, if I want a user on Server to be able to log into Client, I will also have to create the principal host/Client.domain.com.

First, create the principal using kadmin. This can be done from any computer capable of talking to the KDC with an admin principal (my KDC is configured to allow users with the user principal form username/admin to perform administrative actions), or it can be done locally on the KDC without an administrative principal using kadmin.local.

# kadmin -p username/admin
kadmin: addprinc -randkey host/computer1.domain.com

Repeat the addprinc command for each computer you want to SSH into with Kerberos. Now, log into one of these computers and run kadmin again. This time, we’ll use ktadd to add the correct host principal to the local keytab. In this example, we’ll log into Computer1 and add Computer1′s host principal to the local keytab.

# kadmin -p username/admin
kadmin: ktadd host/computer1.domain.com

To configure Computer2, log into Computer2 and repeat the command, substituting Computer2 for Computer1. When you’re done configuring all your computers, each computer will have its own host principal and only its own host principal in its keytab file. You can verify this by running klist -k on each computer. Here is an example output for Computer1 (it’s normal to see multiples, since each one is actually a different encryption key):

# klist -k
   2 host/computer1.domain.com@DOMAIN.COM
   2 host/computer1.domain.com@DOMAIN.COM
   2 host/computer1.domain.com@DOMAIN.COM
   2 host/computer1.domain.com@DOMAIN.COM

The only thing left to do now is to configure /etc/ssh/sshd_config and /etc/ssh/ssh_config on each computer. Find and correct the following lines in /etc/ssh/sshd_config:

GSSAPIAuthentication yes
GSSAPIKeyExchange yes
GSSAPICleanupCredentials yes

After restarting SSHD, test the connection from a different machine with the following command:

ssh -v -K computer1.domain.com

If it worked, you should not be prompted for a password (assuming you have a valid ticket) and it should print out something like the following:

debug1: Next authentication method: gssapi-keyex
debug1: Authentication succeeded (gssapi-keyex).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env XMODIFIERS = @im=none
debug1: Sending env LANG = en_US.utf8
Last login: Tue Jan 31 14:21:21 2012 from computername.domain.com

You should also be able to run klist and see a ticket. If the login worked, but you don’t have a ticket, your KDC was not configured to issue forwardable tickets. This can be fixed. If the login worked, but it says Authentication succeeded (gssapi-with-mic), you’re still using Kerberos, but you’re probably trying to connect to a computer that doesn’t support GSSAPIKeyExchange, which is a relatively new addition to Linux.

You can now either edit /etc/ssh/ssh_config to tell SSH to use Kerberos automatically for every SSH connection or you can edit your personal ~/.ssh/config to enable it for your user account only. The lines and format are the same:

Host *
   GSSAPIAuthentication yes
   GSSAPIKeyExchange yes
   GSSAPIDelegateCredentials yes

To tell SSH to only use these options for local computers on the network (the Kerberos negotiation can add an extra unwanted delay), you can edit the first line to read Host *.domain.com. However, this means you will have to type the FQDN every time you use ssh. Alternatively, you can use the -K flag every time, which would eliminate the need to edit any client configuration files at all.

Kerberizing LDAP with Host Principals

Since Kerberos relies heavily on a user database backend like LDAP, there are a multitude of ways the two services can be configured to interact. You can create your KDC inside an LDAP database, you can store information about Kerberos principals in LDAP user fields, or you can use Kerberos to authenticate LDAP lookups. I'm only going to focus on the last bit there, using ldapsearch as an example to show GSSAPI in action. In RHEL6, ldapsearch tries to use SASL by default (SASL actually supports several mechanisms besides GSSAPI/Kerberos, but we'll configure it to only accept GSSAPI). You can disable SASL by using the -x flag to enable simple authentication, which is what we had been doing prior to Kerberizing our network, but why would anyone want to do that when we could be using glorious Kerberos?

First, we're going to perform the (increasingly familiar) task of creating the host principal for ldap. In this case, we will create one that uses the format ldap/ldapserver.domain.com:

# kadmin -p username/admin
kadmin: addprinc -randkey ldap/ldapserver.domain.com

Log into the ldap server and create a keytab for ldap. We will use an ldap specific keytab in /etc/openldap/ldap.keytab:

# kadmin -p username/admin
kadmin: ktadd -k /etc/openldap/ldap.keytab ldap/ldapserver.domain.com

Change the ownership and permissions for our new keytab:

# chown ldap:ldap /etc/openldap/ldap.keytab
# chmod 640 /etc/openldap/ldap.keytab

We're halfway there. Now we have to configure slapd to use GSSAPI. OpenLDAP has migrated to using a directory structure for slapd configuration. I prefer to edit something more human readable, though, so I edit /etc/openldap/slapd.conf and generate a new /etc/openldap/slapd.d only as necessary. Add or correct the following lines in /etc/openldap/slapd.conf:

sasl-secprops noanonymous,noplain,noactive
sasl-regexp uid=([^/]*),cn=GSSAPI,cn=auth uid=$1,ou=people,dc=domain,dc=com

The first line will prevent SASL from advertising undesired methods such as DIGEST-MD5. The second line will need to be configured to match your environment. When you run ldapsearch with SASL, it sends the user id to slapd in the form uid=username,cn=GSSAPI,cn=auth. This will map it into the form uid=username,ou=people,dc=domain,dc=com, which is the structure most ldap servers use. We throw away anything after the user id with a wildcard, so a principal that looks like username/admin will work just as well as username.

If you have ACLs set up, here is a way to add an access line for Kerberos administrators, shown alongside a more typical access line, assuming you use the syntax username/admin:

access to *
  by dn.regex="uid=.*/admin,cn=GSSAPI,cn=auth" write
  by group/groupOfUniqueNames/uniqueMember="cn=ldapadmins,ou=groups,dc=domain,dc=com" write
  by * read

Here you see the same cn=GSSAPI,cn=auth suffix from the sasl-regexp line, but now we're specifically looking for the /admin suffix after the user id.

The only thing left to do is to generate the new /etc/openldap/slapd.d configuration folder and restart slapd. If you were planning to store Kerberos information in LDAP, this would be an ideal time to add a Kerberos schema. I use the following commands to do the generation and restarting, although I usually paste it on one line, separating each command with semicolons:

# service slapd stop
# rm -rf /etc/openldap/slapd.d/
# mkdir slapd.d
# slaptest -f /etc/openldap/slapd.conf -F /etc/openldap/slapd.d/
# chown -R ldap:ldap /etc/openldap/slapd.d/
# chmod -R go-rwx /etc/openldap/slapd.d/
# service slapd start

To test if Kerberos support is working, you should be able to run ldapsearch with GSSAPI SASL. Try a command like the following:

# ldapsearch uid=username

You should get something that looks like this, followed by the information about the username you're querying:

SASL/GSSAPI authentication started
SASL username: user@DOMAIN.COM
SASL SSF: 56
SASL data security layer installed.
# extended LDIF
#
# LDAPv3
# base  (default) with scope subtree
# filter: uid=username
# requesting: ALL
#

You can check to see if you got a ticket with klist. You should see the host principal ldap/ldapserver.domain.com. If so, you have successfully Kerberized ldap!

Kerberizing Dovecot and Postfix with Host Principals

Dovecot, which is what we use for POP3, happily supports GSSAPI. We're going to give Dovecot its own keytab file, just like we did for LDAP. We'll create host principals with pop/ and smtp/ prefixes. (IMAP users should create a host principal that begins with imap/ instead of pop/.)

# kadmin -p username/admin
kadmin: addprinc -randkey pop/mailserver.domain.com
kadmin: addprinc -randkey smtp/mailserver.domain.com

On the mail server:

# kadmin -p username/admin
kadmin: ktadd -k /etc/dovecot/dovecot.keytab pop/mailserver.domain.com
kadmin: ktadd -k /etc/dovecot/dovecot.keytab smtp/mailserver.domain.com

Change the ownership and permissions for our new keytab:

# chown dovecot /etc/dovecot/dovecot.keytab
# chmod 640 /etc/dovecot/dovecot.keytab

Now we'll configure dovecot to use this keytab and use GSSAPI for user authentication. Edit /etc/dovecot/conf.d/10-auth.conf (or /etc/dovecot/dovecot.conf if 10-auth.conf does not exist) and add or edit the following lines:

auth_gssapi_hostname = mailserver.domain.com
auth_krb5_keytab = /etc/dovecot/dovecot.keytab
auth_mechanisms = gssapi

The value you choose for auth_gssapi_hostname here should match whatever you used in the host principals you created. If you need to support non-Kerberos users, auth_mechanisms will support several options. Here's what you would need to support both Kerberos principals and users with regular shadow passwords:

auth_mechanisms = plain gssapi

Next, we'll edit /etc/dovecot/10-master.conf to add support for authenticating postfix SMTP. Add or edit the following lines:

unix_listener /var/spool/postfix/private/auth {
  mode = 0666
  user = postfix
  group = postfix
}

We're done with configuring dovecot. Time to move on to configuring postfix! We'll edit /etc/postfix/main.cf and add or edit the following lines:

smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination

By default, postfix tries to use Cyrus SASL, but we want to use Dovecot, which we've already configured with a keytab and host principals. Some email clients don't follow the RFC correctly (*cough* old versions of Outlook and Outlook Express *cough*), so if you need to support them, you can add broken_sasl_auth_clients = yes to this list. smtpd_sasl_path tells postfix to use Unix sockets to communicate, but it's possible to use inet instead if you want to listen on a TCP port. If you decide to use inet, use something like inet:127.0.0.1:someport for smtpd_sasl_path. You'll then have to configure dovecot to listen on that port by replacing the unix_listener section we added to 10-master.conf something that looks like this:

inet_listener {
  port = someport
}

That's all you'll need to get postfix to talk to dovecot for SASL. Go ahead and restart the service. Time to try everything out! The version of Thunderbird shipped with RHEL6 supports Kerberos. (Sadly, Claws Mail does not, though I otherwise adore that program.) If you've already been using Thunderbird, open up your Account Settings and look for Authentication Method under Server Settings. Choose Kerberos/GSSAPI from the drop down list. (I'd hope you were already using SSL/TLS or STARTTLS for Connection Security.) Then under Outgoing Server (SMTP), select your server, click edit, and do the same for its Authentication method drop down list. If you'd previously been using passwords and told Thunderbird to remember them for you, you'll have to delete them from its Password Manager. You'll find the list of saved passwords in Edit -> Preferences. Go to the Security tab, then select Passwords. You'll see a button labeled Saved Passwords.

Try "Get Mail." Look at your tickets with klist as usual. You should see a ticket that looks like pop/mailserver.domain.com@DOMAIN.COM. Try sending a message and look at your tickets again. You should see a ticket that looks like smtp/mailserver.domain.com@DOMAIN.COM. If so, you're done! If not, log into the mail server and look at your maillog to find out where it all went wrong.