I found out today that SSSD doesn’t support multiple domains in the way I thought it did. I thought I could tell SSSD to try Kerberos first, and if it failed, try LDAP. What isn’t documented ANYWHERE, but is instead buried in a mailing list post, is that if you use the same LDAP server/id_provider for both domains, SSSD won’t perform the lookup a second time for the LDAP domain. It just dies horribly and silently (I haven’t found SSSD error messages terribly enlightening). In the same mailing list post, it was suggested that different search bases for Kerberos and LDAP users be used, or that usernames be fully qualified. In the first case, different search bases would allow me to use Kerberos as my primary domain, but fully qualified usernames would only work if Kerberos was the second domain. I can see I have more testing to do. In any case, you can read the relevant post on Fedora Hosted, where the poster explains the reason for this decision. In any case, I’m glad I found the post, because man sssd.conf mentions no such limitations.

SSSD supports LDAP and Kerberos with ease. Once I’d ironed out the wrinkle with multiple domains and shelved that problem for another day, I made much better progress on my testbed machine. Here’s an example sssd.conf that will support a Kerberos domain:

[sssd]
config_file_version = 2
services = nss, pam
domains = kerberos

[nss]
filter_groups = root
filter_users = root
reconnection_retries = 3

[pam]
reconnection_retries = 3

[domain/kerberos]
id_provider = ldap
ldap_uri = ldap://ldapserver.domain.com
ldap_search_base = dc=domain,dc=com
ldap_id_use_start_tls = true

auth_provider = krb5
chpass_provider = krb5
krb5_server = kerberos.domain.com
krb5_realm = DOMAIN.COM
krb5_ccname_template = FILE:%d/krb5cc_%U

The [nss] and [pam] sections are straight out of the default config file. The important line to change right at the top of the config file is domains, which should be set to whatever you’ll call your Kerberos section. I named mine [domain/kerberos] because I’m boring that way, so I used domains = kerberos. You could also have named it [domain/awesome] and it would still work as long as your domains line read domains = awesome. The domain/ prefix is the important part.

Once we’ve pointed domains to look at [domain/kerberos], we’ll set up a few options. First, we’ll set up the id_provider to use ldap. SSSD is smart enough to get some of its information from /etc/openldap/openldap.conf, so although we use TLS, I don’t have to tell SSSD where to find our certificate files. Instead, I give it our URI, the search base, and I set ldap_id_use_start_tls to true to make sure it uses TLS. Now we’ll move on to the Kerberos specific part. First, we set auth_provider and chpass_provider to krb5 to make sure it talks to Kerberos instead of falling back to LDAP for authentication or storing passwords. Then we set krb5_server to our Kerberos server (it can be a comma separated list, if you have backup KDCs on your network). krb5_realm should be self explanatory. I’ve modified krb5_ccname_template to use the simpler form /tmp/krb5cc_someuserid. The default is to create a file with a unique suffix, which isn’t what I wanted.

Once you’re done editing /etc/sssd/sssd.conf, start (or restart) the service and ensure it runs at boot:

# /sbin/service sssd start
# /sbin/chkconfig sssd on

SSSD assumes that you’ve configured pam to use its pam_sss module to make everything possible. If you’re using a default RHEL6 install and haven’t touched /etc/pam.d/password-auth-ac or /etc/pam.d/system-auth-ac, everything should be configured correctly. Check both files to make sure you have an auth, an account, a password, and a session line for pam_sss.so in each.

You should now be able to log into your computer and automatically get a Kerberos ticket, either from the command line or from GDM. You can test this by wiping out your tickets with kdestroy, dropping down to a command line, and logging in. You should be able to run klist and see a ticket.

I’ve already discussed how to set up a KDC in my earlier post on NFSv4 (I also explained how to Kerberize NFS). Let’s talk about setting up other services, such as SSH, LDAP, and Postfix (using Dovecot SASL for authentication). RHEL6 makes this dead drop simple, and many services have GSSAPI support built right in. GSSAPI, in turn, talks Kerberos. You create a few host principals, create a few new keytabs, and add a few lines here and there to configuration files. With a working KDC, most services can be configured in a few minutes at most. Rolling it out to every user and computer on a network will take a little more time, but most of that work can be scripted.

Kerberizing SSH with Host Principals

SSH will need a host principal in the form host/computer.domain.com on every computer running SSHD. As an example, assume I have a computer named Server and a computer named Client. I’ll create a host principal on Server called host/Server.domain.com. This will allow a user logged into Client to SSH into Server. It’ll also allow a user on Client1, Client2, and Server2 to do the same. However, if I want a user on Server to be able to log into Client, I will also have to create the principal host/Client.domain.com.

First, create the principal using kadmin. This can be done from any computer capable of talking to the KDC with an admin principal (my KDC is configured to allow users with the user principal form username/admin to perform administrative actions), or it can be done locally on the KDC without an administrative principal using kadmin.local.

# kadmin -p username/admin
kadmin: addprinc -randkey host/computer1.domain.com

Repeat the addprinc command for each computer you want to SSH into with Kerberos. Now, log into one of these computers and run kadmin again. This time, we’ll use ktadd to add the correct host principal to the local keytab. In this example, we’ll log into Computer1 and add Computer1′s host principal to the local keytab.

# kadmin -p username/admin
kadmin: ktadd host/computer1.domain.com

To configure Computer2, log into Computer2 and repeat the command, substituting Computer2 for Computer1. When you’re done configuring all your computers, each computer will have its own host principal and only its own host principal in its keytab file. You can verify this by running klist -k on each computer. Here is an example output for Computer1 (it’s normal to see multiples, since each one is actually a different encryption key):

# klist -k
   2 host/computer1.domain.com@DOMAIN.COM
   2 host/computer1.domain.com@DOMAIN.COM
   2 host/computer1.domain.com@DOMAIN.COM
   2 host/computer1.domain.com@DOMAIN.COM

The only thing left to do now is to configure /etc/ssh/sshd_config and /etc/ssh/ssh_config on each computer. Find and correct the following lines in /etc/ssh/sshd_config:

GSSAPIAuthentication yes
GSSAPIKeyExchange yes
GSSAPICleanupCredentials yes

After restarting SSHD, test the connection from a different machine with the following command:

ssh -v -K computer1.domain.com

If it worked, you should not be prompted for a password (assuming you have a valid ticket) and it should print out something like the following:

debug1: Next authentication method: gssapi-keyex
debug1: Authentication succeeded (gssapi-keyex).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env XMODIFIERS = @im=none
debug1: Sending env LANG = en_US.utf8
Last login: Tue Jan 31 14:21:21 2012 from computername.domain.com

You should also be able to run klist and see a ticket. If the login worked, but you don’t have a ticket, your KDC was not configured to issue forwardable tickets. This can be fixed. If the login worked, but it says Authentication succeeded (gssapi-with-mic), you’re still using Kerberos, but you’re probably trying to connect to a computer that doesn’t support GSSAPIKeyExchange, which is a relatively new addition to Linux.

You can now either edit /etc/ssh/ssh_config to tell SSH to use Kerberos automatically for every SSH connection or you can edit your personal ~/.ssh/config to enable it for your user account only. The lines and format are the same:

Host *
   GSSAPIAuthentication yes
   GSSAPIKeyExchange yes
   GSSAPIDelegateCredentials yes

To tell SSH to only use these options for local computers on the network (the Kerberos negotiation can add an extra unwanted delay), you can edit the first line to read Host *.domain.com. However, this means you will have to type the FQDN every time you use ssh. Alternatively, you can use the -K flag every time, which would eliminate the need to edit any client configuration files at all.

Kerberizing LDAP with Host Principals

Since Kerberos relies heavily on a user database backend like LDAP, there are a multitude of ways the two services can be configured to interact. You can create your KDC inside an LDAP database, you can store information about Kerberos principals in LDAP user fields, or you can use Kerberos to authenticate LDAP lookups. I'm only going to focus on the last bit there, using ldapsearch as an example to show GSSAPI in action. In RHEL6, ldapsearch tries to use SASL by default (SASL actually supports several mechanisms besides GSSAPI/Kerberos, but we'll configure it to only accept GSSAPI). You can disable SASL by using the -x flag to enable simple authentication, which is what we had been doing prior to Kerberizing our network, but why would anyone want to do that when we could be using glorious Kerberos?

First, we're going to perform the (increasingly familiar) task of creating the host principal for ldap. In this case, we will create one that uses the format ldap/ldapserver.domain.com:

# kadmin -p username/admin
kadmin: addprinc -randkey ldap/ldapserver.domain.com

Log into the ldap server and create a keytab for ldap. We will use an ldap specific keytab in /etc/openldap/ldap.keytab:

# kadmin -p username/admin
kadmin: ktadd -k /etc/openldap/ldap.keytab ldap/ldapserver.domain.com

Change the ownership and permissions for our new keytab:

# chown ldap:ldap /etc/openldap/ldap.keytab
# chmod 640 /etc/openldap/ldap.keytab

We're halfway there. Now we have to configure slapd to use GSSAPI. OpenLDAP has migrated to using a directory structure for slapd configuration. I prefer to edit something more human readable, though, so I edit /etc/openldap/slapd.conf and generate a new /etc/openldap/slapd.d only as necessary. Add or correct the following lines in /etc/openldap/slapd.conf:

sasl-secprops noanonymous,noplain,noactive
sasl-regexp uid=([^/]*),cn=GSSAPI,cn=auth uid=$1,ou=people,dc=domain,dc=com

The first line will prevent SASL from advertising undesired methods such as DIGEST-MD5. The second line will need to be configured to match your environment. When you run ldapsearch with SASL, it sends the user id to slapd in the form uid=username,cn=GSSAPI,cn=auth. This will map it into the form uid=username,ou=people,dc=domain,dc=com, which is the structure most ldap servers use. We throw away anything after the user id with a wildcard, so a principal that looks like username/admin will work just as well as username.

If you have ACLs set up, here is a way to add an access line for Kerberos administrators, shown alongside a more typical access line, assuming you use the syntax username/admin:

access to *
  by dn.regex="uid=.*/admin,cn=GSSAPI,cn=auth" write
  by group/groupOfUniqueNames/uniqueMember="cn=ldapadmins,ou=groups,dc=domain,dc=com" write
  by * read

Here you see the same cn=GSSAPI,cn=auth suffix from the sasl-regexp line, but now we're specifically looking for the /admin suffix after the user id.

The only thing left to do is to generate the new /etc/openldap/slapd.d configuration folder and restart slapd. If you were planning to store Kerberos information in LDAP, this would be an ideal time to add a Kerberos schema. I use the following commands to do the generation and restarting, although I usually paste it on one line, separating each command with semicolons:

# service slapd stop
# rm -rf /etc/openldap/slapd.d/
# mkdir slapd.d
# slaptest -f /etc/openldap/slapd.conf -F /etc/openldap/slapd.d/
# chown -R ldap:ldap /etc/openldap/slapd.d/
# chmod -R go-rwx /etc/openldap/slapd.d/
# service slapd start

To test if Kerberos support is working, you should be able to run ldapsearch with GSSAPI SASL. Try a command like the following:

# ldapsearch uid=username

You should get something that looks like this, followed by the information about the username you're querying:

SASL/GSSAPI authentication started
SASL username: user@DOMAIN.COM
SASL SSF: 56
SASL data security layer installed.
# extended LDIF
#
# LDAPv3
# base  (default) with scope subtree
# filter: uid=username
# requesting: ALL
#

You can check to see if you got a ticket with klist. You should see the host principal ldap/ldapserver.domain.com. If so, you have successfully Kerberized ldap!

Kerberizing Dovecot and Postfix with Host Principals

Dovecot, which is what we use for POP3, happily supports GSSAPI. We're going to give Dovecot its own keytab file, just like we did for LDAP. We'll create host principals with pop/ and smtp/ prefixes. (IMAP users should create a host principal that begins with imap/ instead of pop/.)

# kadmin -p username/admin
kadmin: addprinc -randkey pop/mailserver.domain.com
kadmin: addprinc -randkey smtp/mailserver.domain.com

On the mail server:

# kadmin -p username/admin
kadmin: ktadd -k /etc/dovecot/dovecot.keytab pop/mailserver.domain.com
kadmin: ktadd -k /etc/dovecot/dovecot.keytab smtp/mailserver.domain.com

Change the ownership and permissions for our new keytab:

# chown dovecot /etc/dovecot/dovecot.keytab
# chmod 640 /etc/dovecot/dovecot.keytab

Now we'll configure dovecot to use this keytab and use GSSAPI for user authentication. Edit /etc/dovecot/conf.d/10-auth.conf (or /etc/dovecot/dovecot.conf if 10-auth.conf does not exist) and add or edit the following lines:

auth_gssapi_hostname = mailserver.domain.com
auth_krb5_keytab = /etc/dovecot/dovecot.keytab
auth_mechanisms = gssapi

The value you choose for auth_gssapi_hostname here should match whatever you used in the host principals you created. If you need to support non-Kerberos users, auth_mechanisms will support several options. Here's what you would need to support both Kerberos principals and users with regular shadow passwords:

auth_mechanisms = plain gssapi

Next, we'll edit /etc/dovecot/10-master.conf to add support for authenticating postfix SMTP. Add or edit the following lines:

unix_listener /var/spool/postfix/private/auth {
  mode = 0666
  user = postfix
  group = postfix
}

We're done with configuring dovecot. Time to move on to configuring postfix! We'll edit /etc/postfix/main.cf and add or edit the following lines:

smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination

By default, postfix tries to use Cyrus SASL, but we want to use Dovecot, which we've already configured with a keytab and host principals. Some email clients don't follow the RFC correctly (*cough* old versions of Outlook and Outlook Express *cough*), so if you need to support them, you can add broken_sasl_auth_clients = yes to this list. smtpd_sasl_path tells postfix to use Unix sockets to communicate, but it's possible to use inet instead if you want to listen on a TCP port. If you decide to use inet, use something like inet:127.0.0.1:someport for smtpd_sasl_path. You'll then have to configure dovecot to listen on that port by replacing the unix_listener section we added to 10-master.conf something that looks like this:

inet_listener {
  port = someport
}

That's all you'll need to get postfix to talk to dovecot for SASL. Go ahead and restart the service. Time to try everything out! The version of Thunderbird shipped with RHEL6 supports Kerberos. (Sadly, Claws Mail does not, though I otherwise adore that program.) If you've already been using Thunderbird, open up your Account Settings and look for Authentication Method under Server Settings. Choose Kerberos/GSSAPI from the drop down list. (I'd hope you were already using SSL/TLS or STARTTLS for Connection Security.) Then under Outgoing Server (SMTP), select your server, click edit, and do the same for its Authentication method drop down list. If you'd previously been using passwords and told Thunderbird to remember them for you, you'll have to delete them from its Password Manager. You'll find the list of saved passwords in Edit -> Preferences. Go to the Security tab, then select Passwords. You'll see a button labeled Saved Passwords.

Try "Get Mail." Look at your tickets with klist as usual. You should see a ticket that looks like pop/mailserver.domain.com@DOMAIN.COM. Try sending a message and look at your tickets again. You should see a ticket that looks like smtp/mailserver.domain.com@DOMAIN.COM. If so, you're done! If not, log into the mail server and look at your maillog to find out where it all went wrong.

I’ll often install a brand new Mac with OS X 10.6, run puppet, and find myself facing the same two intermittent problems. First, I’ll notice that the SSH host keys in /etc never get created (we turn on Remote Login through puppet, so it’s possible we’re accidentally skipping a step). Second, System Preferences will abruptly decide to no longer accept my administrator password, even though Terminal will happily let me use it to run commands with sudo.

The solution to the first problem is easy: I run /usr/libexec/sshd-keygen-wrapper with sudo. It’s a very simple shell script that checks for the presence of /etc/ssh_host_key, /etc/ssh_host_rsa_key and /etc/ssh_host_dsa_key. It’s unclear why this command doesn’t always get automatically run.

I’m still trying to perfect the solution to my second problem, but wiping out the problematic administrator user’s Library and logging back in fixes it. When it’s a brand new install, this is a fairly safe move, and since we tell our users to log in as regular users, NOT as admins, there’s no chance that this will wipe out any important user settings. It seems to be a user preference corruption error. In the most recent iteration, I ruled out ~/Library/Caches and ~/Library/Preferences before I got bored and wiped the entire directory out again. Each time, I manage to cross a few more folders off the list. As for the cause of the problem, it may be related to MCX settings. We push a small glob of MCX settings out through puppet, and the problem only seems to occur after they’re installed.

I manage a teeny tiny network of computers, mostly running RHEL 6. I decided I would be clever and get Kerberos support working. I’m not done (SSSD is getting me down, so no automatic ticket renewals yet, and I want to support 10.7 macs), but I’m getting close to a fully working system. It took a few weeks of what felt like banging my head against a wall because the documentation for Kerberos is just terrible because it makes too many assumptions.

Right now, I have SSH and NFS4 working with Kerberos. I even have Kerberized NFS working with AES 256, something that used to be impossible. In fact, most documentation on the Internet still tells users to use the ancient and insecure DES. Don’t do this unless you absolutely positively have to support creaky old computers.

You will need a Kerberos server, an NFS server, and an NFS client machine. My Kerberos server is also an LDAP server, but you can get away with local user accounts as long as they’re the same on every system. We’re not doing Active Directory here! Some (all) of these services can be running on the same machine, but that would be pretty strange. In a very small network, perhaps you have two computers, the server running all the necessary services and a client that wants to use those services. For my own sanity, let’s assume iptables is NOT running, but there are only a few ports you need (TCP/UDP 88, 464, and 749 work for me). I’m also not going to talk about setting up LDAP, but honestly, LDAP is MUCH better documented.

Setting Up a KDC

First, we’ll set up Kerberos and install the necessary packages. I borrowed liberally from RHEL’s own documentation, because that’s what I used the first time I set up my own server. Some of these packages might pull in the others automatically. You’re welcome to experiment.

On the Kerberos server (you can skip krb5-server-ldap if you’re not doing LDAP):

# yum install krb5-libs krb5-server krb5-server-ldap krb5-auth-dialog

On the client:

# yum install krb5-libs krb5-workstation krb5-auth-dialog

On the Kerberos server, we’ll need to edit /var/kerberos/krb5kdc/kdc.conf and /etc/krb5.conf. (Clients will only need to edit /etc/krb5.conf.) Here’s my kdc.conf:

[kdcdefaults]
 kdc_ports = 88
 kdc_tcp_ports = 88

[realms]
DOMAIN.COM = {
 acl_file = /var/kerberos/krb5kdc/kadm5.acl
 dict_file = /usr/share/dict/words
 admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
 supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal
}

The KDC is the Key Distribution Center. It’s the service that will run on the Kerberos server and hand out tickets. Here we set the port to 88 (which is the Kerberos 5 default). If you don’t specify a TCP port, it’ll only use UDP connections. The acl_file will have a list of principals (users, in Kerberos speak) that have special Kerberos privileges and it will list what those principals are allowed to do. The admin_keytab file is where Kerberos principals will store their credentials by default. My supported_enctypes will look different from the default RHEL 6 file because I deleted all the really terrible encryption types you don’t need and don’t want.

Here’s /etc/krb5.conf. It can be exactly the same on both clients and servers:

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 default_realm = DOMAIN.COM
 dns_lookup_realm = false
 dns_lookup_kdc = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true

[realms]
 DOMAIN.COM = {
  kdc = kerberos.domain.com
  admin_server = kerberos.domain.com
  default_domain = domain.com
 }

[domain_realm]
 .domain.com = DOMAIN.COM
 domain.com = DOMAIN.COM

You don’t have to capitalize things, but you should since it makes the file more readable. The [logging] section is boring until things start to go wrong. Most client Kerberos logging is actually going to be through the logs for the programs you are trying to “kerberize,” so the logging section here is just for the server. The options in [libdefaults] are all documented in man krb5.conf(5) and can be adjusted later. The forwardable option is important for SSH and some other services. In [realms], I’ve set things to use the FQDN kerberos.domain.com. If your Kerberos server is called binky, you can make this binky.domain.com. There’s no special meaning behind the domain names used here.

The last file you’ll need to edit directly is kadm5.acl. Here’s mine:

*/admin@DOMAIN.COM *

This means that any principal that looks like username/admin (*/admin) will be allowed to do anything they want (the * at the end of the line) when using kadmin. kadmin is the program used to create principals, set password policy, write credentials into various keytabs, and some other administrative nonsense. When you’re logged into the Kerberos server, you can use kadmin.local as root, but if you want to use kadmin from a client machine, you can make yourself a Kerberos admin principal without having to create a root user principal. You can also add individual users and limit their permissions to specific tasks. Read man kadmind(5) and scroll down to the part about ACL FILE SYNTAX.

The last thing to do before firing up the Kerberos server is to create the Kerberos database that will store its keys.

# /usr/sbin/kdb5_util create -s

It’ll ask you to pick a master password. Pick something strong, like a Diceware password. You might never have to type this password ever again. If you forget it, you’ll lose your user database and have to start over, so feel free to write this one down and store it somewhere safe. When you’re done, it’s time to start up the KDC and the kadmin backend.

# /sbin/service krb5kdc start
# /sbin/service kadmin start
# /sbin/chkconfig --add krb5kdc
# /sbin/chkconfig --add kadmin

Test it by running kadmin:

# /usr/sbin/kadmin.local

You’ll get a prompt that looks like this:

kadmin.local:

You can type ? at any time to get a list of options. If you type a command without any options, it will print out a normal usage message like it would for any shell command. To add a principal (probably your regular user account), we can do the following:

kadmin.local: addprinc myuseraccount

It’ll prompt you to create a password. You can make it the same as your existing password or something else. To make an admin account, assuming you used the same kadm5.acl file I did, add an admin principal:

kadmin.local: addprinc myuseraccount/admin

This step only has to be done once for each principal. On each computer, you can add the principals’ keys to the local keytab to avoid typing a password, but this is optional. Right now, there’s only the server. Here’s how you’d do it:

kadmin.local: ktadd myuseraccount

Now it’s time to configure the client. Log into the client and copy or retype /etc/krb5.conf. We’ll test to see if our kadmin backend is working properly by using kadmin to log in as our admin principal. kadmin can be run as a regular user and that user would still be able to perform administrative Kerberos functions, but if you will need to edit keytab files or have logging work properly, you will have to run it with sudo or as root. If you did run it as a regular user, you wouldn’t need to specify the principal, since it’s been configured to automatically try appending /admin to your local user name.

# kadmin -p myuseraccount/admin

If that worked, the next step is to see if we can get a ticket from the server with kinit. If your local user account and Kerberos principal have the same name, you won’t have to specify a principal name. If they’re different, you’d just have to append the principal name after the command. Run kinit to get a ticket and type your password when requested.

# kinit

To see if you got a ticket, run:

# klist

If you see a line with some time and date information that ends in krbtgt/DOMAIN.COM@DOMAIN.COM, you win! Well, you have a working KDC server anyway. If it didn’t work, try disabling any firewalls and making sure your DNS (or /etc/hosts) is set up correctly. If you told /etc/krb5.conf to try talking to kerberos.domain.com, make sure you can talk to kerberos.domain.com (nslookup kerberos.domain.com should spit out an ip address).

Adding Host Principals/Kerberizing Services

Usually, normal user principals don’t use a keytab and are prompted for a password. Obviously, services can’t keep stopping to ask for passwords every time they need a new ticket. To kerberize a service, you need a host principal. They’re actually just normal Kerberos principals that are stored in keytabs throughout your network, but they follow a special naming convention that varies by which service you’re trying to kerberize. However, since you’ll never need to log in as a host principal manually, you don’t even have to assign them passwords. You’ll use -randkey instead to use a random value. From the Kerberos server, I’ll create a server and client principal for each service I want to kerberize. Here is an example for SSH:

kadmin: addprinc -randkey host/kerberos.domain.com
kadmin: addprinc -randkey host/nfsserver.domain.com
kadmin: addprinc -randkey host/clientmachine.domain.com

SSH needs principals that start with host/ and look like host/computername.domain.com. You can force it to use something else, but there’s no real point. NFS is similar. It looks for principals that start with nfs/ and look like nfs/computername.domain.com. You can allow NFS to use host/ principals (in fact, if you omit nfs/clientmachine.domain.com, it will actually fall back to host/clientmachine.domain.com if it’s available), but I believe in separation of services and that means separation of credentials.

kadmin: addprinc -randkey nfs/kerberos.domain.com
kadmin: addprinc -randkey nfs/nfsserver.domain.com
kadmin: addprinc -randkey nfs/clientmachine.domain.com

Now that you’ve created all these principals, they need to be added to keytabs. Each computer should have its own principals stashed in its keytab. So from the NFS server, run the following to add both an SSH and an NFS principal for nfsserver.domain.com:

kadmin: ktadd host/nfsserver.domain.com
kadmin: ktadd nfs/nfsserver.domain.com

And on the kerberos server, you would run this (assuming the Kerberos server is on a different machine and will be mounting or sharing NFS shares):

kadmin: ktadd host/kerberos.domain.com
kadmin: ktadd nfs/kerberos.domain.com

And from the client machine that wants to use NFS (and presumably also allow users to log in over SSH):

kadmin: ktadd host/clientmachine.domain.com
kadmin: ktadd nfs/clientmachine.domain.com

If this seems unnecessarily wordy, it’s because this was a stumbling block for me and not well documented. At this point, you could get Kerberized SSH working pretty quickly. I’ll document that later (all you need is a few changes to /etc/ssh/sshd_config and /etc/ssh/ssh_config and you are good to go).

Setting Up NFS4 Exports

If you haven’t used NFS before, it’s pretty simple. You install the right server packages, create a file called /etc/exports, and tell /etc/exports which file systems you want to share with users. NFSv4 works the same way, but there are a few new rules to follow and the format for /etc/exports is a little different. Here’s an example of a simple /etc/exports file for NFSv4 that will export a file system (/nfsexports/sharedfiles) that supports multiple users and preserves file permissions:

/nfsexports *.domain.com(ro,fsid=0,insecure)
/nfsexports/sharedfiles *.domain.com(rw,nohide,insecure)

When we actually mount the filesystem, we’ll leave off /nfsexports entirely (mount -t nfs4 nfsserver.domain.com:/sharedfiles, not mount -t nfs4 nfsserver.domain.com:/nfsexports/sharedfiles). That’s because /nfsexports is a pseudo filesystem. If you’ve used sandboxes or chroot jails before, it’s the same concept. Root squashing is turned on by default. If you want root to be able to access this filesystem, you should add the option no_root_squash inside the parentheses on both lines. The option insecure is actually a flag that tells NFS we want to use ports in the “insecure” range. That is, we want to use ports numbered 1024-65535. Read man exports(5) for more options. Anytime you edit /etc/exports, you will have to tell NFS to reread it by running /usr/sbin/exportfs -r.

The last exports file did not use Kerberos. To tell NFS we want to use Kerberos, we add a sec option.

/nfsexports *.domain.com(ro,fsid=0,insecure,sec=krb5)
/nfsexports/sharedfiles *.domain.com(rw,nohide,insecure,sec=krb5)

krb5 is the most basic Kerberos option. Other options are krb5i and krb5p. krb5i and krb5p add additional security features at the cost of speed (krb5i is only a little bit slower, but krb5p can be twice as slow). man nfs(5) has more details. We can also tell Kerberos that we want to support all options, including the default, which is not to use Kerberos (sys):

/nfsexports *.domain.com(ro,fsid=0,insecure,sec=krb5p:krb5i:krb5:sys)
/nfsexports/sharedfiles *.domain.com(rw,nohide,insecure,sec=krb5p:krb5i:krb5:sys)

Now all flavors of Kerberos are possible and users without Kerberos principals can still access directories. You can go a step further, and set up custom configurations for each sec option. For example, you can allow krb5p users to read and write while restricting sys users to read only.

Turn On NFS4 and GSSD on the Server

Settings for the NFS service are stored in /etc/sysconfig/nfs. On the NFS server, open it and make sure SECURE_NFS is set to yes. This will tell the nfs service to start up the GSS service daemon automagically. If you’ll need the idmap service (new in NFS4 and not always required), make sure /etc/idmapd.conf has Domain set to your domain. I use idmapd to map LDAP user and group ids to human readable names and groups using nsswitch (nsswitch has likewise been configured to use SSSD/LDAP).

Then, start idmapd and all the various NFS services:

# /sbin/service nfs start
# /sbin/service nfslock start
# /sbin/service rpcidmapd start
# /sbin/chkconfig nfs on
# /sbin/chkconfig nfslock on
# /sbin/chkconfig rpcidmapd on

Turn On NFS4 and GSSD on the Client

Start idmapd (if necessary) and GSS:

# /sbin/service nfslock start
# /sbin/service rpcgssd start
# /sbin/service rpcidmapd start
# /sbin/chkconfig nfslock on
# /sbin/chkconfig rpcgssd on
# /sbin/chkconfig rpcidmapd on

If you haven’t done so recently, rerun kinit and try mounting the filesystem from the command line as follows:

# mount -t nfs4 -o sec=krb5 nfsserver.domain.com:/sharedfiles /mnt

If it works, huzzah! You can add a line to /etc/fstab if you like to make sure it mounts on boot. The system is able to mount the filesystem unattended because we used host principals and keytabs. Your user must use kinit to access the filesystem.

If it didn’t work, time to troubleshoot.

Troubleshooting

1. Try running mount without “-o sec=krb5″ (this assumes you enabled a fallback like sec=krb5:sys). If it fails, you probably have a syntax error in /etc/exports or you forgot to (re)run /usr/sbin/exportfs -rv after editing the file.
2. Run /usr/sbin/exportfs -v on the server and make sure /nfsexports/sharedfiles shows up and that sec=krb5 shows up somewhere in the listed options. If it’s not there, add it and rerun /usr/sbin/exportfs -rv.
3. On the NFS server, run sudo klist -k /etc/krb5.keytab and make sure nfs/nfsserver.domain.com shows up (probably more than once, since there will be a line for each supported encryption key). Run the same command on the client machine and make sure nfs/clientmachine.domain.com shows up there at least once.
4. Run klist as your regular user account and make sure your user’s ticket hasn’t expired. kinit will get you a new ticket, kinit -R will renew an expired one. On my system, I run kinit as a normal user and mount with sudo. sudo is smart enough to use my regular user’s tickets. It doesn’t try to use a ticket for root.

I knit continental style and HATE purling through the back loop. I only just discovered the secret to twist stitch harmony and wish to record it for all posterity.

PLT/LPT: Insert the right needle into the back of the second stitch from the end of the left needle. Now, pull the left needle out of both stitches ALA cabling without a cable needle and reinsert the left needle into the loose stitch (that isn’t currently sitting on the right needle). Pass the stitch on the right needle back onto the left needle, p1, k1 tbl. To perform a normal LT, knit both stitches through the back loop instead.

PRT/RPT: Slip the next two stitches purlwise (you can keep the yarn in back, but slip as if to purl, i.e., don’t twist the stitches). From the back of the work, slip the left needle into the back of the first stitch you slipped and drop both slipped stitches off the right needle. Catch the loose stitch with the right needle and pass it onto the left needle. Then, p1 tbl, k1. For a normal RT, knit both stitches through the back loop instead.

I just had to have it. A solar powered keyboard. I know what you’re thinking, but it can charge itself off indoor lighting. It’s called the Logitech K750 and it is officially rocking my world right now. It’s 2.4ghz wireless. It’s the thinnest keyboard I have ever owned. Did I mention it’s wireless?

http://www.logitech.com/en-us/keyboards/keyboard/devices/k750-keyboard

It’s the perfect fit for my shiny new Windows 7 multimedia PC.

But getting back to the multimedia PC… Just about every computer geek out there keeps some spare computer parts around the place. Sometimes we’ll even scavenge enough parts to build a spare computer, a slightly creakier computer than whatever it is we’re running, because let’s face it, if the parts were any good, they’d be in your main machine already. I had the usual oddball assortment, nothing at all current, and most of it unsuitable for a multimedia PC. That’s where my dad comes in.

See, my dad is trying to attain homebrew DVR nirvana. He’s gone through about five completely separate iterations and is currently bogged down in hard drive death hell trying to get Windows 7 Media Center up and running. He’s really determined this time, too, since he actually bought two XBOX 360′s to make it happen. It can attempt and fail to record four shows at once with record speed. The craziest part is that he already owns a Tivo, and if all he wanted was HD support, he could have bought three Tivo HDs by now. Reinstalling Windows 7 is my dad’s latest hobby apparently. So when I mention to him casually that it would be nice to have a multimedia PC again (my friend Isaac let me use his when we were roommates), it’s no surprise that he gets very excited. He tells me that he should have some spare parts we can use. I think, OK, I have some parts already, so I probably only need a few more things and I’m set.

As it turned out, my dad had enough spare computer parts sitting around his workroom that I was able to build an entire computer from scratch. And it’s faster than my current computer.

So here’s to you, Dad, and your crazy DVR dreams (but seriously, next time just buy a TIVO HD).

This is a photograph of my grandmother from when she was just a little younger than I am now.

Some of you may have seen my frantic post on Facebook about my missing Threadless package. Well, I found it, and you’ll never believe where.

This shrub looks perfectly innocent, right?

Up close, still looking good…

But wait, what’s this?

Did they chuck it into the shrub? This is a really sturdy plant, by the way. I was able to rest my camera on it, and the camera stayed upright. You’d have to either hit that gap exactly or apply some kind of force.

I’ve been passing this sign for months now, maybe longer, and I finally decided it was time to get some photographic evidence. Don’t ask me about the “defence” misspelling. Perhaps they’re Canadian expatriates?

And they even take Discover!

Last weekend, my mother delivered a belated birthday present: she reupholstered my couch cushions! In typical cat fashion, Sasha snuck into the shot. I didn’t notice her until after I uploaded the photo to my computer.