I manage a teeny tiny network of computers, mostly running RHEL 6. I decided I would be clever and get Kerberos support working. I’m not done (SSSD is getting me down, so no automatic ticket renewals yet, and I want to support 10.7 macs), but I’m getting close to a fully working system. It took a few weeks of what felt like banging my head against a wall because the documentation for Kerberos is just terrible because it makes too many assumptions.

Right now, I have SSH and NFS4 working with Kerberos. I even have Kerberized NFS working with AES 256, something that used to be impossible. In fact, most documentation on the Internet still tells users to use the ancient and insecure DES. Don’t do this unless you absolutely positively have to support creaky old computers.

You will need a Kerberos server, an NFS server, and an NFS client machine. My Kerberos server is also an LDAP server, but you can get away with local user accounts as long as they’re the same on every system. We’re not doing Active Directory here! Some (all) of these services can be running on the same machine, but that would be pretty strange. In a very small network, perhaps you have two computers, the server running all the necessary services and a client that wants to use those services. For my own sanity, let’s assume iptables is NOT running, but there are only a few ports you need (TCP/UDP 88, 464, and 749 work for me). I’m also not going to talk about setting up LDAP, but honestly, LDAP is MUCH better documented.

Setting Up a KDC

First, we’ll set up Kerberos and install the necessary packages. I borrowed liberally from RHEL’s own documentation, because that’s what I used the first time I set up my own server. Some of these packages might pull in the others automatically. You’re welcome to experiment.

On the Kerberos server (you can skip krb5-server-ldap if you’re not doing LDAP):

# yum install krb5-libs krb5-server krb5-server-ldap krb5-auth-dialog

On the client:

# yum install krb5-libs krb5-workstation krb5-auth-dialog

On the Kerberos server, we’ll need to edit /var/kerberos/krb5kdc/kdc.conf and /etc/krb5.conf. (Clients will only need to edit /etc/krb5.conf.) Here’s my kdc.conf:

[kdcdefaults]
 kdc_ports = 88
 kdc_tcp_ports = 88

[realms]
DOMAIN.COM = {
 acl_file = /var/kerberos/krb5kdc/kadm5.acl
 dict_file = /usr/share/dict/words
 admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
 supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal
}

The KDC is the Key Distribution Center. It’s the service that will run on the Kerberos server and hand out tickets. Here we set the port to 88 (which is the Kerberos 5 default). If you don’t specify a TCP port, it’ll only use UDP connections. The acl_file will have a list of principals (users, in Kerberos speak) that have special Kerberos privileges and it will list what those principals are allowed to do. The admin_keytab file is where Kerberos principals will store their credentials by default. My supported_enctypes will look different from the default RHEL 6 file because I deleted all the really terrible encryption types you don’t need and don’t want.

Here’s /etc/krb5.conf. It can be exactly the same on both clients and servers:

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 default_realm = DOMAIN.COM
 dns_lookup_realm = false
 dns_lookup_kdc = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true

[realms]
 DOMAIN.COM = {
  kdc = kerberos.domain.com
  admin_server = kerberos.domain.com
  default_domain = domain.com
 }

[domain_realm]
 .domain.com = DOMAIN.COM
 domain.com = DOMAIN.COM

You don’t have to capitalize things, but you should since it makes the file more readable. The [logging] section is boring until things start to go wrong. Most client Kerberos logging is actually going to be through the logs for the programs you are trying to “kerberize,” so the logging section here is just for the server. The options in [libdefaults] are all documented in man krb5.conf(5) and can be adjusted later. The forwardable option is important for SSH and some other services. In [realms], I’ve set things to use the FQDN kerberos.domain.com. If your Kerberos server is called binky, you can make this binky.domain.com. There’s no special meaning behind the domain names used here.

The last file you’ll need to edit directly is kadm5.acl. Here’s mine:

*/admin@DOMAIN.COM *

This means that any principal that looks like username/admin (*/admin) will be allowed to do anything they want (the * at the end of the line) when using kadmin. kadmin is the program used to create principals, set password policy, write credentials into various keytabs, and some other administrative nonsense. When you’re logged into the Kerberos server, you can use kadmin.local as root, but if you want to use kadmin from a client machine, you can make yourself a Kerberos admin principal without having to create a root user principal. You can also add individual users and limit their permissions to specific tasks. Read man kadmind(5) and scroll down to the part about ACL FILE SYNTAX.

The last thing to do before firing up the Kerberos server is to create the Kerberos database that will store its keys.

# /usr/sbin/kdb5_util create -s

It’ll ask you to pick a master password. Pick something strong, like a Diceware password. You might never have to type this password ever again. If you forget it, you’ll lose your user database and have to start over, so feel free to write this one down and store it somewhere safe. When you’re done, it’s time to start up the KDC and the kadmin backend.

# /sbin/service krb5kdc start
# /sbin/service kadmin start
# /sbin/chkconfig --add krb5kdc
# /sbin/chkconfig --add kadmin

Test it by running kadmin:

# /usr/sbin/kadmin.local

You’ll get a prompt that looks like this:

kadmin.local:

You can type ? at any time to get a list of options. If you type a command without any options, it will print out a normal usage message like it would for any shell command. To add a principal (probably your regular user account), we can do the following:

kadmin.local: addprinc myuseraccount

It’ll prompt you to create a password. You can make it the same as your existing password or something else. To make an admin account, assuming you used the same kadm5.acl file I did, add an admin principal:

kadmin.local: addprinc myuseraccount/admin

This step only has to be done once for each principal. On each computer, you can add the principals’ keys to the local keytab to avoid typing a password, but this is optional. Right now, there’s only the server. Here’s how you’d do it:

kadmin.local: ktadd myuseraccount

Now it’s time to configure the client. Log into the client and copy or retype /etc/krb5.conf. We’ll test to see if our kadmin backend is working properly by using kadmin to log in as our admin principal. kadmin can be run as a regular user and that user would still be able to perform administrative Kerberos functions, but if you will need to edit keytab files or have logging work properly, you will have to run it with sudo or as root. If you did run it as a regular user, you wouldn’t need to specify the principal, since it’s been configured to automatically try appending /admin to your local user name.

# kadmin -p myuseraccount/admin

If that worked, the next step is to see if we can get a ticket from the server with kinit. If your local user account and Kerberos principal have the same name, you won’t have to specify a principal name. If they’re different, you’d just have to append the principal name after the command. Run kinit to get a ticket and type your password when requested.

# kinit

To see if you got a ticket, run:

# klist

If you see a line with some time and date information that ends in krbtgt/DOMAIN.COM@DOMAIN.COM, you win! Well, you have a working KDC server anyway. If it didn’t work, try disabling any firewalls and making sure your DNS (or /etc/hosts) is set up correctly. If you told /etc/krb5.conf to try talking to kerberos.domain.com, make sure you can talk to kerberos.domain.com (nslookup kerberos.domain.com should spit out an ip address).

Adding Host Principals/Kerberizing Services

Usually, normal user principals don’t use a keytab and are prompted for a password. Obviously, services can’t keep stopping to ask for passwords every time they need a new ticket. To kerberize a service, you need a host principal. They’re actually just normal Kerberos principals that are stored in keytabs throughout your network, but they follow a special naming convention that varies by which service you’re trying to kerberize. However, since you’ll never need to log in as a host principal manually, you don’t even have to assign them passwords. You’ll use -randkey instead to use a random value. From the Kerberos server, I’ll create a server and client principal for each service I want to kerberize. Here is an example for SSH:

kadmin: addprinc -randkey host/kerberos.domain.com
kadmin: addprinc -randkey host/nfsserver.domain.com
kadmin: addprinc -randkey host/clientmachine.domain.com

SSH needs principals that start with host/ and look like host/computername.domain.com. You can force it to use something else, but there’s no real point. NFS is similar. It looks for principals that start with nfs/ and look like nfs/computername.domain.com. You can allow NFS to use host/ principals (in fact, if you omit nfs/clientmachine.domain.com, it will actually fall back to host/clientmachine.domain.com if it’s available), but I believe in separation of services and that means separation of credentials.

kadmin: addprinc -randkey nfs/kerberos.domain.com
kadmin: addprinc -randkey nfs/nfsserver.domain.com
kadmin: addprinc -randkey nfs/clientmachine.domain.com

Now that you’ve created all these principals, they need to be added to keytabs. Each computer should have its own principals stashed in its keytab. So from the NFS server, run the following to add both an SSH and an NFS principal for nfsserver.domain.com:

kadmin: ktadd host/nfsserver.domain.com
kadmin: ktadd nfs/nfsserver.domain.com

And on the kerberos server, you would run this (assuming the Kerberos server is on a different machine and will be mounting or sharing NFS shares):

kadmin: ktadd host/kerberos.domain.com
kadmin: ktadd nfs/kerberos.domain.com

And from the client machine that wants to use NFS (and presumably also allow users to log in over SSH):

kadmin: ktadd host/clientmachine.domain.com
kadmin: ktadd nfs/clientmachine.domain.com

If this seems unnecessarily wordy, it’s because this was a stumbling block for me and not well documented. At this point, you could get Kerberized SSH working pretty quickly. I’ll document that later (all you need is a few changes to /etc/ssh/sshd_config and /etc/ssh/ssh_config and you are good to go).

Setting Up NFS4 Exports

If you haven’t used NFS before, it’s pretty simple. You install the right server packages, create a file called /etc/exports, and tell /etc/exports which file systems you want to share with users. NFSv4 works the same way, but there are a few new rules to follow and the format for /etc/exports is a little different. Here’s an example of a simple /etc/exports file for NFSv4 that will export a file system (/nfsexports/sharedfiles) that supports multiple users and preserves file permissions:

/nfsexports *.domain.com(ro,fsid=0,insecure)
/nfsexports/sharedfiles *.domain.com(rw,nohide,insecure)

When we actually mount the filesystem, we’ll leave off /nfsexports entirely (mount -t nfs4 nfsserver.domain.com:/sharedfiles, not mount -t nfs4 nfsserver.domain.com:/nfsexports/sharedfiles). That’s because /nfsexports is a pseudo filesystem. If you’ve used sandboxes or chroot jails before, it’s the same concept. Root squashing is turned on by default. If you want root to be able to access this filesystem, you should add the option no_root_squash inside the parentheses on both lines. The option insecure is actually a flag that tells NFS we want to use ports in the “insecure” range. That is, we want to use ports numbered 1024-65535. Read man exports(5) for more options. Anytime you edit /etc/exports, you will have to tell NFS to reread it by running /usr/sbin/exportfs -r.

The last exports file did not use Kerberos. To tell NFS we want to use Kerberos, we add a sec option.

/nfsexports *.domain.com(ro,fsid=0,insecure,sec=krb5)
/nfsexports/sharedfiles *.domain.com(rw,nohide,insecure,sec=krb5)

krb5 is the most basic Kerberos option. Other options are krb5i and krb5p. krb5i and krb5p add additional security features at the cost of speed (krb5i is only a little bit slower, but krb5p can be twice as slow). man nfs(5) has more details. We can also tell Kerberos that we want to support all options, including the default, which is not to use Kerberos (sys):

/nfsexports *.domain.com(ro,fsid=0,insecure,sec=krb5p:krb5i:krb5:sys)
/nfsexports/sharedfiles *.domain.com(rw,nohide,insecure,sec=krb5p:krb5i:krb5:sys)

Now all flavors of Kerberos are possible and users without Kerberos principals can still access directories. You can go a step further, and set up custom configurations for each sec option. For example, you can allow krb5p users to read and write while restricting sys users to read only.

Turn On NFS4 and GSSD on the Server

Settings for the NFS service are stored in /etc/sysconfig/nfs. On the NFS server, open it and make sure SECURE_NFS is set to yes. This will tell the nfs service to start up the GSS service daemon automagically. If you’ll need the idmap service (new in NFS4 and not always required), make sure /etc/idmapd.conf has Domain set to your domain. I use idmapd to map LDAP user and group ids to human readable names and groups using nsswitch (nsswitch has likewise been configured to use SSSD/LDAP).

Then, start idmapd and all the various NFS services:

# /sbin/service nfs start
# /sbin/service nfslock start
# /sbin/service rpcidmapd start
# /sbin/chkconfig nfs on
# /sbin/chkconfig nfslock on
# /sbin/chkconfig rpcidmapd on

Turn On NFS4 and GSSD on the Client

Start idmapd (if necessary) and GSS:

# /sbin/service nfslock start
# /sbin/service rpcgssd start
# /sbin/service rpcidmapd start
# /sbin/chkconfig nfslock on
# /sbin/chkconfig rpcgssd on
# /sbin/chkconfig rpcidmapd on

If you haven’t done so recently, rerun kinit and try mounting the filesystem from the command line as follows:

# mount -t nfs4 -o sec=krb5 nfsserver.domain.com:/sharedfiles /mnt

If it works, huzzah! You can add a line to /etc/fstab if you like to make sure it mounts on boot. The system is able to mount the filesystem unattended because we used host principals and keytabs. Your user must use kinit to access the filesystem.

If it didn’t work, time to troubleshoot.

Troubleshooting

1. Try running mount without “-o sec=krb5″ (this assumes you enabled a fallback like sec=krb5:sys). If it fails, you probably have a syntax error in /etc/exports or you forgot to (re)run /usr/sbin/exportfs -rv after editing the file.
2. Run /usr/sbin/exportfs -v on the server and make sure /nfsexports/sharedfiles shows up and that sec=krb5 shows up somewhere in the listed options. If it’s not there, add it and rerun /usr/sbin/exportfs -rv.
3. On the NFS server, run sudo klist -k /etc/krb5.keytab and make sure nfs/nfsserver.domain.com shows up (probably more than once, since there will be a line for each supported encryption key). Run the same command on the client machine and make sure nfs/clientmachine.domain.com shows up there at least once.
4. Run klist as your regular user account and make sure your user’s ticket hasn’t expired. kinit will get you a new ticket, kinit -R will renew an expired one. On my system, I run kinit as a normal user and mount with sudo. sudo is smart enough to use my regular user’s tickets. It doesn’t try to use a ticket for root.

This entry was posted on Friday, January 27th, 2012 at 5:21 pm and is filed under HOWTOs, Reading Materials. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.