Configuring RHEL for Kerberized NFSv4 mounts

Configuration to enable mounting

These steps are required in all cases.

Install packages that are frequently missing (Use Puppet when appropriate, e.g. DAC servers)
```
# yum install adcli sssd nfs4-acl-tools
```

Configure /etc/krb5.conf with at least these lines (Use Puppet when appropriate, e.g. DAC servers)

[libdefaults]
default_realm = KIEWIT.DARTMOUTH.EDU
dns_lookup_realm = false
dns_lookup_kdc = true
forwardable = true
rdns = false

Edit the Domain line in /etc/idmapd.conf so it matches the NFSv4 Domain setting on the Isilon (KIEWIT.DARTMOUTH.EDU).
```
Domain = KIEWIT.DARTMOUTH.EDU
```
Join the system to the KIEWIT domain with adcli. This requires an AD account with admin privileges in the OU where you are creating the computer account. Unprivileged NetIDs can create computer accounts but seemingly not with the SPNs we need. In this command replace adminaccount as appropriate and for Research Computing systems change OU=Servers to OU=Research.
```
# adcli join kiewit.dartmouth.edu -S lexington2.dartmouth.edu -U adminaccount -O \
'OU=Servers,dc=Kiewit,dc=Dartmouth,dc=edu' --service-name=nfs --service-name=host
```

Since domain membership is critical, check that /etc/krb5.keytab was created with the right stuff.

# klist -k -t /etc/krb5.keytab
Keytab name: FILE:/etc/krb5.keytab
KVNO Timestamp Principal
---- ------------------- ------------------------------------------------------
2 10/17/2018 14:40:17 MYHOST$@KIEWIT.DARTMOUTH.EDU
[...]

Edit /etc/sysconfig/nfs and add a line so the kernel rechecks TGTs once an hour. The default is to only check when it would have expired (10 hours).
```
RPCGSSDARGS="-t 3600"
```
The rpcgssd and rpcidmapd services must be running. They should start automatically at boot time now that /etc/krb5.keytab exists but if we start them manually we should be able to mount things right away.
```
# systemctl start rpcidmapd
# systemctl start rpcgssd
```

Make a mount!

# mount -t nfs -o sec=krb5,nfsvers=4.0 server:/ifs/path/to/export /mount_point

Configuring the sssd service

Configuring the sssd service enables NetID logins (and the automatic acquisition of a Kerberos TGT) based on group membership defined in /etc/sssd/sssd.conf. Running sssd is not necessary for mounting the Kerberized NFSv4 storage but without that you’ll need to manually acquire the TGT for accessing anything (use the kinit command).

This authconfig command adds an auth, account, password and session line to password-auth-ac and system-auth-ac; and an account and session line to fingerprint-auth-ac, and smartcard-auth-ac. It also adds sss to the end of the passwd, shadow, group, services, netgroup, and autmount lines in /etc/nsswitch.conf
```
# authconfig --enablesssd --enablesssdauth --update
```

Create /etc/sssd/sssd.conf with ownership root:root and 0400 permissions. Change the group on the ad_access_filter line as appropriate to allow only those NetIDs to login. Without that line any NetID can login. Also change override_homedir as appropriate.

[sssd]
debug_level = 0x0270
services = nss, pam
domains = KIEWIT.DARTMOUTH.EDU

[nss]
debug_level = 0x0270
# exclude root from the sss NSS database
filter_groups = root
filter_users = root
# loginShell not set in AD
override_shell = /bin/bash
# May want to override home directories too
# override_homedir = /path/%u

[domain/KIEWIT.DARTMOUTH.EDU]
debug_level = 0x0270
# type of provider
id_provider = ad
auth_provider = ad
access_provider = ad
# e.g. allows 'getent passwd user' without specifying domain
use_fully_qualified_names = False
# same as false, which is required for AD, but preserves case in NSS ops
case_sensitive = Preserving
# Explicitly disable sudo from AD
sudo_provider = none
# UID/GID lower limit - sss ignores if lower
min_id = 10000
# Default domain
ldap_idmap_default_domain = KIEWIT.DARTMOUTH.EDU
# Don't map to objectSID, use uidNumber/gidNumber
ldap_id_mapping = False
# Allow members of "rc-login" group to login
ad_access_filter = DOM:KIEWIT.DARTMOUTH.EDU:(memberOf=CN=rc-login,OU=Groups,OU=Research,DC=KIEWIT,DC=DARTMOUTH,DC=EDU)
# Don't try to update AD DNS server
dyndns_update = False
# Don't try to update machine password
ad_maximum_machine_account_password_age = 0

Enable and start the sssd service

# systemctl enable sssd
# systemctl start sssd

You can check that sssd is doing something by looking up a NetID (that isn’t in your passwd file). You should get back a passwd file entry with a home directory that looks like /home/Kiewit/netid (or whatever you specified in /etc/sssd/sssd.conf).
```
$ getent passwd dz99918
dz99918:*:17020:10001:Research Computing:/home/Kiewit/dz99918:
```

Configuring Duo Multi-factor authentication in PAM

Not required. Need to flesh this section out

Modify /etc/pam.d/sshd
Modify /etc/pam.d/system-auth-ac

Troubleshooting

Date/time must be the same on the client as the server. Too much clock skew breaks everything.
The Security Type (or “Flavor” in the GUI) of the export on the Isilon must match what the client requests with the “sec=” mount option. krb5 is Kerberos5; krb5i is Kerberos5 Integrity; krb5p is Kerberos5 Privacy.
```
$ isi nfs exports view ExportNum --zone ZoneName | grep Security
Security Type: krb5
```
The Isilon virtual server and the client each need nfs and HOST service principals in Active Directory (actually I am not certain about an nfs SPN for the client but we have always made them). The server SPNs are part of the ISILON-EDC and/or ISILON-BMR computer object(s) (i.e. those are CNs to look for in AD). Client has his own computer object with CN equal to hostname.

The key version number on the client (in /etc/krb5.keytab) must match what’s in AD. If they don’t match, the easiest thing to do is: remove the system from the domain; delete /etc/krb5.keytab; rejoin the domain. In the example below 2 = 2 so life is good.

# kinit -k -t /etc/krb5.keytab HOST$
# kvno HOST$
HOST$@KIEWIT.DARTMOUTH.EDU: kvno  2
# klist -k -t /etc/krb5.keytab
Keytab name: FILE:/etc/krb5.keytab
KVNO Timestamp Principal
---- ------------------- ------------------------------------------------------
2 12/15/2017 22:44:46 HOST$@KIEWIT.DARTMOUTH.EDU
[...]

Issues with joining the server to domain with adcli results in error like:

adcli: joining domain kiewit.dartmouth.edu failed: Couldn’t create computer account: CN=SASSER1,OU=Servers,dc=Kiewit,dc=Dartmouth,dc=edu: 00002083: AtrErr: DSID-031519D9, #2:
0: 00002083: DSID-031519D9, problem 1006 (ATT_OR_VALUE_EXISTS), data 0, Att 90303 (servicePrincipalName):len 22
1: 00002083: DSID-031519D9, problem 1006 (ATT_OR_VALUE_EXISTS), data 0, Att 90303 (servicePrincipalName):len 24

This error can be caused if gethost() does not return the FQDN. Try setting using:

sudo hostnamectl set-hostname servername.dartmouth.edu

For additional troubleshooing information while joining the domain add –verbose to the adcli command
If when mounting you get and “invalid mount option” error with correct looking mount options, check that rpc.gssd is running. If it’s not, then “sec=krb5” will cause this error.
Reboot. There are services that must be running and that may have ordering dependencies. We’ve never bothered to figure put exactly what they are because the system gets it right at boot time assuming the configuration files are all in place. e.g. krb5.conf, rcp-gssd, rpcidmapd, gssproxy

Creating a web site in your DartFS account

On this page: