Set NFS with Kerberos authentication and encryption

Last updated: Dec 2, 2024

I needed to create a share on my network. Being in a GNU/Linux environment, my natural choice was NFS.

Unfortunately, by default, this protocol offers no security. The only thing the protocol offers is IP filtering, which remains rather weak, as there is no authentication and no encryption.

To overcome these two weaknesses, we have a choice between setting up a vpn or using the kerberos protocol. As I've never worked with the latter, I've chosen kerberos.

In this guide, I'm going to set up an NFS+Kerberos architecture with the following properties:
- Debian Server: will be the NFS Server + Ticket-Granting Service and Authentication Server
- ArchLinux Client: NFS Client
- Debian Client: NFS Client

Network Architecture

Server

Install

Edit the /etc/network/interfaces file and set the static IP address:

allow-hotplug ens192
iface ens192 inet static
        address 192.168.1.100
        netmask 255.255.255.0
        gateway 192.168.1.254
        dns-nameservers 192.168.1.254

Install the necessary packages:

root@debian:~# apt update && apt install nfs-kernel-server nfs-common krb5-user libpam-krb5 krb5-admin-server krb5-kdc

Note: you can answer STD.LOCAL and nfs.std.local to the questions asked. In fact, it doesn't matter as we'll define these parameters in the configuration files later.

Add an nfs group:

root@debian:~# groupadd nfs

Add the user nobody to the nfs group:

root@debian:~# usermod -a -G nfs nobody

Create the folder you want to share:

root@debian:~# mkdir /nfs

Define authorizations:

root@debian:~# chmod 770 /nfs && chgrp nfs /nfs

Edit /etc/hosts and add these entries:

192.168.1.100   nfs.std.local nfs
192.168.1.10    arch.std.local arch
192.168.1.11    debian.std.local debian

NFS service configuration

Edit the /etc/exports file with the following options:
- rw: read/write access
- sync: server waits until data is on disk before responding
- anongid: explicitly defines gid 1001 (nfs group created ealier). Default values for uid and gid are: 65534 which corresponds to user/group nobody.
- root_squash: prevents remote root users from having root privileges on remote NFS-mounted volumes
- no_subtree_check: this option disables subtree checking, which has few security implications, but may improve reliability in certain circumstances.

Note:
- krb5: uses Kerberos for authentication only.
- krb5i: uses Kerberos for authentication, and include a hash in each transaction to guarantee integrity. Traffic can still be intercepted and examined, but changes to traffic will be apparent.
- krb5p: uses Kerberos for authentication, integrity control and encrypts all traffic between client and server. This is the most secure solution, but also the most CPU-intensive.

/nfs *(rw,sync,anongid=1001,root_squash,no_subtree_check,sec=krb5p)

Or even more secure with the addition of IP address filtering:

/nfs    192.168.1.10(rw,sync,anongid=1001,root_squash,no_subtree_check,sec=krb5p) 192.168.1.11(rw,sync,anongid=1001,root_squash,no_subtree_check,sec=krb5p)

Edit /etc/default/nfs-kernel-server to enable the svcgssd daemon, which is required by kerberos:

NEED_SVCGSSD=yes

Restart and reload/export NFS share:

root@debian:~# systemctl restart nfs-kernel-server.service && exportfs -arv

Configure Kerberos

Edit /etc/krb5.conf:

[libdefaults]
	default_realm = STD.LOCAL
	kdc_timesync = 1
	ccache_type = 4
	forwardable = true
	proxiable = true
	fcc-mit-ticketflags = true
	allow_weak_crypto = false
[realms]
	STD.LOCAL = {
		kdc = nfs.std.local
		admin_server = nfs.std.local
		default_domain = std.local
	}
[domain_realm]
	.std.local = STD.LOCAL
	std.local = STD.LOCAL
[logging]
    kdc          = SYSLOG:NOTICE
    admin_server = SYSLOG:NOTICE
    default      = SYSLOG:NOTICE

Create a new database and define the KDC database master key:

root@debian:~# kdb5_util -r STD.LOCAL create -s
You will be prompted for the database Master Password.
It is important that you NOT FORGET this password.
Enter KDC database master key: MasterOfTheDomain:p
Re-enter KDC database master key to verify: MasterOfTheDomain:p

Edit the /etc/krb5kdc/kadm5.acl file:

#*/admin@STD.LOCAL *
kdcadmin/admin@STD.LOCAL *

Start the following services:

root@debian:~# systemctl start krb5-kdc.service root@debian:~# systemctl start krb5-admin-server.service

Create a admin principal user for administration:

root@debian:~# kadmin.local
Authenticating as principal root/admin@STD.LOCAL with password.
kadmin.local:  add_principal kdcadmin/admin@STD.LOCAL
No policy specified for kdcadmin/admin@STD.LOCAL; defaulting to no policy
Enter password for principal "kdcadmin/admin@STD.LOCAL":$superKDC$2000
Re-enter password for principal "kdcadmin/admin@STD.LOCAL":$superKDC$2000
Principal "kdcadmin/admin@STD.LOCAL" created.
kadmin:  exit

Create an nfs user:

root@debian:~# kadmin.local
Authenticating as principal root/admin@STD.LOCAL with password.
kadmin.local:  add_principal nfsuser@STD.LOCAL
No policy specified for nfsuser@STD.LOCAL; defaulting to no policy
Enter password for principal "nfsuser@STD.LOCAL":NFScher$$
Re-enter password for principal "nfsuser@STD.LOCAL":NFScher$$
Principal "nfsuser@STD.LOCAL" created.
kadmin:  exit

Create nfs service records for server and clients:

root@debian:~# kadmin.local
kadmin.local:  addprinc -randkey nfs/nfs.std.local@STD.LOCAL
No policy specified for nfs/nfs.std.local@STD.LOCAL; defaulting to no policy
Principal "nfs/nfs.std.local@STD.LOCAL" created.
kadmin.local:  addprinc -randkey nfs/arch.std.local@STD.LOCAL
No policy specified for nfs/arch.std.local@STD.LOCAL; defaulting to no policy
Principal "nfs/arch.std.local@STD.LOCAL" created.
kadmin.local:  addprinc -randkey nfs/debian.std.local@STD.LOCAL
No policy specified for nfs/debian.std.local@STD.LOCAL; defaulting to no policy
Principal "nfs/debian.std.local@STD.LOCAL" created.
kadmin:  exit

Add nfs entries to the /etc/krb5.keytab file:

root@debian:~# kadmin.local
kadmin.local:  ktadd nfs/nfs.std.local
kadmin.local:  ktadd nfs/arch.std.local
kadmin.local:  ktadd nfs/debian.std.local
kadmin.local:  exit

Finally, restart the operating system (important!):

root@debian:~# reboot

Client (ArchLinux)

Install packages:

[root@arch ~]# pacman -S nfs-utils

Enable the nfs service to start at boot time:

[root@arch ~]# systemctl enable nfs-client.target

Start the nfs service:

[root@arch ~]# systemctl start nfs-client.target

Add the following lines to the /etc/hosts file:

192.168.1.100   nfs.std.local nfs
192.168.1.10    arch.std.local arch
192.168.1.11    debian.std.local debian

Edit the /etc/krb5.conf file, which is identical to the server file:

[libdefaults]
	default_realm = STD.LOCAL
	kdc_timesync = 1
	ccache_type = 4
	forwardable = true
	proxiable = true
	fcc-mit-ticketflags = true
	allow_weak_crypto = false
[realms]
	STD.LOCAL = {
		kdc = nfs.std.local
		admin_server = nfs.std.local
		default_domain = std.local
	}
[domain_realm]
	.std.local = STD.LOCAL
	std.local = STD.LOCAL

Add the nfs/arch.std.local entry to /etc/krb5.keytab

We can now copy the /etc/krb5.keytab file from the server or generate it from client via Kerberos.

Add it from the client using Kerberos:

[root@arch ~]# kadmin -p kdcadmin/admin@STD.LOCAL
Authenticating as principal kdcadmin/admin@STD.LOCAL with password.
Password for kdcadmin/admin@STD.LOCAL: $superKDC$2000
kadmin:  ktadd nfs/arch.std.local
Entry for principal nfs/arch.std.local with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab FILE:/etc/krb5.keytab.
Entry for principal nfs/arch.std.local with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab FILE:/etc/krb5.keytab.
kadmin:  exit

Restart host:

Note: I got the following error: mount.nfs: Operation not permitted until the machine was completely rebooted.

[root@arch ~]# reboot

Mount the nfs share on /mnt:

[root@arch ~]# mkdir /mnt/nfs [root@arch ~]# mount -t nfs -o sec=krb5p nfs.std.local:/nfs /mnt/nfs/

Note: You should now have access to the nfs share from the root user, but you get the following error: cannot access '/mnt/nfs/': Permission denied from a user account. You need a kerberos ticket for this to work.

Request a kerberos ticket:

[user@arch ~]$ kinit nfsuser@STD.LOCAL
Password for nfsuser@STD.LOCAL: NFScher$$

You now have access to the nfs share with a simple user account.

Check the status of the kerberos ticket:

[user@arch ~]$ klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: nfsuser@STD.LOCAL

Valid starting       Expires              Service principal
18/09/2022 20:14:38  19/09/2022 06:14:38  krbtgt/STD.LOCAL@STD.LOCAL
	renew until 19/09/2022 15:14:33

To enable the user to mount /mnt/nfs once authenticated, add this line to /etc/fstab:

nfs.std.local:/nfs /mnt/nfs nfs defaults,timeo=900,retrans=5,_netdev,sec=krb5p,user,noauto 0 0

Client (Debian)

Install the following packages:

root@debian:~# apt update && apt install nfs-common krb5-user

Note: you can answer STD.LOCAL and nfs.std.local to the questions asked. In fact, it doesn't matter as we we'll define these parameters in the configuration files later.

Enable the nfs service to start at boot time:

root@debian:~# systemctl enable nfs-client.target

Start the nfs service:

root@debian:~# systemctl start nfs-client.target

Add these lines to /etc/hosts:

192.168.1.100   nfs.std.local nfs
192.168.1.10    arch.std.local arch
192.168.1.11    debian.std.local debian

Edit the /etc/krb5.conf file, which is identical to the server file:

[libdefaults]
	default_realm = STD.LOCAL
	kdc_timesync = 1
	ccache_type = 4
	forwardable = true
	proxiable = true
	fcc-mit-ticketflags = true
	allow_weak_crypto = false
[realms]
	STD.LOCAL = {
		kdc = nfs.std.local
		admin_server = nfs.std.local
		default_domain = std.local
	}
[domain_realm]
	.std.local = STD.LOCAL
	std.local = STD.LOCAL

Add the nfs/debian.std.local entry to /etc/krb5.keytab

We can copy the /etc/krb5.keytab file from the server or generate it from the client via kerberos.

Add from client with Kerberos:

root@debian:~# kadmin -p kdcadmin/admin@STD.LOCAL
Authenticating as principal kdcadmin/admin@STD.LOCAL with password.
Password for kdcadmin/admin@STD.LOCAL: $superKDC$2000
kadmin:  ktadd nfs/debian.std.local
Entry for principal nfs/arch.std.local with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab FILE:/etc/krb5.keytab.
Entry for principal nfs/arch.std.local with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab FILE:/etc/krb5.keytab.
kadmin:  exit

Restart the host:

Note: I got the following error: mount.nfs: an incorrect mount option was specified until the machine was completly rebooted.

[root@arch ~]# reboot

Mount the nfs share on /mnt:

[root@arch ~]# mkdir /mnt/nfs [root@arch ~]# mount -t nfs -o sec=krb5p nfs.std.local:/nfs /mnt/nfs/

Note: ou should now have access to the nfs share from the root user, but get the error: cannot open directory '/mnt/nfs/': Stale file handle from a user account. You need a Kerberos ticket for this to work.

Request a Kerberos ticket:

[user@arch ~]$ kinit nfsuser@STD.LOCAL
Password for nfsuser@STD.LOCAL: NFScher$$

The user now has access to the nfs share.

Check Kerberos ticket status:

[user@arch ~]$ klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: nfsuser@STD.LOCAL

Valid starting       Expires              Service principal
09/21/2022 18:12:01  09/22/2022 04:12:01  krbtgt/STD.LOCAL@STD.LOCAL
	renew until 09/22/2022 18:11:39
09/21/2022 18:12:03  09/22/2022 04:12:01  nfs/nfs.std.local@STD.LOCAL
	renew until 09/22/2022 18:11:39

To enable the user to mount /mnt/nfs once authenticated, add this line to /etc/fstab:

nfs.std.local:/nfs /mnt/nfs nfs defaults,timeo=900,retrans=5,_netdev,sec=krb5p,user,noauto 0 0

Miscellaneous

Commands

Save a specific key in a keytab:

root@debian:~# ktadd -k krb5.keytab nfs/archtoi.std.local

List cached Kerberos tickets:

root@debian:~# klist

Destroy Kerberos tickets:

root@debian:~# kdestroy

Renew Kerberos ticket:

root@debian:~# kinit -R

List principals:

root@debian:~# kadmin.local
Authenticating as principal root/admin@STD.LOCAL with password.
kadmin.local:  listprincs
K/M@STD.LOCAL
kadmin/admin@STD.LOCAL
kadmin/changepw@STD.LOCAL
kadmin/nfskerb@STD.LOCAL
kdcadmin/admin@STD.LOCAL
kiprop/nfskerb@STD.LOCAL
krbtgt/STD.LOCAL@STD.LOCAL
nfs/arch.std.local@STD.LOCAL
nfs/debian.std.local@STD.LOCAL
nfs/nfs.std.local@STD.LOCAL
nfsuser@STD.LOCAL

Remove principals:

root@debian:~# kadmin.local
Authenticating as principal root/admin@STD.LOCAL with password.
kadmin.local:  delete_principal nfs/arch.std.local@STD.LOCA

List entries in /etc/krb5.keytab:

root@debian:~# klist -e -k /etc/krb5.keytab
Keytab name: FILE:/etc/krb5.keytab
KVNO Principal
---- --------------------------------------------------------------------------
   4 nfs/debian.std.local@STD.LOCAL (aes256-cts-hmac-sha1-96) 
   4 nfs/debian.std.local@STD.LOCAL (aes128-cts-hmac-sha1-96)

List entries in /etc/krb5.keytab (alternative):

root@debian:~# ktutil
ktutil:  rkt /etc/krb5.keytab
ktutil:  list
slot KVNO Principal
---- ---- ---------------------------------------------------------------------
   1    2              nfs/nfs.std.local@STD.LOCAL
   2    2              nfs/nfs.std.local@STD.LOCAL
   3    2             nfs/arch.std.local@STD.LOCAL
   4    2             nfs/arch.std.local@STD.LOCAL
   5    2           nfs/debian.std.local@STD.LOCAL
   6    2           nfs/debian.std.local@STD.LOCAL

Delete an entry from /etc/krb5.keytab:

root@debian:~# ktutil
ktutil:  delete_entry nfs/arch.std.local@STD.LOCAL

Setting up a firewall for the server (optional)

I'll show here the nftables rules I use to protect the server.

Ports used

The ports used by the NFS and Kerberos services are:

Service	TCP	UDP
Kerberos	88, 749	88, 749
NFSv4	2049

Disable IPv6

Edit /etc/sysctl.conf and add these lines:

net.ipv6.conf.all.disable_ipv6=1
net.ipv6.conf.all.autoconf=0
net.ipv6.conf.default.disable_ipv6=1
net.ipv6.conf.default.autoconf=0

Take modifications into account:

root@debian:~# sysctl -p /etc/sysctl.conf

nftables

Enable the nftables service:

root@debian:~# systemctl enable nftables.service

Edit the /etc/nftables.conf file:

#!/usr/sbin/nft -f

flush ruleset

# ----- IPv4 -----
table ip filter {
        chain INPUT {
                type filter hook input priority 0; policy drop; #by default, we drop traffic

                iif lo accept comment "Accept any localhost traffic"
                ct state invalid counter drop comment "Drop invalid connections"
                ct state { established, related } counter accept comment "Accept traffic originated from us"
                iif != lo ip daddr 127.0.0.1/8 counter drop comment "drop connections to loopback not coming from loopback"
		tcp flags & (fin | syn | rst | psh | ack | urg) > urg counter drop comment "Drop TCP Null"
                tcp flags & (fin | syn | rst | psh | ack | urg) < fin counter drop comment "Drop TCP XMS"


                ip protocol icmp icmp type { destination-unreachable, router-solicitation, router-advertisement, time-exceeded, parameter-problem, echo-request } accept comment "Accept ICMP"

                iif ens192 ip saddr { 192.168.1.10, 192.168.1.11 } tcp dport { ssh, 88, 749, 2049 } counter accept comment "Accept ssh, kerberos tcp and nfsv4"
                iif ens192 ip saddr { 192.168.1.10, 192.168.1.11 } udp dport { 88, 749 } counter accept comment "Accept kerberos udp"
                iif ens192 tcp sport { http, https, 53, ntp } counter accept comment "Accept http, https, dns and ntp TCP"
                iif ens192 udp sport { 53, ntp } counter accept comment "Accept dns and ntp UDP"
                log counter drop #log (/var/log/kern.log), count and drop any other rules
        }
        chain FORWARD {
                type filter hook forward priority 0; policy drop;
                counter comment "count dropped packets"
        }
        chain OUTPUT {
                type filter hook output priority 0; policy drop;
                oif ens192 ip daddr { 192.168.1.10, 192.168.1.11 } tcp sport { ssh, 88, 749, 2049 } counter accept comment "Accept ssh, kerberos tcp and nfsv4"
                oif ens192 ip daddr { 192.168.1.10, 192.168.1.11 } udp sport { 88, 749 } counter accept comment "Accept kerberos udp"
                tcp dport { http, https, 53, ntp } counter accept comment "Accept http, https, dns and ntp TCP"
                udp dport { 53, ntp } counter accept comment "Accept dns and ntp UDP"
                ip protocol icmp icmp type { destination-unreachable, router-solicitation, router-advertisement, time-exceeded, parameter-problem, echo-request, echo-reply } accept comment "Accept ICMP"
                log counter drop #log (/var/log/kern.log), count and drop any other rules
        }
}

Restart the nftables service:

root@debian:~# systemctl restart nftables.service

Computing

Music

Misc