Set NFS with Kerberos authentication and encryption

Last updated: May 20, 2023

Network share icon with a kerberos ticket

I needed to make a network share on my network. Being in a GNU/Linux environement my choice was naturally to use NFS.

Unfortunately, by default, this protocol doesn't provide any security. The only thing that the protocol proposes is a filtering by ip which is still quite weak because there is no authentication and no encryption.

In order to overcome these two weaknesses we have the choice between setting up a vpn or the kerberos protocol. As I have never worked on the latter before, I chose the protocol.

I will set up an NFS+Kerberos architecture with these properties:
- Debian Server: will be the NFS Server + Ticket-Granting Service and Authentication Server
- ArchLinux Client: NFS Client
- Debian Client: NFS Client

Network Architecture

Server

Install

Edit /etc/network/interfaces and set static ip configuration:

allow-hotplug ens192
iface ens192 inet static
        address 192.168.1.100
        netmask 255.255.255.0
        gateway 192.168.1.254
        dns-nameservers 192.168.1.254

Install needed packages:

root@debian:~# apt update && apt install nfs-kernel-server nfs-common krb5-user libpam-krb5 krb5-admin-server krb5-kdc

Note: you can answer STD.LOCAL and nfs.std.local to questions asked. In fact it doesn't matter because we will define this parameters inside the configuration files later.

Add a nfs group:

root@debian:~# groupadd nfs

Add nobody user to the nfs group:

root@debian:~# usermod -a -G nfs nobody

Create folder which will be shared:

root@debian:~# mkdir /nfs

Set permissions:

root@debian:~# chmod 770 /nfs && chgrp nfs /nfs

Edit /etc/hosts and add these entries:

192.168.1.100   nfs.std.local nfs
192.168.1.10    arch.std.local arch
192.168.1.11    debian.std.local debian

Configure NFS Service

Edit the /etc/exports file with the following options:
- rw: read/write access
- sync: le serveur attend que les données soient sur le disque avant de répondre
- anongid: Explicitly set the gid 1001 (nfs group created before). uid and gid default values are: 65534 which is the nobody user/group.
- root_squash: prevents remote root users from having superuser (root) privileges on remote NFS-mounted volumes
- no_subtree_check: This option disables subtree checking, which has mild security implications, but can improve reliability in some circumstances.

Note:
- krb5 Use Kerberos for authentication only.
- krb5i Use Kerberos for authentication, and include a hash with each transaction to ensure integrity. Traffic can still be intercepted and examined, but modifications to the traffic will be apparent.
- krb5p Use Kerberos for authentication, integrity checking and encrypt all traffic between the client and server. This is the most secure, but also incurs the most load.

/nfs *(rw,sync,anongid=1001,root_squash,no_subtree_check,sec=krb5p)

Or even more secure with the addition of ip filtering:

/nfs    192.168.1.10(rw,sync,anongid=1001,root_squash,no_subtree_check,sec=krb5p) 192.168.1.11(rw,sync,anongid=1001,root_squash,no_subtree_check,sec=krb5p)

Edit the /etc/default/nfs-kernel-server file to enable svcgssd daemon which is needed by kerberos:

NEED_SVCGSSD=yes

Restart and reload/export NFS share:

root@debian:~# systemctl restart nfs-kernel-server.service && exportfs -arv

Configure Kerberos

Edit /etc/krb5.conf:

[libdefaults]
	default_realm = STD.LOCAL
	kdc_timesync = 1
	ccache_type = 4
	forwardable = true
	proxiable = true
	fcc-mit-ticketflags = true
	allow_weak_crypto = false
[realms]
	STD.LOCAL = {
		kdc = nfs.std.local
		admin_server = nfs.std.local
		default_domain = std.local
	}
[domain_realm]
	.std.local = STD.LOCAL
	std.local = STD.LOCAL
[logging]
    kdc          = SYSLOG:NOTICE
    admin_server = SYSLOG:NOTICE
    default      = SYSLOG:NOTICE

Create a new database and set KDC database master key:

root@debian:~# kdb5_util -r STD.LOCAL create -s
You will be prompted for the database Master Password.
It is important that you NOT FORGET this password.
Enter KDC database master key: MasterOfTheDomain:p
Re-enter KDC database master key to verify: MasterOfTheDomain:p

Edit /etc/krb5kdc/kadm5.acl:

#*/admin@STD.LOCAL *
kdcadmin/admin@STD.LOCAL *

Start services:

root@debian:~# systemctl start krb5-kdc.service root@debian:~# systemctl start krb5-admin-server.service

Create a admin principal user for administration:

root@debian:~# kadmin.local
Authenticating as principal root/admin@STD.LOCAL with password.
kadmin.local:  add_principal kdcadmin/admin@STD.LOCAL
No policy specified for kdcadmin/admin@STD.LOCAL; defaulting to no policy
Enter password for principal "kdcadmin/admin@STD.LOCAL":$superKDC$2000
Re-enter password for principal "kdcadmin/admin@STD.LOCAL":$superKDC$2000
Principal "kdcadmin/admin@STD.LOCAL" created.
kadmin:  exit

Create a nfs user:

root@debian:~# kadmin.local
Authenticating as principal root/admin@STD.LOCAL with password.
kadmin.local:  add_principal nfsuser@STD.LOCAL
No policy specified for nfsuser@STD.LOCAL; defaulting to no policy
Enter password for principal "nfsuser@STD.LOCAL":NFScher$$
Re-enter password for principal "nfsuser@STD.LOCAL":NFScher$$
Principal "nfsuser@STD.LOCAL" created.
kadmin:  exit

Create nfs service records for server and clients:

root@debian:~# kadmin.local
kadmin.local:  addprinc -randkey nfs/nfs.std.local@STD.LOCAL
No policy specified for nfs/nfs.std.local@STD.LOCAL; defaulting to no policy
Principal "nfs/nfs.std.local@STD.LOCAL" created.
kadmin.local:  addprinc -randkey nfs/arch.std.local@STD.LOCAL
No policy specified for nfs/arch.std.local@STD.LOCAL; defaulting to no policy
Principal "nfs/arch.std.local@STD.LOCAL" created.
kadmin.local:  addprinc -randkey nfs/debian.std.local@STD.LOCAL
No policy specified for nfs/debian.std.local@STD.LOCAL; defaulting to no policy
Principal "nfs/debian.std.local@STD.LOCAL" created.
kadmin:  exit

Add nfs entries in /etc/krb5.keytab file:

root@debian:~# kadmin.local
kadmin.local:  ktadd nfs/nfs.std.local
kadmin.local:  ktadd nfs/arch.std.local
kadmin.local:  ktadd nfs/debian.std.local
kadmin.local:  exit

Restart (important):

root@debian:~# reboot

Client (ArchLinux)

Install packages:

[root@arch ~]# pacman -S nfs-utils

Enable nfs service to make it start at boot:

[root@arch ~]# systemctl enable nfs-client.target

Start nfs service:

[root@arch ~]# systemctl start nfs-client.target

Add these entries to /etc/hosts file:

192.168.1.100   nfs.std.local nfs
192.168.1.10    arch.std.local arch
192.168.1.11    debian.std.local debian

Edit /etc/krb5.conf file which is identical to the server one:

[libdefaults]
	default_realm = STD.LOCAL
	kdc_timesync = 1
	ccache_type = 4
	forwardable = true
	proxiable = true
	fcc-mit-ticketflags = true
	allow_weak_crypto = false
[realms]
	STD.LOCAL = {
		kdc = nfs.std.local
		admin_server = nfs.std.local
		default_domain = std.local
	}
[domain_realm]
	.std.local = STD.LOCAL
	std.local = STD.LOCAL

Add nfs/arch.std.local entry to /etc/krb5.keytab

We can copy /etc/krb5.keytab file from server or generate it from client via Kerberos.

Add from client with Kerberos:

[root@arch ~]# kadmin -p kdcadmin/admin@STD.LOCAL
Authenticating as principal kdcadmin/admin@STD.LOCAL with password.
Password for kdcadmin/admin@STD.LOCAL: $superKDC$2000
kadmin:  ktadd nfs/arch.std.local
Entry for principal nfs/arch.std.local with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab FILE:/etc/krb5.keytab.
Entry for principal nfs/arch.std.local with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab FILE:/etc/krb5.keytab.
kadmin:  exit

Reboot host:

Note: I had the permanant mount.nfs: Operation not permitted error until the machine was completly rebooted.

[root@arch ~]# reboot

Mount nfs share to /mnt:

[root@arch ~]# mkdir /mnt/nfs [root@arch ~]# mount -t nfs -o sec=krb5p nfs.std.local:/nfs /mnt/nfs/

Note: You should now have access to nfs share with root but have the cannot access '/mnt/nfs/': Permission denied from a user. You need a kerberos ticket.

Ask for kerberos ticket:

[user@arch ~]$ kinit nfsuser@STD.LOCAL
Password for nfsuser@STD.LOCAL: NFScher$$

Now you have access to nfs share from user.

Check kerberos ticket status:

[user@arch ~]$ klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: nfsuser@STD.LOCAL

Valid starting       Expires              Service principal
18/09/2022 20:14:38  19/09/2022 06:14:38  krbtgt/STD.LOCAL@STD.LOCAL
	renew until 19/09/2022 15:14:33

To enable the user to mount /mnt/nfs once authenticated, add this line to /etc/fstab:

nfs.std.local:/nfs /mnt/nfs nfs defaults,timeo=900,retrans=5,_netdev,sec=krb5p,user,noauto 0 0

Client (Debian)

Install packages:

root@debian:~# apt update && apt install nfs-common krb5-user

Note: you can answer STD.LOCAL and nfs.std.local to questions asked. In fact it doesn't matter because we will define this parameters inside the configuration files later.

Enable nfs service to make it start at boot:

root@debian:~# systemctl enable nfs-client.target

Start nfs service:

root@debian:~# systemctl start nfs-client.target

Add these entries to /etc/hosts:

192.168.1.100   nfs.std.local nfs
192.168.1.10    arch.std.local arch
192.168.1.11    debian.std.local debian

Edit the /etc/krb5.conf file which is identical to the server one:

[libdefaults]
	default_realm = STD.LOCAL
	kdc_timesync = 1
	ccache_type = 4
	forwardable = true
	proxiable = true
	fcc-mit-ticketflags = true
	allow_weak_crypto = false
[realms]
	STD.LOCAL = {
		kdc = nfs.std.local
		admin_server = nfs.std.local
		default_domain = std.local
	}
[domain_realm]
	.std.local = STD.LOCAL
	std.local = STD.LOCAL

Add nfs/debian.std.local entry to /etc/krb5.keytab

We can copy /etc/krb5.keytab file from server or generate it from client via Kerberos.

Add from client with Kerberos:

root@debian:~# kadmin -p kdcadmin/admin@STD.LOCAL
Authenticating as principal kdcadmin/admin@STD.LOCAL with password.
Password for kdcadmin/admin@STD.LOCAL: $superKDC$2000
kadmin:  ktadd nfs/debian.std.local
Entry for principal nfs/arch.std.local with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab FILE:/etc/krb5.keytab.
Entry for principal nfs/arch.std.local with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab FILE:/etc/krb5.keytab.
kadmin:  exit

Reboot host:

Note: I had the permanant mount.nfs: an incorrect mount option was specified error until the machine was completly rebooted.

[root@arch ~]# reboot

Mount nfs share to /mnt:

[root@arch ~]# mkdir /mnt/nfs [root@arch ~]# mount -t nfs -o sec=krb5p nfs.std.local:/nfs /mnt/nfs/

Note: We should now have access to nfs share with root but have the cannot open directory '/mnt/nfs/': Stale file handle from user. You need a Kerberos ticket.

Ask for a Kerberos ticket:

[user@arch ~]$ kinit nfsuser@STD.LOCAL
Password for nfsuser@STD.LOCAL: NFScher$$

Now we have access to nfs share from user.

Check the Kerberos ticket status:

[user@arch ~]$ klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: nfsuser@STD.LOCAL

Valid starting       Expires              Service principal
09/21/2022 18:12:01  09/22/2022 04:12:01  krbtgt/STD.LOCAL@STD.LOCAL
	renew until 09/22/2022 18:11:39
09/21/2022 18:12:03  09/22/2022 04:12:01  nfs/nfs.std.local@STD.LOCAL
	renew until 09/22/2022 18:11:39

To enable the user to mount /mnt/nfs once authenticated, add this line to /etc/fstab:

nfs.std.local:/nfs /mnt/nfs nfs defaults,timeo=900,retrans=5,_netdev,sec=krb5p,user,noauto 0 0

Misc

Commands

Save specifi key inside a keytab:

root@debian:~# ktadd -k krb5.keytab nfs/archtoi.std.local

List cached Kerberos tickets:

root@debian:~# klist

Destroy Kerberos tickets:

root@debian:~# kdestroy

Renew Kerberos ticket:

root@debian:~# kinit -R

List principals:

root@debian:~# kadmin.local
Authenticating as principal root/admin@STD.LOCAL with password.
kadmin.local:  listprincs
K/M@STD.LOCAL
kadmin/admin@STD.LOCAL
kadmin/changepw@STD.LOCAL
kadmin/nfskerb@STD.LOCAL
kdcadmin/admin@STD.LOCAL
kiprop/nfskerb@STD.LOCAL
krbtgt/STD.LOCAL@STD.LOCAL
nfs/arch.std.local@STD.LOCAL
nfs/debian.std.local@STD.LOCAL
nfs/nfs.std.local@STD.LOCAL
nfsuser@STD.LOCAL

Remove principals:

root@debian:~# kadmin.local
Authenticating as principal root/admin@STD.LOCAL with password.
kadmin.local:  delete_principal nfs/arch.std.local@STD.LOCA

List the /etc/krb5.keytab entries:

root@debian:~# klist -e -k /etc/krb5.keytab
Keytab name: FILE:/etc/krb5.keytab
KVNO Principal
---- --------------------------------------------------------------------------
   4 nfs/debian.std.local@STD.LOCAL (aes256-cts-hmac-sha1-96) 
   4 nfs/debian.std.local@STD.LOCAL (aes128-cts-hmac-sha1-96)

List the /etc/krb5.keytab entries (alternative):

root@debian:~# ktutil
ktutil:  rkt /etc/krb5.keytab
ktutil:  list
slot KVNO Principal
---- ---- ---------------------------------------------------------------------
   1    2              nfs/nfs.std.local@STD.LOCAL
   2    2              nfs/nfs.std.local@STD.LOCAL
   3    2             nfs/arch.std.local@STD.LOCAL
   4    2             nfs/arch.std.local@STD.LOCAL
   5    2           nfs/debian.std.local@STD.LOCAL
   6    2           nfs/debian.std.local@STD.LOCAL

Remove a /etc/krb5.keytab entry:

root@debian:~# ktutil
ktutil:  delete_entry nfs/arch.std.local@STD.LOCAL

Firewall Server (Optional)

I will show here nftables rules that I use to protect the server.

Ports used

The ports used by NFS and Kerberos services are the following:

Service	TCP	UDP
Kerberos	88, 749	88, 749
NFSv4	2049

Disable IPv6

Edit the /etc/sysctl.conf file and add:

net.ipv6.conf.all.disable_ipv6=1
net.ipv6.conf.all.autoconf=0
net.ipv6.conf.default.disable_ipv6=1
net.ipv6.conf.default.autoconf=0

Validate change:

root@debian:~# sysctl -p /etc/sysctl.conf

nftables

Enable nft service:

root@debian:~# systemctl enable nftables.service

Edit the /etc/nftables.conffile:

#!/usr/sbin/nft -f

flush ruleset

# ----- IPv4 -----
table ip filter {
        chain INPUT {
                type filter hook input priority 0; policy drop; #by default, we drop traffic

                iif lo accept comment "Accept any localhost traffic"
                ct state invalid counter drop comment "Drop invalid connections"
                ct state { established, related } counter accept comment "Accept traffic originated from us"
                iif != lo ip daddr 127.0.0.1/8 counter drop comment "drop connections to loopback not coming from loopback"
		tcp flags & (fin | syn | rst | psh | ack | urg) > urg counter drop comment "Drop TCP Null"
                tcp flags & (fin | syn | rst | psh | ack | urg) < fin counter drop comment "Drop TCP XMS"


                ip protocol icmp icmp type { destination-unreachable, router-solicitation, router-advertisement, time-exceeded, parameter-problem, echo-request } accept comment "Accept ICMP"

                iif ens192 ip saddr { 192.168.1.10, 192.168.1.11 } tcp dport { ssh, 88, 749, 2049 } counter accept comment "Accept ssh, kerberos tcp and nfsv4"
                iif ens192 ip saddr { 192.168.1.10, 192.168.1.11 } udp dport { 88, 749 } counter accept comment "Accept kerberos udp"
                iif ens192 tcp sport { http, https, 53, ntp } counter accept comment "Accept http, https, dns and ntp TCP"
                iif ens192 udp sport { 53, ntp } counter accept comment "Accept dns and ntp UDP"
                log counter drop #log (/var/log/kern.log), count and drop any other rules
        }
        chain FORWARD {
                type filter hook forward priority 0; policy drop;
                counter comment "count dropped packets"
        }
        chain OUTPUT {
                type filter hook output priority 0; policy drop;
                oif ens192 ip daddr { 192.168.1.10, 192.168.1.11 } tcp sport { ssh, 88, 749, 2049 } counter accept comment "Accept ssh, kerberos tcp and nfsv4"
                oif ens192 ip daddr { 192.168.1.10, 192.168.1.11 } udp sport { 88, 749 } counter accept comment "Accept kerberos udp"
                tcp dport { http, https, 53, ntp } counter accept comment "Accept http, https, dns and ntp TCP"
                udp dport { 53, ntp } counter accept comment "Accept dns and ntp UDP"
                ip protocol icmp icmp type { destination-unreachable, router-solicitation, router-advertisement, time-exceeded, parameter-problem, echo-request, echo-reply } accept comment "Accept ICMP"
                log counter drop #log (/var/log/kern.log), count and drop any other rules
        }
}

Restart the nftables service:

root@debian:~# systemctl restart nftables.service

Computing

Music

Misc

Set NFS with Kerberos authentication and encryption

Network Architecture

Server

Install

Configure NFS Service

Configure Kerberos

Client (ArchLinux)

Add nfs/arch.std.local entry to /etc/krb5.keytab

Client (Debian)

Add nfs/debian.std.local entry to /etc/krb5.keytab

Misc

Commands

Firewall Server (Optional)

Ports used

Disable IPv6

nftables

References