Azure Red Hat OpenShift clusters can be provisioned with persistent storage using NFS. Persistent volumes (PVs) and persistent volume claims (PVCs) provide a convenient method for sharing a volume across a project. While the NFS-specific information contained in a PV definition could also be defined directly in a Pod definition, doing so does not create the volume as a distinct cluster resource, making the volume more susceptible to conflicts.

Additional resources


Storage must exist in the underlying infrastructure before it can be mounted as a volume in Azure Red Hat OpenShift. To provision NFS volumes, a list of NFS servers and export paths are all that is required.

  1. Create an object definition for the PV:

    apiVersion: v1
    kind: PersistentVolume
      name: pv0001 (1)
        storage: 5Gi (2)
      - ReadWriteOnce (3)
      nfs: (4)
        path: /tmp (5)
        server: (6)
      persistentVolumeReclaimPolicy: Retain (7)
    2 The amount of storage allocated to this volume.
    5 The path that is exported by the NFS server.
    6 The host name or IP address of the NFS server.


  2. Verify that the PV was created:

    $ oc get pv
    pv0001   <none>    5Gi          RWO           Available                    31s
  3. Create a persistent volume claim that binds to the new PV:

    apiVersion: v1
    kind: PersistentVolumeClaim
      name: nfs-claim1
        - ReadWriteOnce (1)
          storage: 5Gi (2)
    2 This claim looks for PVs offering 5Gi or greater capacity.
  4. Verify that the persistent volume claim was created:

    $ oc get pvc
    nfs-claim1   Bound    pv0001   5Gi        RWO            gp2            2m

Enforcing disk quotas

You can use disk partitions to enforce disk quotas and size constraints. Each partition can be its own export. Each export is one PV. Azure Red Hat OpenShift enforces unique names for PVs, but the uniqueness of the NFS volume’s server and path is up to the administrator.

Enforcing quotas in this way allows the developer to request persistent storage by a specific amount, such as 10Gi, and be matched with a corresponding volume of equal or greater capacity.

NFS volume security


Developers request NFS storage by referencing either a PVC by name or the NFS volume plug-in directly in the volumes section of their Pod definition.

The /etc/exports file on the NFS server contains the accessible NFS directories. The target NFS directory has POSIX owner and group IDs. The Azure Red Hat OpenShift NFS plug-in mounts the container’s NFS directory with the same POSIX ownership and permissions found on the exported NFS directory. However, the container is not run with its effective UID equal to the owner of the NFS mount, which is the desired behavior.


$ ls -lZ /opt/nfs -d
drwxrws---. nfsnobody 5555 unconfined_u:object_r:usr_t:s0   /opt/nfs

$ id nfsnobody

Then the container must match SELinux labels, and either run with a UID of 65534, the nfsnobody owner, or with 5555 in its supplemental groups in order to access the directory.

The owner ID of 65534 is used as an example. Even though NFS’s root_squash maps root, uid 0, to nfsnobody, uid 65534, NFS exports can have arbitrary owner IDs. Owner 65534 is not required for NFS exports.

Group IDs

The recommended way to handle NFS access, assuming it is not an option to change permissions on the NFS export, is to use supplemental groups. Supplemental groups in Azure Red Hat OpenShift are used for shared storage, of which NFS is an example. In contrast block storage, such as iSCSI, use the fsGroup SCC strategy and the fsGroup value in the Pod’s securityContext.

To gain access to persistent storage, it is generally preferable to use supplemental group IDs versus user IDs.

Because the group ID on the example target NFS directory is 5555, the Pod can define that group ID using supplementalGroups under the Pod’s securityContext definition. For example:

    - name:
  securityContext: (1)
    supplementalGroups: [5555] (2)
1 securityContext must be defined at the Pod level, not under a specific container.
2 An array of GIDs defined for the Pod. In this case, there is one element in the array. Additional GIDs would be comma-separated.

Assuming there are no custom SCCs that might satisfy the Pod’s requirements, the Pod likely matches the restricted SCC. This SCC has the supplementalGroups strategy set to RunAsAny, meaning that any supplied group ID is accepted without range checking.

As a result, the above Pod passes admissions and is launched. However, if group ID range checking is desired, a custom SCC is the preferred solution. A custom SCC can be created such that minimum and maximum group IDs are defined, group ID range checking is enforced, and a group ID of 5555 is allowed.

To use a custom SCC, you must first add it to the appropriate service account. For example, use the default service account in the given project unless another has been specified on the Pod specification.

User IDs

User IDs can be defined in the container image or in the Pod definition.


In the example target NFS directory shown above, the container needs its UID set to 65534, ignoring group IDs for the moment, so the following can be added to the Pod definition:

  containers: (1)
  - name:
      runAsUser: 65534 (2)
1 Pods contain a securityContext specific to each container and a Pod’s securityContext which applies to all containers defined in the Pod.
2 65534 is the nfsnobody user.

Assuming the default project and the restricted SCC, the Pod’s requested user ID of 65534 is not allowed, and therefore the Pod fails. The Pod fails for the following reasons:

  • It requests 65534 as its user ID.

  • All SCCs available to the Pod are examined to see which SCC allows a user ID of 65534. While all policies of the SCCs are checked, the focus here is on user ID.



It is generally considered a good practice not to modify the predefined SCCs. The preferred way to fix this situation is to create a custom SCC A custom SCC can be created such that minimum and maximum user IDs are defined, UID range checking is still enforced, and the UID of 65534 is allowed.



By default, SELinux does not allow writing from a Pod to a remote NFS server. The NFS volume mounts correctly, but is read-only.

To enable writing to a remote NFS server, follow the below procedure.

  • The container-selinux package must be installed. This package provides the virt_use_nfs SELinux boolean.

  • Enable the virt_use_nfs boolean using the following command. The -P option makes this boolean persistent across reboots.

    # setsebool -P virt_use_nfs 1

Export settings


  • Every export must be exported using the following format:

    /<example_fs> *(rw,root_squash)

    • For NFSv4, configure the default port 2049 (nfs).


  • The NFS export and directory must be set up so that they are accessible by the target Pods. Either set the export to be owned by the container’s primary UID, or supply the Pod group access using supplementalGroups, as shown in group IDs above.

Reclaiming resources

NFS implements the Azure Red Hat OpenShift Recyclable plug-in interface. Automatic processes handle reclamation tasks based on policies set on each persistent volume.

By default, PVs are set to Retain.

Once claim to a PVC is deleted, and the PV is released, the PV object should not be reused. Instead, a new PV should be created with the same basic volume details as the original.

For example, the administrator creates a PV named nfs1:

apiVersion: v1
kind: PersistentVolume
  name: nfs1
    storage: 1Mi
    - ReadWriteMany
    path: "/"

The user creates PVC1, which binds to nfs1. The user then deletes PVC1, releasing claim to nfs1. This results in nfs1 being Released. If the administrator wants to make the same NFS share available, they should create a new PV with the same NFS server details, but a different PV name:

apiVersion: v1
kind: PersistentVolume
  name: nfs2
    storage: 1Mi
    - ReadWriteMany
    path: "/"


Additional configuration and troubleshooting


NFSv4 mount incorrectly shows all files with ownership of nobody:nobody

  • Could be attributed to the ID mapping settings, found in /etc/idmapd.conf on your NFS.

  • See this Red Hat Solution.

Disabling ID mapping on NFSv4

  • On both the NFS client and server, run:
