Storage Automation with Red Hat OpenShift and NetApp
A Red Hat OpenShift Container Platform (OCP) and OpenShift Virtualization (OCPv) deployment uses ephemeral (temporary) and persistent storage to maintain pods/containers in the environment. The ephemeral storage only lasts through the life of the pod, so having permanent storage available for stateful applications to access can become a necessity. This can certainly be the case for writable database applications, Artificial Intelligence (AI) and virtual machines.
So when creating persistent storage, do you do this on local storage or shared storage? File, Block or Object?
Local storage is fully supported in OpenShift by way of using the Local Storage and LVM Storage operators from the Ecosystem Software Catalog found in the OpenShift Console UI. Administrators can also deploy the OpenShift Data Foundation operator with the local storage operator to utilize a Ceph-based shared storage.
Shared storage often uses a Container Storage Interface (CSI) operator found in the same Ecosystem Software Catalog to build and manage storage within OpenShift. Many storage manufacturers have their own out there, including NetApp's open-source Trident operator.
Before we jump into the Trident operator and how to install it, let's first look at some of the basic components for managing storage in Red Hat OpenShift.
OpenShift Storage Components
- Storage Class
- An object that defines how new persistent volumes are created dynamically using metrics like performance, type of storage, and reclaim policies. An example would be to have a storage class that specifies to only use NVMe on the shared storage when creating new volumes when a developer requests that storage class.
- Persistent Volume (PV)
- This is the actual storage resource being provided to the Kubernetes/OpenShift cluster, which is defined cluster-wide. For example, allocating a 1TB volume from shared external storage.
- Persistent Volume Claim (PVC)
- This is the request to use the storage in a persistent volume for a pod in Kubernetes/OpenShift. The PVC is part of a namespace/project, as opposed to being cluster-wide. As an example, a developer creates a PVC for 1TB for a new pod in the namespace "dev01", which can then be mapped to a 1TB PV.
- Container Storage Interface (CSI) Driver
- This is a standardized driver format for allowing OpenShift to communicate with the storage system.
- Volume Snapshot Class
- This object defines which CSI driver to use and the deletion policy for creating snapshots for volumes.
NetApp Trident Operator - NFS Configuration
The steps below provide initial configuration of the NetApp Trident operator in Red Hat OpenShift 4.20 to allow for automated provisioning of volumes on a NetApp array.
- From the Console UI of OpenShift, browse to Ecosystem Hub > Software Catalog. In the search box, type trident. Click the tile for the Certified version of the operator.
- Review the subscription and then click Install.
- Review the namespace location where the operator will be installed. Retain the defaults and click Install.
- When the installation is complete, click View Operator.
- The operator details are displayed. In the Trident Orchestrator tile, click Create Instance.
- The YAML view is displayed for the configuration. Under nodePrep, add "- iscsi" to the path as shown below. Click Create.
kind: TridentOrchestrator
apiVersion: trident.netapp.io/v1
metadata:
name: trident
spec:
IPv6: false
debug: true
nodePrep:
- iscsi
imagePullSecrets: []
imageRegistry: ''
namespace: trident
silenceAutosupport: false- When the installation of the orchestrator is complete, it should appear with Status: Installed.
- The next steps for creating the storage class requires importing a YAML file either through the console UI, or from a desktop that has the 'oc' cli installed and kubeconfig file for the OpenShift cluster applied.
- Modify the following YAML by replacing information in the <> to match the environment where the operator is deployed, but don't forget to remove the brackets. If using command line instead of the console UI, first create a file called trident-backend.yaml with the follow code in it and then run the command 'oc create -f trident-backend.yaml'.
apiVersion: v1
kind: Secret
metadata:
name: trident-secret
type: Opaque
stringData:
username: <svmadmin> #The array user with permissions
password: <passw0rd> #Array user password
---
apiVersion: trident.netapp.io/v1
kind: TridentBackendConfig
metadata:
name: netapp-nas
spec:
version: 1
backendName: netapp-nas-backend
storageDriverName: ontap-nas
managementLIF: <IP Address> #IP address of SVM logical interface
svm: <svm> #Specify Storage Virtual Machine Name
storagePrefix: <svm> #prefix for created volumes
useREST: true
credentials:
name: trident-secret- This should have created a secret object in OpenShift storing the user account information in base64 format, followed by completing the backend configuration so the CSI driver knows how to contact the NetApp system.
- Next, create the Storage Class object using the code below.
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: netapp-sc-nfs
provisioner: csi.trident.netapp.io
parameters:
backendType: ontap-nas
provisioningType: "thin"
snapshots: "true"
allowVolumeExpansion: true
- The last YAML file to apply is to create the Volume Snapshot Class for the storage system.
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotClass
metadata:
name: netapp-snapshotclass
driver: csi.trident.netapp.io
deletionPolicy: Delete- If there will be more than one storage class in the cluster, at least one needs to be defined as default. These are the two commands to annotate the NetApp storage class as the default for deployments.
- oc patch storageclass netapp-sc-nfs -p '{"metadata": {"annotations": {"storageclass.kubernetes.io/is-default-class": "true"}}}'
- The volume snapshot class can also be set as the default using an annotation.
- oc patch volumesnapshotclass netapp-snapshotclass -p '{"metadata": {"annotations": {"snapshot.storage.kubernetes.io/is-default-class": "true"}}}'
That's it. Now when new persistent volumes are created, they'll appear in the Volumes tab of the ONTAP user interface with the prefix defined earlier.