Migrating a Kubernetes PV to a new storage class

Kubernetes provides an abstraction layer for persistent data storage. Volumes can be of various storage classes depending on user requirements. Those requirements can evolve with time, meaning that volumes sometimes need to be migrated from one storage class to another. This article introduces a method to do so.

As an example, let's start with a MinIO pod, where data is stored using a storage class called nfs-csi. This setup could be created with a manifest such as the following:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: minio-nfs
spec:
  storageClassName: nfs-csi
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 500Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: minio
spec:
  volumes:
    - name: minio-data
      persistentVolumeClaim:
        claimName: minio-nfs
  containers:
    - name: minio
      image: quay.io/minio/minio
      command:
        - /bin/bash
        - -c
      args:
        - minio server /data --console-address :9001
      volumeMounts:
        - mountPath: /data
          name: minio-data

Now let's imagine we want MinIO to use a new storage class, Longhorn. Thins involves creating a new set of PVC and PV. Assuming the lonhgorn storage class already exists in the cluster: This can be done using the following manifest:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: minio-longhorn
spec:
  storageClassName: longhorn
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 500Gi

The data now needs to be moved from the original PV to the new one. For this purpose, let's start by deleting the MinIO pod:

kubectl delete pod minio

In its place, we will deploy a Ubuntu pod with the PVC mounted to /mnt

apiVersion: v1
kind: Pod
metadata:
  name: ubuntu
spec:
  containers:
  - image: ubuntu
    command:
      - "sleep"
      - "604800"
    name: ubuntu
    volumeMounts:
      - mountPath: /mnt
        name: mnt
  restartPolicy: Always
  volumes:
    - name: mnt
      persistentVolumeClaim:
        claimName: minio-nfs

With the pod deployed, we can leverage kubectl cp to copy data from the PV to our local machine in a temporary folder:

kubectl cp ubuntu:/mnt ./tmp

Once done, we can delete the Ubuntu pod and recreate it by changing the spec.volumes.persistentVolumeClaim.claimName to minio-longhorn. The newly create Ubuntu pod now has the new PVC mounted to /mnt. This allows us to copy the content of the temporary folder into the PV:

microk8s.kubectl cp ./tmp/. ubuntu:/mnt

After the data has been transferred to the new PV, the Ubuntu pod can once again be deleted.

kubectl delete pod minio

The spec.volumes.persistentVolumeClaim.claimName of the MinIO pod can now be changed to longhorn as well, after which the pod can be redeployed.

The MinIO pod should now be using Longhorn as a storage class and have its data from the original nfs-csi PV.