Copying volumes between clusters using CSI Snapshots

Learn how to copy a persistent volume to a different cluster using CSI VolumeSnapshots

In this post we will be exploring how we can copy a persistent volume across to a different Kubernetes cluster running within the same cloud provider and region using Kubernetes native concepts.

This blog post will assume you are familiar with the following concepts:

• Kubernetes Persistent Volumes & Persistent Volume Claims
• Kubernetes CSI VolumeSnapshots

We have various runbooks available that we will be using to help us with copying the data across to a different kubernetes cluster:

• Creating VolumeSnapshots
• Importing VolumeSnapshots

Our clusters

In our case, we have two Kubernetes clusters called cluster-01 and cluster-02. Both are running within the same Cloud Account and Cloud Region.

❯ acloud clusters get
ID                                  	CLUSTER             	STATUS 	ORGANISATION	ENVIRONMENT	CLOUD ACCOUNT             	REGION       	VERSION
e2b3e65d-ff1c-4601-91dd-bf17c69c7ba3	cluster-01             	running	blog         	demo       	aws                       	eu-west-1    	v1.26.9-u-ame.3
4959cc68-d565-4de7-ad2a-29d3978a2744	cluster-02           	running	blog         	demo    	aws                       	eu-west-1    	v1.28.2-u-ame.3

Our goal is to copy a persistent volume called data-database-01 from cluster-01 to cluster-02.

Creating the Snapshot

In order to do this, we will first create a volumeSnapshot for the data-database-01 persistent volume claim, using acloud-toolkit snapshot create:

Connect to cluster-01:

acloud shell cluster-01

View PersistentVolumeClaims:

kubectl get persistentvolumeclaim -n default

Create the volumeSnapshot:

acloud-toolkit snapshot create -p data-database-01 data-database-01-snapshot-01

The snapshot create command will wait until the volumeSnapshot is ready to use. Once it completes, use snapshot ls to find the SNAPSHOT HANDLE:

❯ acloud-toolkit snapshot ls
NAMESPACE	NAME                                SOURCE                  CONTENT                                         	SIZE	READY	CLASSNAME  	SNAPSHOT HANDLE
platform 	data-database-01-snapshot-01    	pvc/data-database-01	snapcontent-Ec71c632-9c8d-4d87-bb02-472d67f4dcea	10Gi 	true 	ebs     	snap-009753350a1dac31c

In this case, the snapshot handle is snap-009753350a1dac31c. We will be using this to import into cluster-02.

Importing into cluster-02

Connect to cluster-02:

acloud shell cluster-02

Once we are connected, we can import the snapshot using acloud snapshot import:

acloud-toolkit snapshot import --name imported-snapshot snap-009753350a1dac31c
Creating snapshotcontent "imported-snapshot-385ffe5a-1964-45cb-9d4c-08436631ec72"..
Creating snapshot "imported-snapshot"..
Completed importing snapshot "snap-009753350a1dac31c" into CSI snapshot imported-snapshot..

Once it is imported, we can restore it into a persistent volume claim:

acloud-toolkit snapshot restore --restore-pvc-name data-database-01 --restore-storage-class ebs imported-snapshot

And now you have a ready to use persistent volume claim called data-database-01 in cluster-02, that is a copy of the PVC in cluster-01!

The imported volumeSnapshot will have it’s deletionPolicy set to Retain. This means that if we delete the volumeSnapshot in cluster-02, we will only delete the references for it. This can and done after restoring it to the PersistentVolumeClaim, since we no longer need it within this cluster.

Cleaning up

During this proces, we have created VolumeSnapshot and VolumeSnapshotContent resources in both clusters, as well as a snapshot in AWS. We can clean those up using kubectl delete:

In cluster-01, run the following:

kubectl delete volumesnapshot data-database-01-snapshot-01

This will remove the kubernetes resources for the snapshot, as well as clean-up the snapshot in AWS.