Platform Engineering

Appearance

Cookbook: Kubernetes Resource Migration using Velero ​

In this scenario, you have two Kubernetes clusters in completely separate environments (e.g., different clouds or data centers). Each cluster's kubectl and Velero CLI run on separate control machines. Backups will be stored in a shared S3 bucket (or S3-compatible object storage) that is accessible from both environments.

Prerequisites ​

  • Velero CLI installed on the source control machine (managing the source cluster) and on the destination control machine (managing the destination cluster).
  • Velero server (deployment) installed in both the source (Huawei Cloud) and destination (AWS) clusters, each pointing to the shared backup bucket.
  • kubectl configured on each control machine with context for its respective cluster (kubectl config get-contexts).
  • A shared S3 bucket (or S3-compatible object storage) accessible from both control machines and both Kubernetes clusters.
  • Credentials (access keys or IAM roles) for the S3 bucket configured on both control machines and within Velero's backup storage configuration.

Step 1: On the source control machine, set Kubernetes context for the source cluster (Huawei Cloud) ​

bash
kubectl config use-context <huawei-cloud-context>

Replace <huawei-cloud-context> with the actual kubeconfig context name for the Huawei Cloud Kubernetes cluster.

Step 2: On the source control machine, take Velero backups for namespaces ​

  1. Backup all resources in namespaces billing-guance-intl and guance-basis (backups will be uploaded to the shared S3 bucket):
bash
velero backup create billing-guance-intl-backup --include-namespaces billing-guance-intl
velero backup create guance-basis-backup --include-namespaces guance-basis
  1. Backup only the redis-basis namespace resources from the middleware namespace:

If redis-basis is a namespace, back it up directly:

bash
velero backup create redis-basis-backup --include-namespaces redis-basis

If redis-basis is a resource inside the middleware namespace, use label selectors or resource filters to back it up selectively (Velero does not support resource-level backup easily, but typically you would backup the whole namespace or exclude resources):

bash
velero backup create middleware-redis-basis-backup --include-namespaces middleware --include-resources deployments,services,configmaps,ingresses --selector app=redis-basis

Confirm the label selector matches the redis-basis resources.

Step 3: On the source control machine, verify the backups are complete and present in S3 ​

bash
velero backup describe billing-guance-intl-backup
velero backup describe guance-basis-backup
velero backup describe redis-basis-backup

Check backup logs and ensure status is Completed.

Step 4: On the destination control machine, set Kubernetes context for the destination cluster (AWS Singapore) ​

bash
kubectl config use-context <aws-singapore-context>

Replace <aws-singapore-context> with the actual kubeconfig context for the AWS Singapore cluster.

Step 5: On the destination control machine, restore the backups to the AWS cluster ​

  1. Restore billing-guance-intl namespace resources:
bash
velero restore create --from-backup billing-guance-intl-backup
  1. Restore guance-basis namespace resources:
bash
velero restore create --from-backup guance-basis-backup
  1. Restore redis-basis resources in the middleware namespace (adjust if applicable):
bash
velero restore create --from-backup redis-basis-backup

Step 6: On the destination control machine, verify resource restoration ​

bash
kubectl get all -n billing-guance-intl
kubectl get all -n guance-basis
kubectl get all -n middleware

Check if deployments, services, configmaps, ingresses, and other resources are restored as expected.

Notes ​

  • Modify the Velero backup and restore commands as necessary if there are custom resource definitions (CRDs) that need to be included.
  • If persistent volume data migration is required, additional Velero plugins or methods may be needed, but this cookbook focuses on resource migration only.
  • Ensure the shared S3 bucket retention and lifecycle policies meet your organizational requirements.
  • After migration, perform the configmap or secret modifications as defined in the project documentation (e.g., updating database URLs, access endpoints, etc.).