Kubernetes Backup and Restore

Backup/Restore Data

Backing up and restoring data requires using the scripts introduced in Data Backup and Restore. The difference is that metadb backup and restore are executed in the metadb-0 container, engine backup and restore are executed in the master-0 container, and minio backup and restore are executed in the minio-0 container.

Please Note

In the k8s environment, only the local backup method is used, and the default backup is stored in the /opt/hsdata/backup directory.

Preparation Before Data Recovery

Before recovering metadb and engine data, the hengshi-sense service needs to be stopped to avoid any impact on the data recovery results. This step can be skipped for minio data recovery.

• Example of stopping the hengshi-sense service.

kubectl -n hengshi scale --replicas=0 deployment hengshi-sense

Backup metadb

kubectl -n hengshi exec -it metadb-0 -- mkdir -p /opt/hsdata/backup
kubectl -n hengshi exec -it metadb-0 -- /opt/hengshi/bin/dbbackup.sh -m metadb -l /opt/hsdata/backup -t sql.gz

Restore metadb

kubectl -n hengshi exec -it metadb-0 -- /bin/bash ## Enter the metadb-0 container and execute the following operations
/opt/hengshi/bin/hengshi-sense-bin stop metadb single
sudo mv ${HSDATA}/pg_data ${HSDATA}/pg_data.old
/opt/hengshi/bin/hengshi-sense-bin init metadb single
/opt/hengshi/bin/hengshi-sense-bin start metadb single
/opt/hengshi/bin/dbrestore.sh -m metadb -l /opt/hsdata/backup -t metadb_backup.xx-xx-xx_xx-xx-xx[.tar.gz]

• After the data is successfully restored, confirm the data accuracy, and then delete the old historical data.

kubectl -n hengshi exec -it metadb-0 -- sudo rm -rf ${HSDATA}/pg_data.old

Backup and Restore Engine Data

Follow the steps in the example to back up and restore engine data.

Backup engine data

kubectl -n hengshi exec -it master-0 -- mkdir -p /opt/hsdata/backup
kubectl -n hengshi exec -it master-0 -- /opt/hengshi/bin/dbbackup.sh -m engine -l /opt/hsdata/backup -t sql.gz

Restore engine data

kubectl -n hengshi exec -it master-0 -- /bin/bash  ## Enter the master-0 container and execute the following operations
/entrypoint.sh -m stopsystem
for host in ${MASTER} ${SEGMENTS};do ssh $host "sudo mv ${HSDATA}/engine-cluster ${HSDATA}/engine-cluster.old";done
/entrypoint.sh -m initsystem
/entrypoint.sh -m startsystem
/opt/hengshi/bin/dbrestore.sh -m engine -l /opt/hsdata/backup -t engine_backup.xx-xx-xx_xx-xx-xx[.tar.gz]

• After the data is successfully restored, confirm the data accuracy, and then delete the old historical data.

kubectl -n hengshi exec -it master-0 -- for host in ${MASTER} ${SEGMENTS};do ssh $host "sudo rm -rf ${HSDATA}/engine-cluster.old";done

Backup minio

kubectl -n hengshi exec -it inio-0 -- mkdir -p /opt/hsdata/backup
kubectl -n hengshi exec -it inio-0 -- /opt/hengshi/bin/dbbackup.sh -m inio -l /opt/hsdata/backup

Restore Minio

kubectl -n hengshi exec -it minio-0 -- /bin/bash ## Enter the metadb-0 container and execute the following operations
/opt/hengshi/bin/hengshi-sense-bin stop minio single
sudo mv ${HSDATA}/minio-data ${HSDATA}/minio-data.old
/opt/hengshi/bin/dbrestore.sh -m minio -l /opt/hsdata/backup -t minio_backup.xx-xx-xx_xx-xx-xx.tar.gz
/opt/hengshi/bin/hengshi-sense-bin start minio single

• After the data is successfully restored, confirm the data accuracy, and then delete the old historical data.

kubectl -n hengshi exec -it minio-0 -- sudo rm -rf ${HSDATA}/minio-data.old

Operations After Data Recovery Completion

After the data recovery of metadb and engine is completed, proceed to recover the hengshi-sense service. The recovery of minio data can be skipped for this step.

• Example of hengshi-sense service recovery.

hs_pods_num=  # Number of hengshi-sense service pods, fill in according to the number before stopping
kubectl -n hengshi scale --replicas=${hs_pods_num} deployment hengshi-sense