The ibmcom/etcd:3.2.24 image is the default etcd image for IBM Cloud Private 3.1.2. If the environment does not have internet access, replace ibmcom  with the private image registry local to the cluster, for example, mycluster.icp:8500/ibmcom/etcd:3.2.24 . 

3. Create the job: 

[root@icp-ha-boot ~]# kubectl create -f etcd-backup_job.yaml job.batch "etcddump" created 

4. Verify that the job was successful: 

[root@icp-ha-boot ~]# kubectl -n kube-system get jobs etcddump NAME DESIRED SUCCESSFUL AGE etcddump 1 1 1m 

5. Verify that the etcd snapshot exists: 

[root@icp-nfs etcd]# ls -sh etcddump* 30M etcddump.2019-02-26_04:49:06.db 

## Backup MongoDB using kubectl 

Perform the following steps on any machine that has kubectl  configured to access to the cluster: 

1. Get the MongoDB pod name: 

[root@icp-ha-boot ~]# kubectl -n kube-system get pods -l app=icp-mongodb -o=jsonpath='{.items[0].metadata.name}' icp-mongodb-0 

2. Retrieve the primary replica: 

[root@icp-ha-boot ~]# kubectl -n kube-system exec icp-mongodb-0 -n kube-system -- sh -c 'mongo --host rs0/mongodb:27017 --username admin --password $ADMIN_PASSWORD --authenticationDatabase admin --ssl --sslCAFile /data/configdb/tls.crt --sslPEMKeyFile /work-dir/mongo.pem --eval="db.isMaster()["primary"]" | grep ^icp-mongodb-' icp-mongodb-1.icp-mongodb.kube-system.svc.cluster.local:27017 

3. Run the mongodump  command using the primary replica retrieved from step 2: 

[root@icp-ha-boot ~]# kubectl -n kube-system exec icp-mongodb-0 -- sh -c 'mkdir -p /work-dir/backup/; mongodump --host icp-mongodb-1.icp-mongodb.kube-system.svc.cluster.local:27017 --username admin --password $ADMIN_PASSWORD --authenticationDatabase admin --ssl --sslCAFile /data/configdb/tls.crt --sslPEMKeyFile /work-dir/mongo.pem --oplog --quiet --gzip --archive="/work-dir/backup/backup.gz"' 

4. Copy the backup.gz  file from the icp-mongodb-0  container to the local machine: 

[root@icp-ha-boot ~]# kubectl cp kube-system/icp-mongodb-0:/work-dir/backup/backup.gz . 

5. Verify that the backup exists: 

[root@icp-ha-boot ~]# ls -sh . total 972K 972K backup.gz 

## Backup MongoDB using a Kubernetes job 

MongoDB can be backed up to a PersistentVolume using a Kubernetes job. For security and isolation from the worker nodes, the pods created in this job are scheduled to a master node. Perform the following steps on any machine that has kubectl  configured to access to the cluster: 

