## Restoring etcd on local cluster 

Restoring etcd is not a trivial task and there are several steps required, including modifying core files. Ensure the following pre-requisites are met before restoring etcd: 

GLYPH<SM590000> You have backed up the cluster at the infrastructure level 

GLYPH<SM590000> You have access to the master and management nodes 

GLYPH<SM590000> The etcd backup file exists on your local machine 

Most steps are run on the etcd/master nodes but the management nodes also require some actions. 

## Restoring etcd on single and High-Availability installations 

The restore procedure for etcd on both single and multi-master installations (where etcd is running on the master nodes) use the same set of steps. Ensure that any steps directed at master nodes are repeated on all master nodes. It’s also important to perform each step on all specified nodes before moving on to the next step, to ensure all nodes are at the same stage. This helps with reversing the process if a step should fail. Perform the following steps: 

1. On all master  nodes, create a backup directory and move the etcd pods to it. This will stop the etcd container, so verify that it is stopped: 

a. Create the backup directories: 

mkdir -p /etc/cfc/podbackup mkdir -p /var/lib/etcdbackup mkdir -p /var/lib/etcdwalbackup 

b. Move the pod data to the backup directory: 

mv /etc/cfc/pods/* /etc/cfc/podbackup/ 

c. Move the current etcd data to the backup directory: 

mv /var/lib/etcd/* /var/lib/etcdbackup/ mv /var/lib/etcd-wal/* /var/lib/etcdwalbackup/ 

d. Verify the etcd pod has stopped. 

docker ps | grep etcd 

2. On the master and management  nodes, stop kubelet and restart docker to stop all running pods: 

systemctl stop kubelet systemctl restart docker 

3. Copy the etcd snapshot file to all masters . 

4. Copy the script in Example 3-14 and run it on all masters . This script will retrieve the current environment information from the /etc/cfc/podbackup/etcd.json  file on the master nodes, then construct and execute a docker command to generate the new etcd data from the snapshot provided. 

When running the script ensure the etcd snapshot file name is passed as a parameter and it is located in the current directory (where the script is run), as the docker container will mount the current directory and use the snapshot file. 

#!/bin/bash [ -z "$1" ] && { echo "Please provide the etcd snapshot file name"; exit 1; } data_dir="/var/lib/etcd" restore_dir="/var/lib/etcd/restored" 

