Verifying and migrating Elasticsearch data in Docker-based OpenIAM cluster

A need to migrate OpenIAM data might arise in case of VM corruption in one of the development environments. You might decide to build a new Docker-based cluster and, after successfully setting it up, transfer all old audit log data from the old cluster to the new one. To complete the transfer, follow the instructions below.

1. Verify all containers in the OpenIAM cluster. Run the following command to list all running containers.

docker ps

2. Use the following command to filter only the Elasticsearch container.

docker ps | grep elastic

3. Log in to the Elasticsearch container as root. Replace <elasticsearch-container-name> with the actual container name or ID when running the command below.

docker exec -it --user root <elasticsearch-container-name> bash

For example, the command might look as follows.

docker exec -it --user root a82b1cc7cd6d bash

4. Install the Elasticdump utility inside the container. Update the package lists and install Node.js and npm with the following command.

apt update && apt install -y nodejs npm

• Install elasticdump globally.

npm install -g elasticdump

• Verify that elasticdump is installed.

elasticdump --help

5. List all current indices inside the ES container. Retrieve the list of indices using the command below.

curl -X GET http://localhost:9200/_cat/indices?v

6. Check the audit log data. Run the following command to view audit log data.

curl localhost:9200/auditlog/_search?pretty

7. Take a backup (dump) of audit log data and export the audit log data...

elasticdump --input=http://localhost:9200/auditlog --output=auditlog.json --type=data

... and export the mapping of the audit log;

elasticdump --input=http://localhost:9200/auditlog --output=auditlog-mapping.json --type=mapping

... and the settings of the audit log.

elasticdump --input=http://localhost:9200/auditlog --output=auditlog-settings.json --type=settings

8. Copy Elasticsearch dump data from the container to localhost by running the following command.

docker cp <elasticsearch-container-name>:/auditlog.json .

9. Next, transfer the dump data to a destination server. You can use one of the following methods.

• WinSCP (GUI-based)
• SCP with the command below:

scp auditlog.json user@destination-server:/path/to/destination/

• SFTP using the following commands.

sftp user@destination-server
put auditlog.json

10. Import data on the destination server by copying the data from localhost to the container using the following command.

docker cp auditlog.json a82b1cc7cd6d:/

Then, import Elasticsearch dump data into the new server as shown below.

elasticdump --input=auditlog.json --output=http://localhost:9200/auditlog --type=data

11. Finally, log in to the OpenIAM UI, confirm that the migrated data is reflecting correctly, and check if all containers are running with the following command.

docker ps