[null, null, null, {"document_url": "https://severalnines.com/blog/how-and-performance-impact-ssl-encrypted-replication-traffic-galera-cluster-mysql", "unformatted_src": "/sites/default/files/enc_vars.png", "src": "https://severalnines.com/sites/default/files/enc_vars.png", "formatted_filename": "enc vars"}, null, {"document_url": "https://severalnines.com/blog/how-and-performance-impact-ssl-encrypted-replication-traffic-galera-cluster-mysql", "unformatted_src": "/sites/default/files/enc_writes.png", "src": "https://severalnines.com/sites/default/files/enc_writes.png", "formatted_filename": "enc writes"}, null, {"document_url": "https://severalnines.com/blog/how-and-performance-impact-ssl-encrypted-replication-traffic-galera-cluster-mysql", "unformatted_src": "/sites/default/files/enc_writes_host.png", "src": "https://severalnines.com/sites/default/files/enc_writes_host.png", "formatted_filename": "enc writes host"}, null, null, {"document_url": "https://severalnines.com/blog/how-and-performance-impact-ssl-encrypted-replication-traffic-galera-cluster-mysql", "unformatted_src": "/sites/default/files/enc_db.png", "src": "https://severalnines.com/sites/default/files/enc_db.png", "formatted_filename": "enc db"}, null, {"document_url": "https://severalnines.com/blog/how-and-performance-impact-ssl-encrypted-replication-traffic-galera-cluster-mysql", "unformatted_src": "/sites/default/files/enc_db_host.png", "src": "https://severalnines.com/sites/default/files/enc_db_host.png", "formatted_filename": "enc db host"}, null, null, null, null, null, null, null]


How-to and Performance Impact of SSL-Encrypted Replication Traffic in Galera Cluster for MySQL

Performance Management Security & Compliance Deploying Galera Clusters across WAN environments might lead to concerns around data privacy and security - especially as more organisations are having to comply with national and international regulations. You would not want hackers eavesdropping or intercepting replication traffic. Encrypted replication hides what is sent between the Galera nodes, and makes sure each node is only communicating to the ones it trusts. But how expensive is encryption? In this blog, we will show you how to encrypt the replication traffic between your Galera nodes. We will also look into the performance impact of this encryption.

Encrypted Replication

Galera supports SSL for the encryption of replication traffic. When encryption is enabled, Galera group communication and Incremental State Transfer (IST) happens over an SSL-encrypted connection. The default SST methods do not support SSL, however this can be scripted.

All nodes in a Galera Cluster communicate via a group communication protocol (default port is 4567), whereby writesets are replicated from originating nodes to the rest of the nodes for certification and possibly commit. This communication protocol can be encrypted with an SSL certificate and a private key.

Note that this does not encrypt the communication between the MySQL client (the application) and the MySQL instances. Finally, at the time of writing, some extra steps are required when adding nodes automatically via ClusterControl.

Enabling Encrypted Replication

Enabling encryption is now supported by the s9s tools available from our Github repository. This feature will be included in the next release of ClusterControl, scheduled by the end of August. The implementation flow is based on Codership’s Galera Cluster documentation.

The following requirements must be met if you would like to run encrypted Galera replication:

To have this functionality with the current version of ClusterControl (1.2.6 or earlier), clone the s9s-admin Github repository on the ClusterControl host:

Or, if you already have the repository cloned, just pull the latest changes:

Update the s9s_galera located in /usr/bin with the latest one from Github:

To enable Galera encrypted replication with ClusterControl, run the following command:

This action will generate a 2048-bit key and certificate on the ClusterControl node and transfer them to all the Galera nodes. Then, it will stop the Galera cluster (cluster ID 1) and configure wsrep_provider_options in the MySQL config files. Once done, it will start the cluster by re-bootstrapping it.

Due to the HeartBleed bug, the tool will automatically update the OpenSSL package to the latest version using package manager before generating keys and certificates. If your ClusterControl host does not have an internet connection, you may skip this step by appending -s in the command line. Also, the default 2048-bit key can be overridden with -b option.

To verify whether encryption has been enabled, use the status parameter:

You can also verify this using the ClusterControl UI, under Performance > DB Variables > wsrep_provider_options, similar to the screenshot below: From the Galera point-of-view, we can see from the MySQL error log file that SSL handshakes were successful:


Disabling encryption is as easy as enabling it:

This will remove any configuration options in the MySQL configuration file under wsrep_provider_options related to SSL parameters. Take note that adding a new node into an encrypted Galera cluster is yet to be included in the next ClusterControl release. As a workaround, you can disable the encryption prior to adding node and enable it back through command line.


Let’s look at the performance impact of encrypting the replication traffic. We did a simple benchmark using Sysbench 0.5 on a three-node MySQL Galera Cluster 5.6 (Codership) running on AWS EC2 m3.xlarge (4 CPUs, 15 GB RAM, 50 GB SSD - 300 IOPS). We deployed using the Galera Configurator and using the default 2048-bit private key. Sysbench distributes connections between specified MySQL hosts on a round-robin basis.

These tests used the same data set, we prepared 20 million records (~5.5 GB of data) with the following command:

The cluster was re-bootstrapped after each test.

Test 1: Write Performance

We performed a sysbench OLTP complex test and disabled all reads to test the write performance with 5 million requests and 32 threads using the following command:

The above command performed the following number of db operations:

Plain replication took 4042s to complete the test while encrypted replication took 5106s, indicating a 26% performance drop. The following graph captured from ClusterControl shows the number of write operations performed on the cluster (the higher the better): Pay attention to the CPU and network usage between plain and encrypted replication: Test 2: Read Performance

Next, we performed a sysbench OLTP read-only test mode to test the read performance with 1 million requests and 32 threads using the following command:


The buffer pool was pre-warmed prior to this test to get predictable result. Encryption does not really affect the read performance since there was no writeset replication involved during the benchmarking activity. This can be confirmed with the following graphs captured by ClusterControl: Despite the early spike, the rest of the read operations are in the same range, between 3000 to 4000 reads per seconds. In fact, the total execution time for plain replication was 4277s (233 tps) while encrypted replication was 4333s (231 tps). Not much difference for an hour of read operations. This is also expected as only writes are distributed between the nodes and subject to encryption.

Test 3: Mixed Read/Write Performance

This test used OLTP complex test as we tried to produce a mixed workload consisting of 65% reads and 35% writes with 1 million requests and 32 threads using following command:

The above command performed the following number of database operations:

With plain replication, the test took 2549s (392 tps) to complete while with encrypted replication, it took 2824s (354 tps) indicating 11% performance drop from the former. The following graph captured from ClusterControl shows the number of queries performed on the cluster (the higher the better): As expected for the CPU and network usage, we can see the difference in the following graphs: The performance impact of encrypting Galera replication traffic seems to be reasonable. However, this might not necessarily apply to your workload. It depends on your network performance, private key (a key can have 1024, 2048, 4096 bits, higher is slower), configuration parameters as well as your hardware. We would recommend you to set up your own Galera Cluster and give it a try - since SSL can now be turned on/off in one single command using the s9s tools. galera MySQL ssl encryption replication