Test Scaling

More followers can be added to the MySQL Cluster to increase read capacity. This can be done by running the following command.

kubectl -n mysql scale statefulset mysql --replicas=3

You can see the message that StatefulSet “mysql” scaled. 


statefulset "mysql" scaled

Watch the progress of ordered and graceful scaling.

kubectl -n mysql rollout status statefulset mysql


Waiting for 1 pods to be ready...
partitioned roll out complete: 3 new pods have been updated...

It may take few minutes to launch all the pods.

Open another terminal to check loop if you closed it.

kubectl -n mysql run mysql-client-loop --image=mysql:5.7 -i -t --rm --restart=Never --\
   bash -ic "while sleep 1; do mysql -h mysql-read -e 'SELECT @@server_id,NOW()'; done"

You will see 3 servers are running. 


+-------------+---------------------+
| @@server_id | NOW()               |
+-------------+---------------------+
|         100 | 2020-01-25 02:32:43 |
+-------------+---------------------+
+-------------+---------------------+
| @@server_id | NOW()               |
+-------------+---------------------+
|         102 | 2020-01-25 02:32:44 |
+-------------+---------------------+
+-------------+---------------------+
| @@server_id | NOW()               |
+-------------+---------------------+
|         101 | 2020-01-25 02:32:45 |
+-------------+---------------------+

Verify if the newly deployed follower (mysql-2) have the same data set by following command.

kubectl -n mysql run mysql-client --image=mysql:5.7 -i -t --rm --restart=Never --\
 mysql -h mysql-2.mysql -e "SELECT * FROM test.messages"

It will show the same data that the leader has. 


+--------------------------+
| message                  |
+--------------------------+
| hello, from mysql-client |
+--------------------------+

Scale down replicas to 2 by running the following command.

kubectl -n mysql  scale statefulset mysql --replicas=2

You can see StatefulSet “mysql” scaled 


statefulset "mysql" scaled

Note that scale in doesn’t delete the data or PVCs attached to the pods. You have to delete them manually.

Check scale in is completed by following command.

kubectl -n mysql get pods -l app=mysql


NAME      READY     STATUS    RESTARTS   AGE
mysql-0   2/2       Running   0          1d
mysql-1   2/2       Running   0          1d

Check data-mysql-2 PVCs still exist by following command.

kubectl -n mysql  get pvc -l app=mysql


NAME           STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
data-mysql-0   Bound    pvc-fdb74a5e-ba51-4ccf-925b-e64761575059   10Gi       RWO            mysql-gp2      18m
data-mysql-1   Bound    pvc-355b9910-c446-4f66-8da6-629989a34d9a   10Gi       RWO            mysql-gp2      17m
data-mysql-2   Bound    pvc-12c304e4-2b3e-4621-8521-0dc17f41d107   10Gi       RWO            mysql-gp2      9m35s

Challenge

By default, deleting a PersistentVolumeClaim will delete its associated persistent volume. What if you wanted to keep the volume?

Change the reclaim policy of the PersistentVolume associated with PersistentVolumeClaim called “data-mysql-2” to “Retain”. Please see Kubernetes documentation for help

Expand here to see the solution

Delete data-mysql-2 with following commands.

kubectl -n mysql delete pvc data-mysql-2


persistentvolumeclaim "data-mysql-2" deleted