Patch Updates and New Version Upgrades of Your Database Clusters

Database vendors regularly issue critical patch updates to address software bugs or known vulnerabilities, but for a variety of reasons, organizations are often unable to install them in a timely manner, if at all. Evidence suggests that companies are actually getting worse at patching databases, with an increased number violating compliance standards and governance policies¹.

Patching that require database downtime would be of extreme concern in a 24*7 environment, however most cluster upgrades can be performed online. ClusterControl performs a rolling upgrade of the cluster, upgrading and restarting one node at a time. The logical upgrade steps might slightly differ between the different cluster types.

Upgrading your Database

In this post, we are going to show you how to upgrade your database cluster. Some things to be aware of:

• Backup your database before performing any upgrade,
• Upgrade requires a restart of the DB service on each node,
• Verify the installation package is compatible with your machine’s architecture and does not conflict with other packages,
• Note your current database version, available under General Settings page in the ClusterControl UI,
• Make sure you understand the new features in the version you plan to upgrade to.

MySQL Galera Cluster

If you choose Codership as the vendor in our Online Configurator page, you will see there are some available packages listed under Software Packages. Codership does not provide a package manager repository, so a standard package installation is the way.

Prior to upgrading, go to Software P

ackages to create a new package name and upload the latest version of galera and MySQL-server binary release. Select the new package as “Selected For Deployment” and Save:

Next, go to Manage >> Upgrades and start the upgrade process by clicking “Install” button. An upgrade job will be created under Logs >> Jobs and you can monitor the process by clicking refresh button.

Note, in order to upgrade to MySQL Server wsrep 24.8 and galera 24.2.7 you need to upgrade to ClusterControl 1.2.3b first on each node. Assuming you are on 1.2.3, do:

Debian/Ubuntu:

$ wget https://staging1.severalnines.com/downloads/cmon/cmon-1.2.3b-64bit-glibc23-mc70.tar.gz
$ tar xvfz cmon-1.2.3b-64bit-glibc23-mc70.tar.gz -C /usr/local/
$ rm -rf /usr/local/cmon && ln -s /usr/local/cmon-1.2.3b-64bit-glibc23-mc70 /usr/local/cmon
$ service cmon restart

Redhat/CentOS/Fedora:

$ wget https://staging1.severalnines.com/downloads/cmon/cmon-controller-1.2.3b-1.x86_64.rpm
$ rpm -Uvh cmon-controller-1.2.3b-1.x86_64.rpm
$ service cmon restart

Percona XtraDB Cluster/MariaDB Galera Cluster

Percona and MariaDB have their own package repositories which allow the database software to be managed by the package manager. Hence, the database software is not listed under Software Packages unlike the Codership version. ClusterControl will trigger the upgrade process by sending commands to the package manager and monitoring the output of these processes.

To upgrade, just click the “Install” button on Step 1 on Manage >> Upgrades page:

ClusterControl will upgrade the database software on all DB nodes by utilizing package manager, ignoring the selected “Package Name” that appears in the list.

Note , in order to upgrade to MySQL Server wsrep 24.8 and galera 24.2.7 you need to upgrade to ClusterControl 1.2.3b first, please see the end of the previous section “MySQL Galera Cluster”.

MySQL Replication

For Redhat-based systems, download the related RPM packages from the MySQL download site. For Debian-based systems, download the compressed TAR archive. Prior to upgrade, upload the downloaded packages into the ClusterControl repository through Software Packages page.

Note that maximum file upload size will depend on post_max_size and upload_max_filesize as configured in PHP. In some cases, you might need to increase the value higher than 256MB to allow bigger files to be successfully uploaded (as shown in the screenshot above).

To upgrade, just click the “Install” button on Step 1 at Manage >> Upgrades page:

Once the installation has completed, perform a rolling restart to complete the upgrades by clicking the “Rolling Restart” button on Step 2a. Finally, log into each of the DB node to upgrade the MySQL structure by executing this command:

$ mysql_upgrade

Note that a separate mysql_upgrade step will not be required in ClusterControl v1.2.4, as it will be performed automatically.

MySQL Cluster

Similar to MySQL Replication, the Severalnines Configurator deploys MySQL Cluster using the latest binary release available from the MySQL download site. For Redhat-based systems, download the related RPM packages. For Debian-based system, download the compressed TAR archive.

In this example, we are going to upgrade MySQL Cluster from version 7.2.10 to 7.3.2. In this case, it is recommended to upgrade to the latest minor release of 7.2 (which is 7.2.13) before we proceed to upgrade to major release 7.3.

Prior to upgrade, upload the downloaded packages into the ClusterControl repository through the Software Packages page:

Upgrade to 7.2.13, by clicking the “Install” button on Step 1 at Manage >> Upgrades page and make sure you select the correct package on the list. Once the job has completed, perform a rolling restart by clicking “Rolling Restart”. You will notice that the cluster has been upgraded to 7.2.13 under General Settings page:

Finally, log into each of the DB node to upgrade MySQL structure by executing this command:

$ mysql_upgrade

Note that from version 1.2.4, ClusterControl will automatically run the mysql_upgrade command.

7.3 can be considered as a major release, and needs to be done manually. A backup and restore using mysqldump is recommended. Enterprise customers are welcome to contact us for upgrading from 7.2.x to 7.3.x, since it may not be safe in all cases.

MongoDB (10gen)

To upgrade MongoDB, run following commands on each of DB nodes:

Redhat/CentOS/Fedora:

$ yum install -y mongo-10gen mongo-10gen-server

Debian/Ubuntu:

$ sudo apt-get -y install mongodb-10gen

Kill all mongod and mongos processes, and let ClusterControl recover them:

$ killall -15 mongod mongos

You may perform this upgrade one node at a time so as to avoid downtime.

TokuMX (Tokutek)

Upgrading TokuMX requires you to download the latest binary release which available at TokuMX download page. In this example, we are going to upgrade TokuMX from version 1.1.2 to 1.2.1.

On each DB node, perform the upgrade as below:

$ tar -xzf tokumx-1.2.1-linux-x86_64.tgz -C /usr/local
$ rm /usr/local/tokumx
$ ln -sf tokumx-1.2.1-linux-x86_64 /usr/local/tokumx
$ killall -15 mongod

ClusterControl will restart the killed mongod instance for that particular node and report a new PID with a green tick icon indicating the mongod process is started and running.

If you choose to start the instance manually, use following command:

$ /usr/local/tokumx/bin/mongod -f /etc/mongodb.27017.conf --logpath /var/log/mongodb/mongodb_27017.log --pidfilepath /var/run/mongodb/mongodb_27017.pid --shardsvr

Post Upgrade

1. Verify all database instances are running on each DB node under Manage Processes page,
2. Verify that the DB version is correctly updated under General Settings page,
3. Take advantage of the new features of your upgraded database. E.g., for your Replication Cluster, enable GTID by updating your DB configurations under Manage Configurations page,
4. For subscription customers, you are welcome to contact us for assistance with your database upgrades.

¹ Public companies must adhere to data security controls outlined in Sarbanes-Oxley (SOX), organizations handling private medical records are subject to Health Insurance Portability and Accountability Act (HIPAA), and anyone accepting credit cards or debit cards must follow the provisions of the Payment Card Industry Data Security Standard (PCI DSS).