Problem: You have installed Check_MK Raw Edition (CRE) on your server running Ubuntu Server / Debian. You want to perform a release / distribution upgrade (for example from Ubuntu Server 14.04 to 16.04 or from Debian Jessie to Debian Stretch). How to perform the distribution upgrade while Check_MK is installed? Consider that you have to change the installed Check_MK version for the new distribution release as well. How to do this?

Solution:

Important: It should be noted that a distribution / release upgrade is a potentially harmful / dangerous process. It should only be done if you have recent backups of the system and sufficient systems adminstration knowledge to fix potential issues.

Step 1: Upgrading your Check_MK instances

First, ideally you should upgrade all OMD sites (Check_MK instances) on the server to the current version, if possible. While this step is optional in case the Check_MK version you are currently using does have a version for the distribution release you are upgrading to (you can check at the Check_MK download archive), it is highly recommended. To do this:

Run omd sites as root to see all installed sites with their versions:

SITE             VERSION          COMMENTS
monitoring       1.5.0p14.cre     default version

Download the newest Check_MK Raw Edition version from the Check_MK Website. Use the version for your current distribution release (before the upgrade).

wget https://mathias-kettner.de/support/1.5.0p15/check-mk-raw-1.5.0p15_0.trusty_amd64.deb

(The link provided in the above command is an example and will not be updated, please get the newest version for your OS from the Check_MK website.)

Install the new version:

gdebi check-mk-raw-1.5.0p15_0.trusty_amd64.deb

When running omd versions you should now see the new version since it is installed.

Switch into the user of your OMD site:

sudo su - monitoring

Stop the OMD site, then update it:

omd stop
omd update

Check that the new version is now installed for this site:

omd sites

Start the site again:

omd start

Check incompatible werks by logging into WATO and click on the red ball on the top left. Acknowledge them. Proceed in the same way with all other installed sites.

Step 2: Preparing all Check_MK instances for the distribution upgrade

As root, disable all OMD sites:

omd disable <SITE>

For example:

omd disable monitoring

Run the command

ls -l /omd/sites/*/version

This command will show you the symlink „version“ for each OMD site:

lrwxrwxrwx 1 monitoring monitoring 27 Apr 20 00:45 /omd/sites/monitoring/version -> ../../versions/1.5.0p15.cre

Save this output, so that you can restore the correct symlinks later. Now delete the symlinks:

rm /omd/sites/*/version

Now, check for all installed Check_MK packages:

dpkg -l | grep check-mk

Delete all packages with the Status „ii“ using apt-get:

apt-get remove <package-names>

For example:

apt-get remove check-mk-raw-1.5.0p14 check-mk-raw-1.5.0p15

Step 3: Perform the distribution upgrade

You may now perform the distribution / release upgrade, as explained in the instructions provided by your linux distribution. For example, for Ubuntu Server, you would usually run do-release-upgrade. You should check that the upgrade was successful and all services, except for Check_MK, are running and behaving as expected. Then proceed.

Step 4: Re-enabling your Check_MK instances

After performing the distribution / release upgrade, download the Check_MK Version for your new upgraded distribution release and install it via gdebi. If you’ve previously downloaded, installed and activated the newest Check_MK version for all sites, it is enough to install it. Otherwise, you have to install all the versions currently in use on your sites for the newly updated OS release from the Check_MK download archive.

wget https://mathias-kettner.de/support/1.5.0p15/check-mk-raw-1.5.0p15_0.xenial_amd64.deb
gdebi check-mk-raw-1.5.0p15_0.xenial_amd64.deb

(The link provided in the above command is an example and will not be updated, please get the correct versions for your upgraded OS from the Check_MK website.)

Next, recreate the previously deleted symlinks. In our example:

ln -s ../../versions/1.5.0p15.cre /omd/sites/monitoring/version

Change owner / group of the symlinks to match the output of the ls command which you previously ran. Usually, owner and group should be the name of the OMD site. Make sure to pass the --no-dereference option, to change the ownership of the symlink, not of the folder it links to:

chown --no-dereference monitoring:monitoring /omd/sites/monitoring/version

Run ls -l /omd/sites/*/version once again, and make sure it matches the previously saved output.

Re-enable the sites:

omd enable monitoring

and then start them:

omd start monitoring

Your Check_MK instances should now be working again.

Step 5: Remove unneeded, old Check_MK versions

In case you only use the current Check_MK version now, or you have other versions, which are not in use anymore, you can purge old versions, as they are not needed. For example:

# dpkg -l | grep check-mk
rc  check-mk-raw-1.5.0p14                 0.trusty                                      amd64        Check_MK is a full featured system monitoring
ii  check-mk-raw-1.5.0p15                 0.xenial                                      amd64        Check_MK is a full featured system monitoring

# omd sites
SITE             VERSION          COMMENTS
monitoring       1.5.0p15.cre     default version

→ Version 1.5.0p14 is not in use anymore, so we can purge it and do not have to install the older version for our current distribution release:

dpkg --purge check-mk-raw-1.5.0p14