ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-01-31 14:37:18 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
e4a7448935
clearml-docs/docs/deploying_clearml/clearml_server_mongo44_migration.md
pollfly e4a7448935
Add Server Upgrade instructions (#184)
2022-02-13 18:54:59 +02:00

5.0 KiB
Raw Blame History

title
Upgrading Server from v1.1 or Older

:::important If your server is v0.15 or older, you must first go through the v0.16 upgrade procedure and continue here once successfully completed. :::

In v1.2, the MongoDB subsystem of ClearML Server has been upgraded from version 3.6 to version 4.4. This change necessitates the migration of the database contents to be compatible with the new version.

:::note Kubernetes installations ClearML’s helm chart is already running mongodb version 4.4. If your ClearML server had been deployed with this chart (with the default mongodb bitnami chart) - You can stop reading here, as no migration is required. :::

The migration process is comprised of the following:

  • Backing up your data
  • Creating a copy of the data which will be migrated for use with the new version
  • Sequentially running data migration for MongoDB: 3.6->4.0, 4.0->4.2, 4.0->4.4

At the end of the process, your data will be available in the new location, ready to be used with ClearML Server v1.2.

Once ClearML Server v1.2 is up and running, and you are satisfied that your data is intact, you can safely delete the old copy of your data.

Prerequisites

  • Read/write permissions for the default ClearML Server data directory /opt/clearml/data and its subdirectories, or, if a different location had been used, permissions for that directory and its subdirectories.
  • Minimum free disk space of at least three times the size of the data.
  • Python version >=2.7 or >=3.6, and Python accessible from the command-line as python

Before You Begin

To avoid data corruption, shut down your ClearML server before applying the migration procedure:

  • Linux and macOS

    docker-compose -f /opt/clearml/docker-compose.yml down

  • Windows

    docker-compose -f c:\opt\clearml\docker-compose-win10.yml down

Migrating the Data

The following are the instructions for migrating your ClearML data from mongodb 3.6 to mongodb 4.4.

The commands are provided for a Linux environment, and assume ClearML is installed in its default paths.

If you are using a different OS, replace the shell and docker commands to the ones appropriate for your OS. If you’ve installed ClearML under a different path, replace the file locations accordingly.

:::info automating your migration A migration script is available to automatically run this process for all supported OSs. Download the script and run it on your ClearML server. Run clearml-server-1.2.0-migration.py –help to see execution options. Note the script will create a backup archive of your data in the original directory. :::

  1. Backup the source data folder:

    sudo tar -zcf mongodb_upgrade.gz -C /opt/clearml/data/mongo .

  2. Copy the data to the new folder and set permissions (permissions are required on Linux):

    sudo mkdir /opt/clearml/data/mongo_4
sudo tar -xzf mongodb_upgrade.gz -C /opt/clearml/data/mongo_4
sudo chown -R 1000:1000 /opt/clearml/data/mongo_4

  3. Run the mongo 4.0 container:

    sudo docker run -id -v /opt/clearml/data/mongo_4/db:/data/db -v /opt/clearml/data/mongo_4/configdb:/data/configdb --name mongodb_upgrade mongo:4.0.27

  4. Upgrade data to mongo 4.0:

    sudo docker exec mongodb_upgrade bash -c 'mongo --eval "db.adminCommand({setFeatureCompatibilityVersion:\"4.0\"})"'

    This command should return an { "ok" : 1 } response.

  5. Stop the mongo 4.0 container

    sudo docker stop mongodb_upgrade && sudo docker rm mongodb_upgrade

  6. Run the mongo 4.2 container:

    sudo docker run -id -v /opt/clearml/data/mongo_4/db:/data/db -v /opt/clearml/data/mongo_4/configdb:/data/configdb --name mongodb_upgrade mongo:4.2.16

  7. Upgrade data to mongo 4.2:

    sudo docker exec mongodb_upgrade bash -c 'mongo --eval "db.adminCommand({setFeatureCompatibilityVersion:\"4.2\"})"'

    This command should return an { "ok" : 1 } response.

  8. Stop the mongo 4.2 container

    sudo docker stop mongodb_upgrade && sudo docker rm mongodb_upgrade

  9. Run the mongo 4.4 container:

    sudo docker run -id -v /opt/clearml/data/mongo_4/db:/data/db -v /opt/clearml/data/mongo_4/configdb:/data/configdb --name mongodb_upgrade mongo:4.4.9

  10. Upgrade data to mongo 4.4:

    sudo docker exec mongodb_upgrade bash -c 'mongo --eval "db.adminCommand({setFeatureCompatibilityVersion:\"4.4\"})"'

    This command should return an { "ok" : 1 } response.

  11. Stop the mongo 4.4

    sudo docker stop mongodb_upgrade && sudo docker rm mongodb_upgrade

  12. Continue upgrading your ClearML Server: