-Important: Upgrading from v0.14 or older
-
- For Linux only, if upgrading from **Trains Server** v0.14 or older, configure the **ClearML Agent Services**.
-
- * If ``CLEARML_HOST_IP`` is not provided, then **ClearML Agent Services** will use the external public address of the
- **ClearML Server**.
- * If ``CLEARML_AGENT_GIT_USER`` / ``CLEARML_AGENT_GIT_PASS`` are not provided, then **ClearML Agent Services** will
- not be able to access any private repositories for running service tasks.
-
-
- export CLEARML_HOST_IP=server_host_ip_here
- export CLEARML_AGENT_GIT_USER=git_username_here
- export CLEARML_AGENT_GIT_PASS=git_password_here
-
-
-:::note
-For backwards compatibility, the environment variables ``TRAINS_HOST_IP``, ``TRAINS_AGENT_GIT_USER``, and ``TRAINS_AGENT_GIT_PASS`` are supported.
-:::
-
-
-
-
-
-1. We recommend backing up data and, if the configuration folder is not empty, backing up the configuration.
-
- For example, if the data and configuration folders are in `/opt/trains`, then archive all data into `~/trains_backup_data.tgz`,
- and the configuration into `~/trains_backup_config.tgz`:
-
- sudo tar czvf ~/trains_backup_data.tgz -C /opt/trains/data .
- sudo tar czvf ~/trains_backup_config.tgz -C /opt/trains/config .
-
-1. Rename `/opt/trains` and its subdirectories to `/opt/clearml`.
-
- sudo mv /opt/trains /opt/clearml
-
-1. Download the latest `docker-compose.yml` file.
-
- curl https://raw.githubusercontent.com/allegroai/clearml-server/master/docker/docker-compose.yml -o /opt/clearml/docker-compose.yml
-
-1. Startup **ClearML Server**. This automatically pulls the latest **ClearML Server** build.
-
- docker-compose -f /opt/clearml/docker-compose.yml pull
- docker-compose -f /opt/clearml/docker-compose.yml up -d
-
-If issues arise during the upgrade, see the FAQ page, [How do I fix Docker upgrade errors?](../faq.md#common-docker-upgrade-errors).
-
-##### Other Deployment Formats
-
-To conclude the upgrade for deployment formats other than Linux, follow their upgrade instructions:
-
+After verifying the data migration completed successfully, continue upgrading your server:
* [AWS EC2 AMIs](upgrade_server_aws_ec2_ami.md)
* [Google Cloud Platform custom images](upgrade_server_gcp.md)
* [Linux and macOS](upgrade_server_linux_mac.md)
* [Windows](upgrade_server_win.md)
-* [Kubernetes](upgrade_server_kubernetes.md)
-* [Kubernetes Using Helm](upgrade_server_kubernetes_helm.md).
\ No newline at end of file
diff --git a/docs/deploying_clearml/clearml_server_gcp.md b/docs/deploying_clearml/clearml_server_gcp.md
index 536ed7aa..2a29f675 100644
--- a/docs/deploying_clearml/clearml_server_gcp.md
+++ b/docs/deploying_clearml/clearml_server_gcp.md
@@ -2,10 +2,6 @@
title: Google Cloud Platform
---
-:::important
-This documentation page applies to deploying your own open source ClearML Server. It does not apply to ClearML Hosted Service users.
-:::
-
Deploy **ClearML Server** on the Google Cloud Platform (GCP) using one of the pre-built GCP Custom Images. **ClearML**
provides custom images for each released version of **ClearML Server**. For a list of the pre-built custom images, see
[ClearML Server GCP Custom Image](#clearml-server-gcp-custom-image).
diff --git a/docs/deploying_clearml/clearml_server_kubernetes.md b/docs/deploying_clearml/clearml_server_kubernetes.md
deleted file mode 100644
index 0a744301..00000000
--- a/docs/deploying_clearml/clearml_server_kubernetes.md
+++ /dev/null
@@ -1,9 +0,0 @@
----
-title: Kubernetes
----
-
-:::important
-Kubernetes support is being replaced by Kubernetes using Helm. Please see the new and improved [ClearML Helm Charts repository](https://github.com/allegroai/clearml-helm-charts)
-for the most updated instructions
-:::
-
diff --git a/docs/deploying_clearml/clearml_server_kubernetes_helm.md b/docs/deploying_clearml/clearml_server_kubernetes_helm.md
index b5b1913f..17df5ee0 100644
--- a/docs/deploying_clearml/clearml_server_kubernetes_helm.md
+++ b/docs/deploying_clearml/clearml_server_kubernetes_helm.md
@@ -1,5 +1,5 @@
---
-title: Kubernetes Using Helm
+title: Kubernetes
---
:::important
@@ -7,10 +7,6 @@ title: Kubernetes Using Helm
for the most updated instructions.
:::
-:::note
-This documentation page applies to deploying your own open source **ClearML Server**. It does not apply to **ClearML Hosted Service** users.
-:::
-
:::warning
If **ClearML Server** is being reinstalled, we recommend clearing browser cookies for **ClearML Server**. For example,
for Firefox, go to Developer Tools > Storage > Cookies, and for Chrome, go to Developer Tools > Application > Cookies,
diff --git a/docs/deploying_clearml/clearml_server_linux_mac.md b/docs/deploying_clearml/clearml_server_linux_mac.md
index 884d7ac7..7473652b 100644
--- a/docs/deploying_clearml/clearml_server_linux_mac.md
+++ b/docs/deploying_clearml/clearml_server_linux_mac.md
@@ -2,10 +2,6 @@
title: Linux and macOS
---
-:::important
-This documentation page applies to deploying your own open source ClearML Server. It does not apply to ClearML Hosted Service users.
-:::
-
Deploy the **ClearML Server** in Linux or macOS using the pre-built Docker image.
For ClearML docker images, including previous versions, see [https://hub.docker.com/r/allegroai/clearml](https://hub.docker.com/r/allegroai/clearml).
diff --git a/docs/deploying_clearml/clearml_server_mongo44_migration.md b/docs/deploying_clearml/clearml_server_mongo44_migration.md
new file mode 100644
index 00000000..5fd60ebc
--- /dev/null
+++ b/docs/deploying_clearml/clearml_server_mongo44_migration.md
@@ -0,0 +1,128 @@
+---
+title: Upgrading Server from v1.1 or Older
+---
+
+:::important
+If your server is v0.15 or older, you must first go through the [v0.16 upgrade procedure](clearml_server_es7_migration.md)
+and continue here once successfully completed.
+:::
+
+In v1.2, the MongoDB subsystem of ClearML Server has been upgraded from version 3.6 to version 4.4. This change
+necessitates the migration of the database contents to be compatible with the new version.
+
+:::note Kubernetes installations
+[ClearML’s helm chart](https://github.com/allegroai/clearml-helm-charts/tree/main/charts/clearml) is already running
+mongodb version 4.4. If your ClearML server had been deployed with this chart (with the default mongodb bitnami chart) -
+You can stop reading here, as no migration is required.
+:::
+
+The migration process is comprised of the following:
+* Backing up your data
+* Creating a copy of the data which will be migrated for use with the new version
+* Sequentially running data migration for MongoDB: 3.6->4.0, 4.0->4.2, 4.0->4.4
+
+At the end of the process, your data will be available in the new location, ready to be used with ClearML Server v1.2.
+
+Once ClearML Server v1.2 is up and running, and you are satisfied that your data is intact, you can safely delete the
+old copy of your data.
+
+## Prerequisites
+* Read/write permissions for the default ClearML Server data directory `/opt/clearml/data` and its subdirectories, or,
+ if a different location had been used, permissions for that directory and its subdirectories.
+* Minimum free disk space of at least three times the size of the data.
+* Python version >=2.7 or >=3.6, and Python accessible from the command-line as python
+
+## Before You Begin
+To avoid data corruption, shut down your ClearML server before applying the migration procedure:
+* Linux and macOS
+
+ ```bash
+ docker-compose -f /opt/clearml/docker-compose.yml down
+ ```
+* Windows
+
+ ```bash
+ docker-compose -f c:\opt\clearml\docker-compose-win10.yml down
+ ```
+
+## Migrating the Data
+The following are the instructions for migrating your ClearML data from mongodb 3.6 to mongodb 4.4.
+
+The commands are provided for a Linux environment, and assume ClearML is installed in its default paths.
+
+If you are using a different OS, replace the shell and docker commands to the ones appropriate for your OS. If you’ve
+installed ClearML under a different path, replace the file locations accordingly.
+
+:::info automating your migration
+A migration script is available to automatically run this process for all supported OSs.
+[Download the script](https://github.com/allegroai/clearml-server/releases/download/1.2.0/clearml-server-1.2.0-migration.py) and run it on your ClearML server.
+Run `clearml-server-1.2.0-migration.py –help` to see execution options.
+Note the script will create a backup archive of your data in the original directory.
+:::
+
+1. Backup the source data folder:
+
+ ```bash
+ sudo tar -zcf mongodb_upgrade.gz -C /opt/clearml/data/mongo .
+ ```
+1. Copy the data to the new folder and set permissions (permissions are required on Linux):
+
+ ```bash
+ sudo mkdir /opt/clearml/data/mongo_4
+ sudo tar -xzf mongodb_upgrade.gz -C /opt/clearml/data/mongo_4
+ sudo chown -R 1000:1000 /opt/clearml/data/mongo_4
+ ```
+1. Run the mongo 4.0 container:
+
+ ```bash
+ sudo docker run -id -v /opt/clearml/data/mongo_4/db:/data/db -v /opt/clearml/data/mongo_4/configdb:/data/configdb --name mongodb_upgrade mongo:4.0.27
+ ```
+1. Upgrade data to mongo 4.0:
+
+ ```bash
+ sudo docker exec mongodb_upgrade bash -c 'mongo --eval "db.adminCommand({setFeatureCompatibilityVersion:\"4.0\"})"'
+ ```
+ This command should return an `{ "ok" : 1 }` response.
+1. Stop the mongo 4.0 container
+
+ ```bash
+ sudo docker stop mongodb_upgrade && sudo docker rm mongodb_upgrade
+ ```
+1. Run the mongo 4.2 container:
+
+ ```bash
+ sudo docker run -id -v /opt/clearml/data/mongo_4/db:/data/db -v /opt/clearml/data/mongo_4/configdb:/data/configdb --name mongodb_upgrade mongo:4.2.16
+ ```
+1. Upgrade data to mongo 4.2:
+
+ ```bash
+ sudo docker exec mongodb_upgrade bash -c 'mongo --eval "db.adminCommand({setFeatureCompatibilityVersion:\"4.2\"})"'
+ ```
+ This command should return an `{ "ok" : 1 }` response.
+1. Stop the mongo 4.2 container
+
+ ```bash
+ sudo docker stop mongodb_upgrade && sudo docker rm mongodb_upgrade
+ ```
+1. Run the mongo 4.4 container:
+
+ ```bash
+ sudo docker run -id -v /opt/clearml/data/mongo_4/db:/data/db -v /opt/clearml/data/mongo_4/configdb:/data/configdb --name mongodb_upgrade mongo:4.4.9
+ ```
+1. Upgrade data to mongo 4.4:
+
+ ```bash
+ sudo docker exec mongodb_upgrade bash -c 'mongo --eval "db.adminCommand({setFeatureCompatibilityVersion:\"4.4\"})"'
+ ```
+ This command should return an `{ "ok" : 1 }` response.
+1. Stop the mongo 4.4
+
+ ```bash
+ sudo docker stop mongodb_upgrade && sudo docker rm mongodb_upgrade
+ ```
+1. Continue upgrading your ClearML Server:
+ * [AWS EC2 AMIs](upgrade_server_aws_ec2_ami.md)
+ * [Google Cloud Platform custom images](upgrade_server_gcp.md)
+ * [Linux and macOS](upgrade_server_linux_mac.md)
+ * [Windows](upgrade_server_win.md)
+
\ No newline at end of file
diff --git a/docs/deploying_clearml/clearml_server_win.md b/docs/deploying_clearml/clearml_server_win.md
index 3b83956e..0789faf0 100644
--- a/docs/deploying_clearml/clearml_server_win.md
+++ b/docs/deploying_clearml/clearml_server_win.md
@@ -2,10 +2,6 @@
title: Windows 10
---
-:::important
-This documentation page applies to deploying your own open source ClearML Server. It does not apply to ClearML Hosted Service users.
-:::
-
For Windows, we recommend launching the pre-built Docker image on a Linux virtual machine (see [Deploying ClearML Server: Linux or macOS](clearml_server_linux_mac.md)).
However, **ClearML Server** can be launched on Windows 10, using Docker Desktop for Windows (see the Docker [System Requirements](https://docs.docker.com/docker-for-windows/install/#system-requirements)).
diff --git a/docs/deploying_clearml/upgrade_server_aws_ec2_ami.md b/docs/deploying_clearml/upgrade_server_aws_ec2_ami.md
index d40bd30a..6f8ea3b7 100644
--- a/docs/deploying_clearml/upgrade_server_aws_ec2_ami.md
+++ b/docs/deploying_clearml/upgrade_server_aws_ec2_ami.md
@@ -2,10 +2,6 @@
title: AWS EC2 AMIs
---
-:::important
-This documentation page applies to deploying your own open source ClearML Server. It does not apply to ClearML Hosted Service users.
-:::
-
:::note
For upgrade purposes, the terms **Trains Server** and **ClearML Server** are interchangeable.
:::
@@ -34,9 +30,9 @@ Some legacy **Trains Server** AMIs provided an auto-upgrade on restart capabilit
1. We recommend [backing up your data](clearml_server_aws_ec2_ami.md#backing-up-and-restoring-data-and-configuration) and,
if your configuration folder is not empty, backing up your configuration.
-1. If upgrading from **Trains Server** version 0.15 or older, a data migration is required before upgrading,
- First follow these [data migration instructions](clearml_server_es7_migration.md), and then continue this upgrade.
-
+1. If upgrading from ClearML Server version older than 1.2, you need to migrate your data before upgrading your server. See instructions [here](clearml_server_mongo44_migration.md).
+If upgrading from Trains Server version 0.15 or older, a data migration is required before continuing this upgrade. See instructions [here](clearml_server_es7_migration.md).
+
1. If upgrading from **Trains Server** to **ClearML Server**, rename `/opt/trains` to `/opt/clearml`.
1. Download the latest `docker-compose.yml` file. Execute the following command:
diff --git a/docs/deploying_clearml/upgrade_server_gcp.md b/docs/deploying_clearml/upgrade_server_gcp.md
index a47e21f4..b077f8b6 100644
--- a/docs/deploying_clearml/upgrade_server_gcp.md
+++ b/docs/deploying_clearml/upgrade_server_gcp.md
@@ -2,19 +2,16 @@
title: Google Cloud Platform
---
-:::important
-This documentation page applies to deploying your own open source ClearML Server. It does not apply to ClearML Hosted Service users.
-:::
-
**To upgrade ClearML Server Docker deployment:**
1. Shut down the docker containers with the following command:
docker-compose -f docker-compose.yml down
-
+
+1. If upgrading from ClearML Server version older than 1.2, you need to migrate your data before upgrading your server. See instructions [here](clearml_server_mongo44_migration.md).
1. If upgrading from **Trains Server** version 0.15 or older to **ClearML Server**, do the following:
- 1. A data migration is required before upgrading. First follow these [data migration instructions](clearml_server_es7_migration.md),
+ 1. Follow these [data migration instructions](clearml_server_es7_migration.md),
and then continue this upgrade.
1. Rename `/opt/trains` and its subdirectories to `/opt/clearml`.
diff --git a/docs/deploying_clearml/upgrade_server_kubernetes.md b/docs/deploying_clearml/upgrade_server_kubernetes.md
deleted file mode 100644
index 2cdfe0bd..00000000
--- a/docs/deploying_clearml/upgrade_server_kubernetes.md
+++ /dev/null
@@ -1,10 +0,0 @@
----
-title: Kubernetes
----
-
-
-:::important
-Kubernetes support is being replaced by Kubernetes using Helm. Please see the new and improved [ClearML Helm Charts repository](https://github.com/allegroai/clearml-helm-charts)
-for the most updated instructions.
-:::
-
diff --git a/docs/deploying_clearml/upgrade_server_kubernetes_helm.md b/docs/deploying_clearml/upgrade_server_kubernetes_helm.md
index e7eb5bbe..264cde46 100644
--- a/docs/deploying_clearml/upgrade_server_kubernetes_helm.md
+++ b/docs/deploying_clearml/upgrade_server_kubernetes_helm.md
@@ -1,5 +1,5 @@
---
-title: Kubernetes Helm
+title: Kubernetes
---
:::important
@@ -8,9 +8,7 @@ for the most updated instructions.
:::
:::note
-* This documentation page applies to deploying your own open source ClearML Server. It does not apply to ClearML Hosted Service users.
-
-* We strongly encourage to keep the **ClearML Server** up to date, by upgrading to the current release.
+We strongly encourage to keep the **ClearML Server** up to date, by upgrading to the current release.
:::
1. Upgrade using new or upgraded values.yaml
@@ -21,8 +19,8 @@ for the most updated instructions.
helm delete --purge clearml-server
-1. If upgrading from **Trains Server** version 0.15 or older to **ClearML Server**, a data migration is required before
- upgrading. First follow these [data migration instructions](clearml_server_es7_migration.md), and then continue this upgrade.
+1. If upgrading from Trains Server version 0.15 or older, a data migration is required before continuing this upgrade.
+ See instructions [here](clearml_server_es7_migration.md).
1. Upgrade deployment to match repository version.
diff --git a/docs/deploying_clearml/upgrade_server_linux_mac.md b/docs/deploying_clearml/upgrade_server_linux_mac.md
index d3200faa..97dc414d 100644
--- a/docs/deploying_clearml/upgrade_server_linux_mac.md
+++ b/docs/deploying_clearml/upgrade_server_linux_mac.md
@@ -2,12 +2,6 @@
title: Linux or macOS
---
-:::important
-This documentation page applies to deploying your own open source ClearML Server. It does not apply to ClearML Hosted Service users.
-:::
-
-
Important: Upgrading from v0.14 or older
@@ -35,12 +29,13 @@ For backwards compatibility, the environment variables ``TRAINS_HOST_IP``, ``TRA
**To upgrade ClearML Server Docker deployment:**
-1. Shutdown **ClearML Server**. Execute the following command (which assumes the configuration file is in the environment path).
+1. Shutdown ClearML Server. Execute the following command (which assumes the configuration file is in the environment path).
docker-compose -f docker-compose.yml down
-1. If upgrading from **Trains Server** version 0.15 or older to **ClearML Server**, a data migration is required before
- upgrading. First follow these [data migration instructions](clearml_server_es7_migration.md), and then continue this upgrade.
+1. If upgrading from ClearML Server version older than 1.2, you need to migrate your data before upgrading your server. See instructions [here](clearml_server_mongo44_migration.md).
+
+1. If upgrading from **Trains Server** version 0.15 or older, a data migration is required before continuing this upgrade. See instructions [here](clearml_server_es7_migration.md).
1. We recommend [backing up data](clearml_server_linux_mac.md#backing-up-and-restoring-data-and-configuration) and, if the configuration folder is
not empty, backing up the configuration.
diff --git a/docs/deploying_clearml/upgrade_server_win.md b/docs/deploying_clearml/upgrade_server_win.md
index ea7a0d93..9c9a9a8d 100644
--- a/docs/deploying_clearml/upgrade_server_win.md
+++ b/docs/deploying_clearml/upgrade_server_win.md
@@ -2,10 +2,6 @@
title: Windows
---
-:::important
-This documentation page applies to deploying your own open source ClearML Server. It does not apply to ClearML Hosted Service users.
-:::
-
**To upgrade ClearML Server Docker deployment:**
1. Shut down the docker containers.
@@ -20,8 +16,9 @@ This documentation page applies to deploying your own open source ClearML Server
docker-compose -f c:\opt\trains\docker-compose-win10.yml down
-1. If upgrading from **Trains Server** version 0.15 or older to **ClearML Server**, a data migration is required before
- upgrading. First follow these [data migration instructions](clearml_server_es7_migration.md), and then continue this upgrade.
+1. If upgrading from ClearML Server version older than 1.2, you need to migrate your data before upgrading your server. See instructions [here](clearml_server_mongo44_migration.md).
+
+1. If upgrading from **Trains Server** version 0.15 or older, a data migration is required before continuing this upgrade. See instructions [here](clearml_server_es7_migration.md).
1. We recommend backing up data and, if the configuration folder is not empty, backing up the configuration.
diff --git a/sidebars.js b/sidebars.js
index a92bb48e..55a35175 100644
--- a/sidebars.js
+++ b/sidebars.js
@@ -44,14 +44,14 @@ module.exports = {
{'ClearML Server': ['deploying_clearml/clearml_server',
{
'Deploying ClearML Server': ['deploying_clearml/clearml_server_aws_ec2_ami', 'deploying_clearml/clearml_server_gcp',
- 'deploying_clearml/clearml_server_linux_mac', 'deploying_clearml/clearml_server_win', 'deploying_clearml/clearml_server_kubernetes',
+ 'deploying_clearml/clearml_server_linux_mac', 'deploying_clearml/clearml_server_win',
'deploying_clearml/clearml_server_kubernetes_helm']
},
{
'Upgrading ClearML Server':['deploying_clearml/upgrade_server_aws_ec2_ami','deploying_clearml/upgrade_server_gcp',
'deploying_clearml/upgrade_server_linux_mac', 'deploying_clearml/upgrade_server_win',
- 'deploying_clearml/upgrade_server_kubernetes', 'deploying_clearml/upgrade_server_kubernetes_helm',
- 'deploying_clearml/clearml_server_es7_migration']
+ 'deploying_clearml/upgrade_server_kubernetes_helm',
+ 'deploying_clearml/clearml_server_es7_migration', 'deploying_clearml/clearml_server_mongo44_migration']
},
'deploying_clearml/clearml_server_config', 'deploying_clearml/clearml_config_for_clearml_server', 'deploying_clearml/clearml_server_security'
]},