ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-01-31 06:27:22 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
58b2993d73
clearml-docs/docs/deploying_clearml/clearml_server_gcp.md
pollfly 7137669f24
Small edits (#861)
2024-06-30 09:16:11 +03:00

8.9 KiB
Raw Blame History

title
Google Cloud Platform

Deploy ClearML Server on the Google Cloud Platform (GCP) using one of the pre-built GCP Custom Images. ClearML provides custom images for each released version of ClearML Server. For a list of the pre-built custom images, see ClearML Server GCP Custom Image.

To keep track of your experiments and/or data, the clearml package needs to communicate with the server you have deployed. For instruction to connect the ClearML SDK to the server, see Getting Started: First Steps.

:::info In order for clearml to work with a ClearML Server on GCP, set CLEARML_API_DEFAULT_REQ_METHOD=PUT or set api.http.default_method to "PUT" in the clearml.conf file. :::

For information about upgrading ClearML server on GCP, see here.

:::note Reinstallation If ClearML Server is being reinstalled, clearing browser cookies for ClearML Server is recommended. For example, for Firefox, go to Developer Tools > Storage > Cookies, and for Chrome, go to Developer Tools > Application > Cookies, and delete all cookies under the ClearML Server URL. :::

Default ClearML Server Service Ports

After deploying ClearML Server, the services expose the following node ports:

  • Web server on 8080
  • API server on 8008
  • File Server on 8081

Default ClearML Server Storage Paths

The persistent storage configuration:

  • MongoDB: /opt/clearml/data/mongo_4/
  • Elasticsearch: /opt/clearml/data/elastic_7/
  • File Server: /mnt/fileserver/

Importing the Custom Image to your GCP account

Before launching an instance using a ClearML Server GCP Custom Image, import the image to the custom images list.

:::note No upload of the image file is required. Links to image files stored in Google Storage are provided. :::

To import the image to your custom images list:

  1. In the Cloud Console, go to the Images page.

  2. At the top of the page, click Create image.

  3. In Name, specify a unique name for the image.

  4. Optionally, specify an image family for the new image, or configure specific encryption settings for the image.

  5. In the Source menu, select Cloud Storage file.

  6. Enter the ClearML Server image bucket path (see ClearML Server GCP Custom Image), for example: allegro-files/clearml-server/clearml-server.tar.gz.

  7. Click Create to import the image. The process can take several minutes depending on the size of the boot disk image.

For more information see the Compute Engine Documentation.

Launching

:::warning By default, ClearML Server launches with unrestricted access. To restrict ClearML Server access, follow the instructions in the Security page. :::

To launch ClearML Server using a GCP Custom Image, see the Google Cloud Storage documentation. For more information about Custom Images, see Custom Images in the Compute Engine documentation.

The minimum requirements for ClearML Server are:

  • 2 vCPUs
  • 7.5GB RAM

Restarting

To restart ClearML Server Docker deployment:

  • Stop and then restart the Docker containers by executing the following commands:

    docker-compose -f /opt/clearml/docker-compose.yml down
docker-compose -f /opt/clearml/docker-compose.yml up -d

Backing Up and Restoring Data and Configuration

:::warning Stop your server before backing up or restoring data and configuration :::

The commands in this section are an example of how to back up and restore data and configuration.

If data and configuration folders are in /opt/clearml, then archive all data into ~/clearml_backup_data.tgz, and configuration into ~/clearml_backup_config.tgz:

sudo tar czvf ~/clearml_backup_data.tgz -C /opt/clearml/data .
sudo tar czvf ~/clearml_backup_config.tgz -C /opt/clearml/config .

If the data and the configuration need to be restored:

  1. Verify you have the backup files.

  2. Replace any existing data with the backup data:

    sudo rm -fR /opt/clearml/data/* /opt/clearml/config/*
sudo tar -xzf ~/clearml_backup_data.tgz -C /opt/clearml/data
sudo tar -xzf ~/clearml_backup_config.tgz -C /opt/clearml/config

  3. Grant access to the data:

    sudo chown -R 1000:1000 /opt/clearml

ClearML Server GCP Custom Image

The following section contains a list of Custom Image URLs (exported in different formats) for each released ClearML Server version.

Latest Version - v1.13.1

All Release Versions

Next Step

To keep track of your experiments and/or data, the clearml package needs to communicate with your server. For instruction to connect the ClearML SDK to the server, see Getting Started: First Steps.