diff --git a/docs/deploying_clearml/clearml_server.md b/docs/deploying_clearml/clearml_server.md index 3435e176..0977c005 100644 --- a/docs/deploying_clearml/clearml_server.md +++ b/docs/deploying_clearml/clearml_server.md @@ -42,7 +42,7 @@ authentication, sub-domains, and load balancers, and use any of its many configu * Pre-built [Google Cloud Platform custom images](clearml_server_gcp.md) * Pre-built Docker images for [Linux](clearml_server_linux_mac.md), [macOS](clearml_server_linux_mac.md), and [Windows 10](clearml_server_win.md) - * [Kubernetes](clearml_server_kubernetes.md) and [Kubernetes using Helm](clearml_server_kubernetes_helm.md) + * [Kubernetes using Helm](clearml_server_kubernetes_helm.md) 1. Optionally, [configure ClearML Server](clearml_server_config.md) for additional features, including sub-domains and load balancers, web login authentication, and the non-responsive task watchdog. @@ -56,6 +56,6 @@ When necessary, upgrade your ClearML Server on any of the available formats: * [Google Cloud Platform](upgrade_server_gcp.md) * [Linux or MacOS](upgrade_server_linux_mac.md) * [Windows 10](upgrade_server_win.md) -* [Kubernetes](upgrade_server_kubernetes.md) and [Kubernetes using Helm](upgrade_server_kubernetes_helm.md). +* [Kubernetes using Helm](upgrade_server_kubernetes_helm.md). If you are using v0.15 or Older, [upgrade to ClearML Server](clearml_server_es7_migration.md). \ No newline at end of file diff --git a/docs/deploying_clearml/clearml_server_kubernetes.md b/docs/deploying_clearml/clearml_server_kubernetes.md index 8b562fe5..0a744301 100644 --- a/docs/deploying_clearml/clearml_server_kubernetes.md +++ b/docs/deploying_clearml/clearml_server_kubernetes.md @@ -2,145 +2,8 @@ title: Kubernetes --- -:::important -This documentation page applies to deploying your own open source **ClearML Server**. It does not apply to **ClearML Hosted Service** users. -::: - -This page describes the prerequisites and procedures for deploying **ClearML Server** to Kubernetes clusters, using manual -instructions, as well as accessing **ClearML Server**, and port mappings. - -To deploy **ClearML Server** to Kubernetes using Helm, see [Deploying ClearML Server: Kubernetes using Helm](clearml_server_kubernetes_helm.md). - -For more information about upgrading **ClearML Server** in a Kubernetes Cluster, see [here](upgrade_server_kubernetes.md) - :::important -If **ClearML Server** is being reinstalled, we recommend clearing browser cookies for **ClearML Server**. For example, -for Firefox, go to Developer Tools > Storage > Cookies, and for Chrome, go to Developer Tools > Application > Cookies, -and delete all cookies under the **ClearML Server** URL. +Kubernetes support is being replaced by Kubernetes using Helm. Please see the new and improved [ClearML Helm Charts repository](https://github.com/allegroai/clearml-helm-charts) +for the most updated instructions ::: - -## Prerequisites - -* A Kubernetes cluster. -* `kubectl` installed and configured (see [Install and Set Up kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) in the Kubernetes documentation). -* One node labeled `app=clearml`. - - -:::warning -ClearML Server deployment uses node storage. If more than one node is labeled as ``app=clearml``, and the server is later -redeployed or updated, then **ClearML Server** may not locate all the data. -::: - - -## Deploying - - -:::warning -By default, **ClearML Server** launches with unrestricted access. To restrict **ClearML Server** access, follow the instructions -in the [Security](clearml_server_security.md) page. -::: - - - - -### Step 1: Modify Elasticsearch default values in the Docker configuration file - -Before deploying **ClearML Server** in a Kubernetes cluster, modify several Elasticsearch settings in the Docker configuration. -For more information, see [Install Elasticsearch with Docker](https://www.elastic.co/guide/en/elasticsearch/reference/master/docker.html#_notes_for_production_use_and_defaults) -in the Elasticsearch documentation and [Daemon configuration file](https://docs.docker.com/config/daemon/) in the Docker documentation. - -**To modify Elasticsearch default values in the Docker configuration file:** - -1. Connect to the node in the Kubernetes cluster labeled `app=clearml`. -1. Create or edit (if one exists) the `/etc/docker/daemon.json` file, and add or modify the `defaults-ulimits` section - as the following example shows: - - { - "default-ulimits": { - "nofile": { - "name": "nofile", - "hard": 65536, - "soft": 1024 - }, - "memlock": - { - "name": "memlock", - "soft": -1, - "hard": -1 - } - } - } - -1. Elasticsearch requires that the `vm.max_map_count` kernel setting, which is the maximum number of memory map areas a process can use, be set to at least `262144`. - - For CentOS 7, Ubuntu 16.04, Mint 18.3, Ubuntu 18.04 and Mint 19.x, use the following commands to set `vm.max_map_count`: - - echo "vm.max_map_count=262144" > /tmp/99-clearml.conf - sudo mv /tmp/99-clearml.conf /etc/sysctl.d/99-clearml.conf - sudo sysctl -w vm.max_map_count=262144 - -1. Restart docker: - - sudo service docker restart - -### Step 2. Deploy ClearML Server in the Kubernetes Cluster - -After modifying several Elasticsearch settings in the Docker configuration (see Step 1 above), deploy **ClearML -Server**. - -**To deploy ClearML Server in Kubernetes Clusters:** - -1. Clone the `clearml-server-k8s` repository and change to the new `clearml-server-k8s` directory: - - git clone https://github.com/allegroai/clearml-server-k8s.git && cd clearml-server-k8s/clearml-server-k8s - -1. Create the clearml `namespace` and deployments: - - kubectl apply -k overlays/current_version - - :::note - This installs the templates for the current ``clearml-server`` version and updates patch versions whenever the deployment is restarted (or reinstalled). - ::: - - - To use the latest version, which is **_not recommended_**: - - kubectl apply -k base - -## Port Mapping - -After deploying **ClearML Server**, the services expose the following node ports: - -* API server on `30008`. -* Web server on `30080`. -* File server on `30081`. - -## Accessing ClearML Server - -**To access the ClearML Server, do the following:** - -1. Create domain records. - - * Create records for the **ClearML Server** web server, file server, and API access using the following rules: - * `app.` - * `files.` - * `api.` - - For example: - * `app.clearml.mydomainname.com` - * `files.clearml.mydomainname.com` - * `api.clearml.mydomainname.com` - -1. Point the created records to the load balancer. - -1. Configure the load balancer to redirect traffic coming from the records: - - * `app.` should be redirected to k8s cluster nodes on port `30080` - * `files.` should be redirected to k8s cluster nodes on port `30081` - * `api.` should be redirected to k8s cluster nodes on port `30008` - - -## Next Step - -* [Configuring ClearML for ClearML Server](clearml_config_for_clearml_server.md). \ No newline at end of file diff --git a/docs/deploying_clearml/clearml_server_kubernetes_helm.md b/docs/deploying_clearml/clearml_server_kubernetes_helm.md index 86ff93ab..1c762711 100644 --- a/docs/deploying_clearml/clearml_server_kubernetes_helm.md +++ b/docs/deploying_clearml/clearml_server_kubernetes_helm.md @@ -3,6 +3,11 @@ title: Kubernetes Using Helm --- :::important +**This documentation page is being updated.** Please see the new and improved **[ClearML Helm Charts repository](https://github.com/allegroai/clearml-helm-charts)** +for the most updated instructions. +::: + +:::note This documentation page applies to deploying your own open source **ClearML Server**. It does not apply to **ClearML Hosted Service** users. ::: diff --git a/docs/deploying_clearml/upgrade_server_kubernetes.md b/docs/deploying_clearml/upgrade_server_kubernetes.md index 3af6249a..2cdfe0bd 100644 --- a/docs/deploying_clearml/upgrade_server_kubernetes.md +++ b/docs/deploying_clearml/upgrade_server_kubernetes.md @@ -2,26 +2,9 @@ title: Kubernetes --- + :::important -This documentation page applies to deploying your own open source ClearML Server. It does not apply to ClearML Hosted Service users. +Kubernetes support is being replaced by Kubernetes using Helm. Please see the new and improved [ClearML Helm Charts repository](https://github.com/allegroai/clearml-helm-charts) +for the most updated instructions. ::: -:::note -We strongly encourage to keep your **ClearML Server** up to date, by upgrading to the current release. -::: - -To update deployment, edit the yaml file that needs to be updated and then run following command: - -**To update the ClearML Server in Kubernetes clusters:** - -1. If a **ClearML Server** was previously deployed, delete old deployments using the following command: - - kubectl delete -f . - -1. If upgrading from **Trains Server** version 0.15 or older to **ClearML Server**, a data migration is required before - upgrading. First follow these [data migration instructions](clearml_server_es7_migration.md), and then continue this - upgrade. - -1. Edit the YAML file that needs to be updated and then run following command: - - kubectl apply -f .yaml diff --git a/docs/deploying_clearml/upgrade_server_kubernetes_helm.md b/docs/deploying_clearml/upgrade_server_kubernetes_helm.md index de6061cc..e7eb5bbe 100644 --- a/docs/deploying_clearml/upgrade_server_kubernetes_helm.md +++ b/docs/deploying_clearml/upgrade_server_kubernetes_helm.md @@ -3,11 +3,14 @@ title: Kubernetes Helm --- :::important -This documentation page applies to deploying your own open source ClearML Server. It does not apply to ClearML Hosted Service users. -::: +**This documentation page is being updated.** Please see the new and improved **[ClearML Helm Charts repository](https://github.com/allegroai/clearml-helm-charts)** +for the most updated instructions. +::: :::note -We strongly encourage to keep the **ClearML Server** up to date, by upgrading to the current release. +* This documentation page applies to deploying your own open source ClearML Server. It does not apply to ClearML Hosted Service users. + +* We strongly encourage to keep the **ClearML Server** up to date, by upgrading to the current release. ::: 1. Upgrade using new or upgraded values.yaml diff --git a/docs/faq.md b/docs/faq.md index 9c7ab1a9..b71fb78f 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -77,7 +77,6 @@ title: FAQ * [AWS EC2 AMIs?](#aws_ec2_amis) * [Google Cloud Platform?](#google_cloud_platform) * [How do I restart ClearML Server?](#restart) -* [Can I deploy ClearML Server on Kubernetes clusters?](#kubernetes) * [Can I create a Helm Chart for ClearML Server Kubernetes deployment?](#helm) * [My Docker cannot load a local host directory on SELinux?](#selinux) @@ -659,12 +658,6 @@ if you deployed to Linux, see [Restarting](deploying_clearml/clearml_server_linu
-**Can I deploy ClearML Server on Kubernetes clusters?** - -Yes! ClearML Server supports Kubernetes. For detailed instructions, see [Deploying ClearML Server: Kubernetes](deploying_clearml/clearml_server_kubernetes.md) -in the "Deploying ClearML" section. - -
**Can I create a Helm Chart for ClearML Server Kubernetes deployment?**