ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-05-10 07:31:15 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Update k8s deployment instructions

Browse Source
This commit is contained in:
Noam Wasersprung 2025-04-24 18:52:40 +03:00 committed by GitHub
commit 911bf7e7c0
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 67 additions and 106 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

173
docs/deploying_clearml/enterprise_deploy/k8s.md
View File

@ -58,83 +58,46 @@ that will have records pointing to the cluster’s ingress controller (see ingre
::: :::
``` ```yaml
imageCredentials: imageCredentials:
password: "<clearml_enterprise_DockerHub_TOKEN>" password: "<clearml_enterprise_DockerHub_TOKEN>"
clearml: clearml:
cookieDomain: "<BASE_DOMAIN>" cookieDomain: "<BASE_DOMAIN>"
# Set values for improved security # Set values for improved security
apiserverKey: "<GENERATED_API_SERVER_KEY>" apiserverKey: "<GENERATED_API_SERVER_KEY>"
apiserverSecret: "<GENERATED_API_SERVER_SECRET>" apiserverSecret: "<GENERATED_API_SERVER_SECRET>"
fileserverKey: "<GENERATED_FILE_SERVER_KEY>" fileserverKey: "<GENERATED_FILE_SERVER_KEY>"
fileserverSecret: "<GENERATED_FILE_SERVER_SECRET>" fileserverSecret: "<GENERATED_FILE_SERVER_SECRET>"
secureAuthTokenSecret: "<GENERATED_AUTH_TOKEN_SECRET>" secureAuthTokenSecret: "<GENERATED_AUTH_TOKEN_SECRET>"
testUserKey: "<GENERATED_TEST_USER_KEY>" testUserKey: "<GENERATED_TEST_USER_KEY>"
testUserSecret: "<GENERATED_TEST_USER_SECRET>" testUserSecret: "<GENERATED_TEST_USER_SECRET>"
apiserver: apiserver:
ingress: ingress:
enabled: true enabled: true
hostName: "api.<BASE_DOMAIN>" hostName: "api.<BASE_DOMAIN>"
service: service:
type: ClusterIP type: ClusterIP
extraEnvs:
- name: CLEARML__services__organization__features__user_management_advanced
value: "true"
- name: CLEARML__services__auth__ui_features_per_role__user__show_datasets
value: "false"
- name: CLEARML__services__auth__ui_features_per_role__user__show_orchestration
value: "false"
- name: CLEARML__services__workers__resource_usages__supervisor_company
value: "<SUPERVISOR_TENANT_ID>"
- name: CLEARML__secure__credentials__supervisor__role
value: "system"
- name: CLEARML__secure__credentials__supervisor__allow_login
value: "true"
- name: CLEARML__secure__credentials__supervisor__user_key
value: "<SUPERVISOR_USER_KEY>"
- name: CLEARML__secure__credentials__supervisor__user_secret
value: "<SUPERVISOR_USER_SECRET>"
- name: CLEARML__secure__credentials__supervisor__sec_groups
value: "[\"users\", \"admins\", \"queue_admins\"]"
- name: CLEARML__secure__credentials__supervisor__email
value: "\"<SUPERVISOR_USER_EMAIL>\""
- name: CLEARML__apiserver__company__unique_names
value: "true"
fileserver: fileserver:
ingress: ingress:
enabled: true enabled: true
hostName: "file.<BASE_DOMAIN>" hostName: "file.<BASE_DOMAIN>"
service: service:
type: ClusterIP type: ClusterIP
webserver: webserver:
ingress: ingress:
enabled: true enabled: true
hostName: "app.<BASE_DOMAIN>" hostName: "app.<BASE_DOMAIN>"
service: service:
type: ClusterIP type: ClusterIP
clearmlApplications: clearmlApplications:
enabled: true enabled: true
``` ```
The credentials specified in `<SUPERVISOR_USER_KEY>` and `<SUPERVISOR_USER_SECRET>` can be used to log in as the
supervisor user from the ClearML Web UI accessible using the URL `app.<BASE_DOMAIN>`.
Note that the `<SUPERVISOR_USER_EMAIL>` value must be explicitly quoted. To do so, put `\"` around the quoted value.
For example `"\"email@example.com\""`.
#### Additional Configuration Options #### Additional Configuration Options
##### Fixed Users (Simple Login) ##### Fixed Users (Simple Login)
@ -181,7 +144,7 @@ Substitute all `<PLACEHOLDER>`s with the correct value for your configuration.
##### Auth0 Identity Provider ##### Auth0 Identity Provider
``` ```yaml
apiserver: apiserver:
extraEnvs: extraEnvs:
- name: CLEARML__secure__login__sso__oauth_client__auth0__client_id - name: CLEARML__secure__login__sso__oauth_client__auth0__client_id
@ -202,7 +165,7 @@ apiserver:
##### Keycloak Identity Provider ##### Keycloak Identity Provider
``` ```yaml
apiserver: apiserver:
extraEnvs: extraEnvs:
- name: CLEARML__secure__login__sso__oauth_client__keycloak__client_id - name: CLEARML__secure__login__sso__oauth_client__keycloak__client_id
@ -217,8 +180,6 @@ apiserver:
value: "<KC_URL>/realms/<REALM_NAME>/protocol/openid-connect/token" value: "<KC_URL>/realms/<REALM_NAME>/protocol/openid-connect/token"
- name: CLEARML__services__login__sso__oauth_client__keycloak__idp_logout - name: CLEARML__services__login__sso__oauth_client__keycloak__idp_logout
value: "true" value: "true"
``` ```
@ -247,24 +208,24 @@ To configure the agent you will need to choose a Redis password and use that whe
The Helm Chart must be installed with `overrides.yaml`: The Helm Chart must be installed with `overrides.yaml`:
``` ```yaml
imageCredentials: imageCredentials:
password: "<CLEARML_DOCKERHUB_TOKEN>" password: "<CLEARML_DOCKERHUB_TOKEN>"
clearml: clearml:
agentk8sglueKey: "<ACCESS_KEY>" agentk8sglueKey: "<ACCESS_KEY>"
agentk8sglueSecret: "<SECRET_KEY>" agentk8sglueSecret: "<SECRET_KEY>"
agentk8sglue: agentk8sglue:
apiServerUrlReference: "https://api.<BASE_DOMAIN>" apiServerUrlReference: "https://api.<BASE_DOMAIN>"
fileServerUrlReference: "https://files.<BASE_DOMAIN>" fileServerUrlReference: "https://files.<BASE_DOMAIN>"
webServerUrlReference: "https://app.<BASE_DOMAIN>" webServerUrlReference: "https://app.<BASE_DOMAIN>"
defaultContainerImage: "python:3.9" defaultContainerImage: "python:3.9"
``` ```
#### Installing the Chart #### Installing the Chart
``` ```bash
helm install -n <WORKLOAD_NAMESPACE> \ helm install -n <WORKLOAD_NAMESPACE> \
clearml-agent \ clearml-agent \
clearml-enterprise/clearml-enterprise-agent \ clearml-enterprise/clearml-enterprise-agent \
@ -276,7 +237,7 @@ helm install -n <WORKLOAD_NAMESPACE> \
To create a queue by API: To create a queue by API:
``` ```bash
curl $APISERVER_URL/queues.create \ curl $APISERVER_URL/queues.create \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-H "X-Clearml-Impersonate-As:<USER_ID>" \ -H "X-Clearml-Impersonate-As:<USER_ID>" \
@ -294,22 +255,22 @@ curl $APISERVER_URL/queues.create \
The Helm Chart must be installed with `overrides.yaml`: The Helm Chart must be installed with `overrides.yaml`:
``` ```yaml
imageCredentials: imageCredentials:
password: "<DOCKERHUB_TOKEN>" password: "<DOCKERHUB_TOKEN>"
clearml: clearml:
apiServerKey: "" apiServerKey: ""
apiServerSecret: "" apiServerSecret: ""
apiServerUrlReference: "https://api." apiServerUrlReference: "https://api."
authCookieName: "" authCookieName: ""
ingress: ingress:
enabled: true enabled: true
hostName: "task-router.dev" hostName: "task-router.dev"
tcpSession: tcpSession:
routerAddress: "<NODE_IP OR EXTERNAL_NAME>" routerAddress: "<NODE_IP OR EXTERNAL_NAME>"
portRange: portRange:
start: <START_PORT> start: <START_PORT>
end: <END_PORT> end: <END_PORT>
``` ```
@ -330,7 +291,7 @@ tcpSession:
### Installing the Chart ### Installing the Chart
``` ```bash
helm install -n <WORKLOAD_NAMESPACE> \ helm install -n <WORKLOAD_NAMESPACE> \
clearml-ttr \ clearml-ttr \
clearml-enterprise/clearml-enterprise-task-traffic-router \ clearml-enterprise/clearml-enterprise-task-traffic-router \
@ -429,20 +390,20 @@ This example configures a specific queue, but you can include this setting in th
apply it to all tasks. apply it to all tasks.
``` ```yaml
agentk8sglue: agentk8sglue:
queues: queues:
GPUshm: GPUshm:
templateOverrides: templateOverrides:
env: env:
- name: VLLM_SKIP_P2P_CHECK - name: VLLM_SKIP_P2P_CHECK
value: "1" value: "1"
volumeMounts: volumeMounts:
- name: dshm - name: dshm
mountPath: /dev/shm mountPath: /dev/shm
volumes: volumes:
- name: dshm - name: dshm
emptyDir: emptyDir:
medium: Memory medium: Memory
sizeLimit: <SIZE>Gi sizeLimit: <SIZE>Gi
``` ```