Fix glue namespace handling (#63 )

* Changed: namespace handling for glue * Changed: set glue as default agent system * Changed: bump up version
Changed: bump up versions (#62 )
2025-04-17 01:31:13 +00:00 · 2022-04-22 10:19:03 +02:00 · 2022-04-19 08:22:08 +02:00 · 2022-04-04 10:32:51 +02:00 · 2022-03-30 16:33:06 +02:00 · 2022-03-28 09:29:04 +02:00
32 changed files with 638 additions and 161 deletions
--- a/.github/helm-docs.sh
+++ b/.github/helm-docs.sh
@@ -1,7 +1,7 @@
 #!/bin/bash

 CHART_DIRS="$(git diff --find-renames --name-only "$(git rev-parse --abbrev-ref HEAD)" remotes/origin/main -- 'charts' | grep '[cC]hart.yaml' | sed -e 's#/[Cc]hart.yaml##g')"
-HELM_DOCS_VERSION="1.5.0"
+HELM_DOCS_VERSION="1.8.1"

 curl --silent --show-error --fail --location --output /tmp/helm-docs.tar.gz https://github.com/norwoodj/helm-docs/releases/download/v"${HELM_DOCS_VERSION}"/helm-docs_"${HELM_DOCS_VERSION}"_Linux_x86_64.tar.gz
 tar -xf /tmp/helm-docs.tar.gz helm-docs
--- a/.github/workflows/ci.yaml
+++ b/.github/workflows/ci.yaml
@@ -21,8 +21,9 @@ jobs:
    strategy:
      matrix:
        k8s:
-          - v1.20.7
-          - v1.21.1
+          - v1.21.2
+          - v1.22.1
+          - v1.23.0
    steps:
      - name: Checkout
        uses: actions/checkout@v1
--- a/charts/clearml/Chart.lock
+++ b/charts/clearml/Chart.lock
@@ -7,6 +7,6 @@ dependencies:
  version: 10.3.4
 - name: elasticsearch
  repository: https://helm.elastic.co
-  version: 7.10.1
-digest: sha256:aefd3992b2ab085161e4cca35c6f73dd33f8d19272a9405b5ee4e8c2a0e79bba
-generated: "2021-01-05T14:26:33.629164+01:00"
+  version: 7.16.2
+digest: sha256:ac733cb02d50e8398c1d2832988333896f1c7b765c19a0f1eea5e9b24bdb8207
+generated: "2022-01-05T07:52:34.913745+01:00"
--- a/charts/clearml/Chart.yaml
+++ b/charts/clearml/Chart.yaml
@@ -2,8 +2,8 @@ apiVersion: v2
 name: clearml
 description: MLOps platform
 type: application
-version: "2.2.2"
-appVersion: "1.1.1"
+version: "3.10.0"
+appVersion: "1.4.0"
 home: https://clear.ml
 icon: https://raw.githubusercontent.com/allegroai/clearml/master/docs/clearml-logo.svg
 sources:
@@ -18,14 +18,14 @@ keywords:
  - mlops
 dependencies:
  - name: redis
-    version: "~10.9.0"
+    version: "10.9.0"
    repository: "https://charts.bitnami.com/bitnami"
    condition: redis.enabled
  - name: mongodb
-    version: "~10.3.2"
+    version: "10.3.4"
    repository: "https://charts.bitnami.com/bitnami"
    condition: mongodb.enabled
  - name: elasticsearch
-    version: "~7.10.1"
+    version: "7.16.2"
    repository: "https://helm.elastic.co"
    condition: elasticsearch.enabled
--- a/charts/clearml/README.md
+++ b/charts/clearml/README.md
@@ -1,6 +1,6 @@
 # ClearML Ecosystem for Kubernetes

-![Version: 2.2.2](https://img.shields.io/badge/Version-2.2.2-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.1.1](https://img.shields.io/badge/AppVersion-1.1.1-informational?style=flat-square)
+![Version: 3.10.0](https://img.shields.io/badge/Version-3.10.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.4.0](https://img.shields.io/badge/AppVersion-1.4.0-informational?style=flat-square)

 MLOps platform

@@ -10,7 +10,7 @@ MLOps platform

 | Name | Email | Url |
 | ---- | ------ | --- |
-| valeriano-manassero |  | https://github.com/valeriano-manassero |
+| valeriano-manassero |  | <https://github.com/valeriano-manassero> |

 ## Introduction

@@ -31,22 +31,26 @@ For development/evaluation it's possible to use [kind](https://kind.sigs.k8s.io)
 After installation, following commands will create a complete ClearML insatllation:

 ```
-mkdir -pm 777 /tmp/clearml-kind
-
-cat <<EOF > /tmp/clearml-kind.yaml
+cat <<EOF | kind create cluster --config=-                                                                  ─╯
 kind: Cluster
 apiVersion: kind.x-k8s.io/v1alpha4
 nodes:
 - role: control-plane
  extraPortMappings:
+  # API server's default nodePort is 30008. If you customize it in helm values by
+  # `apiserver.service.nodePort`, `containerPort` should match it
  - containerPort: 30008
    hostPort: 30008
    listenAddress: "127.0.0.1"
    protocol: TCP
+  # Web server's default nodePort is 30080. If you customize it in helm values by
+  # `webserver.service.nodePort`, `containerPort` should match it
  - containerPort: 30080
    hostPort: 30080
    listenAddress: "127.0.0.1"
    protocol: TCP
+  # File server's default nodePort is 30081. If you customize it in helm values by
+  # `fileserver.service.nodePort`, `containerPort` should match it
  - containerPort: 30081
    hostPort: 30081
    listenAddress: "127.0.0.1"
@@ -56,8 +60,6 @@ nodes:
    containerPath: /var/local-path-provisioner
 EOF

-kind create cluster --config /tmp/clearml-kind.yaml
-
 helm install clearml allegroai/clearml
 ```

@@ -83,6 +85,24 @@ This will create 3 ingress rules:

 Just pointing the domain records to the IP where ingress controller is responding will complete the deployment process.

+## Upgrades/ Values upgrades
+
+Updating to latest version of this chart can be done in two steps:
+
+```
+helm repo update
+helm upgrade clearml allegroai/clearml
+```
+
+Changing values on existing installation can be done with:
+
+```
+helm upgrade clearml allegroai/clearml --version <CURRENT CHART VERSION> -f custom_values.yaml
+```
+
+Please note: updating values only should always be done setting explicit chart version to avoid a possible chart update.
+Keeping separate updates procedures between version and values can be a good practice to seprate potential concerns.
+
 ## Additional Configuration for ClearML Server

 You can also configure the **clearml-server** for:
@@ -101,9 +121,9 @@ For detailed instructions, see the [Optional Configuration](https://github.com/a

 | Repository | Name | Version |
 |------------|------|---------|
-| https://charts.bitnami.com/bitnami | mongodb | ~10.3.2 |
-| https://charts.bitnami.com/bitnami | redis | ~10.9.0 |
-| https://helm.elastic.co | elasticsearch | ~7.10.1 |
+| https://charts.bitnami.com/bitnami | mongodb | 10.3.4 |
+| https://charts.bitnami.com/bitnami | redis | 10.9.0 |
+| https://helm.elastic.co | elasticsearch | 7.16.2 |

 ## Values

@@ -121,6 +141,7 @@ For detailed instructions, see the [Optional Configuration](https://github.com/a
 | agentGroups.agent-group-cpu.clearmlGitPassword | string | `nil` |  |
 | agentGroups.agent-group-cpu.clearmlGitUser | string | `nil` |  |
 | agentGroups.agent-group-cpu.clearmlSecretKey | string | `nil` |  |
+| agentGroups.agent-group-cpu.enabled | bool | `false` |  |
 | agentGroups.agent-group-cpu.image.pullPolicy | string | `"IfNotPresent"` |  |
 | agentGroups.agent-group-cpu.image.repository | string | `"ubuntu"` |  |
 | agentGroups.agent-group-cpu.image.tag | string | `"18.04"` |  |
@@ -144,6 +165,7 @@ For detailed instructions, see the [Optional Configuration](https://github.com/a
 | agentGroups.agent-group-gpu.clearmlGitPassword | string | `nil` |  |
 | agentGroups.agent-group-gpu.clearmlGitUser | string | `nil` |  |
 | agentGroups.agent-group-gpu.clearmlSecretKey | string | `nil` |  |
+| agentGroups.agent-group-gpu.enabled | bool | `false` |  |
 | agentGroups.agent-group-gpu.image.pullPolicy | string | `"IfNotPresent"` |  |
 | agentGroups.agent-group-gpu.image.repository | string | `"nvidia/cuda"` |  |
 | agentGroups.agent-group-gpu.image.tag | string | `"11.0-base-ubuntu18.04"` |  |
@@ -155,6 +177,19 @@ For detailed instructions, see the [Optional Configuration](https://github.com/a
 | agentGroups.agent-group-gpu.replicaCount | int | `0` |  |
 | agentGroups.agent-group-gpu.tolerations | list | `[]` |  |
 | agentGroups.agent-group-gpu.updateStrategy | string | `"Recreate"` |  |
+| agentk8sglue.defaultDockerImage | string | `"nvidia/cuda:11.3.1-cudnn8-runtime-ubuntu20.04"` |  |
+| agentk8sglue.enabled | bool | `true` |  |
+| agentk8sglue.id | string | `"k8s-agent"` |  |
+| agentk8sglue.image.repository | string | `"allegroai/clearml-agent-k8s"` |  |
+| agentk8sglue.image.tag | string | `"latest"` |  |
+| agentk8sglue.maxPods | int | `10` |  |
+| agentk8sglue.podTemplate.env | list | `[]` |  |
+| agentk8sglue.podTemplate.nodeSelector | object | `{}` |  |
+| agentk8sglue.podTemplate.resources | object | `{}` |  |
+| agentk8sglue.podTemplate.tolerations | list | `[]` |  |
+| agentk8sglue.podTemplate.volumes | list | `[]` |  |
+| agentk8sglue.queue | string | `"default"` |  |
+| agentk8sglue.serviceAccountName | string | `"default"` |  |
 | agentservices.affinity | object | `{}` |  |
 | agentservices.agentVersion | string | `""` |  |
 | agentservices.awsAccessKeyId | string | `nil` |  |
@@ -168,6 +203,7 @@ For detailed instructions, see the [Optional Configuration](https://github.com/a
 | agentservices.clearmlHostIp | string | `nil` |  |
 | agentservices.clearmlWebHost | string | `nil` |  |
 | agentservices.clearmlWorkerId | string | `"clearml-services"` |  |
+| agentservices.enabled | bool | `false` |  |
 | agentservices.extraEnvs | list | `[]` |  |
 | agentservices.googleCredentials | string | `nil` |  |
 | agentservices.image.pullPolicy | string | `"IfNotPresent"` |  |
@@ -180,12 +216,14 @@ For detailed instructions, see the [Optional Configuration](https://github.com/a
 | agentservices.storage.data.class | string | `"standard"` |  |
 | agentservices.storage.data.size | string | `"50Gi"` |  |
 | agentservices.tolerations | list | `[]` |  |
+| apiserver.additionalConfigs | object | `{}` | additional configurations that can be used by api server; check examples in values.yaml file |
 | apiserver.affinity | object | `{}` |  |
+| apiserver.authCookiesMaxAge | int | `864000` | Amount of seconds the authorization cookie will last in user browser |
 | apiserver.configDir | string | `"/opt/clearml/config"` |  |
 | apiserver.extraEnvs | list | `[]` |  |
 | apiserver.image.pullPolicy | string | `"IfNotPresent"` |  |
 | apiserver.image.repository | string | `"allegroai/clearml"` |  |
-| apiserver.image.tag | string | `"1.1.1"` |  |
+| apiserver.image.tag | string | `"1.4.0"` |  |
 | apiserver.livenessDelay | int | `60` |  |
 | apiserver.nodeSelector | object | `{}` |  |
 | apiserver.podAnnotations | object | `{}` |  |
@@ -195,11 +233,9 @@ For detailed instructions, see the [Optional Configuration](https://github.com/a
 | apiserver.readinessDelay | int | `60` |  |
 | apiserver.replicaCount | int | `1` |  |
 | apiserver.resources | object | `{}` |  |
+| apiserver.service.nodePort | int | `30008` | If service.type set to NodePort, this will be set to service's nodePort field. If service.type is set to others, this field will be ignored |
 | apiserver.service.port | int | `8008` |  |
-| apiserver.service.type | string | `"NodePort"` |  |
-| apiserver.storage.config.class | string | `"standard"` |  |
-| apiserver.storage.config.size | string | `"1Gi"` |  |
-| apiserver.storage.enableConfigVolume | bool | `false` |  |
+| apiserver.service.type | string | `"NodePort"` | This will set to service's spec.type field |
 | apiserver.tolerations | list | `[]` |  |
 | clearml.defaultCompany | string | `"d1bd92a3b039400cbafc60a7a5b1e52b"` |  |
 | elasticsearch.clusterHealthCheckParams | string | `"wait_for_status=yellow&timeout=1s"` |  |
@@ -237,28 +273,44 @@ For detailed instructions, see the [Optional Configuration](https://github.com/a
 | elasticsearch.roles.remote_cluster_client | string | `"true"` |  |
 | elasticsearch.volumeClaimTemplate.accessModes[0] | string | `"ReadWriteOnce"` |  |
 | elasticsearch.volumeClaimTemplate.resources.requests.storage | string | `"50Gi"` |  |
+| externalServices.elasticsearchHost | string | `""` | Existing ElasticSearch Hostname to use if elasticsearch.enabled is false |
+| externalServices.elasticsearchPort | int | `9200` | Existing ElasticSearch Port to use if elasticsearch.enabled is false |
+| externalServices.mongodbHost | string | `""` | Existing MongoDB Hostname to use if elasticsearch.enabled is false |
+| externalServices.mongodbPort | int | `27017` | Existing MongoDB Port to use if elasticsearch.enabled is false |
+| externalServices.redisHost | string | `""` | Existing Redis Hostname to use if elasticsearch.enabled is false |
+| externalServices.redisPort | int | `6379` | Existing Redis Port to use if elasticsearch.enabled is false |
 | fileserver.affinity | object | `{}` |  |
 | fileserver.extraEnvs | list | `[]` |  |
 | fileserver.image.pullPolicy | string | `"IfNotPresent"` |  |
 | fileserver.image.repository | string | `"allegroai/clearml"` |  |
-| fileserver.image.tag | string | `"1.1.1"` |  |
+| fileserver.image.tag | string | `"1.4.0"` |  |
 | fileserver.nodeSelector | object | `{}` |  |
 | fileserver.podAnnotations | object | `{}` |  |
 | fileserver.replicaCount | int | `1` |  |
 | fileserver.resources | object | `{}` |  |
+| fileserver.service.nodePort | int | `30081` | If service.type set to NodePort, this will be set to service's nodePort field. If service.type is set to others, this field will be ignored |
 | fileserver.service.port | int | `8081` |  |
-| fileserver.service.type | string | `"NodePort"` |  |
+| fileserver.service.type | string | `"NodePort"` | This will set to service's spec.type field |
 | fileserver.storage.data.class | string | `"standard"` |  |
 | fileserver.storage.data.size | string | `"50Gi"` |  |
 | fileserver.tolerations | list | `[]` |  |
 | ingress.annotations | object | `{}` |  |
-| ingress.enabled | bool | `false` |  |
-| ingress.host | string | `""` |  |
-| ingress.hostPrefixApi | string | `"api."` |  |
-| ingress.hostPrefixApp | string | `"app."` |  |
-| ingress.hostPrefixFiles | string | `"files."` |  |
+| ingress.api.annotations | object | `{}` |  |
+| ingress.api.enabled | bool | `false` |  |
+| ingress.api.hostName | string | `"api.clearml.127-0-0-1.nip.io"` |  |
+| ingress.api.path | string | `"/"` |  |
+| ingress.api.tlsSecretName | string | `""` |  |
+| ingress.app.annotations | object | `{}` |  |
+| ingress.app.enabled | bool | `false` |  |
+| ingress.app.hostName | string | `"app.clearml.127-0-0-1.nip.io"` |  |
+| ingress.app.path | string | `"/"` |  |
+| ingress.app.tlsSecretName | string | `""` |  |
+| ingress.files.annotations | object | `{}` |  |
+| ingress.files.enabled | bool | `false` |  |
+| ingress.files.hostName | string | `"files.clearml.127-0-0-1.nip.io"` |  |
+| ingress.files.path | string | `"/"` |  |
+| ingress.files.tlsSecretName | string | `""` |  |
 | ingress.name | string | `"clearml-server-ingress"` |  |
-| ingress.tls.secretName | string | `""` |  |
 | mongodb.architecture | string | `"standalone"` |  |
 | mongodb.auth.enabled | bool | `false` |  |
 | mongodb.enabled | bool | `true` |  |
@@ -279,15 +331,23 @@ For detailed instructions, see the [Optional Configuration](https://github.com/a
 | redis.master.persistence.size | string | `"5Gi"` |  |
 | redis.master.port | int | `6379` |  |
 | redis.usePassword | bool | `false` |  |
+| secret.authToken | string | `"1SCf0ov3Nm544Td2oZ0gXSrsNx5XhMWdVlKz1tOgcx158bD5RV"` | Set for auth_token field |
+| secret.credentials.apiserver.accessKey | string | `"5442F3443MJMORWZA3ZH"` | Set for apiserver_key field |
+| secret.credentials.apiserver.secretKey | string | `"BxapIRo9ZINi8x25CRxz8Wdmr2pQjzuWVB4PNASZqCtTyWgWVQ"` | Set for apiserver_secret field |
+| secret.credentials.tests.accessKey | string | `"ENP39EQM4SLACGD5FXB7"` | Set for tests_user_key field |
+| secret.credentials.tests.secretKey | string | `"lPcm0imbcBZ8mwgO7tpadutiS3gnJD05x9j7afwXPS35IKbpiQ"` | Set for tests_user_secret field |
+| secret.httpSession | string | `"9Tw20RbhJ1bLBiHEOWXvhplKGUbTgLzAtwFN2oLQvWwS0uRpD5"` | Set for http_session field |
+| webserver.additionalConfigs | object | `{}` |  |
 | webserver.affinity | object | `{}` |  |
 | webserver.extraEnvs | list | `[]` |  |
 | webserver.image.pullPolicy | string | `"IfNotPresent"` |  |
 | webserver.image.repository | string | `"allegroai/clearml"` |  |
-| webserver.image.tag | string | `"1.1.1"` |  |
+| webserver.image.tag | string | `"1.4.0"` |  |
 | webserver.nodeSelector | object | `{}` |  |
 | webserver.podAnnotations | object | `{}` |  |
 | webserver.replicaCount | int | `1` |  |
 | webserver.resources | object | `{}` |  |
+| webserver.service.nodePort | int | `30080` | If service.type set to NodePort, this will be set to service's nodePort field. If service.type is set to others, this field will be ignored |
 | webserver.service.port | int | `80` |  |
-| webserver.service.type | string | `"NodePort"` |  |
+| webserver.service.type | string | `"NodePort"` | This will set to service's spec.type field |
 | webserver.tolerations | list | `[]` |  |
--- a/charts/clearml/README.md.gotmpl
+++ b/charts/clearml/README.md.gotmpl
@@ -28,22 +28,26 @@ For development/evaluation it's possible to use [kind](https://kind.sigs.k8s.io)
 After installation, following commands will create a complete ClearML insatllation:

 ```
-mkdir -pm 777 /tmp/clearml-kind
-
-cat <<EOF > /tmp/clearml-kind.yaml
+cat <<EOF | kind create cluster --config=-                                                                  ─╯
 kind: Cluster
 apiVersion: kind.x-k8s.io/v1alpha4
 nodes:
 - role: control-plane
  extraPortMappings:
+  # API server's default nodePort is 30008. If you customize it in helm values by
+  # `apiserver.service.nodePort`, `containerPort` should match it
  - containerPort: 30008
    hostPort: 30008
    listenAddress: "127.0.0.1"
    protocol: TCP
+  # Web server's default nodePort is 30080. If you customize it in helm values by
+  # `webserver.service.nodePort`, `containerPort` should match it
  - containerPort: 30080
    hostPort: 30080
    listenAddress: "127.0.0.1"
    protocol: TCP
+  # File server's default nodePort is 30081. If you customize it in helm values by
+  # `fileserver.service.nodePort`, `containerPort` should match it
  - containerPort: 30081
    hostPort: 30081
    listenAddress: "127.0.0.1"
@@ -53,8 +57,6 @@ nodes:
    containerPath: /var/local-path-provisioner
 EOF

-kind create cluster --config /tmp/clearml-kind.yaml
-
 helm install clearml allegroai/clearml
 ```

@@ -80,6 +82,24 @@ This will create 3 ingress rules:

 Just pointing the domain records to the IP where ingress controller is responding will complete the deployment process.

+## Upgrades/ Values upgrades
+
+Updating to latest version of this chart can be done in two steps:
+
+```
+helm repo update
+helm upgrade clearml allegroai/clearml
+```
+
+Changing values on existing installation can be done with:
+
+```
+helm upgrade clearml allegroai/clearml --version <CURRENT CHART VERSION> -f custom_values.yaml
+```
+
+Please note: updating values only should always be done setting explicit chart version to avoid a possible chart update. 
+Keeping separate updates procedures between version and values can be a good practice to seprate potential concerns.
+
 ## Additional Configuration for ClearML Server

 You can also configure the **clearml-server** for:
--- a/charts/clearml/charts/elasticsearch-7.10.1.tgz
+++ b/charts/clearml/charts/elasticsearch-7.10.1.tgz
--- a/charts/clearml/charts/elasticsearch-7.16.2.tgz
+++ b/charts/clearml/charts/elasticsearch-7.16.2.tgz
--- a/charts/clearml/ci/README.md
+++ b/charts/clearml/ci/README.md
@@ -0,0 +1,7 @@
+Place values files with different values in this directory to ensure these cases are tested by the CI as well.
+
+https://github.com/helm/chart-testing/blob/main/doc/ct_install.md
+
+```
+"Charts may have multiple custom values files matching the glob pattern '*-values.yaml' in a directory named 'ci' in the root of the chart's directory. The chart is installed and tested for each of these files. If no custom values file is present, the chart is installed and tested with defaults."
+```
--- a/charts/clearml/ci/default-values.yaml
+++ b/charts/clearml/ci/default-values.yaml
@@ -0,0 +1 @@
+# empty so default values.yaml gets tested
--- a/charts/clearml/ci/k8sagent-values.yaml
+++ b/charts/clearml/ci/k8sagent-values.yaml
@@ -0,0 +1,2 @@
+agentk8sglue:
+  enabled: true
--- a/charts/clearml/templates/_helpers.tpl
+++ b/charts/clearml/templates/_helpers.tpl
@@ -101,13 +101,13 @@ Create the name of the App service to use
 */}}
 {{- define "clearml.serviceApp" -}}
 {{- if .Values.ingress.enabled }}
-{{- if .Values.ingress.tls.secretName }}
-{{- printf "%s%s%s" "https://" .Values.ingress.hostPrefixApp .Values.ingress.host }}
+{{- if .Values.ingress.app.tlsSecretName }}
+{{- printf "%s%s" "https://" .Values.ingress.app.hostName }}
 {{- else }}
-{{- printf "%s%s%s" "http://" .Values.ingress.hostPrefixApp .Values.ingress.host }}
+{{- printf "%s%s" "http://" .Values.ingress.app.hostName }}
 {{- end }}
 {{- else }}
-{{- printf "%s%s%s%s" "http://" (include "clearml.fullname" .) "-webserver:" (.Values.webserver.service.port | quote) }}
+{{- printf "%s%s%s%s" "http://" (include "clearml.fullname" .) "-webserver:" (.Values.webserver.service.port | toString) }}
 {{- end }}
 {{- end }}

@@ -116,13 +116,13 @@ Create the name of the Api service to use
 */}}
 {{- define "clearml.serviceApi" -}}
 {{- if .Values.ingress.enabled }}
-{{- if .Values.ingress.tls.secretName }}
-{{- printf "%s%s%s" "https://" .Values.ingress.hostPrefixApi .Values.ingress.host }}
+{{- if .Values.ingress.api.tlsSecretName }}
+{{- printf "%s%s" "https://" .Values.ingress.api.hostName }}
 {{- else }}
-{{- printf "%s%s%s" "http://" .Values.ingress.hostPrefixApi .Values.ingress.host }}
+{{- printf "%s%s" "http://" .Values.ingress.api.hostName }}
 {{- end }}
 {{- else }}
-{{- printf "%s%s%s%s" "http://" (include "clearml.fullname" .) "-apiserver:" (.Values.apiserver.service.port | quote) }}
+{{- printf "%s%s%s%s" "http://" (include "clearml.fullname" .) "-apiserver:" (.Values.apiserver.service.port | toString) }}
 {{- end }}
 {{- end }}

@@ -131,12 +131,26 @@ Create the name of the Files service to use
 */}}
 {{- define "clearml.serviceFiles" -}}
 {{- if .Values.ingress.enabled }}
-{{- if .Values.ingress.tls.secretName }}
-{{- printf "%s%s%s" "https://" .Values.ingress.hostPrefixFiles .Values.ingress.host }}
+{{- if .Values.ingress.files.tlsSecretName }}
+{{- printf "%s%s" "https://" .Values.ingress.files.hostName }}
 {{- else }}
-{{- printf "%s%s%s" "http://" .Values.ingress.hostPrefixFiles .Values.ingress.host }}
+{{- printf "%s%s" "http://" .Values.ingress.files.hostName }}
 {{- end }}
 {{- else }}
-{{- printf "%s%s%s%s" "http://" (include "clearml.fullname" .) "-fileserver:" (.Values.fileserver.service.port | quote) }}
+{{- printf "%s%s%s%s" "http://" (include "clearml.fullname" .) "-fileserver:" (.Values.fileserver.service.port | toString) }}
 {{- end }}
 {{- end }}
+
+{{/*
+Return the proper Docker Image Registry Secret Names
+*/}}
+{{- define "clearml.imagePullSecrets" -}}
+{{- if .Values.global }}
+{{- if .Values.global.imagePullSecrets }}
+imagePullSecrets:
+{{- range .Values.global.imagePullSecrets }}
+  - name: {{ . }}
+{{- end }}
+{{- end -}}
+{{- end -}}
+{{- end -}}
--- a/charts/clearml/templates/configmap-agentk8s-template.yaml
+++ b/charts/clearml/templates/configmap-agentk8s-template.yaml
@@ -0,0 +1,57 @@
+{{- if .Values.agentk8sglue.enabled }}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: k8sagent-pod-template
+data:
+  template.yaml: |
+    apiVersion: v1
+    metadata:
+      namespace: {{ .Release.Namespace }}
+    spec:
+      serviceAccountName: {{ .Values.agentk8sglue.serviceAccountName }}
+      volumes:
+      {{- range .Values.agentk8sglue.podTemplate.volumes }}
+      - name: {{ .name }}
+        persistentVolumeClaim:
+          claimName: {{ .name }}
+      {{- end }}
+      containers:
+      - resources:
+          {{- toYaml .Values.agentk8sglue.podTemplate.resources | nindent 10 }}
+        ports:
+        - containerPort: 10022
+        volumeMounts:
+        {{- range .Values.agentk8sglue.podTemplate.volumes }}
+        - mountPath: {{ .path }}
+          name: {{ .name }}
+        {{- end }}
+        env:
+        - name: CLEARML_API_HOST
+          value: "http://{{ include "clearml.fullname" . }}-apiserver:{{ .Values.apiserver.service.port }}"
+        - name: CLEARML_WEB_HOST
+          value: "http://{{ include "clearml.fullname" . }}-webserver"
+        - name: CLEARML_FILES_HOST
+          value: "http://{{ include "clearml.fullname" . }}-fileserver:{{ .Values.fileserver.service.port }}"
+        - name: CLEARML_API_ACCESS_KEY
+          valueFrom:
+            secretKeyRef:
+              name: clearml-conf
+              key: apiserver_key
+        - name: CLEARML_API_SECRET_KEY
+          valueFrom:
+            secretKeyRef:
+              name: clearml-conf
+              key: apiserver_secret
+        {{- if .Values.agentk8sglue.podTemplate.env }}
+        {{ toYaml .Values.agentk8sglue.podTemplate.env | nindent 8 }}
+        {{- end }}
+        {{- with .Values.agentk8sglue.podTemplate.nodeSelector}}
+        nodeSelector:
+          {{- toYaml . | nindent 8 }}
+        {{- end }}
+        {{- with .Values.agentk8sglue.podTemplate.tolerations }}
+        tolerations:
+          {{- toYaml . | nindent 8 }}
+        {{- end }}
+{{- end }}
--- a/charts/clearml/templates/configmap-apiserver.yaml
+++ b/charts/clearml/templates/configmap-apiserver.yaml
@@ -0,0 +1,13 @@
+{{- if .Values.apiserver.additionalConfigs -}}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: "{{ include "clearml.fullname" . }}-apiserver-configmap"
+  labels:
+    {{- include "clearml.labels" . | nindent 4 }}
+data:
+  {{- range $key, $val := .Values.apiserver.additionalConfigs }}
+  {{ $key }}: |
+    {{- $val | nindent 4 }}
+  {{- end }}
+{{- end -}}
--- a/charts/clearml/templates/configmap-webserver.yaml
+++ b/charts/clearml/templates/configmap-webserver.yaml
@@ -0,0 +1,13 @@
+{{- if .Values.webserver.additionalConfigs -}}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: "{{ include "clearml.fullname" . }}-webserver-configmap"
+  labels:
+    {{- include "clearml.labels" . | nindent 4 }}
+data:
+  {{- range $key, $val := .Values.webserver.additionalConfigs }}
+  {{ $key }}: |
+    {{- $val | nindent 4 }}
+  {{- end }}
+{{- end -}}
--- a/charts/clearml/templates/deployment-agent.yaml
+++ b/charts/clearml/templates/deployment-agent.yaml
@@ -1,5 +1,6 @@
 {{- range $key, $value := .Values.agentGroups }}
 {{- with $value }}
+{{- if .enabled }}
 ---
 apiVersion: apps/v1
 kind: Deployment
@@ -16,8 +17,9 @@ spec:
      {{- include "clearml.selectorLabelsAgent" $ | nindent 6 }}
  template:
    metadata:
-      {{- with .podAnnotations }}
      annotations:
+        checksum/secret: {{ include (print $.Template.BasePath "/secrets.yaml") $ | sha256sum }}
+      {{- with .podAnnotations }}
        {{- toYaml . | nindent 8 }}
      {{- end }}
      labels:
@@ -92,7 +94,7 @@ spec:
              apt-get install -y curl python3-pip git;
              python3 -m pip install -U pip ;
              python3 -m pip install clearml-agent{{ .agentVersion}} ;
-              CLEARML_AGENT_K8S_HOST_MOUNT=/root/.clearml:/root/.clearml clearml-agent daemon --queue {{ .queues}}"
+              CLEARML_AGENT_K8S_HOST_MOUNT=/root/.clearml:/root/.clearml clearml-agent daemon --foreground --queue {{ .queues}}"
          {{ if .clearmlConfig }}
          volumeMounts:
           - name: agent-clearml-conf-volume
@@ -114,3 +116,4 @@ spec:
      {{- end }}
 {{- end }}
 {{- end }}
+{{- end }}
--- a/charts/clearml/templates/deployment-agentk8s.yaml
+++ b/charts/clearml/templates/deployment-agentk8s.yaml
@@ -0,0 +1,64 @@
+{{- if .Values.agentk8sglue.enabled }}
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: "{{ include "clearml.fullname" . }}-k8sagent"
+  labels:
+    app: k8sagent
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: k8sagent
+  template:
+    metadata:
+      labels:
+        app: k8sagent
+    spec:
+      containers:
+      - name: k8s-glue-container
+        image: "{{ .Values.agentk8sglue.image.repository }}:{{ .Values.agentk8sglue.image.tag }}"
+        imagePullPolicy: Always
+        command: ["/bin/bash", "-c", "export PATH=$PATH:$HOME/bin; source /root/.bashrc && /root/entrypoint.sh"]
+        volumeMounts:
+          - name: k8sagent-pod-template
+            mountPath: /root/template
+        env:
+          - name: CLEARML_API_HOST
+            value: "http://{{ include "clearml.fullname" . }}-apiserver:{{ .Values.apiserver.service.port }}"
+          - name: CLEARML_WEB_HOST
+            value: "http://{{ include "clearml.fullname" . }}-webserver"
+          - name: CLEARML_FILES_HOST
+            value: "http://{{ include "clearml.fullname" . }}-fileserver:{{ .Values.fileserver.service.port }}"
+          - name: K8S_GLUE_MAX_PODS
+            value: "{{.Values.agentk8sglue.maxPods}}"
+          - name: K8S_GLUE_QUEUE
+            value: "{{.Values.agentk8sglue.queue}}"
+          - name: K8S_GLUE_EXTRA_ARGS
+            value: "--namespace {{ .Release.Namespace }} --template-yaml /root/template/template.yaml"
+          - name: K8S_DEFAULT_NAMESPACE
+            value: "{{ .Release.Namespace }}"
+          - name: CLEARML_API_ACCESS_KEY
+            valueFrom:
+              secretKeyRef:
+                name: clearml-conf
+                key: apiserver_key
+          - name: CLEARML_API_SECRET_KEY
+            valueFrom:
+              secretKeyRef:
+                name: clearml-conf
+                key: apiserver_secret
+          - name: CLEARML_WORKER_ID
+            value: "{{.Values.agentk8sglue.id}}"
+          - name: CLEARML_AGENT_UPDATE_REPO
+            value: ""
+          - name: FORCE_CLEARML_AGENT_REPO
+            value: ""
+          - name: CLEARML_DOCKER_IMAGE
+            value: "{{.Values.agentk8sglue.defaultDockerImage}}"
+      volumes:
+        - name: k8sagent-pod-template
+          configMap: 
+            name: k8sagent-pod-template
+{{- end }}
+
--- a/charts/clearml/templates/deployment-agentservices.yaml
+++ b/charts/clearml/templates/deployment-agentservices.yaml
@@ -1,3 +1,4 @@
+{{- if .Values.agentservices.enabled }}
 apiVersion: apps/v1
 kind: Deployment
 metadata:
@@ -11,8 +12,9 @@ spec:
      {{- include "clearml.selectorLabelsAgentServices" . | nindent 6 }}
  template:
    metadata:
-      {{- with .Values.agentservices.podAnnotations }}
      annotations:
+        checksum/secret: {{ include (print $.Template.BasePath "/secrets.yaml") . | sha256sum }}
+      {{- with .Values.agentservices.podAnnotations }}
        {{- toYaml . | nindent 8 }}
      {{- end }}
      labels:
@@ -98,3 +100,4 @@ spec:
      tolerations:
        {{- toYaml . | nindent 8 }}
      {{- end }}
+{{- end }}
--- a/charts/clearml/templates/deployment-apiserver.yaml
+++ b/charts/clearml/templates/deployment-apiserver.yaml
@@ -11,19 +11,15 @@ spec:
      {{- include "clearml.selectorLabelsApiServer" . | nindent 6 }}
  template:
    metadata:
-      {{- with .Values.apiserver.podAnnotations }}
      annotations:
+        checksum/secret: {{ include (print $.Template.BasePath "/secrets.yaml") . | sha256sum }}
+      {{- with .Values.apiserver.podAnnotations }}
        {{- toYaml . | nindent 8 }}
      {{- end }}
      labels:
        {{- include "clearml.selectorLabelsApiServer" . | nindent 8 }}
    spec:
-      {{- if .Values.apiserver.storage.enableConfigVolume }}
-      volumes:
-        - name: apiserver-config
-          persistentVolumeClaim:
-            claimName: {{ include "clearml.fullname" . }}-apiserver-config
-      {{- end }}
+{{- include "clearml.imagePullSecrets" . | indent 6 }}
      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.apiserver.image.repository }}:{{ .Values.apiserver.image.tag | default .Chart.AppVersion }}"
@@ -34,23 +30,49 @@ spec:
              protocol: TCP
          env:
          - name: CLEARML_ELASTIC_SERVICE_HOST
+          {{- if .Values.elasticsearch.enabled }}
            value: "{{ .Values.elasticsearch.clusterName }}-master"
+          {{- else }}
+            value: "{{ .Values.externalServices.elasticsearchHost }}"
+          {{- end }}
          - name: CLEARML_ELASTIC_SERVICE_PORT
+          {{- if .Values.elasticsearch.enabled }}
            value: "{{ .Values.elasticsearch.httpPort }}"
+          {{- else }}
+            value: "{{ .Values.externalServices.elasticsearchPort }}"
+          {{- end }}
          - name: CLEARML_MONGODB_SERVICE_HOST
+          {{- if .Values.mongodb.enabled }}
            value: "{{ tpl .Values.mongodb.service.name . }}"
+          {{- else }}
+            value: "{{ .Values.externalServices.mongodbHost }}"
+          {{- end }}
          - name: CLEARML_MONGODB_SERVICE_PORT
+          {{- if .Values.mongodb.enabled }}
            value: "{{ .Values.mongodb.service.port }}"
+          {{- else }}
+            value: "{{ .Values.externalServices.mongodbPort }}"
+          {{- end }}
          - name: CLEARML_REDIS_SERVICE_HOST
+          {{- if .Values.redis.enabled }}
            value: "{{ tpl .Values.redis.master.name . }}"
+          {{- else }}
+            value: "{{ .Values.externalServices.redisHost }}"
+          {{- end }}
          - name: CLEARML_REDIS_SERVICE_PORT
+          {{- if .Values.redis.enabled }}
            value: "{{ .Values.redis.master.port }}"
+          {{- else }}
+            value: "{{ .Values.externalServices.redisPort }}"
+          {{- end }}
          - name: CLEARML__APISERVER__PRE_POPULATE__ENABLED
            value: "{{ .Values.apiserver.prepopulateEnabled }}"
          - name: CLEARML__APISERVER__PRE_POPULATE__ZIP_FILES
            value: "{{ .Values.apiserver.prepopulateZipFiles }}"
          - name: CLEARML_SERVER_DEPLOYMENT_TYPE
            value: "helm-cloud"
+          - name: CLEARML__APISERVER__AUTH__COOKIES__MAX_AGE
+            value: "{{ .Values.apiserver.authCookiesMaxAge }}"
          - name: CLEARML_CONFIG_DIR
            value: /opt/clearml/config
          - name: CLEARML__APISERVER__DEFAULT_COMPANY
@@ -101,13 +123,19 @@ spec:
            httpGet:
              path: /debug.ping
              port: 8008
-          {{- if .Values.apiserver.storage.enableConfigVolume }}
+          {{- if .Values.apiserver.additionalConfigs }}
          volumeMounts:
           - name: apiserver-config
             mountPath: /opt/clearml/config
          {{- end }}
          resources:
            {{- toYaml .Values.apiserver.resources | nindent 12 }}
+      {{- if .Values.apiserver.additionalConfigs }}
+      volumes:
+        - name: apiserver-config
+          configMap:
+            name: "{{ include "clearml.fullname" . }}-apiserver-configmap"
+      {{- end }}
      {{- with .Values.apiserver.nodeSelector }}
      nodeSelector:
        {{- toYaml . | nindent 8 }}
--- a/charts/clearml/templates/deployment-fileserver.yaml
+++ b/charts/clearml/templates/deployment-fileserver.yaml
@@ -22,6 +22,7 @@ spec:
        - name: fileserver-data
          persistentVolumeClaim:
            claimName: {{ include "clearml.fullname" . }}-fileserver-data
+{{- include "clearml.imagePullSecrets" . | indent 6 }}
      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.fileserver.image.repository }}:{{ .Values.fileserver.image.tag | default .Chart.AppVersion }}"
--- a/charts/clearml/templates/deployment-webserver.yaml
+++ b/charts/clearml/templates/deployment-webserver.yaml
@@ -18,6 +18,7 @@ spec:
      labels:
        {{- include "clearml.selectorLabelsWebServer" . | nindent 8 }}
    spec:
+{{- include "clearml.imagePullSecrets" . | indent 6 }}
      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.webserver.image.repository }}:{{ .Values.webserver.image.tag | default .Chart.AppVersion }}"
@@ -38,6 +39,11 @@ spec:
                - curl
                - -X OPTIONS
                - http://0.0.0.0:80/
+          {{- if .Values.webserver.additionalConfigs }}
+          volumeMounts:
+          - name: webserver-config
+            mountPath: /opt/clearml/config
+          {{- end }}
          env:
          - name: NGINX_APISERVER_ADDRESS
            value: "http://{{ include "clearml.fullname" . }}-apiserver:{{ .Values.apiserver.service.port }}"
@@ -50,6 +56,12 @@ spec:
            - webserver
          resources:
            {{- toYaml .Values.webserver.resources | nindent 12 }}
+      {{- if .Values.webserver.additionalConfigs }}
+      volumes:
+        - name: webserver-config
+          configMap:
+            name: "{{ include "clearml.fullname" . }}-webserver-configmap"
+      {{- end }}
      {{- with .Values.webserver.nodeSelector }}
      nodeSelector:
        {{- toYaml . | nindent 8 }}
@@ -61,4 +73,4 @@ spec:
      {{- with .Values.webserver.tolerations }}
      tolerations:
        {{- toYaml . | nindent 8 }}
-      {{- end }}
+      {{- end }}
--- a/charts/clearml/templates/ingress-api.yaml
+++ b/charts/clearml/templates/ingress-api.yaml
@@ -1,33 +1,45 @@
-{{- if .Values.ingress.enabled -}}
-{{- $fullName := include "clearml.fullname" . -}}
-{{- if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
+{{- if .Values.ingress.api.enabled -}}
+{{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion -}}
+apiVersion: networking.k8s.io/v1
+{{- else if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
 apiVersion: networking.k8s.io/v1beta1
 {{- else -}}
 apiVersion: extensions/v1beta1
 {{- end }}
 kind: Ingress
 metadata:
-  name: {{ $fullName }}-api
+  name: {{ include "clearml.fullname" . }}-api
  labels:
    {{- include "clearml.labels" . | nindent 4 }}
-  {{- with .Values.ingress.annotations }}
-  annotations:
-    {{- toYaml . | nindent 4 }}
+  {{- $annotations := .Values.ingress.annotations }}
+  {{- if .Values.ingress.api.annotations }}
+  {{- $annotations = mergeOverwrite $annotations .Values.ingress.api.annotations }}
  {{- end }}
+  annotations:
+    {{- toYaml $annotations | nindent 4 }}
+
 spec:
-  {{- if .Values.ingress.tls.secretName }}
+  {{- if .Values.ingress.api.tlsSecretName }}
  tls:
    - hosts:
-        - "{{ .Values.ingress.hostPrefixApi }}{{ .Values.ingress.host }}"
-      secretName: {{ .Values.ingress.tls.secretName }}
+        - {{ .Values.ingress.api.hostName }}
+      secretName: {{ .Values.ingress.api.tlsSecretName }}
  {{- end }}
  rules:
-    - host: "{{ .Values.ingress.hostPrefixApi }}{{ .Values.ingress.host }}"
-      http:
-        paths:
-          - path: "/"
-            pathType: Prefix
-            backend:
-              serviceName: {{ include "clearml.fullname" . }}-apiserver
-              servicePort: {{ .Values.apiserver.service.port }}
+  - host: {{ .Values.ingress.api.hostName }}
+    http:
+      paths:
+      - path: {{ .Values.ingress.api.path }}
+  {{ if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion }}
+        pathType: Prefix
+        backend:
+          service:
+            name: {{ include "clearml.fullname" . }}-apiserver
+            port:
+              number: {{ .Values.apiserver.service.port }}
+  {{ else }}
+        backend:
+          serviceName: {{ include "clearml.fullname" . }}-apiserver
+          servicePort: {{ .Values.apiserver.service.port }}
+  {{ end }}
 {{- end }}
--- a/charts/clearml/templates/ingress-app.yaml
+++ b/charts/clearml/templates/ingress-app.yaml
@@ -1,33 +1,44 @@
-{{- if .Values.ingress.enabled -}}
-{{- $fullName := include "clearml.fullname" . -}}
-{{- if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
+{{- if .Values.ingress.app.enabled -}}
+{{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion -}}
+apiVersion: networking.k8s.io/v1
+{{- else if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
 apiVersion: networking.k8s.io/v1beta1
 {{- else -}}
 apiVersion: extensions/v1beta1
 {{- end }}
 kind: Ingress
 metadata:
-  name: {{ $fullName }}-app
+  name: {{ include "clearml.fullname" . }}-app
  labels:
    {{- include "clearml.labels" . | nindent 4 }}
-  {{- with .Values.ingress.annotations }}
-  annotations:
-    {{- toYaml . | nindent 4 }}
+  {{- $annotations := .Values.ingress.annotations }}
+  {{- if .Values.ingress.app.annotations }}
+  {{- $annotations = mergeOverwrite $annotations .Values.ingress.app.annotations }}
  {{- end }}
+  annotations:
+    {{- toYaml $annotations | nindent 4 }}
 spec:
-  {{- if .Values.ingress.tls.secretName }}
+  {{- if .Values.ingress.app.tlsSecretName }}
  tls:
    - hosts:
-        - "{{ .Values.ingress.hostPrefixApp }}{{ .Values.ingress.host }}"
-      secretName: {{ .Values.ingress.tls.secretName }}
+        - {{ .Values.ingress.app.hostName }}
+      secretName: {{ .Values.ingress.app.tlsSecretName }}
  {{- end }}
  rules:
-    - host: "{{ .Values.ingress.hostPrefixApp }}{{ .Values.ingress.host }}"
-      http:
-        paths:
-          - path: "/"
-            pathType: Prefix
-            backend:
-              serviceName: {{ include "clearml.fullname" . }}-webserver
-              servicePort: {{ .Values.webserver.service.port }}
+  - host: {{ .Values.ingress.app.hostName }}
+    http:
+      paths:
+      - path: {{ .Values.ingress.app.path }}
+  {{ if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion }}
+        pathType: Prefix
+        backend:
+          service:
+            name: {{ include "clearml.fullname" . }}-webserver
+            port:
+              number: {{ .Values.webserver.service.port }}
+  {{ else }}
+        backend:
+          serviceName: {{ include "clearml.fullname" . }}-webserver
+          servicePort: {{ .Values.webserver.service.port }}
+  {{ end }}
 {{- end }}
--- a/charts/clearml/templates/ingress-files.yaml
+++ b/charts/clearml/templates/ingress-files.yaml
@@ -1,33 +1,44 @@
-{{- if .Values.ingress.enabled -}}
-{{- $fullName := include "clearml.fullname" . -}}
-{{- if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
+{{- if .Values.ingress.files.enabled -}}
+{{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion -}}
+apiVersion: networking.k8s.io/v1
+{{- else if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
 apiVersion: networking.k8s.io/v1beta1
 {{- else -}}
 apiVersion: extensions/v1beta1
 {{- end }}
 kind: Ingress
 metadata:
-  name: {{ $fullName }}-files
+  name: {{ include "clearml.fullname" . }}-files
  labels:
    {{- include "clearml.labels" . | nindent 4 }}
-  {{- with .Values.ingress.annotations }}
-  annotations:
-    {{- toYaml . | nindent 4 }}
+  {{- $annotations := .Values.ingress.annotations }}
+  {{- if .Values.ingress.files.annotations }}
+  {{- $annotations = mergeOverwrite $annotations .Values.ingress.files.annotations }}
  {{- end }}
+  annotations:
+    {{- toYaml $annotations | nindent 4 }}
 spec:
-  {{- if .Values.ingress.tls.secretName }}
+  {{- if .Values.ingress.files.tlsSecretName }}
  tls:
    - hosts:
-        - "{{ .Values.ingress.hostPrefixFiles }}{{ .Values.ingress.host }}"
-      secretName: {{ .Values.ingress.tls.secretName }}
+        - {{ .Values.ingress.files.hostName }}
+      secretName: {{ .Values.ingress.files.tlsSecretName }}
  {{- end }}
  rules:
-    - host: "{{ .Values.ingress.hostPrefixFiles }}{{ .Values.ingress.host }}"
-      http:
-        paths:
-          - path: "/"
-            pathType: Prefix
-            backend:
-              serviceName: {{ include "clearml.fullname" . }}-fileserver
-              servicePort: {{ .Values.fileserver.service.port }}
+  - host: {{ .Values.ingress.files.hostName }}
+    http:
+      paths:
+      - path: {{ .Values.ingress.files.path }}
+  {{ if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion }}
+        pathType: Prefix
+        backend:
+          service:
+            name: {{ include "clearml.fullname" . }}-fileserver
+            port:
+              number: {{ .Values.fileserver.service.port }}
+  {{ else }}
+        backend:
+          serviceName: {{ include "clearml.fullname" . }}-fileserver
+          servicePort: {{ .Values.fileserver.service.port }}
+  {{ end }}
 {{- end }}
--- a/charts/clearml/templates/pvc-agentservices.yaml
+++ b/charts/clearml/templates/pvc-agentservices.yaml
@@ -1,3 +1,4 @@
+{{- if .Values.agentservices.enabled }}
 kind: PersistentVolumeClaim
 apiVersion: v1
 metadata:
@@ -11,3 +12,4 @@ spec:
    requests:
      storage: {{ .Values.agentservices.storage.data.size | quote }}
  storageClassName: {{ .Values.agentservices.storage.data.class | quote }}
+{{- end }}
--- a/charts/clearml/templates/pvc-apiserver.yaml
+++ b/charts/clearml/templates/pvc-apiserver.yaml
@@ -1,15 +0,0 @@
-{{- if .Values.apiserver.storage.enableConfigVolume }}
-kind: PersistentVolumeClaim
-apiVersion: v1
-metadata:
-  name: {{ include "clearml.fullname" . }}-apiserver-config
-  labels:
-    {{- include "clearml.labels" . | nindent 4 }}
-spec:
-  accessModes:
-    - ReadWriteOnce
-  resources:
-    requests:
-      storage: {{ .Values.apiserver.storage.config.size | quote }}
-  storageClassName: {{ .Values.apiserver.storage.config.class | quote }}
-{{- end }}
--- a/charts/clearml/templates/rbac-agentk8s.yaml
+++ b/charts/clearml/templates/rbac-agentk8s.yaml
@@ -0,0 +1,25 @@
+{{- if .Values.agentk8sglue.enabled }}
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name: k8sagent-pods-access
+rules:
+  - apiGroups:
+      - ""
+    resources:
+      - pods
+    verbs: ["get", "list", "watch", "create", "patch", "delete"]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: k8sagent-pods-access
+subjects:
+  - kind: ServiceAccount
+    name: default
+    namespace: {{ .Release.Namespace }}
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: k8sagent-pods-access
+{{- end }}
--- a/charts/clearml/templates/secrets.yaml
+++ b/charts/clearml/templates/secrets.yaml
@@ -2,10 +2,10 @@ apiVersion: v1
 kind: Secret
 metadata:
  name: clearml-conf
-data:
-  apiserver_key: NTQ0MkYzNDQzTUpNT1JXWkEzWkg=
-  apiserver_secret: QnhhcElSbzlaSU5pOHgyNUNSeHo4V2RtcjJwUWp6dVdWQjRQTkFTWnFDdFR5V2dXVlE=
-  http_session: OVR3MjBSYmhKMWJMQmlIRU9XWHZocGxLR1ViVGdMekF0d0ZOMm9MUXZXd1MwdVJwRDU=
-  auth_token: MVNDZjBvdjNObTU0NFRkMm9aMGdYU3JzTng1WGhNV2RWbEt6MXRPZ2N4MTU4YkQ1UlY=
-  tests_user_key: RU5QMzlFUU00U0xBQ0dENUZYQjc=
-  tests_user_secret: bFBjbTBpbWJjQlo4bXdnTzd0cGFkdXRpUzNnbkpEMDV4OWo3YWZ3WFBTMzVJS2JwaVE=
+stringData:
+  apiserver_key: {{ .Values.secret.credentials.apiserver.accessKey }}
+  apiserver_secret: {{ .Values.secret.credentials.apiserver.secretKey }}
+  http_session: {{ .Values.secret.httpSession }}
+  auth_token: {{ .Values.secret.authToken }}
+  tests_user_key: {{ .Values.secret.credentials.tests.accessKey }}
+  tests_user_secret: {{ .Values.secret.credentials.tests.secretKey }}
--- a/charts/clearml/templates/service-apiserver.yaml
+++ b/charts/clearml/templates/service-apiserver.yaml
@@ -9,7 +9,9 @@ spec:
  ports:
    - port: {{ .Values.apiserver.service.port }}
      targetPort: {{ .Values.apiserver.service.port }}
-      nodePort: 30008
+{{- if eq .Values.apiserver.service.type "NodePort" }}
+      nodePort: {{ .Values.apiserver.service.nodePort }}
+{{- end }}
      protocol: TCP
  selector:
    {{- include "clearml.selectorLabelsApiServer" . | nindent 4 }}
--- a/charts/clearml/templates/service-fileserver.yaml
+++ b/charts/clearml/templates/service-fileserver.yaml
@@ -9,7 +9,9 @@ spec:
  ports:
    - port: {{ .Values.fileserver.service.port }}
      targetPort: {{ .Values.fileserver.service.port }}
-      nodePort: 30081
+{{- if eq .Values.fileserver.service.type "NodePort" }}
+      nodePort: {{ .Values.fileserver.service.nodePort }}
+{{- end }}
      protocol: TCP
  selector:
    {{- include "clearml.selectorLabelsFileServer" . | nindent 4 }}
--- a/charts/clearml/templates/service-webserver.yaml
+++ b/charts/clearml/templates/service-webserver.yaml
@@ -9,7 +9,9 @@ spec:
  ports:
    - port: {{ .Values.webserver.service.port }}
      targetPort: {{ .Values.webserver.service.port }}
-      nodePort: 30080
+{{- if eq .Values.webserver.service.type "NodePort" }}
+      nodePort: {{ .Values.webserver.service.nodePort }}
+{{- end }}
      protocol: TCP
  selector:
    {{- include "clearml.selectorLabelsWebServer" . | nindent 4 }}
--- a/charts/clearml/values.yaml
+++ b/charts/clearml/values.yaml
@@ -1,15 +1,46 @@
+# global:
+#   imagePullSecrets:
+#   - docker-cfg
 clearml:
  defaultCompany: "d1bd92a3b039400cbafc60a7a5b1e52b"
 ingress:
-  enabled: false
  name: clearml-server-ingress
  annotations: {}
-  host: ""
-  hostPrefixApp: "app."
-  hostPrefixApi: "api."
-  hostPrefixFiles: "files."
-  tls:
-    secretName: ""
+  app:
+    enabled: false
+    hostName: "app.clearml.127-0-0-1.nip.io"
+    tlsSecretName: ""
+    annotations: {}
+    path: "/"
+  api:
+    enabled: false
+    hostName: "api.clearml.127-0-0-1.nip.io"
+    tlsSecretName: ""
+    annotations: {}
+    path: "/"
+  files:
+    enabled: false
+    hostName: "files.clearml.127-0-0-1.nip.io"
+    tlsSecretName: ""
+    annotations: {}
+    path: "/"
+
+secret:
+  # -- Set for http_session field
+  httpSession: "9Tw20RbhJ1bLBiHEOWXvhplKGUbTgLzAtwFN2oLQvWwS0uRpD5"
+  # -- Set for auth_token field
+  authToken: "1SCf0ov3Nm544Td2oZ0gXSrsNx5XhMWdVlKz1tOgcx158bD5RV"
+  credentials:
+    apiserver:
+      # -- Set for apiserver_key field
+      accessKey: "5442F3443MJMORWZA3ZH"
+      # -- Set for apiserver_secret field
+      secretKey: "BxapIRo9ZINi8x25CRxz8Wdmr2pQjzuWVB4PNASZqCtTyWgWVQ"
+    tests:
+      # -- Set for tests_user_key field
+      accessKey: "ENP39EQM4SLACGD5FXB7"
+      # -- Set for tests_user_secret field
+      secretKey: "lPcm0imbcBZ8mwgO7tpadutiS3gnJD05x9j7afwXPS35IKbpiQ"

 apiserver:
  prepopulateEnabled: "true"
@@ -17,9 +48,16 @@ apiserver:
  prepopulateArtifactsPath: "/mnt/fileserver"
  configDir: /opt/clearml/config

+  # -- Amount of seconds the authorization cookie will last in user browser
+  authCookiesMaxAge: 864000
+
  service:
+    # -- This will set to service's spec.type field
    type: NodePort
    port: 8008
+    # -- If service.type set to NodePort, this will be set to service's nodePort field.
+    # If service.type is set to others, this field will be ignored
+    nodePort: 30008

  livenessDelay: 60
  readinessDelay: 60
@@ -29,7 +67,7 @@ apiserver:
  image:
    repository: "allegroai/clearml"
    pullPolicy: IfNotPresent
-    tag: "1.1.1"
+    tag: "1.4.0"

  extraEnvs: []

@@ -53,24 +91,52 @@ apiserver:

  affinity: {}

-  # Optional: used in pvc-apiserver containing optional server configuration files
-  storage:
-    enableConfigVolume: false
-    config:
-      class: "standard"
-      size: 1Gi
+  # -- additional configurations that can be used by api server; check examples in values.yaml file
+  additionalConfigs: {}
+    # services.conf: |
+    #   tasks {
+    #     non_responsive_tasks_watchdog {
+    #       # In-progress tasks that haven't been updated for at least 'value' seconds will be stopped by the watchdog
+    #       threshold_sec: 21000
+    #       # Watchdog will sleep for this number of seconds after each cycle
+    #       watch_interval_sec: 900
+    #     }
+    #   }
+    # apiserver.conf: |
+    #   auth {
+    #     fixed_users {
+    #       enabled: true
+    #       pass_hashed: false
+    #       users: [
+    #         {
+    #           username: "jane"
+    #           password: "12345678"
+    #           name: "Jane Doe"
+    #         },
+    #         {
+    #           username: "john"
+    #           password: "12345678"
+    #           name: "John Doe"
+    #         },
+    #       ]
+    #     }
+    #   }

 fileserver:
  service:
+    # -- This will set to service's spec.type field
    type: NodePort
    port: 8081
+    # -- If service.type set to NodePort, this will be set to service's nodePort field.
+    # If service.type is set to others, this field will be ignored
+    nodePort: 30081

  replicaCount: 1

  image:
    repository: "allegroai/clearml"
    pullPolicy: IfNotPresent
-    tag: "1.1.1"
+    tag: "1.4.0"

  extraEnvs: []

@@ -103,15 +169,19 @@ webserver:
  extraEnvs: []

  service:
+    # -- This will set to service's spec.type field
    type: NodePort
    port: 80
+    # -- If service.type set to NodePort, this will be set to service's nodePort field.
+    # If service.type is set to others, this field will be ignored
+    nodePort: 30080

  replicaCount: 1

  image:
    repository: "allegroai/clearml"
    pullPolicy: IfNotPresent
-    tag: "1.1.1"
+    tag: "1.4.0"

  podAnnotations: {}

@@ -133,7 +203,10 @@ webserver:

  affinity: {}

+  additionalConfigs: {}
+
 agentservices:
+  enabled: false
  clearmlHostIp: null
  agentVersion: ""
  clearmlWebHost: null
@@ -184,6 +257,7 @@ agentservices:

 agentGroups:
  agent-group-cpu:
+    enabled: false
    name: agent-group-cpu
    replicaCount: 1
    updateStrategy: Recreate
@@ -217,6 +291,7 @@ agentGroups:
    affinity: {}

  agent-group-gpu:
+    enabled: false
    name: agent-group-gpu
    replicaCount: 0
    updateStrategy: Recreate
@@ -249,6 +324,57 @@ agentGroups:

    affinity: {}

+# This agent will spawn queued experiments in new pods, a good use case is to combine this with
+# GPU autoscaling nodes.
+# https://github.com/allegroai/clearml-agent/tree/master/docker/k8s-glue
+agentk8sglue:
+  enabled: true
+  image:
+    repository: "allegroai/clearml-agent-k8s"
+    tag: "latest"
+  serviceAccountName: default
+  maxPods: 10
+  defaultDockerImage: nvidia/cuda:11.3.1-cudnn8-runtime-ubuntu20.04   # default docker image that is spawned as new pod
+  queue: default
+  id: k8s-agent
+  podTemplate:
+    volumes: []
+      # - name: "yourvolume"
+      #   path: "/yourpath"
+    env: []
+      # # to setup access to private repo, setup secret with git credentials:
+      # - name: CLEARML_AGENT_GIT_USER
+      #   value: mygitusername
+      # - name: CLEARML_AGENT_GIT_PASS
+      #   valueFrom:
+      #     secretKeyRef:
+      #       name: git-password
+      #       key: git-password
+    resources: {}
+      # limits:
+      #   nvidia.com/gpu: 1
+    tolerations: []
+      # - key: "nvidia.com/gpu"
+      #   operator: Exists
+      #   effect: "NoSchedule"
+    nodeSelector: {}
+      # fleet: gpu-nodes
+
+
+externalServices:
+  # -- Existing ElasticSearch Hostname to use if elasticsearch.enabled is false
+  elasticsearchHost: ""
+  # -- Existing ElasticSearch Port to use if elasticsearch.enabled is false
+  elasticsearchPort: 9200
+  # -- Existing MongoDB Hostname to use if elasticsearch.enabled is false
+  mongodbHost: ""
+  # -- Existing MongoDB Port to use if elasticsearch.enabled is false
+  mongodbPort: 27017
+  # -- Existing Redis Hostname to use if elasticsearch.enabled is false
+  redisHost: ""
+  # -- Existing Redis Port to use if elasticsearch.enabled is false
+  redisPort: 6379
+
 redis:  # configuration from https://github.com/bitnami/charts/blob/master/bitnami/redis/values.yaml
  enabled: true
  usePassword: false
@@ -281,7 +407,7 @@ mongodb:  # configuration from https://github.com/bitnami/charts/blob/master/bit
    port: 27017
    portName: mongo-service

-elasticsearch:  # configuration from https://github.com/elastic/helm-charts/blob/7.10/elasticsearch/values.yaml
+elasticsearch:  # configuration from https://github.com/elastic/helm-charts/blob/7.16/elasticsearch/values.yaml
  enabled: true
  httpPort: 9200
  roles:
Author	SHA1	Message	Date
Valeriano Manassero	7a7bd930f8	Fix glue namespace handling (#63 ) * Changed: namespace handling for glue * Changed: set glue as default agent system * Changed: bump up version	2022-04-22 10:19:03 +02:00
Valeriano Manassero	25dfbd12d6	Changed: bump up versions (#62 ) * Changed: bump up versions * Changed: helmpdocs to 1.8.1	2022-04-19 08:22:08 +02:00
Valeriano Manassero	d7c3b9d5d9	Added: upgrade procedures (#61 ) * Added: upgrade procedures * Changed: template * Changed: updated chart version	2022-04-04 10:32:51 +02:00
Valeriano Manassero	e16060f2ad	Fix empty glue configs (#59 ) * Added: use empty values without breaking glue agent * Added: release namespace * Changed: bump up version	2022-03-30 16:33:06 +02:00
Valeriano Manassero	27a666d2ae	Clarml app 1.3.0 (#57 ) * Changed: clarml app version * Changed: chart version bump * Added: comment on additional configs	2022-03-28 09:29:04 +02:00
Valeriano Manassero	d7bef0ff9d	Add authentication example (#56 ) * Added: auth enabled example in additionalConfigs * Changed: bump up version * Fixed: remove trailing spaces	2022-03-25 10:27:40 +01:00
Zied ANDOLSI	049e609ce0	add image pull secret + add ingress path (#55 )	2022-03-16 18:04:56 +01:00
Niels ten Boom	fa3739b643	Improvements k8sagent (#54 )	2022-03-01 17:48:33 +01:00
Valeriano Manassero	018348bc1d	Fix image versions (#53 ) * Fixed: image versions * Changed: chart version * Changed: readme update by helm-docs	2022-02-22 11:42:23 +01:00
Valeriano Manassero	57b85cbfce	Update clearml image 1.2.0 (#52 )	2022-02-17 15:33:30 +01:00
Niels ten Boom	9c15a8a348	fix: faulty service values references in k8s agent (#50 ) * add k8s glue deployment * more docs * bump * disabled by default * run helm-docs * fix service references * fix readme * add values file where k8sagent enabled * empty files * newline * fix linter Co-authored-by: Valeriano Manassero <14011549+valeriano-manassero@users.noreply.github.com>	2022-01-21 16:15:09 +01:00
Niels ten Boom	cd7f22f7d8	feat: Add k8s glue agent deployment (#49 )	2022-01-18 23:27:12 +01:00
Shaun Howell	078e394e24	update ingress templates to accept per-service annotation overrides (#48 ) * update ingress yamls to accept annotation overrides * bump version to 3.3.1 * update readme via helm-docs	2022-01-18 18:06:01 +01:00
Valeriano Manassero	70b07c637a	Update Elasticsearch (#47 ) * update elasticsearch * update elasticsearch reference * bump up chart version	2022-01-05 08:26:57 +01:00
Valeriano Manassero	7b8e40c626	Agent foreground mode (#46 ) * use foreground to push output on console * bump up version	2021-12-13 09:04:02 +01:00
Valeriano Manassero	d8117eeb0d	add k8s 1.22.1 to ci procedure (#44 ) After some tests I found 1.22.1 doesn't have ulimit issue so I can include it into the ci process	2021-12-09 11:47:13 +01:00
Valeriano Manassero	4c09ae2c92	Fix env typo (#39 ) * typo fix * bump up version	2021-12-09 11:39:04 +01:00
Valeriano Manassero	478eecd5f2	remove k8s 1.22 from ci (#43 ) It looks 1.22 k8s image from kind has a very low ulimit preventing elastic search from installing, removing it waiting for a fix.	2021-12-09 11:30:15 +01:00
Valeriano Manassero	43f4c44219	test one single kind cluster at time to avoid pressure fails (#42 )	2021-12-09 11:00:57 +01:00
Valeriano Manassero	b83c8cd0e8	indentation fix (#41 )	2021-12-09 10:32:42 +01:00
Valeriano Manassero	97f219228d	update kind k8s versions (#40 )	2021-12-09 10:31:17 +01:00
Valeriano Manassero	1b5b9407f6	Configurable auth cookies age (#38 ) * configurable auth cookies age * version bump up	2021-12-09 08:14:09 +01:00
Valeriano Manassero	b494a8c0cf	External services (#36 ) * use external services switch * bump up version * readme update	2021-11-26 08:11:55 +01:00
Weixiao Huang	266a1e3c41	feat: make service nodePort configurable and add some doc descriptions (#33 ) * feat: make service nodePort configurable * feat: bump version to 3.0.6 * docs: add descriptions for secret and service fields * feat: add comments in clearml-kind.yaml of README.md Co-authored-by: 黄维啸 <huangweixiao@megvii.com>	2021-11-08 14:23:10 +01:00
Weixiao Huang	bba5c0769f	feat: make secret configurable and add secret annotations to deployment (#32 )	2021-11-04 20:36:21 +01:00
Valeriano Manassero	b7f73e3bd9	Switch enabler agentservices (#31 ) * switch to enable/disable agentservices * bump up version	2021-09-21 14:16:30 +02:00
Valeriano Manassero	d3f6f3e50d	Fix helper typo (#30 ) * fix helper typo on api service name * bump up version	2021-09-16 11:21:25 +02:00
Valeriano Manassero	979e73fe3d	Fix ingress compat (#29 ) * fix ingress compatibility with different k8s version * bump up version	2021-09-16 10:54:25 +02:00
Valeriano Manassero	7352f35836	Helpers fix (#28 ) * fix wrong service names * bump up version	2021-09-16 09:11:58 +02:00
Valeriano Manassero	82ad17860d	New ingress style (#27 ) * new ingress style * bump up version * hostName fix * helm-docs update	2021-09-16 08:51:07 +02:00
Valeriano Manassero	aa761dd450	Agent enable switch (#26 ) * enable/disable switch * bump up chart	2021-09-15 08:13:01 +02:00
Valeriano Manassero	7ff2f94d1a	Apiserver configmap (#25 ) * metadata name fix * use toString * use configmap for apiserver configs * bump up version * indentation fix * fix trailing whitespaces	2021-09-14 15:43:10 +02:00