Fix service url generation (#21 )

* service url generation functions * use generation functions * bump up version
Use many ingresses (#20 )
2025-04-17 01:31:13 +00:00 · 2021-08-26 10:58:06 +02:00 · 2021-08-25 14:49:43 +02:00 · 2021-08-17 14:59:13 +02:00 · 2021-08-14 12:00:23 +02:00 · 2021-08-12 12:02:26 +03:00
10 changed files with 152 additions and 41 deletions
--- a/README.md
+++ b/README.md
@@ -1,12 +1,8 @@
 # ClearML Helm Charts Library for Kubernetes

-Helm charts provided by [Allegro AI](https://clear.ml), ready to launch on Kubernetes using [Kubernetes Helm](https://github.com/helm/helm).
-
 ##  Auto-Magical Experiment Manager & Version Control for AI

-[![GitHub license](https://img.shields.io/badge/license-SSPL-green.svg)](https://img.shields.io/badge/license-SSPL-green.svg)
-[![GitHub version](https://img.shields.io/github/release-pre/allegroai/clearml-server.svg)](https://img.shields.io/github/release-pre/allegroai/clearml-server.svg)
-[![PyPI status](https://img.shields.io/badge/status-beta-yellow.svg)](https://img.shields.io/badge/status-beta-yellow.svg)
+Helm charts provided by [Allegro AI](https://clear.ml), ready to launch on Kubernetes using [Kubernetes Helm](https://github.com/helm/helm).

 ## Introduction

@@ -84,4 +80,4 @@ Additionally, you can always find us at *clearml@allegro.ai*
 **PRs are always welcomed** :heart: See more details in the ClearML [Guidelines for Contributing](https://github.com/allegroai/clearml-helm-charts/blob/main/CONTRIBUTING.md).


-_May the force (and the goddess of learning rates) be with you!_
+_May the force (and the goddess of learning rates) be with you!_
--- a/charts/clearml/Chart.yaml
+++ b/charts/clearml/Chart.yaml
@@ -2,8 +2,8 @@ apiVersion: v2
 name: clearml
 description: MLOps platform
 type: application
-version: "2.0.0"
-appVersion: "1.1.0"
+version: "2.2.2"
+appVersion: "1.1.1"
 home: https://clear.ml
 icon: https://raw.githubusercontent.com/allegroai/clearml/master/docs/clearml-logo.svg
 sources:
--- a/charts/clearml/README.md
+++ b/charts/clearml/README.md
@@ -1,6 +1,6 @@
 # ClearML Ecosystem for Kubernetes

-![Version: 2.0.0](https://img.shields.io/badge/Version-2.0.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.1.0](https://img.shields.io/badge/AppVersion-1.1.0-informational?style=flat-square)
+![Version: 2.2.2](https://img.shields.io/badge/Version-2.2.2-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.1.1](https://img.shields.io/badge/AppVersion-1.1.1-informational?style=flat-square)

 MLOps platform

@@ -131,6 +131,7 @@ For detailed instructions, see the [Optional Configuration](https://github.com/a
 | agentGroups.agent-group-cpu.queues | string | `"default"` |  |
 | agentGroups.agent-group-cpu.replicaCount | int | `1` |  |
 | agentGroups.agent-group-cpu.tolerations | list | `[]` |  |
+| agentGroups.agent-group-cpu.updateStrategy | string | `"Recreate"` |  |
 | agentGroups.agent-group-gpu.affinity | object | `{}` |  |
 | agentGroups.agent-group-gpu.agentVersion | string | `""` |  |
 | agentGroups.agent-group-gpu.awsAccessKeyId | string | `nil` |  |
@@ -153,6 +154,7 @@ For detailed instructions, see the [Optional Configuration](https://github.com/a
 | agentGroups.agent-group-gpu.queues | string | `"default"` |  |
 | agentGroups.agent-group-gpu.replicaCount | int | `0` |  |
 | agentGroups.agent-group-gpu.tolerations | list | `[]` |  |
+| agentGroups.agent-group-gpu.updateStrategy | string | `"Recreate"` |  |
 | agentservices.affinity | object | `{}` |  |
 | agentservices.agentVersion | string | `""` |  |
 | agentservices.awsAccessKeyId | string | `nil` |  |
@@ -183,7 +185,7 @@ For detailed instructions, see the [Optional Configuration](https://github.com/a
 | apiserver.extraEnvs | list | `[]` |  |
 | apiserver.image.pullPolicy | string | `"IfNotPresent"` |  |
 | apiserver.image.repository | string | `"allegroai/clearml"` |  |
-| apiserver.image.tag | string | `"1.1.0"` |  |
+| apiserver.image.tag | string | `"1.1.1"` |  |
 | apiserver.livenessDelay | int | `60` |  |
 | apiserver.nodeSelector | object | `{}` |  |
 | apiserver.podAnnotations | object | `{}` |  |
@@ -239,7 +241,7 @@ For detailed instructions, see the [Optional Configuration](https://github.com/a
 | fileserver.extraEnvs | list | `[]` |  |
 | fileserver.image.pullPolicy | string | `"IfNotPresent"` |  |
 | fileserver.image.repository | string | `"allegroai/clearml"` |  |
-| fileserver.image.tag | string | `"1.1.0"` |  |
+| fileserver.image.tag | string | `"1.1.1"` |  |
 | fileserver.nodeSelector | object | `{}` |  |
 | fileserver.podAnnotations | object | `{}` |  |
 | fileserver.replicaCount | int | `1` |  |
@@ -252,6 +254,9 @@ For detailed instructions, see the [Optional Configuration](https://github.com/a
 | ingress.annotations | object | `{}` |  |
 | ingress.enabled | bool | `false` |  |
 | ingress.host | string | `""` |  |
+| ingress.hostPrefixApi | string | `"api."` |  |
+| ingress.hostPrefixApp | string | `"app."` |  |
+| ingress.hostPrefixFiles | string | `"files."` |  |
 | ingress.name | string | `"clearml-server-ingress"` |  |
 | ingress.tls.secretName | string | `""` |  |
 | mongodb.architecture | string | `"standalone"` |  |
@@ -278,7 +283,7 @@ For detailed instructions, see the [Optional Configuration](https://github.com/a
 | webserver.extraEnvs | list | `[]` |  |
 | webserver.image.pullPolicy | string | `"IfNotPresent"` |  |
 | webserver.image.repository | string | `"allegroai/clearml"` |  |
-| webserver.image.tag | string | `"1.1.0"` |  |
+| webserver.image.tag | string | `"1.1.1"` |  |
 | webserver.nodeSelector | object | `{}` |  |
 | webserver.podAnnotations | object | `{}` |  |
 | webserver.replicaCount | int | `1` |  |
--- a/charts/clearml/templates/_helpers.tpl
+++ b/charts/clearml/templates/_helpers.tpl
@@ -95,3 +95,48 @@ Create the name of the service account to use
 {{- default "default" .Values.serviceAccount.name }}
 {{- end }}
 {{- end }}
+
+{{/*
+Create the name of the App service to use
+*/}}
+{{- define "clearml.serviceApp" -}}
+{{- if .Values.ingress.enabled }}
+{{- if .Values.ingress.tls.secretName }}
+{{- printf "%s%s%s" "https://" .Values.ingress.hostPrefixApp .Values.ingress.host }}
+{{- else }}
+{{- printf "%s%s%s" "http://" .Values.ingress.hostPrefixApp .Values.ingress.host }}
+{{- end }}
+{{- else }}
+{{- printf "%s%s%s%s" "http://" (include "clearml.fullname" .) "-webserver:" (.Values.webserver.service.port | quote) }}
+{{- end }}
+{{- end }}
+
+{{/*
+Create the name of the Api service to use
+*/}}
+{{- define "clearml.serviceApi" -}}
+{{- if .Values.ingress.enabled }}
+{{- if .Values.ingress.tls.secretName }}
+{{- printf "%s%s%s" "https://" .Values.ingress.hostPrefixApi .Values.ingress.host }}
+{{- else }}
+{{- printf "%s%s%s" "http://" .Values.ingress.hostPrefixApi .Values.ingress.host }}
+{{- end }}
+{{- else }}
+{{- printf "%s%s%s%s" "http://" (include "clearml.fullname" .) "-apiserver:" (.Values.apiserver.service.port | quote) }}
+{{- end }}
+{{- end }}
+
+{{/*
+Create the name of the Files service to use
+*/}}
+{{- define "clearml.serviceFiles" -}}
+{{- if .Values.ingress.enabled }}
+{{- if .Values.ingress.tls.secretName }}
+{{- printf "%s%s%s" "https://" .Values.ingress.hostPrefixFiles .Values.ingress.host }}
+{{- else }}
+{{- printf "%s%s%s" "http://" .Values.ingress.hostPrefixFiles .Values.ingress.host }}
+{{- end }}
+{{- else }}
+{{- printf "%s%s%s%s" "http://" (include "clearml.fullname" .) "-fileserver:" (.Values.fileserver.service.port | quote) }}
+{{- end }}
+{{- end }}
--- a/charts/clearml/templates/deployment-agent.yaml
+++ b/charts/clearml/templates/deployment-agent.yaml
@@ -9,6 +9,8 @@ metadata:
    {{- include "clearml.labels" $ | nindent 4 }}
 spec:
  replicas: {{ .replicaCount }}
+  strategy:
+    type: {{ .updateStrategy }}
  selector:
    matchLabels:
      {{- include "clearml.selectorLabelsAgent" $ | nindent 6 }}
@@ -38,7 +40,7 @@ spec:
            - -c
            - >
              set -x;
-              while [ $(curl -sw '%{http_code}' "http://{{ include "clearml.fullname" $ }}-apiserver:{{ $.Values.apiserver.service.port }}/debug.ping" -o /dev/null) -ne 200 ] ; do
+              while [ $(curl -sw '%{http_code}' "{{ include "clearml.serviceApi" $ }}/debug.ping" -o /dev/null) -ne 200 ] ; do
                echo "waiting for apiserver" ;
                sleep 5 ;
              done
@@ -54,11 +56,11 @@ spec:
                {{ .nvidiaGpusPerAgent }}
          env:
          - name: CLEARML_API_HOST
-            value: 'http://{{ include "clearml.fullname" $ }}-apiserver:{{ $.Values.apiserver.service.port }}'
+            value: {{ include "clearml.serviceApi" $ }}
          - name: CLEARML_WEB_HOST
-            value: 'http://{{ include "clearml.fullname" $ }}-webserver:{{ $.Values.webserver.service.port }}'
+            value: {{ include "clearml.serviceApp" $ }}
          - name: CLEARML_FILES_HOST
-            value: 'http://{{ include "clearml.fullname" $ }}-fileserver:{{ $.Values.fileserver.service.port }}'
+            value: {{ include "clearml.serviceFiles" $ }}
          - name: CLEARML_AGENT_GIT_USER
            value: {{ .clearmlGitUser}}
          - name: CLEARML_AGENT_GIT_PASS
@@ -91,6 +93,13 @@ spec:
              python3 -m pip install -U pip ;
              python3 -m pip install clearml-agent{{ .agentVersion}} ;
              CLEARML_AGENT_K8S_HOST_MOUNT=/root/.clearml:/root/.clearml clearml-agent daemon --queue {{ .queues}}"
+          {{ if .clearmlConfig }}
+          volumeMounts:
+           - name: agent-clearml-conf-volume
+             mountPath: /root/clearml.conf
+             subPath: clearml.conf
+             readOnly: true
+          {{- end }}
      {{- with .nodeSelector }}
      nodeSelector:
        {{- toYaml . | nindent 8 }}
--- a/charts/clearml/templates/deployment-agentservices.yaml
+++ b/charts/clearml/templates/deployment-agentservices.yaml
@@ -30,7 +30,7 @@ spec:
              - -c
              - >
                set -x;
-                while [ $(curl -sw '%{http_code}' "http://{{ include "clearml.fullname" . }}-apiserver:{{ .Values.apiserver.service.port }}/debug.ping" -o /dev/null) -ne 200 ] ; do
+                while [ $(curl -sw '%{http_code}' "{{ include "clearml.serviceApi" $ }}/debug.ping" -o /dev/null) -ne 200 ] ; do
                  echo "waiting for apiserver" ;
                  sleep 5 ;
                done
@@ -42,7 +42,7 @@ spec:
          - name: CLEARML_HOST_IP
            value: {{ .Values.agentservices.clearmlHostIp }}
          - name: CLEARML_API_HOST
-            value: "http://{{ include "clearml.fullname" . }}-apiserver:{{ .Values.apiserver.service.port }}"
+            value: {{ include "clearml.serviceApi" $ }}
          - name: CLEARML_WEB_HOST
            value: {{ .Values.agentservices.clearmlWebHost }}
          - name: CLEARML_FILES_HOST
--- a/charts/clearml/templates/ingress-api.yaml
+++ b/charts/clearml/templates/ingress-api.yaml
@@ -0,0 +1,33 @@
+{{- if .Values.ingress.enabled -}}
+{{- $fullName := include "clearml.fullname" . -}}
+{{- if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
+apiVersion: networking.k8s.io/v1beta1
+{{- else -}}
+apiVersion: extensions/v1beta1
+{{- end }}
+kind: Ingress
+metadata:
+  name: {{ $fullName }}-api
+  labels:
+    {{- include "clearml.labels" . | nindent 4 }}
+  {{- with .Values.ingress.annotations }}
+  annotations:
+    {{- toYaml . | nindent 4 }}
+  {{- end }}
+spec:
+  {{- if .Values.ingress.tls.secretName }}
+  tls:
+    - hosts:
+        - "{{ .Values.ingress.hostPrefixApi }}{{ .Values.ingress.host }}"
+      secretName: {{ .Values.ingress.tls.secretName }}
+  {{- end }}
+  rules:
+    - host: "{{ .Values.ingress.hostPrefixApi }}{{ .Values.ingress.host }}"
+      http:
+        paths:
+          - path: "/"
+            pathType: Prefix
+            backend:
+              serviceName: {{ include "clearml.fullname" . }}-apiserver
+              servicePort: {{ .Values.apiserver.service.port }}
+{{- end }}
--- a/charts/clearml/templates/ingress-app.yaml
+++ b/charts/clearml/templates/ingress-app.yaml
@@ -0,0 +1,33 @@
+{{- if .Values.ingress.enabled -}}
+{{- $fullName := include "clearml.fullname" . -}}
+{{- if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
+apiVersion: networking.k8s.io/v1beta1
+{{- else -}}
+apiVersion: extensions/v1beta1
+{{- end }}
+kind: Ingress
+metadata:
+  name: {{ $fullName }}-app
+  labels:
+    {{- include "clearml.labels" . | nindent 4 }}
+  {{- with .Values.ingress.annotations }}
+  annotations:
+    {{- toYaml . | nindent 4 }}
+  {{- end }}
+spec:
+  {{- if .Values.ingress.tls.secretName }}
+  tls:
+    - hosts:
+        - "{{ .Values.ingress.hostPrefixApp }}{{ .Values.ingress.host }}"
+      secretName: {{ .Values.ingress.tls.secretName }}
+  {{- end }}
+  rules:
+    - host: "{{ .Values.ingress.hostPrefixApp }}{{ .Values.ingress.host }}"
+      http:
+        paths:
+          - path: "/"
+            pathType: Prefix
+            backend:
+              serviceName: {{ include "clearml.fullname" . }}-webserver
+              servicePort: {{ .Values.webserver.service.port }}
+{{- end }}
--- a/charts/clearml/templates/ingress-files.yaml
+++ b/charts/clearml/templates/ingress-files.yaml
@@ -7,7 +7,7 @@ apiVersion: extensions/v1beta1
 {{- end }}
 kind: Ingress
 metadata:
-  name: {{ $fullName }}
+  name: {{ $fullName }}-files
  labels:
    {{- include "clearml.labels" . | nindent 4 }}
  {{- with .Values.ingress.annotations }}
@@ -18,30 +18,15 @@ spec:
  {{- if .Values.ingress.tls.secretName }}
  tls:
    - hosts:
-        - "app.{{ .Values.ingress.host }}"
-        - "files.{{ .Values.ingress.host }}"
-        - "api.{{ .Values.ingress.host }}"
+        - "{{ .Values.ingress.hostPrefixFiles }}{{ .Values.ingress.host }}"
      secretName: {{ .Values.ingress.tls.secretName }}
  {{- end }}
  rules:
-    - host: "app.{{ .Values.ingress.host }}"
+    - host: "{{ .Values.ingress.hostPrefixFiles }}{{ .Values.ingress.host }}"
      http:
        paths:
-          - path: "/*"
-            backend:
-              serviceName: {{ include "clearml.fullname" . }}-webserver
-              servicePort: {{ .Values.webserver.service.port }}
-    - host: "api.{{ .Values.ingress.host }}"
-      http:
-        paths:
-          - path: "/*"
-            backend:
-              serviceName: {{ include "clearml.fullname" . }}-apiserver
-              servicePort: {{ .Values.apiserver.service.port }}
-    - host: "files.{{ .Values.ingress.host }}"
-      http:
-        paths:
-          - path: "/*"
+          - path: "/"
+            pathType: Prefix
            backend:
              serviceName: {{ include "clearml.fullname" . }}-fileserver
              servicePort: {{ .Values.fileserver.service.port }}
--- a/charts/clearml/values.yaml
+++ b/charts/clearml/values.yaml
@@ -5,6 +5,9 @@ ingress:
  name: clearml-server-ingress
  annotations: {}
  host: ""
+  hostPrefixApp: "app."
+  hostPrefixApi: "api."
+  hostPrefixFiles: "files."
  tls:
    secretName: ""

@@ -26,7 +29,7 @@ apiserver:
  image:
    repository: "allegroai/clearml"
    pullPolicy: IfNotPresent
-    tag: "1.1.0"
+    tag: "1.1.1"

  extraEnvs: []

@@ -67,7 +70,7 @@ fileserver:
  image:
    repository: "allegroai/clearml"
    pullPolicy: IfNotPresent
-    tag: "1.1.0"
+    tag: "1.1.1"

  extraEnvs: []

@@ -108,7 +111,7 @@ webserver:
  image:
    repository: "allegroai/clearml"
    pullPolicy: IfNotPresent
-    tag: "1.1.0"
+    tag: "1.1.1"

  podAnnotations: {}

@@ -183,6 +186,7 @@ agentGroups:
  agent-group-cpu:
    name: agent-group-cpu
    replicaCount: 1
+    updateStrategy: Recreate
    nvidiaGpusPerAgent: 0
    agentVersion: ""  # if set, it *MUST* include comparison operator (e.g. ">=0.16.1")
    queues: "default"  # multiple queues can be specified separated by a space (e.g. "important_jobs default")
@@ -215,6 +219,7 @@ agentGroups:
  agent-group-gpu:
    name: agent-group-gpu
    replicaCount: 0
+    updateStrategy: Recreate
    nvidiaGpusPerAgent: 1
    agentVersion: ""  # if set, it *MUST* include comparison operator (e.g. ">=0.16.1")
    queues: "default"  # multiple queues can be specified separated by a space (e.g. "important_jobs default")
Author	SHA1	Message	Date
Valeriano Manassero	618a269c97	Fix service url generation (#21 ) * service url generation functions * use generation functions * bump up version	2021-08-26 10:58:06 +02:00
Valeriano Manassero	3f215d2d90	Use many ingresses (#20 ) * use many ingresses * bump up version	2021-08-25 14:49:43 +02:00
Valeriano Manassero	03223fc1c1	Use Recreate as strategy (#19 ) * use Recreate as strategybump up version * fix strategy indentation and position * updatesStrategy configurable * updateStrategy parameter * use 2.2.0 instead of patch release	2021-08-17 14:59:13 +02:00
Valeriano Manassero	898089b7fb	Fix agent clearml conf (#18 ) * fix agent mount clearml.conf * bump up version	2021-08-14 12:00:23 +02:00
Valeriano Manassero	732bb970aa	Configurable prefix ingress (#17 ) * ingress configurable prefixes * chart version bump up * fix version number	2021-08-12 12:02:26 +03:00
Valeriano Manassero	91d45281fa	bump up app version to 1.1.1 (#16 )	2021-08-06 14:18:53 +02:00
Valeriano Manassero	28b6e9f4e4	Remove badges from root README since we have them in chart one (#14 )	2021-07-27 16:18:02 +02:00