Merge branch 'main' into add-job-support

.github/workflows/inactive-issues.yaml

@@ -0,0 +1,22 @@
+name: Close inactive issues
+on:
+  schedule:
+    - cron: "30 1 * * *"
+
+jobs:
+  close-issues:
+    runs-on: ubuntu-latest
+    permissions:
+      issues: write
+      pull-requests: write
+    steps:
+      - uses: actions/stale@v7
+        with:
+          days-before-issue-stale: 14
+          days-before-issue-close: 7
+          stale-issue-label: "stale"
+          stale-issue-message: "This issue is stale because it has been open for 14 days with no activity."
+          close-issue-message: "This issue was closed because it has been inactive for 7 days since being marked as stale."
+          days-before-pr-stale: -1
+          days-before-pr-close: -1
+          repo-token: ${{ secrets.GITHUB_TOKEN }}

INSTALL.md

@@ -0,0 +1,57 @@
+# ClearML Helm Charts Installation guide
+
+## Requirements
+
+### Setup a Kubernetes Cluster
+
+For setting up Kubernetes on various platforms refer to the Kubernetes [getting started guide](http://kubernetes.io/docs/getting-started-guides/).
+
+#### Setup a single node LOCAL Kubernetes on laptop/desktop (development)
+
+For setting up Kubernetes on your laptop/desktop we suggest [kind](https://kind.sigs.k8s.io).
+
+#### [Kubernetes Tanzu users only] Additional setup requirements
+
+For setting up Clear.ML on a Tanzu cluster, check [prerequisites](https://github.com/allegroai/clearml-helm-charts/tree/main/platform-specific-configs/tanzu).
+
+#### [Kubernetes Openshift users only] Additional setup requirements
+
+For setting up Clear.ML on a Openshift cluster, check [prerequisites](https://github.com/allegroai/clearml-helm-charts/tree/main/platform-specific-configs/openshift).
+
+### Install Helm
+
+Helm is a tool for managing Kubernetes charts. Charts are packages of pre-configured Kubernetes resources.
+
+To install Helm, refer to the [Helm install guide](https://github.com/helm/helm#install) and ensure that the `helm` binary is in the `PATH` of your shell.
+
+## Helm charts installation
+
+### Helm Repo
+
+```bash
+$ helm repo add allegroai https://allegroai.github.io/clearml-helm-charts
+$ helm repo update
+```
+### ClearML server ecosystem
+
+```bash
+$ helm install clearml allegroai/clearml
+```
+
+### ClearML agent
+
+Agent is always related a ClearML server ecosystem (by default using `app.clear.ml` public service but can be on same or another Kubernetes cluster or a single server installation).
+
+On ClearML UI, Settings -> Workspace and Create new Credentials.
+
+In following Helm chart install command:
+
+* set ACCESSKEY to resuted credentials access_key
+* set SECRETKEY to resuted credentials secret_key
+* set APIERVERURL to resuted credentials api_server
+* set FILESSERVERURL to resuted credentials files_server
+* set WEBSERVERURL to resuted credentials web_server
+
+```bash
+$ helm install clearml-agent allegroai/clearml-agent --set clearml.agentk8sglueKey=ACCESSKEY --set clearml.agentk8sglueSecret=SECRETKEY --set agentk8sglue.apiServerUrlReference=APISERVERURL --set agentk8sglue.fileServerUrlReference=FILESERVERURL --set agentk8sglue.webServerUrlReference=WEBSERVERURL
+```

README.md

@@ -1,4 +1,4 @@
-# ClearML Helm Charts Library for Kubernetes
+# ClearML Helm Charts for Kubernetes
 
 ##  Auto-Magical Experiment Manager & Version Control for AI
 
@@ -23,7 +23,11 @@ Use this repository to deploy **clearml-server** on Kubernetes clusters.
 
 ## Provided in this repository
 
-### [All around Helm Chart](https://github.com/allegroai/clearml-helm-charts/tree/main/charts/clearml)
+### [ClearML server chart](https://github.com/allegroai/clearml-helm-charts/tree/main/charts/clearml)
+
+### [ClearML agent chart](https://github.com/allegroai/clearml-helm-charts/tree/main/charts/clearml-agent)
+
+### [ClearML serving chart](https://github.com/allegroai/clearml-helm-charts/tree/main/charts/clearml-serving)
 
 ## Who We Are
 
@@ -40,30 +44,9 @@ will always upgrade with you.
 
 Apache License, Version 2.0, (see the [LICENSE](https://www.apache.org/licenses/LICENSE-2.0) for more information)
 
-## Requirements
+## Installation guide
 
-### Setup a Kubernetes Cluster
-
-For setting up Kubernetes on various platforms refer to the Kubernetes [getting started guide](http://kubernetes.io/docs/getting-started-guides/).
-
-### Setup a single node LOCAL Kubernetes on laptop/desktop
-
-For setting up Kubernetes on your laptop/desktop we suggest [kind](https://kind.sigs.k8s.io).
-
-### Install Helm
-
-Helm is a tool for managing Kubernetes charts. Charts are packages of pre-configured Kubernetes resources.
-
-To install Helm, refer to the [Helm install guide](https://github.com/helm/helm#install) and ensure that the `helm` binary is in the `PATH` of your shell.
-
-## Usage
-
-```bash
-$ helm repo add allegroai https://allegroai.github.io/clearml-helm-charts
-$ helm repo update
-$ helm search repo allegroai
-$ helm install <release-name> allegroai/<chart>
-```
+For installation instruction, follow related [Installation Guide](INSTALL.md).
 
 ## Documentation, Community & Support
 

charts/clearml-agent/templates/_helpers.tpl

@@ -1,23 +1,23 @@
 {{/*
 Expand the name of the chart.
 */}}
-{{- define "clearml.name" -}}
+{{- define "clearmlAgent.name" -}}
 {{- .Release.Name | trunc 59 | trimSuffix "-" }}
 {{- end }}
 
 {{/*
 Create chart name and version as used by the chart label.
 */}}
-{{- define "clearml.chart" -}}
+{{- define "clearmlAgent.chart" -}}
 {{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 59 | trimSuffix "-" }}
 {{- end }}
 
 {{/*
 Common labels
 */}}
-{{- define "clearml.labels" -}}
-helm.sh/chart: {{ include "clearml.chart" . }}
-{{ include "clearml.selectorLabels" . }}
+{{- define "clearmlAgent.labels" -}}
+helm.sh/chart: {{ include "clearmlAgent.chart" . }}
+{{ include "clearmlAgent.selectorLabels" . }}
 {{- if .Chart.AppVersion }}
 app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
 {{- end }}
@@ -30,7 +30,7 @@ app.kubernetes.io/managed-by: {{ .Release.Service }}
 {{/*
 Common annotations
 */}}
-{{- define "clearml.annotations" -}}
+{{- define "clearmlAgent.annotations" -}}
 {{- if $.Values.agentk8sglue.annotations }}
 {{ toYaml $.Values.agentk8sglue.annotations }}
 {{- end }}
@@ -39,8 +39,8 @@ Common annotations
 {{/*
 Selector labels
 */}}
-{{- define "clearml.selectorLabels" -}}
-app.kubernetes.io/name: {{ include "clearml.name" . }}
+{{- define "clearmlAgent.selectorLabels" -}}
+app.kubernetes.io/name: {{ include "clearmlAgent.name" . }}
 app.kubernetes.io/instance: {{ .Release.Name }}
 {{- end }}
 
@@ -48,18 +48,18 @@ app.kubernetes.io/instance: {{ .Release.Name }}
 Selector labels (agentk8sglue)
 */}}
 {{- define "agentk8sglue.selectorLabels" -}}
-app.kubernetes.io/name: {{ include "clearml.name" . }}
-app.kubernetes.io/instance: {{ include "clearml.name" . }}
+app.kubernetes.io/name: {{ include "clearmlAgent.name" . }}
+app.kubernetes.io/instance: {{ include "clearmlAgent.name" . }}
 {{- end }}
 
 {{/*
 Create the name of the service account to use
 */}}
-{{- define "clearml.serviceAccountName" -}}
+{{- define "clearmlAgent.serviceAccountName" -}}
 {{- if .Values.agentk8sglue.serviceExistingAccountName }}
 {{- .Values.agentk8sglue.serviceExistingAccountName }}
 {{- else }}
-{{- include "clearml.name" . }}-sa
+{{- include "clearmlAgent.name" . }}-sa
 {{- end }}
 {{- end }}
 

charts/clearml-agent/templates/agentk8sglue-configmap.yaml

@@ -1,7 +1,7 @@
 apiVersion: v1
 kind: ConfigMap
 metadata:
-  name: {{ include "clearml.name" . }}-pt
+  name: {{ include "clearmlAgent.name" . }}-pt
 data:
 {{- if .Values.enterpriseFeatures.enabled }}
   template.yaml: |
@@ -14,9 +14,9 @@ data:
     {{- range $key, $value := $.Values.enterpriseFeatures.queues }}
     {{ $key }}:
       {{- if $value.templateOverrides.fileMounts }}
-        - {{ include "clearml.name" $ }}-{{ $key }}-fm
+        - {{ include "clearmlAgent.name" $ }}-{{ $key }}-fm
       {{- else if $.Values.agentk8sglue.basePodTemplate.fileMounts }}
-        - {{ include "clearml.name" $ }}-fm
+        - {{ include "clearmlAgent.name" $ }}-fm
       {{- end }}
     {{- end }}
 {{- else }}
@@ -34,14 +34,14 @@ data:
       {{- if .Values.imageCredentials.existingSecret }}
       - name: {{ .Values.imageCredentials.existingSecret }}
       {{- else }}
-      - name: name: {{ include "clearml.name" $ }}-ark
+      - name: name: {{ include "clearmlAgent.name" $ }}-ark
       {{- end }}
       {{- end }}
       {{- with .Values.agentk8sglue.basePodTemplate.volumes }}
       volumes:
       {{- toYaml . | nindent 8 }}
       {{- end }}
-      serviceAccountName: {{ include "clearml.serviceAccountName" $ }}
+      serviceAccountName: {{ include "clearmlAgent.serviceAccountName" $ }}
       containers:
       - resources:
           {{- toYaml .Values.agentk8sglue.basePodTemplate.resources | nindent 10 }}
@@ -64,7 +64,7 @@ data:
               {{- if .Values.clearml.existingAgentk8sglueSecret }}
               name: {{ .Values.clearml.existingAgentk8sglueSecret }}
               {{- else }}
-              name: {{ include "clearml.name" . }}-ac
+              name: {{ include "clearmlAgent.name" . }}-ac
               {{- end }}
               key: agentk8sglue_key
         - name: CLEARML_API_SECRET_KEY
@@ -73,7 +73,7 @@ data:
               {{- if .Values.clearml.existingAgentk8sglueSecret }}
               name: {{ .Values.clearml.existingAgentk8sglueSecret }}
               {{- else }}
-              name: {{ include "clearml.name" . }}-ac
+              name: {{ include "clearmlAgent.name" . }}-ac
               {{- end }}
               key: agentk8sglue_secret
         {{- if .Values.agentk8sglue.basePodTemplate.env }}
@@ -100,7 +100,7 @@ data:
     metadata:
       name: clearml-session-{{ . }}
       labels:
-        {{- include "clearml.labels" $ | nindent 8 }}
+        {{- include "clearmlAgent.labels" $ | nindent 8 }}
       {{- with $.Values.sessions.svcAnnotations }}
       annotations:
         {{- toYaml . | nindent 8 }}

charts/clearml-agent/templates/agentk8sglue-deployment.yaml

@@ -1,11 +1,11 @@
 apiVersion: apps/v1
 kind: Deployment
 metadata:
-  name: {{ include "clearml.name" . }}
+  name: {{ include "clearmlAgent.name" . }}
   labels:
-    {{- include "clearml.labels" . | nindent 4 }}
+    {{- include "clearmlAgent.labels" . | nindent 4 }}
   annotations:
-    {{- include "clearml.annotations" . | nindent 4 }}
+    {{- include "clearmlAgent.annotations" . | nindent 4 }}
 spec:
   replicas: {{ .Values.agentk8sglue.replicaCount }}
   selector:
@@ -15,19 +15,19 @@ spec:
     metadata:
       annotations:
         checksum/config: {{ printf "%s%s" .Values.clearml .Values.agentk8sglue | sha256sum }}
-        {{- include "clearml.annotations" . | nindent 8 }}
+        {{- include "clearmlAgent.annotations" . | nindent 8 }}
       labels:
-        {{- include "clearml.labels" . | nindent 8 }}
+        {{- include "clearmlAgent.labels" . | nindent 8 }}
     spec:
       {{- if .Values.imageCredentials.enabled }}
       imagePullSecrets:
       {{- if .Values.imageCredentials.existingSecret }}
       - name: {{ .Values.imageCredentials.existingSecret }}
       {{- else }}
-      - name: {{ include "clearml.name" . }}-ark
+      - name: {{ include "clearmlAgent.name" . }}-ark
       {{- end }}
       {{- end }}
-      serviceAccountName: {{ include "clearml.serviceAccountName" . }}
+      serviceAccountName: {{ include "clearmlAgent.serviceAccountName" . }}
       securityContext: {{ toYaml .Values.agentk8sglue.securityContext | nindent 8 }}
       initContainers:
       - name: init-k8s-glue
@@ -68,7 +68,7 @@ spec:
             export PATH=$PATH:$HOME/bin;
             source /root/.bashrc && /root/entrypoint.sh
         volumeMounts:
-          - name: {{ include "clearml.name" . }}-pt
+          - name: {{ include "clearmlAgent.name" . }}-pt
             mountPath: /root/template
           {{ if .Values.clearml.clearmlConfig }}
           - name: k8sagent-clearml-conf-volume
@@ -122,15 +122,15 @@ spec:
           - name: CLEARML_API_ACCESS_KEY
             valueFrom:
               secretKeyRef:
-                name: {{ include "clearml.name" . }}-ac
+                name: {{ include "clearmlAgent.name" . }}-ac
                 key: agentk8sglue_key
           - name: CLEARML_API_SECRET_KEY
             valueFrom:
               secretKeyRef:
-                name: {{ include "clearml.name" . }}-ac
+                name: {{ include "clearmlAgent.name" . }}-ac
                 key: agentk8sglue_secret
           - name: CLEARML_WORKER_ID
-            value: {{ include "clearml.name" . }}
+            value: {{ include "clearmlAgent.name" . }}
           - name: CLEARML_AGENT_UPDATE_REPO
             value: ""
           - name: FORCE_CLEARML_AGENT_REPO
@@ -193,13 +193,13 @@ spec:
         {{- toYaml . | nindent 8 }}
       {{- end }}
       volumes:
-        - name: {{ include "clearml.name" . }}-pt
+        - name: {{ include "clearmlAgent.name" . }}-pt
           configMap:
-            name: {{ include "clearml.name" . }}-pt
+            name: {{ include "clearmlAgent.name" . }}-pt
         {{ if .Values.clearml.clearmlConfig }}
         - name: k8sagent-clearml-conf-volume
           secret:
-            secretName: {{ include "clearml.name" . }}-ac
+            secretName: {{ include "clearmlAgent.name" . }}-ac
             items:
             - key: clearml.conf
               path: clearml.conf
@@ -210,5 +210,5 @@ spec:
         {{ if .Values.agentk8sglue.fileMounts }}
         - name: filemounts
           secret:
-            secretName: {{ include "clearml.name" . }}-afm
+            secretName: {{ include "clearmlAgent.name" . }}-afm
         {{- end }}

charts/clearml-agent/templates/agentk8sglue-rbac.yaml

@@ -2,7 +2,7 @@
 apiVersion: v1
 kind: ServiceAccount
 metadata:
-  name: {{ include "clearml.serviceAccountName" . }}
+  name: {{ include "clearmlAgent.serviceAccountName" . }}
   namespace: {{ .Release.Namespace }}
 {{- end }}
 {{- if .Values.enterpriseFeatures.serviceAccountClusterAccess }}
@@ -10,7 +10,7 @@ metadata:
 apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRole
 metadata:
-  name: {{ include "clearml.name" . }}-kpa
+  name: {{ include "clearmlAgent.name" . }}-kpa
 rules:
   - apiGroups:
       - ""
@@ -36,21 +36,21 @@ rules:
 apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRoleBinding
 metadata:
-  name: {{ include "clearml.name" . }}-kpa
+  name: {{ include "clearmlAgent.name" . }}-kpa
 subjects:
   - kind: ServiceAccount
-    name: {{ include "clearml.serviceAccountName" . }}
+    name: {{ include "clearmlAgent.serviceAccountName" . }}
     namespace: {{ .Release.Namespace }}
 roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
-  name: {{ include "clearml.name" . }}-kpa
+  name: {{ include "clearmlAgent.name" . }}-kpa
 {{- else }}
 ---
 apiVersion: rbac.authorization.k8s.io/v1
 kind: Role
 metadata:
-  name: {{ include "clearml.name" . }}-kpa
+  name: {{ include "clearmlAgent.name" . }}-kpa
 rules:
   - apiGroups:
       - ""
@@ -76,13 +76,13 @@ rules:
 apiVersion: rbac.authorization.k8s.io/v1
 kind: RoleBinding
 metadata:
-  name: {{ include "clearml.name" . }}-kpa
+  name: {{ include "clearmlAgent.name" . }}-kpa
 subjects:
   - kind: ServiceAccount
-    name: {{ include "clearml.serviceAccountName" . }}
+    name: {{ include "clearmlAgent.serviceAccountName" . }}
     namespace: {{ .Release.Namespace }}
 roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: Role
-  name: {{ include "clearml.name" . }}-kpa
+  name: {{ include "clearmlAgent.name" . }}-kpa
 {{- end }}

charts/clearml-agent/templates/clearml-secrets.yaml

@@ -1,7 +1,7 @@
 apiVersion: v1
 kind: Secret
 metadata:
-  name: {{ include "clearml.name" . }}-ac
+  name: {{ include "clearmlAgent.name" . }}-ac
 data:
   agentk8sglue_key: {{ .Values.clearml.agentk8sglueKey | b64enc }}
   agentk8sglue_secret: {{ .Values.clearml.agentk8sglueSecret | b64enc }}
@@ -12,7 +12,7 @@ data:
 apiVersion: v1
 kind: Secret
 metadata:
-  name: {{ include "clearml.name" . }}-ark
+  name: {{ include "clearmlAgent.name" . }}-ark
 type: kubernetes.io/dockerconfigjson
 data:
   .dockerconfigjson: {{ template "imagePullSecret" . }}

charts/clearml-agent/templates/service-secret.yaml

@@ -2,7 +2,7 @@
 apiVersion: v1
 kind: Secret
 metadata:
-  name: {{ include "clearml.name" . }}-afm
+  name: {{ include "clearmlAgent.name" . }}-afm
 data:
   {{- range .Values.agentk8sglue.fileMounts }}
   {{ .name }}: {{ .fileContent | b64enc }}
@@ -14,7 +14,7 @@ data:
 apiVersion: v1
 kind: Secret
 metadata:
-  name: {{ include "clearml.name" . }}-fm
+  name: {{ include "clearmlAgent.name" . }}-fm
 data:
   {{- range .Values.agentk8sglue.basePodTemplate.fileMounts }}
   {{ .name }}: {{ .fileContent | b64enc }}
@@ -26,7 +26,7 @@ data:
 apiVersion: v1
 kind: Secret
 metadata:
-  name: {{ include "clearml.name" $ }}-{{ $key }}-fm
+  name: {{ include "clearmlAgent.name" $ }}-{{ $key }}-fm
 data:
   {{- range .templateOverrides.fileMounts }}
   {{ .name }}: {{ .fileContent | b64enc }}

charts/clearml-agent/templates/service-sessions.yaml

@@ -7,7 +7,7 @@ kind: Service
 metadata:
   name: clearml-session-{{ . }}
   labels:
-    {{- include "clearml.labels" $ | nindent 4 }}
+    {{- include "clearmlAgent.labels" $ | nindent 4 }}
   {{- with $.Values.sessions.svcAnnotations }}
   annotations:
     {{- toYaml . | nindent 4 }}

charts/clearml/Chart.yaml

@@ -2,7 +2,7 @@ apiVersion: v2
 name: clearml
 description: MLOps platform
 type: application
-version: "5.6.0"
+version: "5.6.1"
 appVersion: "1.9.2"
 kubeVersion: ">= 1.21.0-0 < 1.27.0-0"
 home: https://clear.ml
@@ -32,5 +32,5 @@ dependencies:
     condition: elasticsearch.enabled
 annotations:
   artifacthub.io/changes: |
-    - kind: added
-      description: multi host external elasticsearch support
+    - kind: fixed
+      description: custom cookieName always available

charts/clearml/README.md

@@ -1,6 +1,6 @@
 # ClearML Ecosystem for Kubernetes
 
-![Version: 5.6.0](https://img.shields.io/badge/Version-5.6.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.9.2](https://img.shields.io/badge/AppVersion-1.9.2-informational?style=flat-square)
+![Version: 5.6.1](https://img.shields.io/badge/Version-5.6.1-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.9.2](https://img.shields.io/badge/AppVersion-1.9.2-informational?style=flat-square)
 
 MLOps platform
 

charts/clearml/templates/apiserver-deployment.yaml

@@ -110,9 +110,9 @@ spec:
             value: /opt/clearml/config
           - name: CLEARML__apiserver__default_company_name
             value: "{{ .Values.clearml.defaultCompany }}"
-          {{- if not (eq .Values.clearml.cookieDomain "") }}
           - name: CLEARML__APISERVER__AUTH__SESSION_AUTH_COOKIE_NAME
             value: {{ .Values.clearml.cookieName }}
+          {{- if .Values.clearml.cookieDomain }}
           - name: CLEARML__APISERVER__AUTH__COOKIES__DOMAIN
             value: ".{{ .Values.clearml.cookieDomain }}"
           {{- end }}

charts/clearml/values-production.yaml

@@ -16,6 +16,28 @@ webserver:
   ingress:
     enabled: true
     hostName: "app.clearml.127-0-0-1.nip.io"
+redis:
+  master:
+    name: "{{ .Release.Name }}-redis"
+    persistence:
+      enabled: true
+      accessModes:
+        - ReadWriteOnce
+      size: 5Gi
+      ## If undefined (the default) or set to null, no storageClassName spec is set, choosing the default provisioner
+      storageClass: null
+  slave:
+    persistence:
+      enabled: true
+      accessModes:
+        - ReadWriteOnce
+      size: 5Gi
+      ## If undefined (the default) or set to null, no storageClassName spec is set, choosing the default provisioner
+      storageClass: null
+  cluster:
+    enabled: true
+  sentinel:
+    enabled: true
 mongodb:
   enabled: true
   architecture: replicaset