Compare commits

...

36 Commits

Author SHA1 Message Date
Valeriano Manassero
08b92ba622 Fix apps baseimage (#180)
* Fixed: apps base image

* Changed: bump up version
2023-03-09 08:19:30 +01:00
Valeriano Manassero
5b77cf41c2 Add external clusterrolebinding and rolebinding support (#179)
* Added: external rb and crb support

* Changed: bump up version
2023-03-07 13:09:30 +01:00
Valeriano Manassero
a6db8b4262 Fix init container waits forever pinging a mongodb in production config (#178)
* Fixed: hostname healthcheck for mongodb

* Changed: bump up version
2023-03-07 08:19:22 +01:00
Valeriano Manassero
dd4d8bf086 Filemount apps agent (#176)
* Added: filemounts support for apps agent

* Changed: bump up version
2023-03-06 14:23:50 +01:00
Valeriano Manassero
bf959d2f70 Update apps agent (#175)
* Changed: apps agent version bump

* Changed: chart version bump
2023-03-03 14:09:03 +01:00
Valeriano Manassero
340d261f11 Fixed: openshift examples (#172) 2023-02-20 14:50:48 +01:00
Valeriano Manassero
e1fcc5b466 Enterprise create queue (#171)
* Fixed: typo in env example

* Added: create queues switch

* Added: force configuration file mount

* Changed: bump version

* Fixed: helm docs
2023-02-20 13:52:36 +01:00
pollfly
013734c184 edits (#168) 2023-02-16 13:25:00 +01:00
Valeriano Manassero
fded7aa5b4 165 clearml agent priorityclassname in pod template (#166)
* Added: priorityclass name

* Changed: bump up version
2023-02-16 09:39:23 +01:00
Valeriano Manassero
5540188db1 Add job support for task pod (#162)
* Added: task as job support

* Added: template generator

* Fixed: typo

* Changed: bump version

* Added: changelog reference

* Fixed: include function name

* Fixed: checksum generator

* Added: nindent

* Added: changelog item

* Fixed: job env var switch

* Fixed: double Restart policy removed

* Fixed: job template apiVersion
2023-02-15 15:27:59 +01:00
Valeriano Manassero
1f23bcf7ca 160 fileserver doesnt have an option to be with ephemeral storage (#164)
* Added: fileserver emptyDir support

* Changed: bump up version
2023-02-14 16:31:27 +01:00
Valeriano Manassero
3075f5e280 157 improve documentation (#159)
* Changed: updated installation guide

* Fixed: typo in copy and paste

* Changed: updated install guide

* Fixed: use relative path
2023-02-14 08:44:04 +01:00
Valeriano Manassero
97550c720f Fix cookiename availability (#158)
* Fixed: cookieName availability

* Changed: bump up version
2023-02-14 08:42:26 +01:00
Valeriano Manassero
a29a144119 Changed: redis cluster configuration for production (#156) 2023-02-13 12:22:01 +01:00
Valeriano Manassero
a4f77c624d Create inactive-issues.yaml 2023-02-13 08:58:08 +01:00
Valeriano Manassero
dd1c201eeb Avoid collisions in internal helper variable naming (#154)
* Fixed: helper variable rename to avoid collisions

* Changed: bump version
2023-02-13 08:17:53 +01:00
Valeriano Manassero
7995fc8441 Add external multihost elasticsearch support (#150)
* Changed: elasticsearch connstring creation

* Changed: elasticsearch connstring creation

* Changed: bump up version
2023-02-09 10:29:00 +01:00
Valeriano Manassero
99903085cd Fix existing secret reference (#149)
* Fixed: existingSecret reference

* Changed: bump version

* Changed: bump up version
2023-02-09 10:11:03 +01:00
Valeriano Manassero
9fc2b7ddda Fix existing secret apiserver (#148)
* Fixed: missing brackets

* Changed: bump vesion

* Fixed: trailing space in changelog
2023-02-08 14:20:25 +01:00
Valeriano Manassero
c7b3a28989 146 agentadd affinity config (#147)
* Added: affinity parameter

* Changed: bump version
2023-02-02 12:20:06 +01:00
Valeriano Manassero
12baef0d75 fixed: typos (#145) 2023-02-02 11:50:11 +01:00
Valeriano Manassero
72916e171a Added: specific platform configurations (#144) 2023-01-31 09:25:53 +01:00
Valeriano Manassero
126f313cdf Add agent pod securitycontext (#143)
* Added: securityContext for agent

* Changed: bump up version

* Added: support for k8s 1.26
2023-01-31 09:16:25 +01:00
Valeriano Manassero
9aa1997ebd 141 apiserver init check improvements (#142)
* Added: check also redis and mongo before starting apiserver

* Changed: bump version
2023-01-30 12:44:41 +01:00
Valeriano Manassero
db325a95a0 Fileserver existing pvc support (#140)
* Added: support for existing fileserver PVC

* Changed: bump up version

* Changed: changelog update
2023-01-25 17:12:54 +01:00
Valeriano Manassero
9e97c03b5f Fix override url (#139)
* Fixed: url override generation

* Changed: bump up version

* Changed: supported k8s versions

* Changed: changelog update
2023-01-25 16:34:28 +01:00
Valeriano Manassero
16506130ba Changed: updated version references (#138) 2023-01-25 16:16:23 +01:00
Valeriano Manassero
e2d60312d3 Fix enterprise apps deployment (#137)
* Fixed: apps deployment

* Changed: version bump
2023-01-24 13:24:15 +01:00
Valeriano Manassero
7c3ed7eb72 Fix external mongodb connstring (#135)
* Changed: maongodb.enabled check not needed

* Changed: external MongoDB connection string

* Changed: bump up version

* Added: artifacthub changelog annotation
2023-01-24 09:27:42 +01:00
Valeriano Manassero
67d4b5b95d Enterprise apps sa (#134)
* Changed: don't use cluster wide access

* Changed: bump version
2023-01-20 10:24:34 +01:00
Valeriano Manassero
832090a791 Configurable securitycontext (#133)
* Added: configurable securityContext

* Changed: bump up version

* Changed: bump up version
2023-01-19 15:00:22 +01:00
Valeriano Manassero
e1049fa0ab Ingressclassname (#132)
* Added: ingressclassname

* Changed: bump up version
2023-01-19 07:48:30 +01:00
Valeriano Manassero
5f62daac0f Existing resource for additionalconfigs (#130)
* Added: additionalConfigs reference for existing resurce

* Changed: version bump
2023-01-18 13:34:29 +01:00
Valeriano Manassero
cdcd35c224 Enterprise 3.15.3 (#129)
* Changed: enterprise version bump

* Changed: version bump
2023-01-16 16:46:26 +01:00
Valeriano Manassero
3fd3f30030 Enterprise override tag (#127)
* Added: override for enterprise  image tag

* Changed: version bump

* Added: enterprise image tage overrides

* Changed: bump up version
2023-01-12 09:12:19 +01:00
Valeriano Manassero
bdea0e778b Fix nodeport (#126)
* fixed: agent nodeSelector

* Changed: version bump
2023-01-12 08:21:38 +01:00
37 changed files with 799 additions and 383 deletions

View File

@@ -22,19 +22,18 @@ jobs:
strategy:
matrix:
k8s:
- v1.22.13
- v1.23.10
- v1.24.4
- v1.25.0
- v1.24.7
- v1.25.3
- v1.26.0
steps:
- name: Checkout
uses: actions/checkout@v1
- name: Create kind ${{ matrix.k8s }} cluster
uses: helm/kind-action@v1.3.0
uses: helm/kind-action@v1.5.0
with:
node_image: kindest/node:${{ matrix.k8s }}
- name: Set up chart-testing
uses: helm/chart-testing-action@v2.2.1
uses: helm/chart-testing-action@v2.3.1
- name: Run chart-testing (list-changed)
id: list-changed
run: |

22
.github/workflows/inactive-issues.yaml vendored Normal file
View File

@@ -0,0 +1,22 @@
name: Close inactive issues
on:
schedule:
- cron: "30 1 * * *"
jobs:
close-issues:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/stale@v7
with:
days-before-issue-stale: 14
days-before-issue-close: 7
stale-issue-label: "stale"
stale-issue-message: "This issue is stale because it has been open for 14 days with no activity."
close-issue-message: "This issue was closed because it has been inactive for 7 days since being marked as stale."
days-before-pr-stale: -1
days-before-pr-close: -1
repo-token: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -22,7 +22,7 @@ jobs:
git config user.name "$GITHUB_ACTOR"
git config user.email "$GITHUB_ACTOR@users.noreply.github.com"
- name: Run chart-releaser
uses: helm/chart-releaser-action@v1.2.1
uses: helm/chart-releaser-action@v1.5.0
env:
CR_TOKEN: '${{ secrets.CR_TOKEN }}'
with:

47
INSTALL.md Normal file
View File

@@ -0,0 +1,47 @@
# ClearML Helm Charts Installation guide
## Requirements
* Set up a Kubernetes Cluster - for setting up Kubernetes on various platforms refer to the Kubernetes [getting started guide](http://kubernetes.io/docs/getting-started-guides/).
* Set up a single-node LOCAL Kubernetes on laptop/desktop - for setting up Kubernetes on your laptop/desktop, we suggest [kind](https://kind.sigs.k8s.io).
* For **Kubernetes Tanzu users** - see [prerequisites](https://github.com/allegroai/clearml-helm-charts/tree/main/platform-specific-configs/tanzu)
for setting up ClearML on a Tanzu cluster
* For **Kubernetes Openshift users** - see [prerequisites](https://github.com/allegroai/clearml-helm-charts/tree/main/platform-specific-configs/openshift)
for setting up ClearML on an Openshift cluster,
* Install Helm - Helm is a tool for managing Kubernetes charts. Charts are packages of pre-configured Kubernetes
resources. To install Helm, refer to the [Helm install guide](https://github.com/helm/helm#install) and ensure that the `helm` binary is in the `PATH` of your shell.
## Helm Charts Installation
### Helm Repo
```bash
$ helm repo add allegroai https://allegroai.github.io/clearml-helm-charts
$ helm repo update
```
### ClearML Server Ecosystem
```bash
$ helm install clearml allegroai/clearml
```
### ClearML Agent
A ClearML Agent is always related to a ClearML server ecosystem (by default using the `app.clear.ml` hosted server, but
can be on the same or different Kubernetes cluster or a single server installation).
In the ClearML UI, go to **Settings > Workspace** and click **Create New Credentials**. The dialog that pops up displays
the new credentials.
In the Helm chart `install` command below:
* Set `ACCESSKEY` to the new credentials' `access_key` value
* Set `SECRETKEY` to the new credentials' `secret_key` value
* Set `APISERVERURL` to the new credentials' `api_server` value
* Set `FILESSERVERURL` to the new credentials' `files_server` value
* Set `WEBSERVERURL` to the new credentials' `web_server` value
```bash
$ helm install clearml-agent allegroai/clearml-agent --set clearml.agentk8sglueKey=ACCESSKEY --set clearml.agentk8sglueSecret=SECRETKEY --set agentk8sglue.apiServerUrlReference=APISERVERURL --set agentk8sglue.fileServerUrlReference=FILESERVERURL --set agentk8sglue.webServerUrlReference=WEBSERVERURL
```

View File

@@ -1,4 +1,4 @@
# ClearML Helm Charts Library for Kubernetes
# ClearML Helm Charts for Kubernetes
## Auto-Magical Experiment Manager & Version Control for AI
@@ -23,57 +23,40 @@ Use this repository to deploy **clearml-server** on Kubernetes clusters.
## Provided in this repository
### [All around Helm Chart](https://github.com/allegroai/clearml-helm-charts/tree/main/charts/clearml)
### [ClearML server chart](https://github.com/allegroai/clearml-helm-charts/tree/main/charts/clearml)
### [ClearML agent chart](https://github.com/allegroai/clearml-helm-charts/tree/main/charts/clearml-agent)
### [ClearML serving chart](https://github.com/allegroai/clearml-helm-charts/tree/main/charts/clearml-serving)
## Who We Are
ClearML is supported by the team behind *allegro.ai*,
where we build deep learning pipelines and infrastructure for enterprise companies.
ClearML is supported by you :heart: and the [clear.ml](https://clear.ml) team, which helps enterprise companies build
scalable MLOps.
We built ClearML to track and control the glorious but messy process of training production-grade deep learning models.
We are committed to vigorously supporting and expanding the capabilities of ClearML.
We promise to always be backwardly compatible, making sure all your logs, data and pipelines
We promise to always be backwards compatible, making sure all your logs, data, and pipelines
will always upgrade with you.
## License
Apache License, Version 2.0, (see the [LICENSE](https://www.apache.org/licenses/LICENSE-2.0) for more information)
## Requirements
## Installation Guide
### Setup a Kubernetes Cluster
For setting up Kubernetes on various platforms refer to the Kubernetes [getting started guide](http://kubernetes.io/docs/getting-started-guides/).
### Setup a single node LOCAL Kubernetes on laptop/desktop
For setting up Kubernetes on your laptop/desktop we suggest [kind](https://kind.sigs.k8s.io).
### Install Helm
Helm is a tool for managing Kubernetes charts. Charts are packages of pre-configured Kubernetes resources.
To install Helm, refer to the [Helm install guide](https://github.com/helm/helm#install) and ensure that the `helm` binary is in the `PATH` of your shell.
## Usage
```bash
$ helm repo add allegroai https://allegroai.github.io/clearml-helm-charts
$ helm repo update
$ helm search repo allegroai
$ helm install <release-name> allegroai/<chart>
```
For installation instruction, follow related [Installation Guide](INSTALL.md).
## Documentation, Community & Support
More information in the [official documentation](https://allegro.ai/clearml/docs) and [on YouTube](https://www.youtube.com/c/ClearML).
See more information in the [official documentation](https://clear.ml/docs/latest/docs) and [on YouTube](https://www.youtube.com/c/ClearML).
If you have any questions: post on our [Slack Channel](https://join.slack.com/t/clearml/shared_invite/zt-c0t13pty-aVUZZW1TSSSg2vyIGVPBhg), or tag your questions on [stackoverflow](https://stackoverflow.com/questions/tagged/clearml) with '**[clearml](https://stackoverflow.com/questions/tagged/clearml)**' tag (*previously [trains](https://stackoverflow.com/questions/tagged/trains) tag*).
If you have any questions, post on our [Slack Channel](https://join.slack.com/t/clearml/shared_invite/zt-c0t13pty-aVUZZW1TSSSg2vyIGVPBhg), or tag your questions on [stackoverflow](https://stackoverflow.com/questions/tagged/clearml) with '**[clearml](https://stackoverflow.com/questions/tagged/clearml)**' tag (*previously [trains](https://stackoverflow.com/questions/tagged/trains) tag*).
For feature requests or bug reports, please use [GitHub issues](https://github.com/allegroai/clearml-helm-charts/issues).
Additionally, you can always find us at *clearml@allegro.ai*
Additionally, you can always find us at *support@clear.ml*
## Contributing

View File

@@ -1,10 +1,10 @@
apiVersion: v2
name: clearml-agent
description: MLOps platform
description: MLOps platform Task running agent
type: application
version: "3.1.2"
version: "3.7.0"
appVersion: "1.24"
kubeVersion: ">= 1.19.0-0 < 1.26.0-0"
kubeVersion: ">= 1.21.0-0 < 1.27.0-0"
home: https://clear.ml
icon: https://raw.githubusercontent.com/allegroai/clearml/master/docs/clearml-logo.svg
sources:
@@ -17,3 +17,8 @@ keywords:
- clearml
- "machine learning"
- mlops
- "task agent"
annotations:
artifacthub.io/changes: |
- kind: added
description: support for existing rolebindings and clusterrolebindings

View File

@@ -1,8 +1,8 @@
# ClearML Kubernetes Agent
![Version: 3.1.2](https://img.shields.io/badge/Version-3.1.2-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.24](https://img.shields.io/badge/AppVersion-1.24-informational?style=flat-square)
![Version: 3.7.0](https://img.shields.io/badge/Version-3.7.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.24](https://img.shields.io/badge/AppVersion-1.24-informational?style=flat-square)
MLOps platform
MLOps platform Task running agent
**Homepage:** <https://clear.ml>
@@ -24,23 +24,28 @@ It allows you to schedule distributed experiments on a Kubernetes cluster.
## Requirements
Kubernetes: `>= 1.19.0-0 < 1.26.0-0`
Kubernetes: `>= 1.21.0-0 < 1.27.0-0`
## Values
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| agentk8sglue | object | `{"annotations":{},"apiServerUrlReference":"https://api.clear.ml","basePodTemplate":{"annotations":{},"env":[],"fileMounts":[],"hostAliases":{},"initContainers":[],"labels":{},"nodeSelector":{},"resources":{},"schedulerName":"","securityContext":{},"tolerations":[],"volumeMounts":[],"volumes":[]},"clearmlcheckCertificate":true,"containerCustomBashScript":"","customBashScript":"","debugMode":false,"defaultContainerImage":"ubuntu:18.04","extraEnvs":[],"fileMounts":[],"fileServerUrlReference":"https://files.clear.ml","image":{"repository":"allegroai/clearml-agent-k8s-base","tag":"1.24-21"},"labels":{},"nodeSelector":{},"queue":"default","replicaCount":1,"serviceExistingAccountName":"","volumeMounts":[],"volumes":[],"webServerUrlReference":"https://app.clear.ml"}` | This agent will spawn queued experiments in new pods, a good use case is to combine this with GPU autoscaling nodes. https://github.com/allegroai/clearml-agent/tree/master/docker/k8s-glue |
| agentk8sglue | object | `{"additionalClusterRoleBindings":[],"additionalRoleBindings":[],"affinity":{},"annotations":{},"apiServerUrlReference":"https://api.clear.ml","basePodTemplate":{"affinity":{},"annotations":{},"env":[],"fileMounts":[],"hostAliases":[],"initContainers":[],"labels":{},"nodeSelector":{},"priorityClassName":"","resources":{},"schedulerName":"","securityContext":{},"tolerations":[],"volumeMounts":[],"volumes":[]},"clearmlcheckCertificate":true,"containerCustomBashScript":"","customBashScript":"","debugMode":false,"defaultContainerImage":"ubuntu:18.04","extraEnvs":[],"fileMounts":[],"fileServerUrlReference":"https://files.clear.ml","image":{"repository":"allegroai/clearml-agent-k8s-base","tag":"1.24-21"},"labels":{},"nodeSelector":{},"queue":"default","replicaCount":1,"securityContext":{},"serviceExistingAccountName":"","taskAsJob":false,"tolerations":[],"volumeMounts":[],"volumes":[],"webServerUrlReference":"https://app.clear.ml"}` | This agent will spawn queued experiments in new pods, a good use case is to combine this with GPU autoscaling nodes. https://github.com/allegroai/clearml-agent/tree/master/docker/k8s-glue |
| agentk8sglue.additionalClusterRoleBindings | list | `[]` | additional existing ClusterRoleBindings |
| agentk8sglue.additionalRoleBindings | list | `[]` | additional existing RoleBindings |
| agentk8sglue.affinity | object | `{}` | affinity setup for Agent pod (example in values.yaml comments) |
| agentk8sglue.annotations | object | `{}` | annotations setup for Agent pod (example in values.yaml comments) |
| agentk8sglue.apiServerUrlReference | string | `"https://api.clear.ml"` | Reference to Api server url |
| agentk8sglue.basePodTemplate | object | `{"annotations":{},"env":[],"fileMounts":[],"hostAliases":{},"initContainers":[],"labels":{},"nodeSelector":{},"resources":{},"schedulerName":"","securityContext":{},"tolerations":[],"volumeMounts":[],"volumes":[]}` | base template for pods spawned to consume ClearML Task |
| agentk8sglue.basePodTemplate | object | `{"affinity":{},"annotations":{},"env":[],"fileMounts":[],"hostAliases":[],"initContainers":[],"labels":{},"nodeSelector":{},"priorityClassName":"","resources":{},"schedulerName":"","securityContext":{},"tolerations":[],"volumeMounts":[],"volumes":[]}` | base template for pods spawned to consume ClearML Task |
| agentk8sglue.basePodTemplate.affinity | object | `{}` | affinity setup for pods spawned to consume ClearML Task |
| agentk8sglue.basePodTemplate.annotations | object | `{}` | annotations setup for pods spawned to consume ClearML Task (example in values.yaml comments) |
| agentk8sglue.basePodTemplate.env | list | `[]` | environment variables for pods spawned to consume ClearML Task (example in values.yaml comments) |
| agentk8sglue.basePodTemplate.fileMounts | list | `[]` | file definition for pods spawned to consume ClearML Task (example in values.yaml comments) |
| agentk8sglue.basePodTemplate.hostAliases | object | `{}` | hostAliases setup for pods spawned to consume ClearML Task (example in values.yaml comments) |
| agentk8sglue.basePodTemplate.hostAliases | list | `[]` | hostAliases setup for pods spawned to consume ClearML Task (example in values.yaml comments) |
| agentk8sglue.basePodTemplate.initContainers | list | `[]` | initContainers definition for pods spawned to consume ClearML Task (example in values.yaml comments) |
| agentk8sglue.basePodTemplate.labels | object | `{}` | labels setup for pods spawned to consume ClearML Task (example in values.yaml comments) |
| agentk8sglue.basePodTemplate.nodeSelector | object | `{}` | nodeSelector setup for pods spawned to consume ClearML Task (example in values.yaml comments) |
| agentk8sglue.basePodTemplate.priorityClassName | string | `""` | priorityClassName setup for pods spawned to consume ClearML Task |
| agentk8sglue.basePodTemplate.resources | object | `{}` | resources declaration for pods spawned to consume ClearML Task (example in values.yaml comments) |
| agentk8sglue.basePodTemplate.schedulerName | string | `""` | schedulerName setup for pods spawned to consume ClearML Task |
| agentk8sglue.basePodTemplate.securityContext | object | `{}` | securityContext setup for pods spawned to consume ClearML Task (example in values.yaml comments) |
@@ -59,7 +64,10 @@ Kubernetes: `>= 1.19.0-0 < 1.26.0-0`
| agentk8sglue.nodeSelector | object | `{}` | nodeSelector setup for Agent pod (example in values.yaml comments) |
| agentk8sglue.queue | string | `"default"` | ClearML queue this agent will consume |
| agentk8sglue.replicaCount | int | `1` | Glue Agent number of pods |
| agentk8sglue.securityContext | object | `{}` | Web Server pod security context |
| agentk8sglue.serviceExistingAccountName | string | `""` | if set, don't create a serviceAccountName but use defined existing one |
| agentk8sglue.taskAsJob | bool | `false` | ClearML spawn tasks as jobs instead of pods |
| agentk8sglue.tolerations | list | `[]` | tolerations setup for Agent pod (example in values.yaml comments) |
| agentk8sglue.volumeMounts | list | `[]` | volume mounts definition for Glue Agent (example in values.yaml comments) |
| agentk8sglue.volumes | list | `[]` | volumes definition for Glue Agent (example in values.yaml comments) |
| agentk8sglue.webServerUrlReference | string | `"https://app.clear.ml"` | Reference to Web server url |
@@ -69,8 +77,10 @@ Kubernetes: `>= 1.19.0-0 < 1.26.0-0`
| clearml.clearmlConfig | string | `"sdk {\n}"` | ClearML configuration file |
| clearml.existingAgentk8sglueSecret | string | `""` | If this is set, chart will not generate a secret but will use what is defined here |
| clearml.existingClearmlConfigSecret | string | `""` | If this is set, chart will not generate a secret but will use what is defined here |
| enterpriseFeatures | object | `{"applyVaultEnvVars":true,"enabled":false,"maxPods":10,"monitoredResources":{"maxResources":0,"maxResourcesFieldName":"resources|limits|nvidia.com/gpu","minResourcesFieldName":"resources|limits|nvidia.com/gpu"},"queues":null,"serviceAccountClusterAccess":false,"useOwnerToken":true}` | Enterprise features (work only with an Enterprise license) |
| enterpriseFeatures | object | `{"agentImageTagOverride":"1.24-58","applyVaultEnvVars":true,"createQueues":false,"enabled":false,"maxPods":10,"monitoredResources":{"maxResources":0,"maxResourcesFieldName":"resources|limits|nvidia.com/gpu","minResourcesFieldName":"resources|limits|nvidia.com/gpu"},"queues":null,"serviceAccountClusterAccess":false,"useOwnerToken":true}` | Enterprise features (work only with an Enterprise license) |
| enterpriseFeatures.agentImageTagOverride | string | `"1.24-58"` | Image tag override for enterprise version |
| enterpriseFeatures.applyVaultEnvVars | bool | `true` | push env vars from Clear.ML Vault to task pods |
| enterpriseFeatures.createQueues | bool | `false` | Create queues if they don't exist |
| enterpriseFeatures.enabled | bool | `false` | Enable/Disable Enterprise features |
| enterpriseFeatures.maxPods | int | `10` | maximum concurrent consume ClearML Task pod |
| enterpriseFeatures.monitoredResources | object | `{"maxResources":0,"maxResourcesFieldName":"resources|limits|nvidia.com/gpu","minResourcesFieldName":"resources|limits|nvidia.com/gpu"}` | GPU resource general counters |

View File

@@ -1,23 +1,23 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "clearml.name" -}}
{{- define "clearmlAgent.name" -}}
{{- .Release.Name | trunc 59 | trimSuffix "-" }}
{{- end }}
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "clearml.chart" -}}
{{- define "clearmlAgent.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 59 | trimSuffix "-" }}
{{- end }}
{{/*
Common labels
*/}}
{{- define "clearml.labels" -}}
helm.sh/chart: {{ include "clearml.chart" . }}
{{ include "clearml.selectorLabels" . }}
{{- define "clearmlAgent.labels" -}}
helm.sh/chart: {{ include "clearmlAgent.chart" . }}
{{ include "clearmlAgent.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
@@ -30,7 +30,7 @@ app.kubernetes.io/managed-by: {{ .Release.Service }}
{{/*
Common annotations
*/}}
{{- define "clearml.annotations" -}}
{{- define "clearmlAgent.annotations" -}}
{{- if $.Values.agentk8sglue.annotations }}
{{ toYaml $.Values.agentk8sglue.annotations }}
{{- end }}
@@ -39,8 +39,8 @@ Common annotations
{{/*
Selector labels
*/}}
{{- define "clearml.selectorLabels" -}}
app.kubernetes.io/name: {{ include "clearml.name" . }}
{{- define "clearmlAgent.selectorLabels" -}}
app.kubernetes.io/name: {{ include "clearmlAgent.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
@@ -48,18 +48,18 @@ app.kubernetes.io/instance: {{ .Release.Name }}
Selector labels (agentk8sglue)
*/}}
{{- define "agentk8sglue.selectorLabels" -}}
app.kubernetes.io/name: {{ include "clearml.name" . }}
app.kubernetes.io/instance: {{ include "clearml.name" . }}
app.kubernetes.io/name: {{ include "clearmlAgent.name" . }}
app.kubernetes.io/instance: {{ include "clearmlAgent.name" . }}
{{- end }}
{{/*
Create the name of the service account to use
*/}}
{{- define "clearml.serviceAccountName" -}}
{{- define "clearmlAgent.serviceAccountName" -}}
{{- if .Values.agentk8sglue.serviceExistingAccountName }}
{{- .Values.agentk8sglue.serviceExistingAccountName }}
{{- else }}
{{- include "clearml.name" . }}-sa
{{- include "clearmlAgent.name" . }}-sa
{{- end }}
{{- end }}
@@ -72,6 +72,16 @@ Create secret to access docker registry
{{- end }}
{{- end }}
{{/*
Create a queues parameter
*/}}
{{- define "agentk8sglue.createQueues" -}}
{{- if .Values.enterpriseFeatures.createQueues }}
{{- printf "%d" 1}}
{{- else }}
{{- printf "%d" 0 }}
{{- end }}
{{- end }}
{{/*
Create a string composed by queue names
@@ -83,3 +93,136 @@ Create a string composed by queue names
{{- end }}
{{- join " " $list }}
{{- end }}
{{/*
Create a task container template
*/}}
{{- define "taskContainer.containerTemplate" -}}
{{- if .main.Values.imageCredentials.enabled }}
imagePullSecrets:
- name: {{ .main.Values.imageCredentials.existingSecret | default (printf "%s-ark" (include "clearmlAgent.name" .main )) }}
{{- end }}
schedulerName: {{ .value.templateOverrides.schedulerName | default (.main.Values.agentk8sglue.basePodTemplate.schedulerName) }}
restartPolicy: Never
securityContext:
{{- .value.templateOverrides.securityContext | default .main.Values.agentk8sglue.basePodTemplate.securityContext | toYaml | nindent 2 }}
hostAliases:
{{- .value.templateOverrides.hostAliases | default .main.Values.agentk8sglue.basePodTemplate.hostAliases | toYaml | nindent 2 }}
volumes:
{{ $computedvolumes := (.value.templateOverrides.volumes | default .main.Values.agentk8sglue.basePodTemplate.volumes) }}
{{- if $computedvolumes }}{{- $computedvolumes | toYaml | nindent 2 }}{{- end }}
{{- if .value.templateOverrides.fileMounts }}
- name: filemounts
secret:
secretName: {{ include "clearmlAgent.name" .main }}-{{ .key }}-fm
{{- else if .main.Values.agentk8sglue.basePodTemplate.fileMounts }}
- name: filemounts
secret:
secretName: {{ include "clearmlAgent.name" .main }}-fm
{{- end }}
{{- if not .main.Values.enterpriseFeatures.serviceAccountClusterAccess }}
serviceAccountName: {{ include "clearmlAgent.serviceAccountName" .main }}
{{- end }}
initContainers:
{{- .value.templateOverrides.initContainers | default .main.Values.agentk8sglue.basePodTemplate.initContainers | toYaml | nindent 2 }}
priorityClassName: {{ .value.templateOverrides.priorityClassName | default .main.Values.agentk8sglue.basePodTemplate.priorityClassName }}
containers:
- resources:
{{- .value.templateOverrides.resources | default .main.Values.agentk8sglue.basePodTemplate.resources | toYaml | nindent 4 }}
ports:
- containerPort: 10022
volumeMounts:
{{ $computedvolumemounts := (.value.templateOverrides.volumeMounts | default .main.Values.agentk8sglue.basePodTemplate.volumeMounts) }}
{{- if $computedvolumemounts }}{{- $computedvolumemounts | toYaml | nindent 4 }}{{- end }}
{{- if .value.templateOverrides.fileMounts }}
{{- range .value.templateOverrides.fileMounts }}
- name: filemounts
mountPath: "{{ .folderPath }}/{{ .name }}"
subPath: "{{ .name }}"
readOnly: true
{{- end }}
{{- else if .main.Values.agentk8sglue.basePodTemplate.fileMounts }}
{{- range .main.Values.agentk8sglue.basePodTemplate.fileMounts }}
- name: filemounts
mountPath: "{{ .folderPath }}/{{ .name }}"
subPath: "{{ .name }}"
readOnly: true
{{- end }}
{{- end }}
env:
- name: CLEARML_API_HOST
value: {{ .main.Values.agentk8sglue.apiServerUrlReference }}
- name: CLEARML_WEB_HOST
value: {{ .main.Values.agentk8sglue.webServerUrlReference }}
- name: CLEARML_FILES_HOST
value: {{ .main.Values.agentk8sglue.fileServerUrlReference }}
{{- if not .main.Values.enterpriseFeatures.useOwnerToken }}
- name: CLEARML_API_ACCESS_KEY
valueFrom:
secretKeyRef:
name: {{ .main.Values.clearml.existingAgentk8sglueSecret | default (printf "%s-ac" (include "clearmlAgent.name" .main )) }}
key: agentk8sglue_key
- name: CLEARML_API_SECRET_KEY
valueFrom:
secretKeyRef:
name: {{ .main.Values.clearml.existingAgentk8sglueSecret | default (printf "%s-ac" (include "clearmlAgent.name" .main )) }}
key: agentk8sglue_secret
{{- end }}
- name: PYTHONUNBUFFERED
value: "x"
{{- if not .main.Values.agentk8sglue.clearmlcheckCertificate }}
- name: CLEARML_API_HOST_VERIFY_CERT
value: "false"
{{- end }}
{{ $computedenvs := (.value.templateOverrides.env| default .main.Values.agentk8sglue.basePodTemplate.env) }}
{{- if $computedenvs }}{{- $computedenvs | toYaml | nindent 4 }}{{- end }}
nodeSelector:
{{ .value.templateOverrides.nodeSelector | default .main.Values.agentk8sglue.basePodTemplate.nodeSelector | toYaml | nindent 2 }}
tolerations:
{{ .value.templateOverrides.tolerations | default .main.Values.agentk8sglue.basePodTemplate.tolerations | toYaml | nindent 2 }}
affinity:
{{ .value.templateOverrides.affinity | default .main.Values.agentk8sglue.basePodTemplate.affinity | toYaml | nindent 2 }}
{{- end }}
{{/*
Create a task container template
*/}}
{{- define "taskContainer.podTemplate" -}}
{{- range $key, $value := $.Values.enterpriseFeatures.queues }}
{{ $key }}:
apiVersion: v1
kind: Pod
metadata:
namespace: {{ $.Release.Namespace }}
labels:
{{ $value.templateOverrides.labels | default $.Values.agentk8sglue.basePodTemplate.labels | toYaml }}
annotations:
{{ $value.templateOverrides.annotations | default $.Values.agentk8sglue.basePodTemplate.annotations | toYaml }}
spec:
{{- $data := dict "main" $ "key" $key "value" $value -}}
{{- include "taskContainer.containerTemplate" $data | nindent 4}}
{{- end }}
{{- end }}
{{/*
Create a task container template
*/}}
{{- define "taskContainer.jobTemplate" -}}
{{- range $key, $value := $.Values.enterpriseFeatures.queues }}
{{ $key }}:
apiVersion: batch/v1
kind: Job
metadata:
namespace: {{ $.Release.Namespace }}
labels:
{{ $value.templateOverrides.labels | default $.Values.agentk8sglue.basePodTemplate.labels | toYaml }}
annotations:
{{ $value.templateOverrides.annotations | default $.Values.agentk8sglue.basePodTemplate.annotations | toYaml }}
spec:
template:
spec:
{{- $data := dict "main" $ "key" $key "value" $value -}}
{{- include "taskContainer.containerTemplate" $data | nindent 8 }}
backoffLimit: 0
{{- end }}
{{- end }}

View File

@@ -1,185 +1,22 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "clearml.name" . }}-pt
name: {{ include "clearmlAgent.name" . }}-pt
data:
{{- if .Values.enterpriseFeatures.enabled }}
template.yaml: |
{{- range $key, $value := $.Values.enterpriseFeatures.queues }}
{{ $key }}:
apiVersion: v1
metadata:
namespace: {{ $.Release.Namespace }}
{{- if $value.templateOverrides.labels }}
labels:
{{- toYaml $value.templateOverrides.labels | nindent 10 }}
{{- else if $.Values.agentk8sglue.basePodTemplate.labels }}
labels:
{{- toYaml $.Values.agentk8sglue.basePodTemplate.labels | nindent 10 }}
{{- end}}
{{- if $value.templateOverrides.annotations }}
annotations:
{{- toYaml $value.templateOverrides.annotations | nindent 10 }}
{{- else if $.Values.agentk8sglue.basePodTemplate.annotations }}
annotations:
{{- toYaml $.Values.agentk8sglue.basePodTemplate.annotations | nindent 10 }}
{{- end}}
spec:
{{- if $.Values.imageCredentials.enabled }}
imagePullSecrets:
{{- if $.Values.imageCredentials.existingSecret }}
- name: $.Values.imageCredentials.existingSecret
{{- else }}
- name: {{ include "clearml.name" $ }}-ark
{{- end }}
{{- end }}
{{- if $value.templateOverrides.schedulerName }}
schedulerName: {{ $value.templateOverrides.schedulerName }}
{{- else if $.Values.agentk8sglue.basePodTemplate.schedulerName }}
schedulerName: {{ $.Values.agentk8sglue.basePodTemplate.schedulerName }}
{{- end}}
restartPolicy: Never
{{- if $value.templateOverrides.securityContext }}
securityContext:
{{- toYaml $value.templateOverrides.securityContext | nindent 10 }}
{{- else if $.Values.agentk8sglue.basePodTemplate.securityContext }}
securityContext:
{{- toYaml $.Values.agentk8sglue.basePodTemplate.securityContext | nindent 10 }}
{{- end}}
{{- if $value.templateOverrides.hostAliases }}
{{- with $value.templateOverrides.hostAliases }}
hostAliases:
{{- toYaml . | nindent 10 }}
{{- end }}
{{- else if $.Values.agentk8sglue.basePodTemplate.hostAliases }}
{{- with $.Values.agentk8sglue.basePodTemplate.hostAliases }}
hostAliases:
{{- toYaml . | nindent 10 }}
{{- end }}
{{- end }}
volumes:
{{- if $value.templateOverrides.volumes }}
{{- toYaml $value.templateOverrides.volumes | nindent 10 }}
{{- else if $.Values.agentk8sglue.basePodTemplate.volumes }}
{{- toYaml $.Values.agentk8sglue.basePodTemplate.volumes | nindent 10 }}
{{- end }}
{{- if $value.templateOverrides.fileMounts }}
- name: filemounts
secret:
secretName: {{ include "clearml.name" $ }}-{{ $key }}-fm
{{- else if $.Values.agentk8sglue.basePodTemplate.fileMounts }}
- name: filemounts
secret:
secretName: {{ include "clearml.name" $ }}-fm
{{- end }}
{{- if not $.Values.enterpriseFeatures.serviceAccountClusterAccess }}
serviceAccountName: {{ include "clearml.serviceAccountName" $ }}
{{- end }}
{{- if $value.templateOverrides.initContainers }}
initContainers:
{{- toYaml $value.templateOverrides.initContainers | nindent 10 }}
{{- else if $.Values.agentk8sglue.basePodTemplate.initContainers }}
initContainers:
{{- toYaml $.Values.agentk8sglue.basePodTemplate.initContainers | nindent 10 }}
{{- end }}
containers:
- resources:
{{- if $value.templateOverrides.resources }}
{{- toYaml $value.templateOverrides.resources | nindent 12 }}
{{- else if $.Values.agentk8sglue.basePodTemplate.resources }}
{{- toYaml $.Values.agentk8sglue.basePodTemplate.resources | nindent 12 }}
{{- end}}
ports:
- containerPort: 10022
volumeMounts:
{{- if $value.templateOverrides.volumeMounts }}
{{- toYaml $value.templateOverrides.volumeMounts | nindent 12 }}
{{- else if $.Values.agentk8sglue.basePodTemplate.volumeMounts }}
{{- toYaml $.Values.agentk8sglue.basePodTemplate.volumeMounts | nindent 12 }}
{{- end }}
{{- if $value.templateOverrides.fileMounts }}
{{- range $value.templateOverrides.fileMounts }}
- name: filemounts
mountPath: "{{ .folderPath }}/{{ .name }}"
subPath: "{{ .name }}"
readOnly: true
{{- end }}
{{- else if $.Values.agentk8sglue.basePodTemplate.fileMounts }}
{{- range $.Values.agentk8sglue.basePodTemplate.fileMounts }}
- name: filemounts
mountPath: "{{ .folderPath }}/{{ .name }}"
subPath: "{{ .name }}"
readOnly: true
{{- end }}
{{- end }}
env:
- name: CLEARML_API_HOST
value: {{ $.Values.agentk8sglue.apiServerUrlReference }}
- name: CLEARML_WEB_HOST
value: {{ $.Values.agentk8sglue.webServerUrlReference }}
- name: CLEARML_FILES_HOST
value: {{ $.Values.agentk8sglue.fileServerUrlReference }}
{{- if not $.Values.enterpriseFeatures.useOwnerToken }}
- name: CLEARML_API_ACCESS_KEY
valueFrom:
secretKeyRef:
{{- if $.Values.clearml.existingAgentk8sglueSecret }}
name: {{ $.Values.clearml.existingAgentk8sglueSecret }}
{{- else }}
name: {{ include "clearml.name" $ }}-ac
{{- end }}
key: agentk8sglue_key
- name: CLEARML_API_SECRET_KEY
valueFrom:
secretKeyRef:
{{- if $.Values.clearml.existingAgentk8sglueSecret }}
name: {{ $.Values.clearml.existingAgentk8sglueSecret }}
{{- else }}
name: {{ include "clearml.name" $ }}-ac
{{- end }}
key: agentk8sglue_secret
{{- end }}
- name: PYTHONUNBUFFERED
value: "x"
{{- if not $.Values.agentk8sglue.clearmlcheckCertificate }}
- name: CLEARML_API_HOST_VERIFY_CERT
value: "false"
{{- end }}
{{- if $value.templateOverrides.env }}
{{- toYaml $value.templateOverrides.env | nindent 12 }}
{{- else if $.Values.agentk8sglue.basePodTemplate.env }}
{{- toYaml $.Values.agentk8sglue.basePodTemplate.env | nindent 12 }}
{{- end }}
{{- if $value.templateOverrides.nodeSelector }}
{{- with $value.templateOverrides.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- else if $.Values.agentk8sglue.basePodTemplate.nodeSelector }}
{{- with $.Values.agentk8sglue.basePodTemplate.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 10 }}
{{- end }}
{{- end }}
{{- if $value.templateOverrides.tolerations }}
{{- with $value.templateOverrides.tolerations }}
tolerations:
{{- toYaml . | nindent 10 }}
{{- end }}
{{- else if $.Values.agentk8sglue.basePodTemplate.tolerations }}
{{- with $.Values.agentk8sglue.basePodTemplate.tolerations }}
tolerations:
{{- toYaml . | nindent 10 }}
{{- end }}
{{- end }}
{{- if .Values.agentk8sglue.taskAsJob }}
{{ include "taskContainer.jobTemplate" . | nindent 4}}
{{- else }}
{{ include "taskContainer.podTemplate" . | nindent 4}}
{{- end }}
secrets.yaml: |
{{- range $key, $value := $.Values.enterpriseFeatures.queues }}
{{ $key }}:
{{- if $value.templateOverrides.fileMounts }}
- {{ include "clearml.name" $ }}-{{ $key }}-fm
- {{ include "clearmlAgent.name" $ }}-{{ $key }}-fm
{{- else if $.Values.agentk8sglue.basePodTemplate.fileMounts }}
- {{ include "clearml.name" $ }}-fm
- {{ include "clearmlAgent.name" $ }}-fm
{{- end }}
{{- end }}
{{- else }}
@@ -195,16 +32,17 @@ data:
{{- if .Values.imageCredentials.enabled }}
imagePullSecrets:
{{- if .Values.imageCredentials.existingSecret }}
- name: {{.Values.imageCredentials.existingSecret}}
- name: {{ .Values.imageCredentials.existingSecret }}
{{- else }}
- name: name: {{ include "clearml.name" $ }}-ark
- name: name: {{ include "clearmlAgent.name" $ }}-ark
{{- end }}
{{- end }}
{{- with .Values.agentk8sglue.basePodTemplate.volumes }}
volumes:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "clearml.serviceAccountName" $ }}
serviceAccountName: {{ include "clearmlAgent.serviceAccountName" $ }}
priorityClassName: {{ .Values.agentk8sglue.basePodTemplate.priorityClassName }}
containers:
- resources:
{{- toYaml .Values.agentk8sglue.basePodTemplate.resources | nindent 10 }}
@@ -227,7 +65,7 @@ data:
{{- if .Values.clearml.existingAgentk8sglueSecret }}
name: {{ .Values.clearml.existingAgentk8sglueSecret }}
{{- else }}
name: {{ include "clearml.name" . }}-ac
name: {{ include "clearmlAgent.name" . }}-ac
{{- end }}
key: agentk8sglue_key
- name: CLEARML_API_SECRET_KEY
@@ -236,7 +74,7 @@ data:
{{- if .Values.clearml.existingAgentk8sglueSecret }}
name: {{ .Values.clearml.existingAgentk8sglueSecret }}
{{- else }}
name: {{ include "clearml.name" . }}-ac
name: {{ include "clearmlAgent.name" . }}-ac
{{- end }}
key: agentk8sglue_secret
{{- if .Values.agentk8sglue.basePodTemplate.env }}
@@ -250,6 +88,10 @@ data:
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.agentk8sglue.basePodTemplate.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
{{- if .Values.sessions.portModeEnabled }}
{{- range untilStep 1 ( ( add .Values.sessions.maxServices 1 ) | int ) 1 }}
@@ -259,7 +101,7 @@ data:
metadata:
name: clearml-session-{{ . }}
labels:
{{- include "clearml.labels" $ | nindent 8 }}
{{- include "clearmlAgent.labels" $ | nindent 8 }}
{{- with $.Values.sessions.svcAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}

View File

@@ -1,11 +1,11 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "clearml.name" . }}
name: {{ include "clearmlAgent.name" . }}
labels:
{{- include "clearml.labels" . | nindent 4 }}
{{- include "clearmlAgent.labels" . | nindent 4 }}
annotations:
{{- include "clearml.annotations" . | nindent 4 }}
{{- include "clearmlAgent.annotations" . | nindent 4 }}
spec:
replicas: {{ .Values.agentk8sglue.replicaCount }}
selector:
@@ -14,23 +14,28 @@ spec:
template:
metadata:
annotations:
checksum/config: {{ printf "%s%s" .Values.clearml .Values.agentk8sglue | sha256sum }}
{{- include "clearml.annotations" . | nindent 8 }}
checksum/config: {{ printf "%s" .Values | sha256sum }}
{{- include "clearmlAgent.annotations" . | nindent 8 }}
labels:
{{- include "clearml.labels" . | nindent 8 }}
{{- include "clearmlAgent.labels" . | nindent 8 }}
spec:
{{- if .Values.imageCredentials.enabled }}
imagePullSecrets:
{{- if .Values.imageCredentials.existingSecret }}
- name: .Values.imageCredentials.existingSecret
- name: {{ .Values.imageCredentials.existingSecret }}
{{- else }}
- name: {{ include "clearml.name" . }}-ark
- name: {{ include "clearmlAgent.name" . }}-ark
{{- end }}
{{- end }}
serviceAccountName: {{ include "clearml.serviceAccountName" . }}
serviceAccountName: {{ include "clearmlAgent.serviceAccountName" . }}
securityContext: {{ toYaml .Values.agentk8sglue.securityContext | nindent 8 }}
initContainers:
- name: init-k8s-glue
{{- if .Values.enterpriseFeatures.enabled }}
image: "{{ .Values.agentk8sglue.image.repository }}:{{ .Values.enterpriseFeatures.agentImageTagOverride }}"
{{- else }}
image: "{{ .Values.agentk8sglue.image.repository }}:{{ .Values.agentk8sglue.image.tag }}"
{{- end }}
command:
- /bin/sh
- -c
@@ -50,7 +55,11 @@ spec:
done
containers:
- name: k8s-glue
{{- if .Values.enterpriseFeatures.enabled }}
image: "{{ .Values.agentk8sglue.image.repository }}:{{ .Values.enterpriseFeatures.agentImageTagOverride }}"
{{- else }}
image: "{{ .Values.agentk8sglue.image.repository }}:{{ .Values.agentk8sglue.image.tag }}"
{{- end }}
imagePullPolicy: IfNotPresent
command:
- /bin/bash
@@ -59,7 +68,7 @@ spec:
export PATH=$PATH:$HOME/bin;
source /root/.bashrc && /root/entrypoint.sh
volumeMounts:
- name: {{ include "clearml.name" . }}-pt
- name: {{ include "clearmlAgent.name" . }}-pt
mountPath: /root/template
{{ if .Values.clearml.clearmlConfig }}
- name: k8sagent-clearml-conf-volume
@@ -104,6 +113,10 @@ spec:
value: "--namespace {{ .Release.Namespace }} --template-yaml /root/template/template.yaml \
--max-pods {{.Values.enterpriseFeatures.maxPods}}{{ if .Values.enterpriseFeatures.enabled }}{{ if .Values.enterpriseFeatures.useOwnerToken }} --use-owner-token{{ end }}{{ end }}"
{{- end }}
{{- if .Values.clearml.clearmlConfig }}
- name: CLEARML_CONFIG_FILE
value: /root/clearml.conf
{{- end }}
- name: CLEARML_K8S_GLUE_LIMIT_POD_LABEL
value: "ai.allegro.agent.serial=pod-{pod_number}"
- name: CLEARML_K8S_SECRETS_LIST_FILE
@@ -113,15 +126,15 @@ spec:
- name: CLEARML_API_ACCESS_KEY
valueFrom:
secretKeyRef:
name: {{ include "clearml.name" . }}-ac
name: {{ include "clearmlAgent.name" . }}-ac
key: agentk8sglue_key
- name: CLEARML_API_SECRET_KEY
valueFrom:
secretKeyRef:
name: {{ include "clearml.name" . }}-ac
name: {{ include "clearmlAgent.name" . }}-ac
key: agentk8sglue_secret
- name: CLEARML_WORKER_ID
value: {{ include "clearml.name" . }}
value: {{ include "clearmlAgent.name" . }}
- name: CLEARML_AGENT_UPDATE_REPO
value: ""
- name: FORCE_CLEARML_AGENT_REPO
@@ -149,33 +162,50 @@ spec:
value: "interactive"
{{- end }}
{{- end }}
{{- if .Values.agentk8sglue.taskAsJob }}
- name: "CLEARML_K8S_GLUE_KIND"
value: "job"
{{- else }}
- name: "CLEARML_K8S_GLUE_KIND"
value: "pod"
{{- end }}
{{- if .Values.enterpriseFeatures.enabled }}
- name: K8S_GLUE_QUEUE
value: {{ include "agentk8sglue.queues" . | quote }}
- name: CLEARML_K8S_GLUE_CREATE_QUEUE
value: {{ include "agentk8sglue.createQueues" . | quote }}
- name: CLEARML_K8S_GLUE_APPLY_VAULT_ENV_VARS
value: {{ .Values.enterpriseFeatures.applyVaultEnvVars | quote }}
- name: "CLEARML_K8S_GLUE_POD_MIN_RES_FIELD"
value: {{.Values.enterpriseFeatures.monitoredResources.minResourcesFieldName}}
value: {{ .Values.enterpriseFeatures.monitoredResources.minResourcesFieldName }}
- name: "CLEARML_K8S_GLUE_MAX_RESOURCES"
value: "{{.Values.enterpriseFeatures.monitoredResources.maxResources}}"
- name: "CLEARML_K8S_GLUE_POD_MAX_RES_FIELD"
value: {{.Values.enterpriseFeatures.monitoredResources.maxResourcesFieldName}}
value: {{ .Values.enterpriseFeatures.monitoredResources.maxResourcesFieldName }}
{{- else }}
- name: K8S_GLUE_QUEUE
value: {{ .Values.agentk8sglue.queue }}
{{- end }}
{{- with .Values.agentk8sglue.basePodTemplate.nodeSelector}}
{{- with .Values.agentk8sglue.nodeSelector}}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.agentk8sglue.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.agentk8sglue.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
volumes:
- name: {{ include "clearml.name" . }}-pt
- name: {{ include "clearmlAgent.name" . }}-pt
configMap:
name: {{ include "clearml.name" . }}-pt
name: {{ include "clearmlAgent.name" . }}-pt
{{ if .Values.clearml.clearmlConfig }}
- name: k8sagent-clearml-conf-volume
secret:
secretName: {{ include "clearml.name" . }}-ac
secretName: {{ include "clearmlAgent.name" . }}-ac
items:
- key: clearml.conf
path: clearml.conf
@@ -186,5 +216,5 @@ spec:
{{ if .Values.agentk8sglue.fileMounts }}
- name: filemounts
secret:
secretName: {{ include "clearml.name" . }}-afm
secretName: {{ include "clearmlAgent.name" . }}-afm
{{- end }}

View File

@@ -2,7 +2,7 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ include "clearml.serviceAccountName" . }}
name: {{ include "clearmlAgent.serviceAccountName" . }}
namespace: {{ .Release.Namespace }}
{{- end }}
{{- if .Values.enterpriseFeatures.serviceAccountClusterAccess }}
@@ -10,7 +10,7 @@ metadata:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: {{ include "clearml.name" . }}-kpa
name: {{ include "clearmlAgent.name" . }}-kpa
rules:
- apiGroups:
- ""
@@ -24,25 +24,33 @@ rules:
resources:
- namespaces
verbs: ["list"]
{{- if .Values.agentk8sglue.taskAsJob }}
- apiGroups:
- batch
- extensions
resources:
- jobs
verbs: ["get", "list", "watch", "create", "patch", "delete"]
{{- end }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: {{ include "clearml.name" . }}-kpa
name: {{ include "clearmlAgent.name" . }}-kpa
subjects:
- kind: ServiceAccount
name: {{ include "clearml.serviceAccountName" . }}
name: {{ include "clearmlAgent.serviceAccountName" . }}
namespace: {{ .Release.Namespace }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: {{ include "clearml.name" . }}-kpa
name: {{ include "clearmlAgent.name" . }}-kpa
{{- else }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: {{ include "clearml.name" . }}-kpa
name: {{ include "clearmlAgent.name" . }}-kpa
rules:
- apiGroups:
- ""
@@ -56,17 +64,55 @@ rules:
resources:
- namespaces
verbs: ["list"]
{{- if .Values.agentk8sglue.taskAsJob }}
- apiGroups:
- batch
- extensions
resources:
- jobs
verbs: ["get", "list", "watch", "create", "patch", "delete"]
{{- end }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: {{ include "clearml.name" . }}-kpa
name: {{ include "clearmlAgent.name" . }}-kpa
subjects:
- kind: ServiceAccount
name: {{ include "clearml.serviceAccountName" . }}
name: {{ include "clearmlAgent.serviceAccountName" . }}
namespace: {{ .Release.Namespace }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: {{ include "clearml.name" . }}-kpa
name: {{ include "clearmlAgent.name" . }}-kpa
{{- end }}
{{- range .Values.agentk8sglue.additionalClusterRoleBindings }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: {{ include "clearmlAgent.name" $ }}-kpa-{{ . }}
subjects:
- kind: ServiceAccount
name: {{ include "clearmlAgent.serviceAccountName" $ }}
namespace: {{ $.Release.Namespace }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: {{ . }}
{{- end }}
{{- range .Values.agentk8sglue.additionalRoleBindings }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: {{ include "clearmlAgent.name" $ }}-kpa-{{ . }}
subjects:
- kind: ServiceAccount
name: {{ include "clearmlAgent.serviceAccountName" $ }}
namespace: {{ $.Release.Namespace }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: {{ . }}
{{- end }}

View File

@@ -1,7 +1,7 @@
apiVersion: v1
kind: Secret
metadata:
name: {{ include "clearml.name" . }}-ac
name: {{ include "clearmlAgent.name" . }}-ac
data:
agentk8sglue_key: {{ .Values.clearml.agentk8sglueKey | b64enc }}
agentk8sglue_secret: {{ .Values.clearml.agentk8sglueSecret | b64enc }}
@@ -12,7 +12,7 @@ data:
apiVersion: v1
kind: Secret
metadata:
name: {{ include "clearml.name" . }}-ark
name: {{ include "clearmlAgent.name" . }}-ark
type: kubernetes.io/dockerconfigjson
data:
.dockerconfigjson: {{ template "imagePullSecret" . }}

View File

@@ -2,7 +2,7 @@
apiVersion: v1
kind: Secret
metadata:
name: {{ include "clearml.name" . }}-afm
name: {{ include "clearmlAgent.name" . }}-afm
data:
{{- range .Values.agentk8sglue.fileMounts }}
{{ .name }}: {{ .fileContent | b64enc }}
@@ -14,7 +14,7 @@ data:
apiVersion: v1
kind: Secret
metadata:
name: {{ include "clearml.name" . }}-fm
name: {{ include "clearmlAgent.name" . }}-fm
data:
{{- range .Values.agentk8sglue.basePodTemplate.fileMounts }}
{{ .name }}: {{ .fileContent | b64enc }}
@@ -26,7 +26,7 @@ data:
apiVersion: v1
kind: Secret
metadata:
name: {{ include "clearml.name" $ }}-{{ $key }}-fm
name: {{ include "clearmlAgent.name" $ }}-{{ $key }}-fm
data:
{{- range .templateOverrides.fileMounts }}
{{ .name }}: {{ .fileContent | b64enc }}

View File

@@ -7,7 +7,7 @@ kind: Service
metadata:
name: clearml-session-{{ . }}
labels:
{{- include "clearml.labels" $ | nindent 4 }}
{{- include "clearmlAgent.labels" $ | nindent 4 }}
{{- with $.Values.sessions.svcAnnotations }}
annotations:
{{- toYaml . | nindent 4 }}

View File

@@ -61,6 +61,8 @@ agentk8sglue:
defaultContainerImage: ubuntu:18.04
# -- ClearML queue this agent will consume
queue: default
# -- ClearML spawn tasks as jobs instead of pods
taskAsJob: false
# -- Custom Bash script for the Glue Agent
# -- labels setup for Agent pod (example in values.yaml comments)
labels: {}
@@ -73,12 +75,25 @@ agentk8sglue:
containerCustomBashScript: ""
# -- Extra Environment variables for Glue Agent
extraEnvs: []
# - name: PYTHONPATH
# value: "somepath"
# - name: PYTHONPATH
# value: "somepath"
# -- Web Server pod security context
securityContext: {}
# runAsUser: 1001
# fsGroup: 1001
# -- additional existing ClusterRoleBindings
additionalClusterRoleBindings: []
# - privileged
# -- additional existing RoleBindings
additionalRoleBindings: []
# - privileged
# -- nodeSelector setup for Agent pod (example in values.yaml comments)
nodeSelector: {}
# fleet: agent-nodes
# -- tolerations setup for Agent pod (example in values.yaml comments)
tolerations: []
# -- affinity setup for Agent pod (example in values.yaml comments)
affinity: {}
# -- volumes definition for Glue Agent (example in values.yaml comments)
volumes: []
# - name: "yourvolume"
@@ -154,24 +169,29 @@ agentk8sglue:
# - name: CURL_CA_BUNDLE
# value: ""
# - name: PYTHONWARNINGS
# value: "=\"ignore:Unverified HTTPS request\""
# value: "ignore:Unverified HTTPS request"
# -- resources declaration for pods spawned to consume ClearML Task (example in values.yaml comments)
resources: {}
# limits:
# nvidia.com/gpu: 1
# -- priorityClassName setup for pods spawned to consume ClearML Task
priorityClassName: ""
# -- nodeSelector setup for pods spawned to consume ClearML Task (example in values.yaml comments)
nodeSelector: {}
# fleet: gpu-nodes
# -- tolerations setup for pods spawned to consume ClearML Task (example in values.yaml comments)
tolerations: []
# - key: "nvidia.com/gpu"
# operator: Exists
# effect: "NoSchedule"
# -- nodeSelector setup for pods spawned to consume ClearML Task (example in values.yaml comments)
nodeSelector: {}
# fleet: gpu-nodes
# -- affinity setup for pods spawned to consume ClearML Task
affinity: {}
# -- securityContext setup for pods spawned to consume ClearML Task (example in values.yaml comments)
securityContext: {}
# runAsUser: 1000
# runAsUser: 1001
# fsGroup: 1001
# -- hostAliases setup for pods spawned to consume ClearML Task (example in values.yaml comments)
hostAliases: {}
hostAliases: []
# - ip: "127.0.0.1"
# hostnames:
# - "foo.local"
@@ -200,6 +220,8 @@ sessions:
enterpriseFeatures:
# -- Enable/Disable Enterprise features
enabled: false
# -- Image tag override for enterprise version
agentImageTagOverride: "1.24-58"
# -- service account access every namespace flag
serviceAccountClusterAccess: false
# -- push env vars from Clear.ML Vault to task pods
@@ -216,6 +238,8 @@ enterpriseFeatures:
maxPods: 10
# -- Agent must use owner Token
useOwnerToken: true
# -- Create queues if they don't exist
createQueues: false
# -- ClearML queues and related template OVERRIDES used this agent will consume
queues:
# -- name of the queue will be used for this template

View File

@@ -2,9 +2,9 @@ apiVersion: v2
name: clearml
description: MLOps platform
type: application
version: "5.0.2"
appVersion: "1.9.0"
kubeVersion: ">= 1.21.0-0 < 1.26.0-0"
version: "5.8.2"
appVersion: "1.9.2"
kubeVersion: ">= 1.21.0-0 < 1.27.0-0"
home: https://clear.ml
icon: https://raw.githubusercontent.com/allegroai/clearml/master/docs/clearml-logo.svg
sources:
@@ -30,3 +30,7 @@ dependencies:
version: "7.16.2"
repository: "file://../../dependency_charts/elasticsearch"
condition: elasticsearch.enabled
annotations:
artifacthub.io/changes: |
- kind: fixed
description: app agent base image parameter

View File

@@ -1,6 +1,6 @@
# ClearML Ecosystem for Kubernetes
![Version: 5.0.2](https://img.shields.io/badge/Version-5.0.2-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.9.0](https://img.shields.io/badge/AppVersion-1.9.0-informational?style=flat-square)
![Version: 5.8.2](https://img.shields.io/badge/Version-5.8.2-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.9.2](https://img.shields.io/badge/AppVersion-1.9.2-informational?style=flat-square)
MLOps platform
@@ -130,7 +130,7 @@ For detailed instructions, see the [Optional Configuration](https://github.com/a
## Requirements
Kubernetes: `>= 1.21.0-0 < 1.26.0-0`
Kubernetes: `>= 1.21.0-0 < 1.27.0-0`
| Repository | Name | Version |
|------------|------|---------|
@@ -142,18 +142,21 @@ Kubernetes: `>= 1.21.0-0 < 1.26.0-0`
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| apiserver | object | `{"additionalConfigs":{},"affinity":{},"enabled":true,"extraEnvs":[],"image":{"pullPolicy":"IfNotPresent","repository":"allegroai/clearml","tag":"1.9.1-312"},"indexReplicas":0,"indexShards":1,"ingress":{"annotations":{},"enabled":false,"hostName":"api.clearml.127-0-0-1.nip.io","path":"/","tlsSecretName":""},"nodeSelector":{},"podAnnotations":{},"prepopulateEnabled":true,"processes":{"count":8,"maxRequests":1000,"maxRequestsJitter":300,"timeout":24000},"replicaCount":1,"resources":{"limits":{"cpu":"2000m","memory":"1Gi"},"requests":{"cpu":"100m","memory":"256Mi"}},"service":{"nodePort":30008,"port":8008,"type":"NodePort"},"tolerations":[]}` | Api Server configurations |
| apiserver.additionalConfigs | object | `{}` | files declared in this parameter will be mounted and read by apiserver (examples in values.yaml) |
| apiserver | object | `{"additionalConfigs":{},"affinity":{},"enabled":true,"existingAdditionalConfigsConfigMap":"","existingAdditionalConfigsSecret":"","extraEnvs":[],"image":{"pullPolicy":"IfNotPresent","repository":"allegroai/clearml","tag":"1.9.2-317"},"indexReplicas":0,"indexShards":1,"ingress":{"annotations":{},"enabled":false,"hostName":"api.clearml.127-0-0-1.nip.io","ingressClassName":"","path":"/","tlsSecretName":""},"nodeSelector":{},"podAnnotations":{},"prepopulateEnabled":true,"processes":{"count":8,"maxRequests":1000,"maxRequestsJitter":300,"timeout":24000},"replicaCount":1,"resources":{"limits":{"cpu":"2000m","memory":"1Gi"},"requests":{"cpu":"100m","memory":"256Mi"}},"securityContext":{},"service":{"nodePort":30008,"port":8008,"type":"NodePort"},"tolerations":[]}` | Api Server configurations |
| apiserver.additionalConfigs | object | `{}` | files declared in this parameter will be mounted and read by apiserver (examples in values.yaml) if not overridden by existingAdditionalConfigsSecret |
| apiserver.affinity | object | `{}` | Api Server affinity setup |
| apiserver.enabled | bool | `true` | Enable/Disable component deployment |
| apiserver.existingAdditionalConfigsConfigMap | string | `""` | reference for files declared in existing ConfigMap will be mounted and read by apiserver (examples in values.yaml) |
| apiserver.existingAdditionalConfigsSecret | string | `""` | reference for files declared in existing Secret will be mounted and read by apiserver (examples in values.yaml) if not overridden by existingAdditionalConfigsConfigMap |
| apiserver.extraEnvs | list | `[]` | Api Server extra envrinoment variables |
| apiserver.image | object | `{"pullPolicy":"IfNotPresent","repository":"allegroai/clearml","tag":"1.9.1-312"}` | Api Server image configuration |
| apiserver.image | object | `{"pullPolicy":"IfNotPresent","repository":"allegroai/clearml","tag":"1.9.2-317"}` | Api Server image configuration |
| apiserver.indexReplicas | int | `0` | Number of additional replicas in Elasticsearch indexes |
| apiserver.indexShards | int | `1` | Number of shards in Elasticsearch indexes |
| apiserver.ingress | object | `{"annotations":{},"enabled":false,"hostName":"api.clearml.127-0-0-1.nip.io","path":"/","tlsSecretName":""}` | Ingress configuration for Api Server component |
| apiserver.ingress | object | `{"annotations":{},"enabled":false,"hostName":"api.clearml.127-0-0-1.nip.io","ingressClassName":"","path":"/","tlsSecretName":""}` | Ingress configuration for Api Server component |
| apiserver.ingress.annotations | object | `{}` | Ingress annotations |
| apiserver.ingress.enabled | bool | `false` | Enable/Disable ingress |
| apiserver.ingress.hostName | string | `"api.clearml.127-0-0-1.nip.io"` | Ingress hostname domain |
| apiserver.ingress.ingressClassName | string | `""` | ClassName (must be defined if no default ingressClassName is available) |
| apiserver.ingress.path | string | `"/"` | Ingress root path url |
| apiserver.ingress.tlsSecretName | string | `""` | Reference to secret containing TLS certificate. If set, it enables HTTPS on ingress rule. |
| apiserver.nodeSelector | object | `{}` | Api Server nodeselector |
@@ -166,6 +169,7 @@ Kubernetes: `>= 1.21.0-0 < 1.26.0-0`
| apiserver.processes.timeout | int | `24000` | Api timeout (ms) |
| apiserver.replicaCount | int | `1` | Api Server number of pods |
| apiserver.resources | object | `{"limits":{"cpu":"2000m","memory":"1Gi"},"requests":{"cpu":"100m","memory":"256Mi"}}` | Api Server resources per pod; these are minimal requirements, it's suggested to increase these values in production environments |
| apiserver.securityContext | object | `{}` | Api Server pod security context |
| apiserver.service | object | `{"nodePort":30008,"port":8008,"type":"NodePort"}` | Api Server internal service configuration |
| apiserver.service.nodePort | int | `30008` | If service.type set to NodePort, this will be set to service's nodePort field. If service.type is set to others, this field will be ignored |
| apiserver.tolerations | list | `[]` | Api Server tolerations setup |
@@ -185,20 +189,22 @@ Kubernetes: `>= 1.21.0-0 < 1.26.0-0`
| clearml.testUserKey | string | `"ENP39EQM4SLACGD5FXB7"` | Test Server basic auth key |
| clearml.testUserSecret | string | `"lPcm0imbcBZ8mwgO7tpadutiS3gnJD05x9j7afwXPS35IKbpiQ"` | Test File Server basic auth secret |
| elasticsearch | object | `{"clusterHealthCheckParams":"wait_for_status=yellow&timeout=1s","clusterName":"clearml-elastic","enabled":true,"esConfig":{"elasticsearch.yml":"xpack.security.enabled: false\n"},"esJavaOpts":"-Xmx2g -Xms2g","extraEnvs":[{"name":"bootstrap.memory_lock","value":"false"},{"name":"cluster.routing.allocation.node_initial_primaries_recoveries","value":"500"},{"name":"cluster.routing.allocation.disk.watermark.low","value":"500mb"},{"name":"cluster.routing.allocation.disk.watermark.high","value":"500mb"},{"name":"cluster.routing.allocation.disk.watermark.flood_stage","value":"500mb"},{"name":"http.compression_level","value":"7"},{"name":"reindex.remote.whitelist","value":"*.*"},{"name":"xpack.monitoring.enabled","value":"false"},{"name":"xpack.security.enabled","value":"false"}],"httpPort":9200,"minimumMasterNodes":1,"persistence":{"enabled":true},"replicas":1,"resources":{"limits":{"cpu":"2000m","memory":"4Gi"},"requests":{"cpu":"100m","memory":"2Gi"}},"roles":{"data":"true","ingest":"true","master":"true","remote_cluster_client":"true"},"volumeClaimTemplate":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"50Gi"}},"storageClassName":null}}` | Configuration from https://github.com/elastic/helm-charts/blob/7.16/elasticsearch/values.yaml |
| enterpriseFeatures | object | `{"airGappedDocumentation":{"enabled":false,"image":{"repository":"","tag":""}},"clearmlApplications":{"affinity":{},"agentKey":"GK4PRTVT3706T25K6BA1","agentSecret":"ymLh1ok5k5xNUQfS944Xdx9xjf0wueokqKM2dMZfHuH9ayItG2","basePodImage":{"repository":"","tag":""},"enabled":true,"extraEnvs":[],"gitAgentPass":"git_password","gitAgentUser":"git_user","image":{"pullPolicy":"IfNotPresent","repository":"","tag":""},"nodeSelector":{},"podAnnotations":{},"replicaCount":1,"resources":{"limits":{"cpu":"2000m","memory":"1Gi"},"requests":{"cpu":"100m","memory":"256Mi"}},"tolerations":[]},"defaultCompanyGuid":"d1bd92a3b039400cbafc60a7a5b1e52b","enabled":false,"extraIndexUrl":"","overrideReferenceApiUrl":"","overrideReferenceFileUrl":""}` | Enterprise features (work only with an Enterprise license) |
| enterpriseFeatures.airGappedDocumentation | object | `{"enabled":false,"image":{"repository":"","tag":""}}` | Air gapped documentation configurations |
| enterpriseFeatures | object | `{"airGappedDocumentation":{"enabled":false,"image":{"repository":"","tag":"4"}},"apiserverImageTagOverride":"3.15.3-909","clearmlApplications":{"affinity":{},"agentKey":"GK4PRTVT3706T25K6BA1","agentSecret":"ymLh1ok5k5xNUQfS944Xdx9xjf0wueokqKM2dMZfHuH9ayItG2","basePodImage":{"repository":"","tag":"app-1.1.1-47"},"enabled":true,"extraEnvs":[],"fileMounts":[],"gitAgentPass":"git_password","gitAgentUser":"git_user","image":{"pullPolicy":"IfNotPresent","repository":"","tag":"1.24-58"},"nodeSelector":{},"podAnnotations":{},"replicaCount":1,"resources":{"limits":{"cpu":"2000m","memory":"1Gi"},"requests":{"cpu":"100m","memory":"256Mi"}},"tolerations":[]},"defaultCompanyGuid":"d1bd92a3b039400cbafc60a7a5b1e52b","enabled":false,"extraIndexUrl":"","fileserverImageTagOverride":"3.15.3-909","overrideReferenceApiUrl":"","overrideReferenceFileUrl":"","webserverImageTagOverride":"3.15.3-801"}` | Enterprise features (work only with an Enterprise license) |
| enterpriseFeatures.airGappedDocumentation | object | `{"enabled":false,"image":{"repository":"","tag":"4"}}` | Air gapped documentation configurations |
| enterpriseFeatures.airGappedDocumentation.enabled | bool | `false` | Enable/Disable air gapped documentation deployment |
| enterpriseFeatures.airGappedDocumentation.image | object | `{"repository":"","tag":""}` | Air gapped documentation image configuration |
| enterpriseFeatures.clearmlApplications | object | `{"affinity":{},"agentKey":"GK4PRTVT3706T25K6BA1","agentSecret":"ymLh1ok5k5xNUQfS944Xdx9xjf0wueokqKM2dMZfHuH9ayItG2","basePodImage":{"repository":"","tag":""},"enabled":true,"extraEnvs":[],"gitAgentPass":"git_password","gitAgentUser":"git_user","image":{"pullPolicy":"IfNotPresent","repository":"","tag":""},"nodeSelector":{},"podAnnotations":{},"replicaCount":1,"resources":{"limits":{"cpu":"2000m","memory":"1Gi"},"requests":{"cpu":"100m","memory":"256Mi"}},"tolerations":[]}` | APPS configurations |
| enterpriseFeatures.airGappedDocumentation.image | object | `{"repository":"","tag":"4"}` | Air gapped documentation image configuration |
| enterpriseFeatures.apiserverImageTagOverride | string | `"3.15.3-909"` | Image tag override for apiserver enterprise version |
| enterpriseFeatures.clearmlApplications | object | `{"affinity":{},"agentKey":"GK4PRTVT3706T25K6BA1","agentSecret":"ymLh1ok5k5xNUQfS944Xdx9xjf0wueokqKM2dMZfHuH9ayItG2","basePodImage":{"repository":"","tag":"app-1.1.1-47"},"enabled":true,"extraEnvs":[],"fileMounts":[],"gitAgentPass":"git_password","gitAgentUser":"git_user","image":{"pullPolicy":"IfNotPresent","repository":"","tag":"1.24-58"},"nodeSelector":{},"podAnnotations":{},"replicaCount":1,"resources":{"limits":{"cpu":"2000m","memory":"1Gi"},"requests":{"cpu":"100m","memory":"256Mi"}},"tolerations":[]}` | APPS configurations |
| enterpriseFeatures.clearmlApplications.affinity | object | `{}` | APPS affinity setup |
| enterpriseFeatures.clearmlApplications.agentKey | string | `"GK4PRTVT3706T25K6BA1"` | Apps Server basic auth key |
| enterpriseFeatures.clearmlApplications.agentSecret | string | `"ymLh1ok5k5xNUQfS944Xdx9xjf0wueokqKM2dMZfHuH9ayItG2"` | Apps Server basic auth secret |
| enterpriseFeatures.clearmlApplications.basePodImage | object | `{"repository":"","tag":""}` | APPS base spawning pods image |
| enterpriseFeatures.clearmlApplications.basePodImage | object | `{"repository":"","tag":"app-1.1.1-47"}` | APPS base spawning pods image |
| enterpriseFeatures.clearmlApplications.enabled | bool | `true` | Enable/Disable component deployment |
| enterpriseFeatures.clearmlApplications.extraEnvs | list | `[]` | APPS extra envrinoment variables |
| enterpriseFeatures.clearmlApplications.fileMounts | list | `[]` | file definition |
| enterpriseFeatures.clearmlApplications.gitAgentPass | string | `"git_password"` | Apps Server Git password |
| enterpriseFeatures.clearmlApplications.gitAgentUser | string | `"git_user"` | Apps Server Git user |
| enterpriseFeatures.clearmlApplications.image | object | `{"pullPolicy":"IfNotPresent","repository":"","tag":""}` | APPS image configuration |
| enterpriseFeatures.clearmlApplications.image | object | `{"pullPolicy":"IfNotPresent","repository":"","tag":"1.24-58"}` | APPS image configuration |
| enterpriseFeatures.clearmlApplications.nodeSelector | object | `{}` | APPS nodeselector |
| enterpriseFeatures.clearmlApplications.podAnnotations | object | `{}` | specific annotation for APPS pods |
| enterpriseFeatures.clearmlApplications.replicaCount | int | `1` | APPS number of pods |
@@ -207,34 +213,40 @@ Kubernetes: `>= 1.21.0-0 < 1.26.0-0`
| enterpriseFeatures.defaultCompanyGuid | string | `"d1bd92a3b039400cbafc60a7a5b1e52b"` | Company ID |
| enterpriseFeatures.enabled | bool | `false` | Enable/Disable Enterprise features |
| enterpriseFeatures.extraIndexUrl | string | `""` | extra index URL for Enterprise packages |
| enterpriseFeatures.fileserverImageTagOverride | string | `"3.15.3-909"` | Image tag override for fileserver enterprise version |
| enterpriseFeatures.overrideReferenceApiUrl | string | `""` | set this value AND overrideReferenceFileUrl if external endpoint exposure is in place (like a LoadBalancer) example: "https://api.clearml.local" |
| enterpriseFeatures.overrideReferenceFileUrl | string | `""` | set this value AND overrideReferenceAPIUrl if external endpoint exposure is in place (like a LoadBalancer) example: "https://files.clearml.local" |
| externalServices | object | `{"elasticsearchHost":"","elasticsearchPort":9200,"mongodbConnectionString":"","redisHost":"","redisPort":6379}` | Definition of external services to use if not enabled as dependency charts here |
| externalServices.elasticsearchHost | string | `""` | Existing ElasticSearch Hostname to use if elasticsearch.enabled is false |
| externalServices.elasticsearchPort | int | `9200` | Existing ElasticSearch Port to use if elasticsearch.enabled is false |
| externalServices.mongodbConnectionString | string | `""` | Existing MongoDB connection string to use if mongodb.enabled is false |
| enterpriseFeatures.webserverImageTagOverride | string | `"3.15.3-801"` | Image tag override for webserver enterprise version |
| externalServices | object | `{"elasticsearchConnectionString":"","mongodbConnectionStringAuth":"","mongodbConnectionStringBackend":"","redisHost":"","redisPort":6379}` | Definition of external services to use if not enabled as dependency charts here |
| externalServices.elasticsearchConnectionString | string | `""` | Existing ElasticSearch connectionstring if elasticsearch.enabled is false (example in values.yaml) |
| externalServices.mongodbConnectionStringAuth | string | `""` | Existing MongoDB connection string for BACKEND to use if mongodb.enabled is false |
| externalServices.mongodbConnectionStringBackend | string | `""` | Existing MongoDB connection string for AUTH to use if mongodb.enabled is false |
| externalServices.redisHost | string | `""` | Existing Redis Hostname to use if redis.enabled is false |
| externalServices.redisPort | int | `6379` | Existing Redis Port to use if redis.enabled is false |
| fileserver | object | `{"affinity":{},"enabled":true,"extraEnvs":[],"image":{"pullPolicy":"IfNotPresent","repository":"allegroai/clearml","tag":"1.9.1-312"},"ingress":{"annotations":{},"enabled":false,"hostName":"files.clearml.127-0-0-1.nip.io","path":"/","tlsSecretName":""},"nodeSelector":{},"podAnnotations":{},"replicaCount":1,"resources":{"limits":{"cpu":"2000m","memory":"1Gi"},"requests":{"cpu":"100m","memory":"256Mi"}},"service":{"nodePort":30081,"port":8081,"type":"NodePort"},"storage":{"data":{"accessMode":"ReadWriteOnce","class":"","size":"50Gi"}},"tolerations":[]}` | File Server configurations |
| fileserver | object | `{"affinity":{},"enabled":true,"extraEnvs":[],"image":{"pullPolicy":"IfNotPresent","repository":"allegroai/clearml","tag":"1.9.2-317"},"ingress":{"annotations":{},"enabled":false,"hostName":"files.clearml.127-0-0-1.nip.io","ingressClassName":"","path":"/","tlsSecretName":""},"nodeSelector":{},"podAnnotations":{},"replicaCount":1,"resources":{"limits":{"cpu":"2000m","memory":"1Gi"},"requests":{"cpu":"100m","memory":"256Mi"}},"securityContext":{},"service":{"nodePort":30081,"port":8081,"type":"NodePort"},"storage":{"data":{"accessMode":"ReadWriteOnce","class":"","existingPVC":"","size":"50Gi"},"enabled":true},"tolerations":[]}` | File Server configurations |
| fileserver.affinity | object | `{}` | File Server affinity setup |
| fileserver.enabled | bool | `true` | Enable/Disable component deployment |
| fileserver.extraEnvs | list | `[]` | File Server extra envrinoment variables |
| fileserver.image | object | `{"pullPolicy":"IfNotPresent","repository":"allegroai/clearml","tag":"1.9.1-312"}` | File Server image configuration |
| fileserver.ingress | object | `{"annotations":{},"enabled":false,"hostName":"files.clearml.127-0-0-1.nip.io","path":"/","tlsSecretName":""}` | Ingress configuration for File Server component |
| fileserver.image | object | `{"pullPolicy":"IfNotPresent","repository":"allegroai/clearml","tag":"1.9.2-317"}` | File Server image configuration |
| fileserver.ingress | object | `{"annotations":{},"enabled":false,"hostName":"files.clearml.127-0-0-1.nip.io","ingressClassName":"","path":"/","tlsSecretName":""}` | Ingress configuration for File Server component |
| fileserver.ingress.annotations | object | `{}` | Ingress annotations |
| fileserver.ingress.enabled | bool | `false` | Enable/Disable ingress |
| fileserver.ingress.hostName | string | `"files.clearml.127-0-0-1.nip.io"` | Ingress hostname domain |
| fileserver.ingress.ingressClassName | string | `""` | ClassName (must be defined if no default ingressClassName is available) |
| fileserver.ingress.path | string | `"/"` | Ingress root path url |
| fileserver.ingress.tlsSecretName | string | `""` | Reference to secret containing TLS certificate. If set, it enables HTTPS on ingress rule. |
| fileserver.nodeSelector | object | `{}` | File Server nodeselector |
| fileserver.podAnnotations | object | `{}` | specific annotation for File Server pods |
| fileserver.replicaCount | int | `1` | File Server number of pods |
| fileserver.resources | object | `{"limits":{"cpu":"2000m","memory":"1Gi"},"requests":{"cpu":"100m","memory":"256Mi"}}` | File Server resources per pod; these are minimal requirements, it's suggested to increase these values in production environments |
| fileserver.securityContext | object | `{}` | File Server pod security context |
| fileserver.service | object | `{"nodePort":30081,"port":8081,"type":"NodePort"}` | File Server internal service configuration |
| fileserver.service.nodePort | int | `30081` | If service.type set to NodePort, this will be set to service's nodePort field. If service.type is set to others, this field will be ignored |
| fileserver.storage | object | `{"data":{"accessMode":"ReadWriteOnce","class":"","size":"50Gi"}}` | File server persistence settings |
| fileserver.storage | object | `{"data":{"accessMode":"ReadWriteOnce","class":"","existingPVC":"","size":"50Gi"},"enabled":true}` | File server persistence settings |
| fileserver.storage.data.accessMode | string | `"ReadWriteOnce"` | Access mode (must be ReadWriteMany if fileserver replica > 1) |
| fileserver.storage.data.class | string | `""` | Storage class (use default if empty) |
| fileserver.storage.data.existingPVC | string | `""` | If set, it uses an already existing PVC instead of dynamic provisioning |
| fileserver.storage.enabled | bool | `true` | If set to false no PVC is created and emptyDir is used |
| fileserver.tolerations | list | `[]` | File Server tolerations setup |
| imageCredentials | object | `{"email":"someone@host.com","enabled":false,"existingSecret":"","password":"pwd","registry":"docker.io","username":"someone"}` | Container registry configuration |
| imageCredentials.email | string | `"someone@host.com"` | Email |
@@ -245,20 +257,22 @@ Kubernetes: `>= 1.21.0-0 < 1.26.0-0`
| imageCredentials.username | string | `"someone"` | Registry username |
| mongodb | object | `{"architecture":"standalone","auth":{"enabled":false},"enabled":true,"persistence":{"accessModes":["ReadWriteOnce"],"enabled":true,"size":"50Gi","storageClass":null},"replicaCount":1}` | Configuration from https://github.com/bitnami/charts/blob/master/bitnami/mongodb/values.yaml |
| redis | object | `{"cluster":{"enabled":false},"databaseNumber":0,"enabled":true,"master":{"name":"{{ .Release.Name }}-redis-master","persistence":{"accessModes":["ReadWriteOnce"],"enabled":true,"size":"5Gi","storageClass":null},"port":6379},"usePassword":false}` | Configuration from https://github.com/bitnami/charts/blob/master/bitnami/redis/values.yaml |
| webserver | object | `{"additionalConfigs":{},"affinity":{},"enabled":true,"extraEnvs":[],"image":{"pullPolicy":"IfNotPresent","repository":"allegroai/clearml","tag":"1.9.1-312"},"ingress":{"annotations":{},"enabled":false,"hostName":"app.clearml.127-0-0-1.nip.io","path":"/","tlsSecretName":""},"nodeSelector":{},"podAnnotations":{},"replicaCount":1,"resources":{"limits":{"cpu":"2000m","memory":"1Gi"},"requests":{"cpu":"100m","memory":"256Mi"}},"service":{"nodePort":30080,"port":8080,"type":"NodePort"},"tolerations":[]}` | Web Server configurations |
| webserver | object | `{"additionalConfigs":{},"affinity":{},"enabled":true,"extraEnvs":[],"image":{"pullPolicy":"IfNotPresent","repository":"allegroai/clearml","tag":"1.9.2-317"},"ingress":{"annotations":{},"enabled":false,"hostName":"app.clearml.127-0-0-1.nip.io","ingressClassName":"","path":"/","tlsSecretName":""},"nodeSelector":{},"podAnnotations":{},"podSecurityContext":{},"replicaCount":1,"resources":{"limits":{"cpu":"2000m","memory":"1Gi"},"requests":{"cpu":"100m","memory":"256Mi"}},"service":{"nodePort":30080,"port":8080,"type":"NodePort"},"tolerations":[]}` | Web Server configurations |
| webserver.additionalConfigs | object | `{}` | Additional specific webserver configurations |
| webserver.affinity | object | `{}` | Web Server affinity setup |
| webserver.enabled | bool | `true` | Enable/Disable component deployment |
| webserver.extraEnvs | list | `[]` | Web Server extra envrinoment variables |
| webserver.image | object | `{"pullPolicy":"IfNotPresent","repository":"allegroai/clearml","tag":"1.9.1-312"}` | Web Server image configuration |
| webserver.ingress | object | `{"annotations":{},"enabled":false,"hostName":"app.clearml.127-0-0-1.nip.io","path":"/","tlsSecretName":""}` | Ingress configuration for Web Server component |
| webserver.image | object | `{"pullPolicy":"IfNotPresent","repository":"allegroai/clearml","tag":"1.9.2-317"}` | Web Server image configuration |
| webserver.ingress | object | `{"annotations":{},"enabled":false,"hostName":"app.clearml.127-0-0-1.nip.io","ingressClassName":"","path":"/","tlsSecretName":""}` | Ingress configuration for Web Server component |
| webserver.ingress.annotations | object | `{}` | Ingress annotations |
| webserver.ingress.enabled | bool | `false` | Enable/Disable ingress |
| webserver.ingress.hostName | string | `"app.clearml.127-0-0-1.nip.io"` | Ingress hostname domain |
| webserver.ingress.ingressClassName | string | `""` | ClassName (must be defined if no default ingressClassName is available) |
| webserver.ingress.path | string | `"/"` | Ingress root path url |
| webserver.ingress.tlsSecretName | string | `""` | Reference to secret containing TLS certificate. If set, it enables HTTPS on ingress rule. |
| webserver.nodeSelector | object | `{}` | Web Server nodeselector |
| webserver.podAnnotations | object | `{}` | specific annotation for Web Server pods |
| webserver.podSecurityContext | object | `{}` | Web Server pod security context |
| webserver.replicaCount | int | `1` | Web Server number of pods |
| webserver.resources | object | `{"limits":{"cpu":"2000m","memory":"1Gi"},"requests":{"cpu":"100m","memory":"256Mi"}}` | Web Server resources per pod; these are minimal requirements, it's suggested to increase these values in production environments |
| webserver.service | object | `{"nodePort":30080,"port":8080,"type":"NodePort"}` | Web Server internal service configuration |

View File

@@ -141,21 +141,24 @@ Create readiness probe auth token
Elasticsearch Service name
*/}}
{{- define "elasticsearch.servicename" -}}
{{- if .Values.elasticsearch.enabled }}
{{- .Values.elasticsearch.clusterName }}-master
{{- else }}
{{- .Values.externalServices.elasticsearchHost }}
{{- end }}
{{- end }}
{{/*
Elasticsearch Service port
*/}}
{{- define "elasticsearch.serviceport" -}}
{{- if .Values.elasticsearch.enabled }}
{{- .Values.elasticsearch.httpPort }}
{{- end }}
{{/*
Elasticsearch Comnnection string
*/}}
{{- define "elasticsearch.connectionstring" -}}
{{- if .Values.elasticsearch.enabled }}
{{- printf "[{\"host\":\"%s\",\"port\":%s}]" (include "elasticsearch.servicename" .) (include "elasticsearch.serviceport" .) | quote }}
{{- else }}
{{- .Values.externalServices.elasticsearchPort }}
{{- .Values.externalServices.elasticsearchConnectionString | quote }}
{{- end }}
{{- end }}
@@ -163,7 +166,6 @@ Elasticsearch Service port
MongoDB Comnnection string
*/}}
{{- define "mongodb.connectionstring" -}}
{{- if .Values.mongodb.enabled }}
{{- if eq .Values.mongodb.architecture "standalone" }}
{{- printf "%s%s%s" "mongodb://" .Release.Name "-mongodb:27017" }}
{{- else }}
@@ -173,8 +175,16 @@ MongoDB Comnnection string
{{- end }}
{{- printf "%s" ( trimSuffix "," $connectionString ) }}
{{- end }}
{{- end }}
{{/*
MongoDB hotname
*/}}
{{- define "mongodb.hostname" -}}
{{- if eq .Values.mongodb.architecture "standalone" }}
{{- printf "%s" "mongodb" }}
{{- else }}
{{- .Values.externalServices.mongodbConnectionString }}
{{- printf "%s" "mongodb-headless" }}
{{- end }}
{{- end }}
@@ -206,11 +216,11 @@ clientConfiguration string compose
{{- define "clearml.clientConfiguration" -}}
{{- $clientConfiguration := "" }}
{{- if and (.Values.clearml.clientConfigurationApiUrl) .Values.clearml.clientConfigurationFilesUrl }}
{{- $clientConfiguration = "{\"apiServer\":\"{{ .Values.clearml.clientConfigurationApiUrl }}\",\"filesServer\":\"{{ .Values.clearml.clientConfigurationFilesUrl }}\"}" }}
{{- $clientConfiguration = printf "%s%s%s%s%s" "{\"apiServer\":\"" .Values.clearml.clientConfigurationApiUrl "\",\"filesServer\":\"" .Values.clearml.clientConfigurationFilesUrl "\"}" }}
{{- else if .Values.clearml.clientConfigurationApiUrl }}
{{- $clientConfiguration = "{\"apiServer\":\"{{ .Values.clearml.clientConfigurationApiUrl }}\"}" }}
{{- $clientConfiguration = printf "%s%s%s" "{\"apiServer\":\"" .Values.clearml.clientConfigurationApiUrl "\"}" }}
{{- else if .Values.clearml.clientConfigurationFilesUrl }}
{{- $clientConfiguration = "{\"filesServer\":\"{{ .Values.clearml.clientConfigurationFilesUrl }}\"}" }}
{{- $clientConfiguration = printf "%s%s%s" "{\"filesServer\":\"" .Values.clearml.clientConfigurationFilesUrl "\"}" }}
{{- end }}
{{- $clientConfiguration }}
{{- end }}

View File

@@ -22,44 +22,86 @@ spec:
{{- if .Values.imageCredentials.enabled }}
imagePullSecrets:
{{- if .Values.imageCredentials.existingSecret }}
- name: .Values.imageCredentials.existingSecret
- name: {{ .Values.imageCredentials.existingSecret }}
{{- else }}
- name: clearml-registry-key
{{- end }}
{{- end }}
{{- if .Values.apiserver.additionalConfigs }}
volumes:
{{- if or .Values.apiserver.additionalConfigs .Values.apiserver.existingAdditionalConfigsConfigMap .Values.apiserver.existingAdditionalConfigsSecret }}
volumes:
- name: apiserver-config
{{- if or .Values.apiserver.existingAdditionalConfigsConfigMap }}
configMap:
name: {{ .Values.apiserver.existingAdditionalConfigsConfigMap }}
{{- else if or .Values.apiserver.existingAdditionalConfigsSecret }}
secret:
secretName: {{ .Values.apiserver.existingAdditionalConfigsSecret }}
{{- else if or .Values.apiserver.additionalConfigs }}
configMap:
name: "{{ include "apiserver.referenceName" . }}-configmap"
{{- end }}
{{- end }}
securityContext: {{ toYaml .Values.apiserver.podSecurityContext | nindent 8 }}
initContainers:
- name: init-apiserver
image: "{{ .Values.apiserver.image.repository }}:{{ .Values.apiserver.image.tag | default .Chart.AppVersion }}"
{{- if .Values.enterpriseFeatures.enabled }}
image: "{{ .Values.apiserver.image.repository }}:{{ .Values.enterpriseFeatures.apiserverImageTagOverride }}"
{{- else }}
image: "{{ .Values.apiserver.image.repository }}:{{ .Values.apiserver.image.tag }}"
{{- end }}
command:
- /bin/sh
- -c
- >
set -x;
{{- if .Values.elasticsearch.enabled }}
while [ $(curl -sw '%{http_code}' "http://{{ include "elasticsearch.servicename" . }}:{{ include "elasticsearch.serviceport" . }}/_cluster/health" -o /dev/null) -ne 200 ] ; do
echo "waiting for elasticsearch" ;
sleep 5 ;
done
done ;
{{- end }}
{{- if .Values.mongodb.enabled }}
while [ $(curl --telnet-option BOGUS --connect-timeout 2 -s "telnet://{{ .Release.Name }}-{{ include "mongodb.hostname" . }}:27017" -o /dev/null; echo $?) -ne 49 ] ; do
echo "waiting for mongodb" ;
sleep 5 ;
done ;
{{- end }}
{{- if .Values.redis.enabled }}
while [ $(curl --telnet-option BOGUS --connect-timeout 2 -s "telnet://{{ include "redis.servicename" . }}:{{ include "redis.serviceport" . }}" -o /dev/null; echo $?) -ne 49 ] ; do
echo "waiting for redis" ;
sleep 5 ;
done ;
{{- end }}
containers:
- name: clearml-apiserver
image: "{{ .Values.apiserver.image.repository }}:{{ .Values.apiserver.image.tag | default .Chart.AppVersion }}"
{{- if .Values.enterpriseFeatures.enabled }}
image: "{{ .Values.apiserver.image.repository }}:{{ .Values.enterpriseFeatures.apiserverImageTagOverride }}"
{{- else }}
image: "{{ .Values.apiserver.image.repository }}:{{ .Values.apiserver.image.tag }}"
{{- end }}
imagePullPolicy: {{ .Values.apiserver.image.pullPolicy }}
ports:
- name: http
containerPort: 8008
protocol: TCP
env:
- name: CLEARML_ELASTIC_SERVICE_HOST
value: {{ include "elasticsearch.servicename" . }}
- name: CLEARML_ELASTIC_SERVICE_PORT
value: "{{ include "elasticsearch.serviceport" . }}"
- name: CLEARML__HOSTS__ELASTIC__WORKERS__HOSTS
value: {{ include "elasticsearch.connectionstring" . }}
- name: CLEARML__HOSTS__ELASTIC__EVENTS__HOSTS
value: {{ include "elasticsearch.connectionstring" . }}
- name: CLEARML__HOSTS__ELASTIC__DATASETS__HOSTS
value: {{ include "elasticsearch.connectionstring" . }}
- name: CLEARML__HOSTS__ELASTIC__LOGS__HOSTS
value: {{ include "elasticsearch.connectionstring" . }}
{{- if .Values.mongodb.enabled }}
- name: CLEARML_MONGODB_SERVICE_CONNECTION_STRING
value: {{ include "mongodb.connectionstring" . | quote }}
{{- else }}
- name: CLEARML__HOSTS__MONGO__BACKEND__HOST
value: {{ .Values.externalServices.mongodbConnectionStringBackend | quote }}
- name: CLEARML__HOSTS__MONGO__AUTH__HOST
value: {{ .Values.externalServices.mongodbConnectionStringAuth | quote }}
{{- end }}
- name: CLEARML_REDIS_SERVICE_HOST
value: {{ include "redis.servicename" . }}
- name: CLEARML_REDIS_SERVICE_PORT
@@ -68,9 +110,9 @@ spec:
value: /opt/clearml/config
- name: CLEARML__apiserver__default_company_name
value: "{{ .Values.clearml.defaultCompany }}"
{{- if not (eq .Values.clearml.cookieDomain "") }}
- name: CLEARML__APISERVER__AUTH__SESSION_AUTH_COOKIE_NAME
value: {{ .Values.clearml.cookieName }}
{{- if .Values.clearml.cookieDomain }}
- name: CLEARML__APISERVER__AUTH__COOKIES__DOMAIN
value: ".{{ .Values.clearml.cookieDomain }}"
{{- end }}
@@ -100,8 +142,6 @@ spec:
value: "{{ .Values.enterpriseFeatures.defaultCompanyGuid }}"
- name: APPLY_ES_MAPPINGS
value: "false"
- name: CLEARML__HOSTS__ELASTIC__LOGS__HOSTS
value: "[\"http://{{ include "elasticsearch.servicename" . }}:{{ include "elasticsearch.serviceport" . }}\"]"
- name: NUMBER_OF_GUNICORN_WORKERS
value: "{{ .Values.apiserver.processes.count }}"
- name: GUNICORN_TIMEOUT
@@ -199,7 +239,7 @@ spec:
httpHeaders:
- name: Authorization
value: Basic {{ include "readinessProbeAuth" . }}
{{- if .Values.apiserver.additionalConfigs }}
{{- if or .Values.apiserver.additionalConfigs .Values.apiserver.existingAdditionalConfigsConfigMap .Values.apiserver.existingAdditionalConfigsSecret }}
volumeMounts:
- name: apiserver-config
{{- if .Values.enterpriseFeatures.enabled }}

View File

@@ -20,6 +20,9 @@ metadata:
{{- toYaml $annotations | nindent 4 }}
spec:
{{- if .Values.apiserver.ingress.ingressClassName }}
ingressClassName: {{ .Values.apiserver.ingress.ingressClassName }}
{{- end }}
{{- if .Values.apiserver.ingress.tlsSecretName }}
tls:
- hosts:

View File

@@ -0,0 +1,47 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "clearml.name" . }}-apps-pt
data:
template.yaml: |
apps_queue:
apiVersion: v1
metadata:
namespace: {{ $.Release.Namespace }}
spec:
{{- if $.Values.imageCredentials.enabled }}
imagePullSecrets:
{{- if $.Values.imageCredentials.existingSecret }}
- name: {{ $.Values.imageCredentials.existingSecret }}
{{- else }}
- name: clearml-registry-key
{{- end }}
{{- end }}
serviceAccountName: "clearml-apps-sa"
volumes:
{{- if .Values.enterpriseFeatures.clearmlApplications.fileMounts }}
- name: filemounts
secret:
secretName: {{ include "clearml.name" . }}-apps-fm
{{- end }}
containers:
- resources:
ports:
- containerPort: 10022
volumeMounts:
{{- range .Values.enterpriseFeatures.clearmlApplications.fileMounts }}
- name: filemounts
mountPath: "{{ .folderPath }}/{{ .name }}"
subPath: "{{ .name }}"
readOnly: true
{{- end }}
env:
- name: CLEARML_API_HOST
value: "http://{{ include "apiserver.referenceName" . }}:{{ .Values.apiserver.service.port }}"
- name: CLEARML_FILES_HOST
value: "http://{{ include "fileserver.referenceName" . }}:{{ .Values.fileserver.service.port }}"
- name: CLEARML_WEB_HOST
value: "http://{{ include "webserver.referenceName" . }}:{{ .Values.webserver.service.port }}"
{{- if .Values.enterpriseFeatures.clearmlApplications.extraEnvs }}
{{ toYaml .Values.enterpriseFeatures.clearmlApplications.extraEnvs | nindent 10 }}
{{- end }}

View File

@@ -23,17 +23,25 @@ spec:
{{- if .Values.imageCredentials.enabled }}
imagePullSecrets:
{{- if .Values.imageCredentials.existingSecret }}
- name: .Values.imageCredentials.existingSecret
- name: {{ .Values.imageCredentials.existingSecret }}
{{- else }}
- name: clearml-registry-key
{{- end }}
{{- end }}
volumes:
- name: {{ include "clearml.name" . }}-apps-pt
configMap:
name: {{ include "clearml.name" . }}-apps-pt
{{- if .Values.enterpriseFeatures.clearmlApplications.additionalConfigs }}
volumes:
- name: apps-config
configMap:
name: "{{ include "clearmlApplications.referenceName" . }}-configmap"
{{- end }}
{{- if .Values.enterpriseFeatures.clearmlApplications.fileMounts }}
- name: filemounts
secret:
secretName: {{ include "clearml.name" . }}-apps-fm
{{- end }}
serviceAccountName: "clearml-apps-sa"
initContainers:
- name: init-apps
@@ -62,14 +70,15 @@ spec:
value: "http://{{ include "fileserver.referenceName" . }}:{{ .Values.fileserver.service.port }}"
- name: CLEARML_WEB_HOST
value: "http://{{ include "webserver.referenceName" . }}:{{ .Values.webserver.service.port }}"
- name: CLEARML_AGENT_DEFAULT_BASE_DOCKER
- name: CLEARML_DOCKER_IMAGE
value: "{{ .Values.enterpriseFeatures.clearmlApplications.basePodImage.repository }}:{{ .Values.enterpriseFeatures.clearmlApplications.basePodImage.tag }}"
- name: CLEARML_WORKER_ID
value: "apps-agent-1"
- name: CLEARML_NO_DEFAULT_SERVER
value: "true"
- name: CLEARML_AGENT_DAEMON_OPTIONS
value: "--foreground --create-queue --use-owner-token --child-report-tags application --services-mode=5"
- name: K8S_GLUE_EXTRA_ARGS
value: "--namespace {{ .Release.Namespace }} --template-yaml /root/template/template.yaml \
--child-report-tags application --max-pods 5 --use-owner-token"
- name: K8S_GLUE_QUEUE
value: "apps_queue"
- name: CLEARML_AGENT_DISABLE_SSH_MOUNT
@@ -97,11 +106,19 @@ spec:
{{- if .Values.enterpriseFeatures.clearmlApplications.extraEnvs }}
{{ toYaml .Values.enterpriseFeatures.clearmlApplications.extraEnvs | nindent 10 }}
{{- end }}
volumeMounts:
- name: {{ include "clearml.name" . }}-apps-pt
mountPath: /root/template
{{- if .Values.enterpriseFeatures.clearmlApplications.additionalConfigs }}
volumeMounts:
- name: apps-config
mountPath: /opt/clearml/config/default
{{- end }}
{{- range .Values.enterpriseFeatures.clearmlApplications.fileMounts }}
- name: filemounts
mountPath: "{{ .folderPath }}/{{ .name }}"
subPath: "{{ .name }}"
readOnly: true
{{- end }}
resources:
{{- toYaml .Values.enterpriseFeatures.clearmlApplications.resources | nindent 12 }}
{{- with .Values.enterpriseFeatures.clearmlApplications.nodeSelector }}

View File

@@ -7,31 +7,6 @@ metadata:
namespace: {{ .Release.Namespace }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: {{ include "clearmlApplications.referenceName" . }}-kpa
rules:
- apiGroups:
- ""
resources:
- pods
verbs: ["get", "list", "watch", "create", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: {{ include "clearmlApplications.referenceName" . }}-kpa
subjects:
- kind: ServiceAccount
name: "clearml-apps-sa"
namespace: {{ .Release.Namespace }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: {{ include "clearmlApplications.referenceName" . }}-kpa
{{- else }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: {{ include "clearmlApplications.referenceName" . }}-kpa

View File

@@ -0,0 +1,10 @@
{{ if .Values.enterpriseFeatures.clearmlApplications.fileMounts }}
apiVersion: v1
kind: Secret
metadata:
name: {{ include "clearml.name" . }}-apps-fm
data:
{{- range .Values.enterpriseFeatures.clearmlApplications.fileMounts }}
{{ .name }}: {{ .fileContent | b64enc }}
{{- end }}
{{ end }}

View File

@@ -22,18 +22,34 @@ spec:
{{- if .Values.imageCredentials.enabled }}
imagePullSecrets:
{{- if .Values.imageCredentials.existingSecret }}
- name: .Values.imageCredentials.existingSecret
- name: {{ .Values.imageCredentials.existingSecret }}
{{- else }}
- name: clearml-registry-key
{{- end }}
{{- end }}
volumes:
{{- if .Values.fileserver.storage.enabled }}
{{- if .Values.fileserver.storage.data.existingPVC }}
- name: fileserver-data
persistentVolumeClaim:
claimName: {{ .Values.fileserver.storage.data.existingPVC | quote }}
{{- else }}
- name: fileserver-data
persistentVolumeClaim:
claimName: {{ include "fileserver.referenceName" . }}-data
{{- end }}
{{- else }}
- name: fileserver-data
emptyDir: {}
{{- end }}
securityContext: {{ toYaml .Values.fileserver.podSecurityContext | nindent 8 }}
initContainers:
- name: init-fileserver
image: "{{ .Values.fileserver.image.repository }}:{{ .Values.fileserver.image.tag | default .Chart.AppVersion }}"
{{- if .Values.enterpriseFeatures.enabled }}
image: "{{ .Values.fileserver.image.repository }}:{{ .Values.enterpriseFeatures.fileserverImageTagOverride }}"
{{- else }}
image: "{{ .Values.fileserver.image.repository }}:{{ .Values.fileserver.image.tag }}"
{{- end }}
command:
- /bin/sh
- -c
@@ -45,7 +61,11 @@ spec:
done
containers:
- name: clearml-fileserver
image: "{{ .Values.fileserver.image.repository }}:{{ .Values.fileserver.image.tag | default .Chart.AppVersion }}"
{{- if .Values.enterpriseFeatures.enabled }}
image: "{{ .Values.fileserver.image.repository }}:{{ .Values.enterpriseFeatures.fileserverImageTagOverride }}"
{{- else }}
image: "{{ .Values.fileserver.image.repository }}:{{ .Values.fileserver.image.tag }}"
{{- end }}
imagePullPolicy: {{ .Values.fileserver.image.pullPolicy }}
ports:
- name: http

View File

@@ -19,6 +19,9 @@ metadata:
annotations:
{{- toYaml $annotations | nindent 4 }}
spec:
{{- if .Values.fileserver.ingress.ingressClassName }}
ingressClassName: {{ .Values.fileserver.ingress.ingressClassName }}
{{- end }}
{{- if .Values.fileserver.ingress.tlsSecretName }}
tls:
- hosts:

View File

@@ -1,4 +1,6 @@
{{- if .Values.fileserver.enabled }}
{{- if .Values.fileserver.storage.enabled }}
{{- if not .Values.fileserver.storage.data.existingPVC }}
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
@@ -15,3 +17,5 @@ spec:
storageClassName: {{ .Values.fileserver.storage.data.class | quote }}
{{- end -}}
{{- end }}
{{- end }}
{{- end }}

View File

@@ -17,7 +17,7 @@ data:
},
"docsLink": "https://clear.ml/docs/",
"applicationsBackground": "ui-assets/apps-message.svg"
{{- if and .Values.webserver.overrideReferenceApiUrl .Values.enterpriseFeatures.overrideReferenceFileUrl }}
{{- if and .Values.enterpriseFeatures.overrideReferenceApiUrl .Values.enterpriseFeatures.overrideReferenceFileUrl }}
,
"fileBaseUrl": "{{ .Values.enterpriseFeatures.overrideReferenceFileUrl }}",
"apiBaseUrl": "{{ .Values.enterpriseFeatures.overrideReferenceApiUrl }}"

View File

@@ -22,7 +22,7 @@ spec:
{{- if .Values.imageCredentials.enabled }}
imagePullSecrets:
{{- if .Values.imageCredentials.existingSecret }}
- name: .Values.imageCredentials.existingSecret
- name: {{ .Values.imageCredentials.existingSecret }}
{{- else }}
- name: clearml-registry-key
{{- end }}
@@ -35,6 +35,7 @@ spec:
- name: documentation
emptyDir: {}
{{- end }}
securityContext: {{ toYaml .Values.webserver.podSecurityContext | nindent 8 }}
initContainers:
{{- if .Values.enterpriseFeatures.airGappedDocumentation.enabled }}
- name: init-airgap-docs
@@ -52,7 +53,11 @@ spec:
{{- end }}
{{- end }}
- name: init-webserver
image: "{{ .Values.webserver.image.repository }}:{{ .Values.webserver.image.tag | default .Chart.AppVersion }}"
{{- if .Values.enterpriseFeatures.enabled }}
image: "{{ .Values.webserver.image.repository }}:{{ .Values.enterpriseFeatures.webserverImageTagOverride }}"
{{- else }}
image: "{{ .Values.webserver.image.repository }}:{{ .Values.webserver.image.tag }}"
{{- end }}
command:
- /bin/sh
- -c
@@ -64,7 +69,11 @@ spec:
done
containers:
- name: clearml-webserver
image: "{{ .Values.webserver.image.repository }}:{{ .Values.webserver.image.tag | default .Chart.AppVersion }}"
{{- if .Values.enterpriseFeatures.enabled }}
image: "{{ .Values.webserver.image.repository }}:{{ .Values.enterpriseFeatures.webserverImageTagOverride }}"
{{- else }}
image: "{{ .Values.webserver.image.repository }}:{{ .Values.webserver.image.tag }}"
{{- end }}
imagePullPolicy: {{ .Values.webserver.image.pullPolicy }}
ports:
- name: http

View File

@@ -19,6 +19,9 @@ metadata:
annotations:
{{- toYaml $annotations | nindent 4 }}
spec:
{{- if .Values.webserver.ingress.ingressClassName }}
ingressClassName: {{ .Values.webserver.ingress.ingressClassName }}
{{- end }}
{{- if .Values.webserver.ingress.tlsSecretName }}
tls:
- hosts:

View File

@@ -16,6 +16,28 @@ webserver:
ingress:
enabled: true
hostName: "app.clearml.127-0-0-1.nip.io"
redis:
master:
name: "{{ .Release.Name }}-redis"
persistence:
enabled: true
accessModes:
- ReadWriteOnce
size: 5Gi
## If undefined (the default) or set to null, no storageClassName spec is set, choosing the default provisioner
storageClass: null
slave:
persistence:
enabled: true
accessModes:
- ReadWriteOnce
size: 5Gi
## If undefined (the default) or set to null, no storageClassName spec is set, choosing the default provisioner
storageClass: null
cluster:
enabled: true
sentinel:
enabled: true
mongodb:
enabled: true
architecture: replicaset

View File

@@ -54,7 +54,7 @@ apiserver:
image:
repository: "allegroai/clearml"
pullPolicy: IfNotPresent
tag: "1.9.1-312"
tag: "1.9.2-317"
# -- Api Server internal service configuration
service:
type: NodePort
@@ -68,6 +68,8 @@ apiserver:
ingress:
# -- Enable/Disable ingress
enabled: false
# -- ClassName (must be defined if no default ingressClassName is available)
ingressClassName: ""
# -- Ingress hostname domain
hostName: "api.clearml.127-0-0-1.nip.io"
# -- Reference to secret containing TLS certificate. If set, it enables HTTPS on ingress rule.
@@ -109,7 +111,15 @@ apiserver:
tolerations: []
# -- Api Server affinity setup
affinity: {}
# -- files declared in this parameter will be mounted and read by apiserver (examples in values.yaml)
# -- Api Server pod security context
securityContext: {}
# runAsUser: 1001
# fsGroup: 1001
# -- reference for files declared in existing ConfigMap will be mounted and read by apiserver (examples in values.yaml)
existingAdditionalConfigsConfigMap: ""
# -- reference for files declared in existing Secret will be mounted and read by apiserver (examples in values.yaml) if not overridden by existingAdditionalConfigsConfigMap
existingAdditionalConfigsSecret: ""
# -- files declared in this parameter will be mounted and read by apiserver (examples in values.yaml) if not overridden by existingAdditionalConfigsSecret
additionalConfigs: {}
# services.conf: |
# tasks {
@@ -148,7 +158,7 @@ fileserver:
image:
repository: "allegroai/clearml"
pullPolicy: IfNotPresent
tag: "1.9.1-312"
tag: "1.9.2-317"
# -- File Server internal service configuration
service:
type: NodePort
@@ -162,6 +172,8 @@ fileserver:
ingress:
# -- Enable/Disable ingress
enabled: false
# -- ClassName (must be defined if no default ingressClassName is available)
ingressClassName: ""
# -- Ingress hostname domain
hostName: "files.clearml.127-0-0-1.nip.io"
# -- Reference to secret containing TLS certificate. If set, it enables HTTPS on ingress rule.
@@ -189,9 +201,17 @@ fileserver:
tolerations: []
# -- File Server affinity setup
affinity: {}
# -- File Server pod security context
securityContext: {}
# runAsUser: 1001
# fsGroup: 1001
# -- File server persistence settings
storage:
# -- If set to false no PVC is created and emptyDir is used
enabled: true
data:
# -- If set, it uses an already existing PVC instead of dynamic provisioning
existingPVC: ""
# -- Storage class (use default if empty)
class: ""
# -- Access mode (must be ReadWriteMany if fileserver replica > 1)
@@ -206,7 +226,7 @@ webserver:
image:
repository: "allegroai/clearml"
pullPolicy: IfNotPresent
tag: "1.9.1-312"
tag: "1.9.2-317"
# -- Web Server internal service configuration
service:
type: NodePort
@@ -220,6 +240,8 @@ webserver:
ingress:
# -- Enable/Disable ingress
enabled: false
# -- ClassName (must be defined if no default ingressClassName is available)
ingressClassName: ""
# -- Ingress hostname domain
hostName: "app.clearml.127-0-0-1.nip.io"
# -- Reference to secret containing TLS certificate. If set, it enables HTTPS on ingress rule.
@@ -247,17 +269,22 @@ webserver:
tolerations: []
# -- Web Server affinity setup
affinity: {}
# -- Web Server pod security context
podSecurityContext: {}
# runAsUser: 1001
# fsGroup: 1001
# -- Additional specific webserver configurations
additionalConfigs: {}
# -- Definition of external services to use if not enabled as dependency charts here
externalServices:
# -- Existing ElasticSearch Hostname to use if elasticsearch.enabled is false
elasticsearchHost: ""
# -- Existing ElasticSearch Port to use if elasticsearch.enabled is false
elasticsearchPort: 9200
# -- Existing MongoDB connection string to use if mongodb.enabled is false
mongodbConnectionString: ""
# -- Existing ElasticSearch connectionstring if elasticsearch.enabled is false (example in values.yaml)
elasticsearchConnectionString: ""
# [{"host":"hostname1","port":9200},{"host":"hostname2","port":9200},{"host":"hostname3","port":9200}]
# -- Existing MongoDB connection string for BACKEND to use if mongodb.enabled is false
mongodbConnectionStringAuth: ""
# -- Existing MongoDB connection string for AUTH to use if mongodb.enabled is false
mongodbConnectionStringBackend: ""
# -- Existing Redis Hostname to use if redis.enabled is false
redisHost: ""
# -- Existing Redis Port to use if redis.enabled is false
@@ -357,6 +384,12 @@ enterpriseFeatures:
enabled: false
# -- Company ID
defaultCompanyGuid: "d1bd92a3b039400cbafc60a7a5b1e52b"
# -- Image tag override for apiserver enterprise version
apiserverImageTagOverride: "3.15.3-909"
# -- Image tag override for fileserver enterprise version
fileserverImageTagOverride: "3.15.3-909"
# -- Image tag override for webserver enterprise version
webserverImageTagOverride: "3.15.3-801"
# -- Air gapped documentation configurations
airGappedDocumentation:
# -- Enable/Disable air gapped documentation deployment
@@ -364,7 +397,7 @@ enterpriseFeatures:
# -- Air gapped documentation image configuration
image:
repository: ""
tag: ""
tag: "4"
# -- set this value AND overrideReferenceFileUrl if external endpoint exposure is in place (like a LoadBalancer)
# example: "https://api.clearml.local"
overrideReferenceApiUrl: ""
@@ -389,15 +422,17 @@ enterpriseFeatures:
image:
repository: ""
pullPolicy: IfNotPresent
tag: ""
tag: "1.24-58"
# -- APPS base spawning pods image
basePodImage:
repository: ""
tag: ""
tag: "app-1.1.1-47"
# -- APPS number of pods
replicaCount: 1
# -- APPS extra envrinoment variables
extraEnvs: []
# -- file definition
fileMounts: []
# -- specific annotation for APPS pods
podAnnotations: {}
# -- APPS resources per pod; these are minimal requirements, it's suggested to increase

View File

@@ -0,0 +1,3 @@
# Openshift specific configuration
Use override files when deploying ClearML. Proposed files in this folder require setup of `<USER>` and `<FSUSER>` values to uids accepted by specific openshift configuration.

View File

@@ -0,0 +1,7 @@
agentk8sglue:
securityContext:
fsGroup: <FSUSER>
runAsUser: <USER>
basePodTemplate:
securityContext:
runAsUser: 0

View File

@@ -0,0 +1,36 @@
apiserver:
securityContext:
fsGroup: <FSUSER>
runAsUser: <USER>
runAsNonRoot: true
fileserver:
securityContext:
fsGroup: <FSUSER>
runAsUser: <USER>
runAsNonRoot: true
webserver:
securityContext:
fsGroup: <FSUSER>
runAsUser: <USER>
runAsNonRoot: true
elasticsearch:
securityContext:
runAsUser: <USER>
podSecurityContext:
fsGroup: <FSUSER>
runAsUser: <USER>
sysctlInitContainer:
enabled: false
volumeClaimTemplate:
redis:
securityContext:
fsGroup: <FSUSER>
runAsUser: <USER>
mongodb:
podSecurityContext:
enabled: true
fsGroup: <FSUSER>
containerSecurityContext:
enabled: true
runAsUser: <USER>
runAsNonRoot: true

View File

@@ -0,0 +1,3 @@
# Tanzu specific configuration
Before installing any ClearML chart, apply `rolebinding.yaml` file after setting needed `<NAMESPACE>` in it.

View File

@@ -2,7 +2,7 @@ kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: clearml-tanzu-rolebinding
namespace: clearml
namespace: <NAMESPACE>
roleRef:
kind: ClusterRole
name: psp:vmware-system-privileged