Update Elasticsearch (#47 )

* update elasticsearch * update elasticsearch reference * bump up chart version
Agent foreground mode (#46 )
2025-04-17 01:31:13 +00:00 · 2022-01-05 08:26:57 +01:00 · 2021-12-13 09:04:02 +01:00 · 2021-12-09 11:47:13 +01:00 · 2021-12-09 11:39:04 +01:00 · 2021-12-09 11:30:15 +01:00
9 changed files with 66 additions and 15 deletions
--- a/.github/workflows/ci.yaml
+++ b/.github/workflows/ci.yaml
@@ -21,8 +21,9 @@ jobs:
    strategy:
      matrix:
        k8s:
-          - v1.20.7
-          - v1.21.1
+          - v1.21.2
+          - v1.22.1
+          - v1.23.0
    steps:
      - name: Checkout
        uses: actions/checkout@v1
--- a/charts/clearml/Chart.lock
+++ b/charts/clearml/Chart.lock
@@ -7,6 +7,6 @@ dependencies:
  version: 10.3.4
 - name: elasticsearch
  repository: https://helm.elastic.co
-  version: 7.10.1
-digest: sha256:aefd3992b2ab085161e4cca35c6f73dd33f8d19272a9405b5ee4e8c2a0e79bba
-generated: "2021-01-05T14:26:33.629164+01:00"
+  version: 7.16.2
+digest: sha256:ac733cb02d50e8398c1d2832988333896f1c7b765c19a0f1eea5e9b24bdb8207
+generated: "2022-01-05T07:52:34.913745+01:00"
--- a/charts/clearml/Chart.yaml
+++ b/charts/clearml/Chart.yaml
@@ -2,7 +2,7 @@ apiVersion: v2
 name: clearml
 description: MLOps platform
 type: application
-version: "3.0.6"
+version: "3.3.0"
 appVersion: "1.1.1"
 home: https://clear.ml
 icon: https://raw.githubusercontent.com/allegroai/clearml/master/docs/clearml-logo.svg
@@ -18,14 +18,14 @@ keywords:
  - mlops
 dependencies:
  - name: redis
-    version: "~10.9.0"
+    version: "10.9.0"
    repository: "https://charts.bitnami.com/bitnami"
    condition: redis.enabled
  - name: mongodb
-    version: "~10.3.2"
+    version: "10.3.4"
    repository: "https://charts.bitnami.com/bitnami"
    condition: mongodb.enabled
  - name: elasticsearch
-    version: "~7.10.1"
+    version: "7.16.2"
    repository: "https://helm.elastic.co"
    condition: elasticsearch.enabled
--- a/charts/clearml/README.md
+++ b/charts/clearml/README.md
@@ -1,6 +1,6 @@
 # ClearML Ecosystem for Kubernetes

-![Version: 3.0.6](https://img.shields.io/badge/Version-3.0.6-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.1.1](https://img.shields.io/badge/AppVersion-1.1.1-informational?style=flat-square)
+![Version: 3.3.0](https://img.shields.io/badge/Version-3.3.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.1.1](https://img.shields.io/badge/AppVersion-1.1.1-informational?style=flat-square)

 MLOps platform

@@ -107,9 +107,9 @@ For detailed instructions, see the [Optional Configuration](https://github.com/a

 | Repository | Name | Version |
 |------------|------|---------|
-| https://charts.bitnami.com/bitnami | mongodb | ~10.3.2 |
-| https://charts.bitnami.com/bitnami | redis | ~10.9.0 |
-| https://helm.elastic.co | elasticsearch | ~7.10.1 |
+| https://charts.bitnami.com/bitnami | mongodb | 10.3.4 |
+| https://charts.bitnami.com/bitnami | redis | 10.9.0 |
+| https://helm.elastic.co | elasticsearch | 7.16.2 |

 ## Values

@@ -191,6 +191,7 @@ For detailed instructions, see the [Optional Configuration](https://github.com/a
 | agentservices.tolerations | list | `[]` |  |
 | apiserver.additionalConfigs | object | `{}` |  |
 | apiserver.affinity | object | `{}` |  |
+| apiserver.authCookiesMaxAge | int | `864000` | Amount of seconds the authorization cookie will last in user browser |
 | apiserver.configDir | string | `"/opt/clearml/config"` |  |
 | apiserver.extraEnvs | list | `[]` |  |
 | apiserver.image.pullPolicy | string | `"IfNotPresent"` |  |
@@ -245,6 +246,12 @@ For detailed instructions, see the [Optional Configuration](https://github.com/a
 | elasticsearch.roles.remote_cluster_client | string | `"true"` |  |
 | elasticsearch.volumeClaimTemplate.accessModes[0] | string | `"ReadWriteOnce"` |  |
 | elasticsearch.volumeClaimTemplate.resources.requests.storage | string | `"50Gi"` |  |
+| externalServices.elasticsearchHost | string | `""` | Existing ElasticSearch Hostname to use if elasticsearch.enabled is false |
+| externalServices.elasticsearchPort | int | `9200` | Existing ElasticSearch Port to use if elasticsearch.enabled is false |
+| externalServices.mongodbHost | string | `""` | Existing MongoDB Hostname to use if elasticsearch.enabled is false |
+| externalServices.mongodbPort | int | `27017` | Existing MongoDB Port to use if elasticsearch.enabled is false |
+| externalServices.redisHost | string | `""` | Existing Redis Hostname to use if elasticsearch.enabled is false |
+| externalServices.redisPort | int | `6379` | Existing Redis Port to use if elasticsearch.enabled is false |
 | fileserver.affinity | object | `{}` |  |
 | fileserver.extraEnvs | list | `[]` |  |
 | fileserver.image.pullPolicy | string | `"IfNotPresent"` |  |
--- a/charts/clearml/charts/elasticsearch-7.10.1.tgz
+++ b/charts/clearml/charts/elasticsearch-7.10.1.tgz
--- a/charts/clearml/charts/elasticsearch-7.16.2.tgz
+++ b/charts/clearml/charts/elasticsearch-7.16.2.tgz
--- a/charts/clearml/templates/deployment-agent.yaml
+++ b/charts/clearml/templates/deployment-agent.yaml
@@ -94,7 +94,7 @@ spec:
              apt-get install -y curl python3-pip git;
              python3 -m pip install -U pip ;
              python3 -m pip install clearml-agent{{ .agentVersion}} ;
-              CLEARML_AGENT_K8S_HOST_MOUNT=/root/.clearml:/root/.clearml clearml-agent daemon --queue {{ .queues}}"
+              CLEARML_AGENT_K8S_HOST_MOUNT=/root/.clearml:/root/.clearml clearml-agent daemon --foreground --queue {{ .queues}}"
          {{ if .clearmlConfig }}
          volumeMounts:
           - name: agent-clearml-conf-volume
--- a/charts/clearml/templates/deployment-apiserver.yaml
+++ b/charts/clearml/templates/deployment-apiserver.yaml
@@ -29,23 +29,49 @@ spec:
              protocol: TCP
          env:
          - name: CLEARML_ELASTIC_SERVICE_HOST
+          {{- if .Values.elasticsearch.enabled }}
            value: "{{ .Values.elasticsearch.clusterName }}-master"
+          {{- else }}
+            value: "{{ .Values.externalServices.elasticsearchHost }}"
+          {{- end }}
          - name: CLEARML_ELASTIC_SERVICE_PORT
+          {{- if .Values.elasticsearch.enabled }}
            value: "{{ .Values.elasticsearch.httpPort }}"
+          {{- else }}
+            value: "{{ .Values.externalServices.elasticsearchPort }}"
+          {{- end }}
          - name: CLEARML_MONGODB_SERVICE_HOST
+          {{- if .Values.mongodb.enabled }}
            value: "{{ tpl .Values.mongodb.service.name . }}"
+          {{- else }}
+            value: "{{ .Values.externalServices.mongodbHost }}"
+          {{- end }}
          - name: CLEARML_MONGODB_SERVICE_PORT
+          {{- if .Values.mongodb.enabled }}
            value: "{{ .Values.mongodb.service.port }}"
+          {{- else }}
+            value: "{{ .Values.externalServices.mongodbPort }}"
+          {{- end }}
          - name: CLEARML_REDIS_SERVICE_HOST
+          {{- if .Values.redis.enabled }}
            value: "{{ tpl .Values.redis.master.name . }}"
+          {{- else }}
+            value: "{{ .Values.externalServices.redisHost }}"
+          {{- end }}
          - name: CLEARML_REDIS_SERVICE_PORT
+          {{- if .Values.redis.enabled }}
            value: "{{ .Values.redis.master.port }}"
+          {{- else }}
+            value: "{{ .Values.externalServices.redisPort }}"
+          {{- end }}
          - name: CLEARML__APISERVER__PRE_POPULATE__ENABLED
            value: "{{ .Values.apiserver.prepopulateEnabled }}"
          - name: CLEARML__APISERVER__PRE_POPULATE__ZIP_FILES
            value: "{{ .Values.apiserver.prepopulateZipFiles }}"
          - name: CLEARML_SERVER_DEPLOYMENT_TYPE
            value: "helm-cloud"
+          - name: CLEARML__APISERVER__AUTH__COOKIES__MAX_AGE
+            value: "{{ .Values.apiserver.authCookiesMaxAge }}"
          - name: CLEARML_CONFIG_DIR
            value: /opt/clearml/config
          - name: CLEARML__APISERVER__DEFAULT_COMPANY
--- a/charts/clearml/values.yaml
+++ b/charts/clearml/values.yaml
@@ -37,6 +37,9 @@ apiserver:
  prepopulateArtifactsPath: "/mnt/fileserver"
  configDir: /opt/clearml/config

+  # -- Amount of seconds the authorization cookie will last in user browser
+  authCookiesMaxAge: 864000
+
  service:
    # -- This will set to service's spec.type field
    type: NodePort
@@ -288,6 +291,20 @@ agentGroups:

    affinity: {}

+externalServices:
+  # -- Existing ElasticSearch Hostname to use if elasticsearch.enabled is false
+  elasticsearchHost: ""
+  # -- Existing ElasticSearch Port to use if elasticsearch.enabled is false
+  elasticsearchPort: 9200
+  # -- Existing MongoDB Hostname to use if elasticsearch.enabled is false
+  mongodbHost: ""
+  # -- Existing MongoDB Port to use if elasticsearch.enabled is false
+  mongodbPort: 27017
+  # -- Existing Redis Hostname to use if elasticsearch.enabled is false
+  redisHost: ""
+  # -- Existing Redis Port to use if elasticsearch.enabled is false
+  redisPort: 6379
+
 redis:  # configuration from https://github.com/bitnami/charts/blob/master/bitnami/redis/values.yaml
  enabled: true
  usePassword: false
@@ -320,7 +337,7 @@ mongodb:  # configuration from https://github.com/bitnami/charts/blob/master/bit
    port: 27017
    portName: mongo-service

-elasticsearch:  # configuration from https://github.com/elastic/helm-charts/blob/7.10/elasticsearch/values.yaml
+elasticsearch:  # configuration from https://github.com/elastic/helm-charts/blob/7.16/elasticsearch/values.yaml
  enabled: true
  httpPort: 9200
  roles:
Author	SHA1	Message	Date
Valeriano Manassero	70b07c637a	Update Elasticsearch (#47 ) * update elasticsearch * update elasticsearch reference * bump up chart version	2022-01-05 08:26:57 +01:00
Valeriano Manassero	7b8e40c626	Agent foreground mode (#46 ) * use foreground to push output on console * bump up version	2021-12-13 09:04:02 +01:00
Valeriano Manassero	d8117eeb0d	add k8s 1.22.1 to ci procedure (#44 ) After some tests I found 1.22.1 doesn't have ulimit issue so I can include it into the ci process	2021-12-09 11:47:13 +01:00
Valeriano Manassero	4c09ae2c92	Fix env typo (#39 ) * typo fix * bump up version	2021-12-09 11:39:04 +01:00
Valeriano Manassero	478eecd5f2	remove k8s 1.22 from ci (#43 ) It looks 1.22 k8s image from kind has a very low ulimit preventing elastic search from installing, removing it waiting for a fix.	2021-12-09 11:30:15 +01:00
Valeriano Manassero	43f4c44219	test one single kind cluster at time to avoid pressure fails (#42 )	2021-12-09 11:00:57 +01:00
Valeriano Manassero	b83c8cd0e8	indentation fix (#41 )	2021-12-09 10:32:42 +01:00
Valeriano Manassero	97f219228d	update kind k8s versions (#40 )	2021-12-09 10:31:17 +01:00
Valeriano Manassero	1b5b9407f6	Configurable auth cookies age (#38 ) * configurable auth cookies age * version bump up	2021-12-09 08:14:09 +01:00
Valeriano Manassero	b494a8c0cf	External services (#36 ) * use external services switch * bump up version * readme update	2021-11-26 08:11:55 +01:00