From eff1f2f21c598714e8dbd2daf1a68b51d4af1f7d Mon Sep 17 00:00:00 2001 From: revital Date: Tue, 3 Jun 2025 07:33:27 +0300 Subject: [PATCH 1/2] Edit custom billing --- .../enterprise_deploy/custom_billing.md | 172 +++++++++--------- .../extra_configs/custom_events.md | 115 ++++++++++++ 2 files changed, 201 insertions(+), 86 deletions(-) create mode 100644 docs/deploying_clearml/enterprise_deploy/extra_configs/custom_events.md diff --git a/docs/deploying_clearml/enterprise_deploy/custom_billing.md b/docs/deploying_clearml/enterprise_deploy/custom_billing.md index 484bff77..620e4a29 100644 --- a/docs/deploying_clearml/enterprise_deploy/custom_billing.md +++ b/docs/deploying_clearml/enterprise_deploy/custom_billing.md @@ -1,13 +1,13 @@ --- -title: Custom Billing Events +title: Custom Events --- :::important Enterprise Feature -Sending custom billing events is available under the ClearML Enterprise plan. +Sending custom events is available under the ClearML Enterprise plan. ::: ClearML supports sending custom events to selected Kafka topics. Event sending is triggered by API calls and -is available only for the companies with the `custom_events` settings set. +is available only to tenants with the `custom_events` feature configured. ## Enabling Custom Events in ClearML Server @@ -15,61 +15,53 @@ is available only for the companies with the `custom_events` settings set. **Precondition**: Customer Kafka for custom events is installed and reachable from the `apiserver`. ::: -Set the following environment variables in the ClearML Enterprise helm chart under the `apiserver.extraEnv`: +Modify the server's `clearml-values.override.yaml` file, then upgrade the chart: -* Enable custom events: +```yaml +apiserver: + extraEnvs: + # Enable custom events. + - name: CLEARML__services__custom_events__enabled + value: "true" + # Folder where the ClearML Apiserver container can find the custom message templates. + - name: CLEARML__services__custom_events__template_folder + value: "/mnt/custom_events/templates" + # Kafka host configuration for custom events. + - name: CLEARML__hosts__kafka__custom_events__host + value: "[:]" + # Kafka security parameters. Below is the example for SASL plaintext security. + - name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__security_protocol + value: "SASL_PLAINTEXT" + - name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__sasl_mechanism + value: "SCRAM-SHA-512" + - name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__sasl_plain_username + value: "" + - name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__sasl_plain_password + value: "" + # Kafka topics names for lifecycle and inventory messages. + - name: CLEARML__services__custom_events__channels__main__topics__service_instance_lifecycle + value: "lifecycle" + - name: CLEARML__services__custom_events__channels__main__topics__service_instance_inventory + value: "inventory" +``` - ``` - - name: CLEARML__services__custom_events__enabled - value: "true" - ``` -* Mount custom message template files into `/mnt/custom_events/templates` folder in the `apiserver` container and point - the `apiserver` into it: +### Configure Per-Tenant Custom Event Fields - ``` - - name: CLEARML__services__custom_events__template_folder - value: "/mnt/custom_events/templates" - ``` -* Configure the Kafka host for sending events: +For each tenant, configure the values required by the message templates using the following API call: - ``` - - name: CLEARML__hosts__kafka__custom_events__host - value: "[]" - ``` - Configure Kafka security parameters. Below is the example for SASL plaintext security: +```bash +curl $APISERVER_URL/system.update_company_custom_events_config -H "Content-Type: application/json" -u $APISERVER_KEY:$APISERVER_SECRET -d'{ + "company": "", + "fields": { + "service_instance_id": "", + "service_instance_name": "", + "service_instance_customer_tenant_name": "", + "service_instance_customer_space_name": "", + "service_instance_customer_space_id": "", + "parameters_connection_points": ["", ""] +}}' +``` - ``` - - name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__security_protocol - value: "SASL_PLAINTEXT" - - name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__sasl_mechanism - value: "SCRAM-SHA-512" - - name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__sasl_plain_username - value: "" - - name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__sasl_plain_password - value: "" - ``` -* Define Kafka topics for lifecycle and inventory messages: - - ``` - - name: CLEARML__services__custom_events__channels__main__topics__service_instance_lifecycle - value: "lifecycle" - - name: CLEARML__services__custom_events__channels__main__topics__service_instance_inventory - value: "inventory" - ``` -* For the desired companies set up the custom events properties required by the event message templates: - - ``` - curl $APISERVER_URL/system.update_company_custom_events_config -H "Content-Type: application/json" -u $APISERVER_KEY:$APISERVER_SECRET -d'{ - "company": "", - "fields": { - "service_instance_id": "", - "service_instance_name": "", - "service_instance_customer_tenant_name": "", - "service_instance_customer_space_name": "", - "service_instance_customer_space_id": "", - "parameters_connection_points": ["", ""] - }}' - ``` ## Sending Custom Events to the API Server @@ -77,46 +69,54 @@ Set the following environment variables in the ClearML Enterprise helm chart und **Precondition:** Dedicated custom-events Redis instance installed and reachable from all the custom events deployments. ::: -Environment lifecycle events are sent directly by the `apiserver`. Other event types are emitted by the following helm charts: +Environment lifecycle events are sent directly by the ClearML `apiserver`. Other event types are emitted by the following +services (Helm charts in Kubernetes): -* `clearml-pods-monitor-exporter` - Monitors running pods and sends container lifecycle events (should run one per cluster with a unique identifier, a UUID is required for the installation): - ``` - # -- Universal Unique string to identify Pods Monitor instances across worker clusters. It cannot be empty. - # Uniqueness is required across different cluster installations to preserve the reported data status. - podsMonitorUUID: "" - # Interval - checkIntervalSeconds: 60 - ``` +* `clearml-pods-monitor-exporter` - Monitors running pods (tasks) and sends lifecycle events. Should run one per cluster + with a unique identifier (a UUID is required for the installation): + * `clearml-pods-inventory` - Periodically sends inventory events about running pods. - ``` - # Cron schedule - https://crontab.guru/ - cronJob: - schedule: "@daily" - ``` -* `clearml-company-inventory` - Monitors Clearml companies and sends environment inventory events. +* `clearml-company-inventory` - Monitors ClearML companies and sends environment inventory events. - ``` - # Cron schedule - https://crontab.guru/ - cronJob: - schedule: "@daily" - ``` +### Install custom-events -For every script chart add the below configuration to enable redis access and connection to the `apiserver`: +The `clearml-custom-events` is the Kubernetes "Custom Events" umbrella Helm chart for ClearML. It includes custom-events +components and dependencies (Redis) to kick-start the installation. -``` -clearml: - apiServerUrlReference: "" - apiServerKey: "" - apiServerSecret: "" -redisConnection: - host: "" - port: - password: "" +#### Prepare Values + +Create a `clearml-custom-events-values.override.yaml` file with the following content (make sure to replace the ``): + +```yaml +global: + imageCredentials: + password: "" + clearml: + apiServerUrlReference: "" + apiServerKey: "" + apiServerSecret: "" + +clearml-pods-monitor-exporter: + checkIntervalSeconds: 60 + # -- *REQUIRED* Universal Unique string to identify Pods Monitor instances across worker clusters. Cannot be empty. + # Uniqueness is required across different clusters installations to preserve the reported data status. + podsMonitorUUID: "" + +clearml-pods-inventory: + cronJob: + schedule: "@daily" + +clearml-company-inventory: + cronJob: + schedule: "@daily" ``` -See all other available options to customize the `custom-events` charts by running: -``` -helm show readme allegroai-enterprise/ +#### Install + +Install the Custom Events umbrella chart: + +```bash +helm install clearml-custom-events clearml-enterprise/clearml-custom-events -f clearml-custom-events-values.override.yaml ``` \ No newline at end of file diff --git a/docs/deploying_clearml/enterprise_deploy/extra_configs/custom_events.md b/docs/deploying_clearml/enterprise_deploy/extra_configs/custom_events.md new file mode 100644 index 00000000..9b6ff94e --- /dev/null +++ b/docs/deploying_clearml/enterprise_deploy/extra_configs/custom_events.md @@ -0,0 +1,115 @@ +--- +title: Custom Events +--- + +ClearML Enterprise supports sending custom events to selected Kafka topics. These events are triggered by API calls +and are only available to tenants with the `custom_events` feature configured. + + +## Prerequisites + +* A Kafka instance must be deployed and reachable from the ClearML `apiserver` + +## Enabling Custom Events on the Server + +To enable custom evens, edit ClearML Server' `clearml-values.override.yaml` file adding the following: + +```yaml +apiserver: + extraEnvs: + # Enable custom events. + - name: CLEARML__services__custom_events__enabled + value: "true" + # Folder where the ClearML Apiserver container can find the custom message templates. + - name: CLEARML__services__custom_events__template_folder + value: "/mnt/custom_events/templates" + # Kafka host configuration for custom events. + - name: CLEARML__hosts__kafka__custom_events__host + value: "[:]" + # Kafka security parameters. Below is the example for SASL plaintext security. + - name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__security_protocol + value: "SASL_PLAINTEXT" + - name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__sasl_mechanism + value: "SCRAM-SHA-512" + - name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__sasl_plain_username + value: "" + - name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__sasl_plain_password + value: "" + # Kafka topics names for lifecycle and inventory messages. + - name: CLEARML__services__custom_events__channels__main__topics__service_instance_lifecycle + value: "lifecycle" + - name: CLEARML__services__custom_events__channels__main__topics__service_instance_inventory + value: "inventory" +``` + +After editing the configuration, upgrade the Helm chart to apply the changes. + +## Enabling Custom Events for Tenants + + +Each tenant must be explicitly configured to use custom events. Use the following API call to set up the custom events +properties required by the event message templates: + +```bash +curl $APISERVER_URL/system.update_company_custom_events_config -H "Content-Type: application/json" -u $APISERVER_KEY:$APISERVER_SECRET -d'{ + "company": "", + "fields": { + "service_instance_id": "", + "service_instance_name": "", + "service_instance_customer_tenant_name": "", + "service_instance_customer_space_name": "", + "service_instance_customer_space_id": "", + "parameters_connection_points": ["", ""] +}}' +``` + +## Event Emitters + +ClearML services that send custom events include: + +* `apiserver` — Sends environment lifecycle events. +* `clearml-pods-monitor-exporter` - Monitors running Pods (Tasks) and sends lifecycle events. Should run one per cluster + with a unique identifier (an explicit UUID is required for the installation). +* `clearml-pods-inventory` - Sends periodic inventory events about active Pods. +* `clearml-company-inventory` - Monitors ClearML tenants and emits environment-level inventory events. + +## Install custom-events + +The `clearml-custom-events` Helm chart bundles all custom event services and their dependencies (e.g., Redis). It serves +as the umbrella chart for custom event functionality. + +### Prepare Values + +Create a `clearml-custom-events-values.override.yaml` file with the following content (make sure to replace ``): + +```yaml +global: + imageCredentials: + password: "" + clearml: + apiServerUrlReference: "" + apiServerKey: "" + apiServerSecret: "" + +clearml-pods-monitor-exporter: + checkIntervalSeconds: 60 + # -- *REQUIRED* Universal Unique string to identify Pods Monitor instances across worker clusters. Cannot be empty. + # Uniqueness is required across different clusters installations to preserve the reported data status. + podsMonitorUUID: "" + +clearml-pods-inventory: + cronJob: + schedule: "@daily" + +clearml-company-inventory: + cronJob: + schedule: "@daily" +``` + +### Install + +Install the Custom Events chart: + +```bash +helm install clearml-custom-events clearml-enterprise/clearml-custom-events -f clearml-custom-events-values.override.yaml +``` \ No newline at end of file From 16608985450e664e6312ab22abc37ca8646f7831 Mon Sep 17 00:00:00 2001 From: revital Date: Sun, 8 Jun 2025 09:07:33 +0300 Subject: [PATCH 2/2] update custom billing --- .../{ => extra_configs}/custom_billing.md | 4 +- .../extra_configs/custom_events.md | 115 ------------------ .../enterprise_deploy/multi_tenant_k8s.md | 2 +- sidebars.js | 1 - 4 files changed, 3 insertions(+), 119 deletions(-) rename docs/deploying_clearml/enterprise_deploy/{ => extra_configs}/custom_billing.md (97%) delete mode 100644 docs/deploying_clearml/enterprise_deploy/extra_configs/custom_events.md diff --git a/docs/deploying_clearml/enterprise_deploy/custom_billing.md b/docs/deploying_clearml/enterprise_deploy/extra_configs/custom_billing.md similarity index 97% rename from docs/deploying_clearml/enterprise_deploy/custom_billing.md rename to docs/deploying_clearml/enterprise_deploy/extra_configs/custom_billing.md index 620e4a29..a1eb82b8 100644 --- a/docs/deploying_clearml/enterprise_deploy/custom_billing.md +++ b/docs/deploying_clearml/enterprise_deploy/extra_configs/custom_billing.md @@ -82,8 +82,8 @@ services (Helm charts in Kubernetes): ### Install custom-events -The `clearml-custom-events` is the Kubernetes "Custom Events" umbrella Helm chart for ClearML. It includes custom-events -components and dependencies (Redis) to kick-start the installation. +The `clearml-custom-events` is the Kubernetes "Custom Events" umbrella Helm chart for ClearML. +It bundles all necessary services and sets up a dedicated Redis instance. #### Prepare Values diff --git a/docs/deploying_clearml/enterprise_deploy/extra_configs/custom_events.md b/docs/deploying_clearml/enterprise_deploy/extra_configs/custom_events.md deleted file mode 100644 index 9b6ff94e..00000000 --- a/docs/deploying_clearml/enterprise_deploy/extra_configs/custom_events.md +++ /dev/null @@ -1,115 +0,0 @@ ---- -title: Custom Events ---- - -ClearML Enterprise supports sending custom events to selected Kafka topics. These events are triggered by API calls -and are only available to tenants with the `custom_events` feature configured. - - -## Prerequisites - -* A Kafka instance must be deployed and reachable from the ClearML `apiserver` - -## Enabling Custom Events on the Server - -To enable custom evens, edit ClearML Server' `clearml-values.override.yaml` file adding the following: - -```yaml -apiserver: - extraEnvs: - # Enable custom events. - - name: CLEARML__services__custom_events__enabled - value: "true" - # Folder where the ClearML Apiserver container can find the custom message templates. - - name: CLEARML__services__custom_events__template_folder - value: "/mnt/custom_events/templates" - # Kafka host configuration for custom events. - - name: CLEARML__hosts__kafka__custom_events__host - value: "[:]" - # Kafka security parameters. Below is the example for SASL plaintext security. - - name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__security_protocol - value: "SASL_PLAINTEXT" - - name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__sasl_mechanism - value: "SCRAM-SHA-512" - - name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__sasl_plain_username - value: "" - - name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__sasl_plain_password - value: "" - # Kafka topics names for lifecycle and inventory messages. - - name: CLEARML__services__custom_events__channels__main__topics__service_instance_lifecycle - value: "lifecycle" - - name: CLEARML__services__custom_events__channels__main__topics__service_instance_inventory - value: "inventory" -``` - -After editing the configuration, upgrade the Helm chart to apply the changes. - -## Enabling Custom Events for Tenants - - -Each tenant must be explicitly configured to use custom events. Use the following API call to set up the custom events -properties required by the event message templates: - -```bash -curl $APISERVER_URL/system.update_company_custom_events_config -H "Content-Type: application/json" -u $APISERVER_KEY:$APISERVER_SECRET -d'{ - "company": "", - "fields": { - "service_instance_id": "", - "service_instance_name": "", - "service_instance_customer_tenant_name": "", - "service_instance_customer_space_name": "", - "service_instance_customer_space_id": "", - "parameters_connection_points": ["", ""] -}}' -``` - -## Event Emitters - -ClearML services that send custom events include: - -* `apiserver` — Sends environment lifecycle events. -* `clearml-pods-monitor-exporter` - Monitors running Pods (Tasks) and sends lifecycle events. Should run one per cluster - with a unique identifier (an explicit UUID is required for the installation). -* `clearml-pods-inventory` - Sends periodic inventory events about active Pods. -* `clearml-company-inventory` - Monitors ClearML tenants and emits environment-level inventory events. - -## Install custom-events - -The `clearml-custom-events` Helm chart bundles all custom event services and their dependencies (e.g., Redis). It serves -as the umbrella chart for custom event functionality. - -### Prepare Values - -Create a `clearml-custom-events-values.override.yaml` file with the following content (make sure to replace ``): - -```yaml -global: - imageCredentials: - password: "" - clearml: - apiServerUrlReference: "" - apiServerKey: "" - apiServerSecret: "" - -clearml-pods-monitor-exporter: - checkIntervalSeconds: 60 - # -- *REQUIRED* Universal Unique string to identify Pods Monitor instances across worker clusters. Cannot be empty. - # Uniqueness is required across different clusters installations to preserve the reported data status. - podsMonitorUUID: "" - -clearml-pods-inventory: - cronJob: - schedule: "@daily" - -clearml-company-inventory: - cronJob: - schedule: "@daily" -``` - -### Install - -Install the Custom Events chart: - -```bash -helm install clearml-custom-events clearml-enterprise/clearml-custom-events -f clearml-custom-events-values.override.yaml -``` \ No newline at end of file diff --git a/docs/deploying_clearml/enterprise_deploy/multi_tenant_k8s.md b/docs/deploying_clearml/enterprise_deploy/multi_tenant_k8s.md index 14f74dfa..dd4e7190 100644 --- a/docs/deploying_clearml/enterprise_deploy/multi_tenant_k8s.md +++ b/docs/deploying_clearml/enterprise_deploy/multi_tenant_k8s.md @@ -287,7 +287,7 @@ For each group you define in the configuration, you need to specify the followin | `name` | Display name for the group (should match `` used in the variable path) | `CLEARML__services__auth__default_groups____name` | `"My Group Name"`, `"MLOps Team"` | | `features` | JSON list of features to enable for this group . For the complete list of available features, see [Available Features](#available-features). Note that the features must be defined for the tenant or for the entire server in order to affect the group. By default, all the features of the tenant are available to all users. | `CLEARML__services__auth__default_groups____features` | `'["applications", "experiments", "pipelines", "reports", "show_dashboard", "show_projects"]'` (Note the single quotes wrapping the JSON string if setting via YAML/environment variables). | | `assignable` | Whether admins can assign users to this group from the ClearML Web UI (`true`/`false`). If `false`, group membership is managed externally or implicitly. | `CLEARML__services__auth__default_groups____assignable` | `"false"` | -| `system` | Always set to `"false"` for custom groups | ``CLEARML__services__auth__default_groups____system` | `"false"` | +| `system` | Always set to `"false"` for custom groups | `CLEARML__services__auth__default_groups____system` | `"false"` | ##### Example Configuration diff --git a/sidebars.js b/sidebars.js index 2b37dd5a..739d136d 100644 --- a/sidebars.js +++ b/sidebars.js @@ -667,7 +667,6 @@ module.exports = { 'deploying_clearml/enterprise_deploy/appgw_install_k8s', ] }, - 'deploying_clearml/enterprise_deploy/custom_billing', {'UI Applications': [ 'deploying_clearml/enterprise_deploy/app_install_ubuntu_on_prem', 'deploying_clearml/enterprise_deploy/app_install_ex_server',