mirror of
https://github.com/clearml/clearml-docs
synced 2025-03-05 03:29:29 +00:00
Add custom billing events (#1058)
This commit is contained in:
parent
0090010fd2
commit
69513f6908
118
docs/deploying_clearml/enterprise_deploy/custom_billing.md
Normal file
118
docs/deploying_clearml/enterprise_deploy/custom_billing.md
Normal file
@ -0,0 +1,118 @@
|
|||||||
|
---
|
||||||
|
title: Custom Billing Events
|
||||||
|
---
|
||||||
|
|
||||||
|
ClearML supports sending custom events to selected Kafka topics. Event sending is triggered by API calls and
|
||||||
|
is available only for the companies with the `custom_events` settings set.
|
||||||
|
|
||||||
|
## Enabling Custom Events in ClearML Server
|
||||||
|
|
||||||
|
:::important Prerequisite
|
||||||
|
**Precondition**: Customer Kafka for custom events is installed and reachable from the `apiserver`.
|
||||||
|
:::
|
||||||
|
|
||||||
|
Set the following environment variables in the ClearML Enterprise helm chart under the `apiserver.extraEnv`:
|
||||||
|
|
||||||
|
* Enable custom events:
|
||||||
|
|
||||||
|
```
|
||||||
|
- name: CLEARML__services__custom_events__enabled
|
||||||
|
value: "true"
|
||||||
|
```
|
||||||
|
* Mount custom message template files into `/mnt/custom_events/templates` folder in the `apiserver` container and point
|
||||||
|
the `apiserver` into it:
|
||||||
|
|
||||||
|
```
|
||||||
|
- name: CLEARML__services__custom_events__template_folder
|
||||||
|
value: "/mnt/custom_events/templates"
|
||||||
|
```
|
||||||
|
* Configure the Kafka host for sending events:
|
||||||
|
|
||||||
|
```
|
||||||
|
- name: CLEARML__hosts__kafka__custom_events__host
|
||||||
|
value: "[<KAFKA host address:port>]"
|
||||||
|
```
|
||||||
|
Configure Kafka security parameters. Below is the example for SASL plaintext security:
|
||||||
|
|
||||||
|
```
|
||||||
|
- name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__security_protocol
|
||||||
|
value: "SASL_PLAINTEXT"
|
||||||
|
- name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__sasl_mechanism
|
||||||
|
value: "SCRAM-SHA-512"
|
||||||
|
- name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__sasl_plain_username
|
||||||
|
value: "<username>"
|
||||||
|
- name: CLEARML__SECURE__KAFKA__CUSTOM_EVENTS__sasl_plain_password
|
||||||
|
value: "<password>"
|
||||||
|
```
|
||||||
|
* Define Kafka topics for lifecycle and inventory messages:
|
||||||
|
|
||||||
|
```
|
||||||
|
- name: CLEARML__services__custom_events__channels__main__topics__service_instance_lifecycle
|
||||||
|
value: "lifecycle"
|
||||||
|
- name: CLEARML__services__custom_events__channels__main__topics__service_instance_inventory
|
||||||
|
value: "inventory"
|
||||||
|
```
|
||||||
|
* For the desired companies set up the custom events properties required by the event message templates:
|
||||||
|
|
||||||
|
```
|
||||||
|
curl $APISERVER_URL/system.update_company_custom_events_config -H "Content-Type: application/json" -u $APISERVER_KEY:$APISERVER_SECRET -d'{
|
||||||
|
"company": "<company_id>",
|
||||||
|
"fields": {
|
||||||
|
"service_instance_id": "<value>",
|
||||||
|
"service_instance_name": "<value>",
|
||||||
|
"service_instance_customer_tenant_name": "<value>",
|
||||||
|
"service_instance_customer_space_name": "<value>",
|
||||||
|
"service_instance_customer_space_id": "<value>",
|
||||||
|
"parameters_connection_points": ["<value1>", "<value2>"]
|
||||||
|
}}'
|
||||||
|
```
|
||||||
|
|
||||||
|
## Sending Custom Events to the API Server
|
||||||
|
|
||||||
|
:::important Prerequisite
|
||||||
|
**Precondition:** Dedicated custom-events Redis instance installed and reachable from all the custom events deployments.
|
||||||
|
:::
|
||||||
|
|
||||||
|
Environment lifecycle events are sent directly by the `apiserver`. Other event types are emitted by the following helm charts:
|
||||||
|
|
||||||
|
* `clearml-pods-monitor-exporter` - Monitors running pods and sends container lifecycle events (should run one per cluster with a unique identifier, a UUID is required for the installation):
|
||||||
|
|
||||||
|
```
|
||||||
|
# -- Universal Unique string to identify Pods Monitor instances across worker clusters. It cannot be empty.
|
||||||
|
# Uniqueness is required across different cluster installations to preserve the reported data status.
|
||||||
|
podsMonitorUUID: "<Unique ID>"
|
||||||
|
# Interval
|
||||||
|
checkIntervalSeconds: 60
|
||||||
|
```
|
||||||
|
* `clearml-pods-inventory` - Periodically sends inventory events about running pods.
|
||||||
|
|
||||||
|
```
|
||||||
|
# Cron schedule - https://crontab.guru/
|
||||||
|
cronJob:
|
||||||
|
schedule: "@daily"
|
||||||
|
```
|
||||||
|
* `clearml-company-inventory` - Monitors Clearml companies and sends environment inventory events.
|
||||||
|
|
||||||
|
```
|
||||||
|
# Cron schedule - https://crontab.guru/
|
||||||
|
cronJob:
|
||||||
|
schedule: "@daily"
|
||||||
|
```
|
||||||
|
|
||||||
|
For every script chart add the below configuration to enable redis access and connection to the `apiserver`:
|
||||||
|
|
||||||
|
```
|
||||||
|
clearml:
|
||||||
|
apiServerUrlReference: "<APISERVER_URL>"
|
||||||
|
apiServerKey: "<APISERVER_KEY>"
|
||||||
|
apiServerSecret: "<APISERVER_SECRET>"
|
||||||
|
redisConnection:
|
||||||
|
host: "<REDIS_HOST>"
|
||||||
|
port: <REDIS_PORT>
|
||||||
|
password: "<REDIS_PWD>"
|
||||||
|
```
|
||||||
|
|
||||||
|
See all other available options to customize the `custom-events` charts by running:
|
||||||
|
```
|
||||||
|
helm show readme allegroai-enterprise/<CHART_NAME>
|
||||||
|
```
|
@ -651,6 +651,7 @@ module.exports = {
|
|||||||
'deploying_clearml/enterprise_deploy/appgw_install_k8s',
|
'deploying_clearml/enterprise_deploy/appgw_install_k8s',
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
|
'deploying_clearml/enterprise_deploy/custom_billing',
|
||||||
'deploying_clearml/enterprise_deploy/delete_tenant',
|
'deploying_clearml/enterprise_deploy/delete_tenant',
|
||||||
'deploying_clearml/enterprise_deploy/import_projects',
|
'deploying_clearml/enterprise_deploy/import_projects',
|
||||||
'deploying_clearml/enterprise_deploy/change_artifact_links',
|
'deploying_clearml/enterprise_deploy/change_artifact_links',
|
||||||
|
Loading…
Reference in New Issue
Block a user