diff --git a/docs/clearml_serving/clearml_serving_setup.md b/docs/clearml_serving/clearml_serving_setup.md index 05219921..4820a946 100644 --- a/docs/clearml_serving/clearml_serving_setup.md +++ b/docs/clearml_serving/clearml_serving_setup.md @@ -15,7 +15,7 @@ The following page goes over how to set up and upgrade `clearml-serving`. [free hosted service](https://app.clear.ml) 1. Connect `clearml` SDK to the server, see instructions [here](../clearml_sdk/clearml_sdk_setup#install-clearml) -1. Install clearml-serving CLI: +1. Install the `clearml-serving` CLI: ```bash pip3 install clearml-serving @@ -27,21 +27,22 @@ The following page goes over how to set up and upgrade `clearml-serving`. clearml-serving create --name "serving example" ``` - The new serving service UID should be printed + This command prints the Serving Service UID: ```console New Serving Service created: id=aa11bb22aa11bb22 ``` - Write down the Serving Service UID + Copy the Serving Service UID (e.g., `aa11bb22aa11bb22`), as you will need it in the next steps. 1. Clone the `clearml-serving` repository: ```bash git clone https://github.com/clearml/clearml-serving.git ``` -1. Edit the environment variables file (docker/example.env) with your clearml-server credentials and Serving Service UID. - For example, you should have something like +1. Edit the environment variables file (`docker/example.env`) with your `clearml-server` API credentials and Serving Service UID. + For example: + ```bash cat docker/example.env ``` @@ -55,31 +56,30 @@ The following page goes over how to set up and upgrade `clearml-serving`. CLEARML_SERVING_TASK_ID="" ``` -1. Spin up the `clearml-serving` containers with `docker-compose` (or if running on Kubernetes, use the helm chart) +1. Spin up the `clearml-serving` containers with `docker-compose` (or if running on Kubernetes, use the helm chart): ```bash cd docker && docker-compose --env-file example.env -f docker-compose.yml up ``` - If you need Triton support (keras/pytorch/onnx etc.), use the triton docker-compose file + If you need Triton support (Keras/PyTorch/ONNX etc.), use the triton `docker-compose` file: ```bash cd docker && docker-compose --env-file example.env -f docker-compose-triton.yml up ``` - If running on a GPU instance with Triton support (keras/pytorch/onnx etc.), use the triton gpu docker-compose file: + If running on a GPU instance with Triton support (Keras/PyTorch/ONNX etc.), use the triton gpu docker-compose file: ```bash cd docker && docker-compose --env-file example.env -f docker-compose-triton-gpu.yml up ``` :::note -Any model that registers with Triton engine will run the pre/post-processing code on the Inference service container, +Any model that registers with Triton engine will run the pre/post-processing code in the Inference service container, and the model inference itself will be executed on the Triton Engine container. ::: ## Advanced Setup - S3/GS/Azure Access (Optional) -To add access credentials and allow the inference containers to download models from your S3/GS/Azure object-storage, -add the respective environment variables to your env files (example.env). For further details, see -[Configuring Storage](../integrations/storage.md#configuring-storage). +To enable inference containers to download models from S3, Google Cloud Storage (GS), or Azure, +add access credentials in the respective environment variables to your env files (`example.env`): ``` AWS_ACCESS_KEY_ID @@ -92,14 +92,21 @@ AZURE_STORAGE_ACCOUNT AZURE_STORAGE_KEY ``` +For further details, see [Configuring Storage](../integrations/storage.md#configuring-storage). + ## Upgrading ClearML Serving **Upgrading to v1.1** -1. Take down the serving containers (`docker-compose` or k8s) -1. Update the `clearml-serving` CLI `pip3 install -U clearml-serving` +1. Shut down the serving containers (`docker-compose` or k8s) +1. Update the `clearml-serving` CLI: + + ``` + pip3 install -U clearml-serving + ``` + 1. Re-add a single existing endpoint with `clearml-serving model add ...` (press yes when asked). It will upgrade the - `clearml-serving` session definitions + `clearml-serving` session definitions. 1. Pull the latest serving containers (`docker-compose pull ...` or k8s) 1. Re-spin serving containers (`docker-compose` or k8s) diff --git a/docs/getting_started/project_progress.md b/docs/getting_started/project_progress.md index 01f12893..1c534e47 100644 --- a/docs/getting_started/project_progress.md +++ b/docs/getting_started/project_progress.md @@ -6,9 +6,22 @@ ClearML provides a comprehensive set of monitoring tools to help effectively tra These tools offer both high-level overviews and detailed insights into task execution, resource utilization, and project performance. -## Offerings -### Project Dashboard +## Project Overview + +A project's **OVERVIEW** tab in the UI presents a general picture of a project: +* Metric Snapshot – A graphical representation of selected metric values across project tasks, offering a quick assessment of progress. +* Task Status Tracking – When a single metric variant is selected for the snapshot, task status is color-coded (e.g., +Completed, Aborted, Published, Failed) for better visibility. + +Use the Metric Snapshot to track project progress and identify trends in task performance. + +For more information, see [Project Overview](../webapp/webapp_project_overview.md). + +![Project Overview](../img/webapp_project_overview.png#light-mode-only) +![Project Overview](../img/webapp_project_overview_dark.png#dark-mode-only) + +## Project Dashboard :::info Pro Plan Offering The Project Dashboard app is available under the ClearML Pro plan. @@ -28,16 +41,22 @@ For more information, see [Project Dashboard](../webapp/applications/apps_dashbo ![Project Dashboard](../img/apps_dashboard.png#light-mode-only) ![Project Dashboard](../img/apps_dashboard_dark.png#dark-mode-only) -### Project Overview +## Task Monitoring -A project's **OVERVIEW** tab in the UI presents a general picture of a project: -* Metric Snapshot – A graphical representation of selected metric values across project tasks, offering a quick assessment of progress. -* Task Status Tracking – When a single metric variant is selected for the snapshot, task status is color-coded (e.g., -Completed, Aborted, Published, Failed) for better visibility. +ClearML provides task monitoring capabilities through the [`clearml.automation.Monitor`](https://github.com/clearml/clearml/blob/master/clearml/automation/monitor.py) +class. With this class you can implement monitoring workflows such as: -Use the Metric Snapshot to track project progress and identify trends in task performance. +* Send notifications via Slack or other channels +* Trigger automated responses based on specific task conditions -For more information, see [Project Overview](../webapp/webapp_project_overview.md). +For a practical example, see the [Slack Alerts Example](../guides/services/slack_alerts.md), which demonstrates how to: + +* Track task status (completion, failure, etc.) +* Send notifications to a specified Slack channel +* Retrieve task details such as status, console logs, and links to the ClearML Web UI + +You can also configure filters for task types and projects to reduce unnecessary notifications. + +![Slack Alerts](../img/examples_slack_alerts.png#light-mode-only) +![Slack Alerts](../img/examples_slack_alerts_dark.png#dark-mode-only) -![Project Overview](../img/webapp_project_overview.png#light-mode-only) -![Project Overview](../img/webapp_project_overview_dark.png#dark-mode-only) diff --git a/docs/guides/services/slack_alerts.md b/docs/guides/services/slack_alerts.md index 8a3bbd6a..d3913146 100644 --- a/docs/guides/services/slack_alerts.md +++ b/docs/guides/services/slack_alerts.md @@ -6,11 +6,12 @@ The [Slack alerts example](https://github.com/clearml/clearml/blob/master/exampl demonstrates how to use the `clearml.automation.monitor` class to implement a service that monitors the completion and failure of tasks, and posts alert messages on a Slack channel. -![Slack alert example](../../img/examples_slack_alerts.png) +![Slack alert example](../../img/examples_slack_alerts.png#light-mode-only) +![Slack alert example](../../img/examples_slack_alerts_dark.png#dark-mode-only) -## Creating a Slack Bot +## Creating a Slackbot -Before configuring and running the Slack alert service, create a Slack Bot (**ClearML Bot**). +Before configuring and running the Slack alert service, create a Slackbot (**ClearML Bot**). :::important The Slack API token and channel you create are required to configure the Slack alert service. diff --git a/docs/img/examples_slack_alerts.png b/docs/img/examples_slack_alerts.png index 6c38d3b0..9bba329c 100644 Binary files a/docs/img/examples_slack_alerts.png and b/docs/img/examples_slack_alerts.png differ diff --git a/docs/img/examples_slack_alerts_dark.png b/docs/img/examples_slack_alerts_dark.png new file mode 100644 index 00000000..4d29a69d Binary files /dev/null and b/docs/img/examples_slack_alerts_dark.png differ diff --git a/docs/integrations/transformers.md b/docs/integrations/transformers.md index c6f4ff07..9483eeaa 100644 --- a/docs/integrations/transformers.md +++ b/docs/integrations/transformers.md @@ -77,12 +77,12 @@ cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents: and shuts down instances as needed, according to a resource budget that you set. -### Reproducing Tasks +### Reproducing Task Runs ![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5.gif#light-mode-only) ![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5_dark.gif#dark-mode-only) -Use ClearML's web interface to reproduce tasks and edit their details, like hyperparameters or input models, then execute the tasks +Use ClearML's web interface to reproduce task runs and edit their details, like hyperparameters or input models, then execute the tasks with the new configuration on a remote machine. When ClearML is integrated into a script, it captures and stores configurations, such as hyperparameters diff --git a/docs/release_notes/clearml_server/enterprise/ver_3_24.md b/docs/release_notes/clearml_server/enterprise/ver_3_24.md index 8c1b4e44..0eaf5082 100644 --- a/docs/release_notes/clearml_server/enterprise/ver_3_24.md +++ b/docs/release_notes/clearml_server/enterprise/ver_3_24.md @@ -3,6 +3,33 @@ title: Version 3.24 --- +### Enterprise Server 3.24.2 + +**New Features** +* Add support for additional billing event formats +* Improve login to multi-tenant service + +**Bug Fixes** +* Security fixes when exporting data to CSV +* Fix access permissions to UI Applications + +### Enterprise Server 3.24.1 + +**New Features** +* Add service for presigning AWS S3 URLs +* Add support for additional billing event formats +* Add support for user-customized UI themes +* Add configuration to disable adding users to the workspace via the UI +* Add on-demand refresh to UI applications +* Add grouped same-event view to UI "Latest Task Events" + +**Bug Fixes** +* Fix downloaded CSV file of UI “Latest Task Events” missing some events +* Fix access permissions to UI Reports +* Fix configuration modal of UI application instance displays incorrect values +* Fix UI Hyper-Dataset frame viewer navigation controls not displaying + + ### Enterprise Server 3.24.0 **Default Behavior Change: Access Rules Enabled** diff --git a/package-lock.json b/package-lock.json index 5a47cfd3..0788e644 100644 --- a/package-lock.json +++ b/package-lock.json @@ -737,11 +737,13 @@ } }, "node_modules/@babel/helpers": { - "version": "7.26.7", + "version": "7.26.10", + "resolved": "https://registry.npmjs.org/@babel/helpers/-/helpers-7.26.10.tgz", + "integrity": "sha512-UPYc3SauzZ3JGgj87GgZ89JVdC5dj0AoetR5Bw6wj4niittNyFh6+eOGonYvJ1ao6B8lEa3Q3klS7ADZ53bc5g==", "license": "MIT", "dependencies": { - "@babel/template": "^7.25.9", - "@babel/types": "^7.26.7" + "@babel/template": "^7.26.9", + "@babel/types": "^7.26.10" }, "engines": { "node": ">=6.9.0" @@ -826,10 +828,12 @@ } }, "node_modules/@babel/parser": { - "version": "7.26.8", + "version": "7.26.10", + "resolved": "https://registry.npmjs.org/@babel/parser/-/parser-7.26.10.tgz", + "integrity": "sha512-6aQR2zGE/QFi8JpDLjUZEPYOs7+mhKXm86VaKFiLP35JQwQb6bwUE+XbvkH0EptsYhbNBSUGaUBLKqxH1xSgsA==", "license": "MIT", "dependencies": { - "@babel/types": "^7.26.8" + "@babel/types": "^7.26.10" }, "bin": { "parser": "bin/babel-parser.js" @@ -1939,7 +1943,9 @@ } }, "node_modules/@babel/runtime": { - "version": "7.26.7", + "version": "7.26.10", + "resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.26.10.tgz", + "integrity": "sha512-2WJMeRQPHKSPemqk/awGrAiuFfzBmOIPXKizAsVhWH9YJqLZ0H+HS4c8loHGgW6utJ3E/ejXQUsiGaQy2NZ9Fw==", "license": "MIT", "dependencies": { "regenerator-runtime": "^0.14.0" @@ -1960,12 +1966,14 @@ } }, "node_modules/@babel/template": { - "version": "7.26.8", + "version": "7.26.9", + "resolved": "https://registry.npmjs.org/@babel/template/-/template-7.26.9.tgz", + "integrity": "sha512-qyRplbeIpNZhmzOysF/wFMuP9sctmh2cFzRAZOn1YapxBsE1i9bJIY586R/WBLfLcmcBlM8ROBiQURnnNy+zfA==", "license": "MIT", "dependencies": { "@babel/code-frame": "^7.26.2", - "@babel/parser": "^7.26.8", - "@babel/types": "^7.26.8" + "@babel/parser": "^7.26.9", + "@babel/types": "^7.26.9" }, "engines": { "node": ">=6.9.0" @@ -1988,7 +1996,9 @@ } }, "node_modules/@babel/types": { - "version": "7.26.8", + "version": "7.26.10", + "resolved": "https://registry.npmjs.org/@babel/types/-/types-7.26.10.tgz", + "integrity": "sha512-emqcG3vHrpxUKTrxcblR36dcrcoRDvKmnL/dCL6ZsHaShW80qxCAcNhzQZrpeM765VzEos+xOi4s+r4IXzTwdQ==", "license": "MIT", "dependencies": { "@babel/helper-string-parser": "^7.25.9", @@ -15567,7 +15577,9 @@ } }, "node_modules/prismjs": { - "version": "1.29.0", + "version": "1.30.0", + "resolved": "https://registry.npmjs.org/prismjs/-/prismjs-1.30.0.tgz", + "integrity": "sha512-DEvV2ZF2r2/63V+tK8hQvrR2ZGn10srHbXviTlcv7Kpzw8jWiNTqbVgjO3IY8RxrrOUF8VPMQQFysYYYv0YZxw==", "license": "MIT", "engines": { "node": ">=6" diff --git a/sidebars.js b/sidebars.js index d76a812e..d467048b 100644 --- a/sidebars.js +++ b/sidebars.js @@ -634,60 +634,55 @@ module.exports = { ]}, ] }, -/* {'Getting Started': [ - 'getting_started/architecture', - ]},*/ - { - 'Enterprise Server': { - 'Deployment Options': [ - 'deploying_clearml/enterprise_deploy/multi_tenant_k8s', - 'deploying_clearml/enterprise_deploy/vpc_aws', - 'deploying_clearml/enterprise_deploy/on_prem_ubuntu', - ], - 'Maintenance and Migration': [ - 'deploying_clearml/enterprise_deploy/import_projects', - 'deploying_clearml/enterprise_deploy/change_artifact_links', - 'deploying_clearml/enterprise_deploy/delete_tenant', - ] - - } - }, { type: 'category', collapsible: true, collapsed: true, - label: 'ClearML Application Gateway', + label: 'Enterprise Server', items: [ - 'deploying_clearml/enterprise_deploy/appgw_install_compose', - 'deploying_clearml/enterprise_deploy/appgw_install_k8s', - ] - }, - 'deploying_clearml/enterprise_deploy/custom_billing', - { - 'UI Applications': [ - 'deploying_clearml/enterprise_deploy/app_install_ubuntu_on_prem', - 'deploying_clearml/enterprise_deploy/app_install_ex_server', - 'deploying_clearml/enterprise_deploy/app_custom', - ] - }, - { - 'User Management': [ - 'user_management/user_groups', - 'user_management/access_rules', - 'user_management/admin_vaults', - { - type: 'category', - collapsible: true, - collapsed: true, - label: 'Identity Provider Integration', - link: {type: 'doc', id: 'user_management/identity_providers'}, - items: [ - 'deploying_clearml/enterprise_deploy/sso_multi_tenant_login', - 'deploying_clearml/enterprise_deploy/sso_saml_k8s', - 'deploying_clearml/enterprise_deploy/sso_keycloak', - 'deploying_clearml/enterprise_deploy/sso_active_directory' + {'Deployment Options': [ + 'deploying_clearml/enterprise_deploy/multi_tenant_k8s', + 'deploying_clearml/enterprise_deploy/vpc_aws', + 'deploying_clearml/enterprise_deploy/on_prem_ubuntu', ] }, + {'Maintenance and Migration': [ + 'deploying_clearml/enterprise_deploy/import_projects', + 'deploying_clearml/enterprise_deploy/change_artifact_links', + 'deploying_clearml/enterprise_deploy/delete_tenant', + ] + }, + {'ClearML Application Gateway': [ + 'deploying_clearml/enterprise_deploy/appgw_install_compose', + 'deploying_clearml/enterprise_deploy/appgw_install_k8s', + ] + }, + 'deploying_clearml/enterprise_deploy/custom_billing', + {'UI Applications': [ + 'deploying_clearml/enterprise_deploy/app_install_ubuntu_on_prem', + 'deploying_clearml/enterprise_deploy/app_install_ex_server', + 'deploying_clearml/enterprise_deploy/app_custom', + ] + }, + {'User Management': [ + 'user_management/user_groups', + 'user_management/access_rules', + 'user_management/admin_vaults', + { + type: 'category', + collapsible: true, + collapsed: true, + label: 'Identity Provider Integration', + link: {type: 'doc', id: 'user_management/identity_providers'}, + items: [ + 'deploying_clearml/enterprise_deploy/sso_multi_tenant_login', + 'deploying_clearml/enterprise_deploy/sso_saml_k8s', + 'deploying_clearml/enterprise_deploy/sso_keycloak', + 'deploying_clearml/enterprise_deploy/sso_active_directory' + ] + }, + ] + }, ] }, ],