ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-05-29 09:39:23 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Merge branch 'main' of https://github.com/allegroai/clearml-docs into edits_3

Browse Source
This commit is contained in:
revital 2025-03-17 11:30:57 +02:00
commit a11208a8ce
9 changed files with 150 additions and 89 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

37
docs/clearml_serving/clearml_serving_setup.md
View File

@ -15,7 +15,7 @@ The following page goes over how to set up and upgrade `clearml-serving`.
[free hosted service](https://app.clear.ml) [free hosted service](https://app.clear.ml)
1. Connect `clearml` SDK to the server, see instructions [here](../clearml_sdk/clearml_sdk_setup#install-clearml) 1. Connect `clearml` SDK to the server, see instructions [here](../clearml_sdk/clearml_sdk_setup#install-clearml)
1. Install clearml-serving CLI: 1. Install the `clearml-serving` CLI:
```bash ```bash
pip3 install clearml-serving pip3 install clearml-serving
@ -27,21 +27,22 @@ The following page goes over how to set up and upgrade `clearml-serving`.
clearml-serving create --name "serving example" clearml-serving create --name "serving example"
``` ```
The new serving service UID should be printed This command prints the Serving Service UID:
```console ```console
New Serving Service created: id=aa11bb22aa11bb22 New Serving Service created: id=aa11bb22aa11bb22
``` ```
Write down the Serving Service UID Copy the Serving Service UID (e.g., `aa11bb22aa11bb22`), as you will need it in the next steps.
1. Clone the `clearml-serving` repository: 1. Clone the `clearml-serving` repository:
```bash ```bash
git clone https://github.com/clearml/clearml-serving.git git clone https://github.com/clearml/clearml-serving.git
``` ```
1. Edit the environment variables file (docker/example.env) with your clearml-server credentials and Serving Service UID. 1. Edit the environment variables file (`docker/example.env`) with your `clearml-server` API credentials and Serving Service UID.
For example, you should have something like For example:
```bash ```bash
cat docker/example.env cat docker/example.env
``` ```
@ -55,31 +56,30 @@ The following page goes over how to set up and upgrade `clearml-serving`.
CLEARML_SERVING_TASK_ID="<serving_service_id_here>" CLEARML_SERVING_TASK_ID="<serving_service_id_here>"
``` ```
1. Spin up the `clearml-serving` containers with `docker-compose` (or if running on Kubernetes, use the helm chart) 1. Spin up the `clearml-serving` containers with `docker-compose` (or if running on Kubernetes, use the helm chart):
```bash ```bash
cd docker && docker-compose --env-file example.env -f docker-compose.yml up cd docker && docker-compose --env-file example.env -f docker-compose.yml up
``` ```
If you need Triton support (keras/pytorch/onnx etc.), use the triton docker-compose file If you need Triton support (Keras/PyTorch/ONNX etc.), use the triton `docker-compose` file:
```bash ```bash
cd docker && docker-compose --env-file example.env -f docker-compose-triton.yml up cd docker && docker-compose --env-file example.env -f docker-compose-triton.yml up
``` ```
If running on a GPU instance with Triton support (keras/pytorch/onnx etc.), use the triton gpu docker-compose file: If running on a GPU instance with Triton support (Keras/PyTorch/ONNX etc.), use the triton gpu docker-compose file:
```bash ```bash
cd docker && docker-compose --env-file example.env -f docker-compose-triton-gpu.yml up cd docker && docker-compose --env-file example.env -f docker-compose-triton-gpu.yml up
``` ```
:::note :::note
Any model that registers with Triton engine will run the pre/post-processing code on the Inference service container, Any model that registers with Triton engine will run the pre/post-processing code in the Inference service container,
and the model inference itself will be executed on the Triton Engine container. and the model inference itself will be executed on the Triton Engine container.
::: :::
## Advanced Setup - S3/GS/Azure Access (Optional) ## Advanced Setup - S3/GS/Azure Access (Optional)
To add access credentials and allow the inference containers to download models from your S3/GS/Azure object-storage, To enable inference containers to download models from S3, Google Cloud Storage (GS), or Azure,
add the respective environment variables to your env files (example.env). For further details, see add access credentials in the respective environment variables to your env files (`example.env`):
[Configuring Storage](../integrations/storage.md#configuring-storage).
``` ```
AWS_ACCESS_KEY_ID AWS_ACCESS_KEY_ID
@ -92,14 +92,21 @@ AZURE_STORAGE_ACCOUNT
AZURE_STORAGE_KEY AZURE_STORAGE_KEY
``` ```
For further details, see [Configuring Storage](../integrations/storage.md#configuring-storage).
## Upgrading ClearML Serving ## Upgrading ClearML Serving
**Upgrading to v1.1** **Upgrading to v1.1**
1. Take down the serving containers (`docker-compose` or k8s) 1. Shut down the serving containers (`docker-compose` or k8s)
1. Update the `clearml-serving` CLI `pip3 install -U clearml-serving` 1. Update the `clearml-serving` CLI:
```
pip3 install -U clearml-serving
```
1. Re-add a single existing endpoint with `clearml-serving model add ...` (press yes when asked). It will upgrade the 1. Re-add a single existing endpoint with `clearml-serving model add ...` (press yes when asked). It will upgrade the
`clearml-serving` session definitions `clearml-serving` session definitions.
1. Pull the latest serving containers (`docker-compose pull ...` or k8s) 1. Pull the latest serving containers (`docker-compose pull ...` or k8s)
1. Re-spin serving containers (`docker-compose` or k8s) 1. Re-spin serving containers (`docker-compose` or k8s)

41
docs/getting_started/project_progress.md
View File

@ -6,9 +6,22 @@ ClearML provides a comprehensive set of monitoring tools to help effectively tra
These tools offer both high-level overviews and detailed insights into task execution, resource These tools offer both high-level overviews and detailed insights into task execution, resource
utilization, and project performance. utilization, and project performance.
## Offerings
### Project Dashboard ## Project Overview
A project's **OVERVIEW** tab in the UI presents a general picture of a project:
* Metric Snapshot – A graphical representation of selected metric values across project tasks, offering a quick assessment of progress.
* Task Status Tracking – When a single metric variant is selected for the snapshot, task status is color-coded (e.g.,
Completed, Aborted, Published, Failed) for better visibility.
Use the Metric Snapshot to track project progress and identify trends in task performance.
For more information, see [Project Overview](../webapp/webapp_project_overview.md).
![Project Overview](../img/webapp_project_overview.png#light-mode-only)
![Project Overview](../img/webapp_project_overview_dark.png#dark-mode-only)
## Project Dashboard
:::info Pro Plan Offering :::info Pro Plan Offering
The Project Dashboard app is available under the ClearML Pro plan. The Project Dashboard app is available under the ClearML Pro plan.
@ -28,16 +41,22 @@ For more information, see [Project Dashboard](../webapp/applications/apps_dashbo
![Project Dashboard](../img/apps_dashboard.png#light-mode-only) ![Project Dashboard](../img/apps_dashboard.png#light-mode-only)
![Project Dashboard](../img/apps_dashboard_dark.png#dark-mode-only) ![Project Dashboard](../img/apps_dashboard_dark.png#dark-mode-only)
### Project Overview ## Task Monitoring
A project's **OVERVIEW** tab in the UI presents a general picture of a project: ClearML provides task monitoring capabilities through the [`clearml.automation.Monitor`](https://github.com/clearml/clearml/blob/master/clearml/automation/monitor.py)
* Metric Snapshot – A graphical representation of selected metric values across project tasks, offering a quick assessment of progress. class. With this class you can implement monitoring workflows such as:
* Task Status Tracking – When a single metric variant is selected for the snapshot, task status is color-coded (e.g.,
Completed, Aborted, Published, Failed) for better visibility.
Use the Metric Snapshot to track project progress and identify trends in task performance. * Send notifications via Slack or other channels
* Trigger automated responses based on specific task conditions
For more information, see [Project Overview](../webapp/webapp_project_overview.md). For a practical example, see the [Slack Alerts Example](../guides/services/slack_alerts.md), which demonstrates how to:
* Track task status (completion, failure, etc.)
* Send notifications to a specified Slack channel
* Retrieve task details such as status, console logs, and links to the ClearML Web UI
You can also configure filters for task types and projects to reduce unnecessary notifications.
![Slack Alerts](../img/examples_slack_alerts.png#light-mode-only)
![Slack Alerts](../img/examples_slack_alerts_dark.png#dark-mode-only)
![Project Overview](../img/webapp_project_overview.png#light-mode-only)
![Project Overview](../img/webapp_project_overview_dark.png#dark-mode-only)

7
docs/guides/services/slack_alerts.md
View File

@ -6,11 +6,12 @@ The [Slack alerts example](https://github.com/clearml/clearml/blob/master/exampl
demonstrates how to use the `clearml.automation.monitor` class to implement a service that monitors the completion and demonstrates how to use the `clearml.automation.monitor` class to implement a service that monitors the completion and
failure of tasks, and posts alert messages on a Slack channel. failure of tasks, and posts alert messages on a Slack channel.
![Slack alert example](../../img/examples_slack_alerts.png) ![Slack alert example](../../img/examples_slack_alerts.png#light-mode-only)
![Slack alert example](../../img/examples_slack_alerts_dark.png#dark-mode-only)
## Creating a Slack Bot ## Creating a Slackbot
Before configuring and running the Slack alert service, create a Slack Bot (**ClearML Bot**). Before configuring and running the Slack alert service, create a Slackbot (**ClearML Bot**).
:::important :::important
The Slack API token and channel you create are required to configure the Slack alert service. The Slack API token and channel you create are required to configure the Slack alert service.

BIN
docs/img/examples_slack_alerts.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 231 KiB

After

Width:  |  Height:  |  Size: 459 KiB

BIN
docs/img/examples_slack_alerts_dark.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 458 KiB

4
docs/integrations/transformers.md
View File

@ -77,12 +77,12 @@ cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents:
and shuts down instances as needed, according to a resource budget that you set. and shuts down instances as needed, according to a resource budget that you set.
### Reproducing Tasks ### Reproducing Task Runs
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5.gif#light-mode-only) ![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5.gif#light-mode-only)
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5_dark.gif#dark-mode-only) ![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5_dark.gif#dark-mode-only)
Use ClearML's web interface to reproduce tasks and edit their details, like hyperparameters or input models, then execute the tasks Use ClearML's web interface to reproduce task runs and edit their details, like hyperparameters or input models, then execute the tasks
with the new configuration on a remote machine. with the new configuration on a remote machine.
When ClearML is integrated into a script, it captures and stores configurations, such as hyperparameters When ClearML is integrated into a script, it captures and stores configurations, such as hyperparameters

27
docs/release_notes/clearml_server/enterprise/ver_3_24.md
View File

@ -3,6 +3,33 @@ title: Version 3.24
--- ---
### Enterprise Server 3.24.2
**New Features**
* Add support for additional billing event formats
* Improve login to multi-tenant service
**Bug Fixes**
* Security fixes when exporting data to CSV
* Fix access permissions to UI Applications
### Enterprise Server 3.24.1
**New Features**
* Add service for presigning AWS S3 URLs
* Add support for additional billing event formats
* Add support for user-customized UI themes
* Add configuration to disable adding users to the workspace via the UI
* Add on-demand refresh to UI applications
* Add grouped same-event view to UI "Latest Task Events"
**Bug Fixes**
* Fix downloaded CSV file of UI “Latest Task Events” missing some events
* Fix access permissions to UI Reports
* Fix configuration modal of UI application instance displays incorrect values
* Fix UI Hyper-Dataset frame viewer navigation controls not displaying
### Enterprise Server 3.24.0 ### Enterprise Server 3.24.0
**Default Behavior Change: Access Rules Enabled** **Default Behavior Change: Access Rules Enabled**

34
package-lock.json generated
View File

@ -737,11 +737,13 @@
} }
}, },
"node_modules/@babel/helpers": { "node_modules/@babel/helpers": {
"version": "7.26.7", "version": "7.26.10",
"resolved": "https://registry.npmjs.org/@babel/helpers/-/helpers-7.26.10.tgz",
"integrity": "sha512-UPYc3SauzZ3JGgj87GgZ89JVdC5dj0AoetR5Bw6wj4niittNyFh6+eOGonYvJ1ao6B8lEa3Q3klS7ADZ53bc5g==",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@babel/template": "^7.25.9", "@babel/template": "^7.26.9",
"@babel/types": "^7.26.7" "@babel/types": "^7.26.10"
}, },
"engines": { "engines": {
"node": ">=6.9.0" "node": ">=6.9.0"
@ -826,10 +828,12 @@
} }
}, },
"node_modules/@babel/parser": { "node_modules/@babel/parser": {
"version": "7.26.8", "version": "7.26.10",
"resolved": "https://registry.npmjs.org/@babel/parser/-/parser-7.26.10.tgz",
"integrity": "sha512-6aQR2zGE/QFi8JpDLjUZEPYOs7+mhKXm86VaKFiLP35JQwQb6bwUE+XbvkH0EptsYhbNBSUGaUBLKqxH1xSgsA==",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@babel/types": "^7.26.8" "@babel/types": "^7.26.10"
}, },
"bin": { "bin": {
"parser": "bin/babel-parser.js" "parser": "bin/babel-parser.js"
@ -1939,7 +1943,9 @@
} }
}, },
"node_modules/@babel/runtime": { "node_modules/@babel/runtime": {
"version": "7.26.7", "version": "7.26.10",
"resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.26.10.tgz",
"integrity": "sha512-2WJMeRQPHKSPemqk/awGrAiuFfzBmOIPXKizAsVhWH9YJqLZ0H+HS4c8loHGgW6utJ3E/ejXQUsiGaQy2NZ9Fw==",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"regenerator-runtime": "^0.14.0" "regenerator-runtime": "^0.14.0"
@ -1960,12 +1966,14 @@
} }
}, },
"node_modules/@babel/template": { "node_modules/@babel/template": {
"version": "7.26.8", "version": "7.26.9",
"resolved": "https://registry.npmjs.org/@babel/template/-/template-7.26.9.tgz",
"integrity": "sha512-qyRplbeIpNZhmzOysF/wFMuP9sctmh2cFzRAZOn1YapxBsE1i9bJIY586R/WBLfLcmcBlM8ROBiQURnnNy+zfA==",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@babel/code-frame": "^7.26.2", "@babel/code-frame": "^7.26.2",
"@babel/parser": "^7.26.8", "@babel/parser": "^7.26.9",
"@babel/types": "^7.26.8" "@babel/types": "^7.26.9"
}, },
"engines": { "engines": {
"node": ">=6.9.0" "node": ">=6.9.0"
@ -1988,7 +1996,9 @@
} }
}, },
"node_modules/@babel/types": { "node_modules/@babel/types": {
"version": "7.26.8", "version": "7.26.10",
"resolved": "https://registry.npmjs.org/@babel/types/-/types-7.26.10.tgz",
"integrity": "sha512-emqcG3vHrpxUKTrxcblR36dcrcoRDvKmnL/dCL6ZsHaShW80qxCAcNhzQZrpeM765VzEos+xOi4s+r4IXzTwdQ==",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@babel/helper-string-parser": "^7.25.9", "@babel/helper-string-parser": "^7.25.9",
@ -15567,7 +15577,9 @@
} }
}, },
"node_modules/prismjs": { "node_modules/prismjs": {
"version": "1.29.0", "version": "1.30.0",
"resolved": "https://registry.npmjs.org/prismjs/-/prismjs-1.30.0.tgz",
"integrity": "sha512-DEvV2ZF2r2/63V+tK8hQvrR2ZGn10srHbXviTlcv7Kpzw8jWiNTqbVgjO3IY8RxrrOUF8VPMQQFysYYYv0YZxw==",
"license": "MIT", "license": "MIT",
"engines": { "engines": {
"node": ">=6" "node": ">=6"

41
sidebars.js
View File

@ -634,44 +634,37 @@ module.exports = {
]}, ]},
] ]
}, },
/* {'Getting Started': [
'getting_started/architecture',
]},*/
{
'Enterprise Server': {
'Deployment Options': [
'deploying_clearml/enterprise_deploy/multi_tenant_k8s',
'deploying_clearml/enterprise_deploy/vpc_aws',
'deploying_clearml/enterprise_deploy/on_prem_ubuntu',
],
'Maintenance and Migration': [
'deploying_clearml/enterprise_deploy/import_projects',
'deploying_clearml/enterprise_deploy/change_artifact_links',
'deploying_clearml/enterprise_deploy/delete_tenant',
]
}
},
{ {
type: 'category', type: 'category',
collapsible: true, collapsible: true,
collapsed: true, collapsed: true,
label: 'ClearML Application Gateway', label: 'Enterprise Server',
items: [ items: [
{'Deployment Options': [
'deploying_clearml/enterprise_deploy/multi_tenant_k8s',
'deploying_clearml/enterprise_deploy/vpc_aws',
'deploying_clearml/enterprise_deploy/on_prem_ubuntu',
]
},
{'Maintenance and Migration': [
'deploying_clearml/enterprise_deploy/import_projects',
'deploying_clearml/enterprise_deploy/change_artifact_links',
'deploying_clearml/enterprise_deploy/delete_tenant',
]
},
{'ClearML Application Gateway': [
'deploying_clearml/enterprise_deploy/appgw_install_compose', 'deploying_clearml/enterprise_deploy/appgw_install_compose',
'deploying_clearml/enterprise_deploy/appgw_install_k8s', 'deploying_clearml/enterprise_deploy/appgw_install_k8s',
] ]
}, },
'deploying_clearml/enterprise_deploy/custom_billing', 'deploying_clearml/enterprise_deploy/custom_billing',
{ {'UI Applications': [
'UI Applications': [
'deploying_clearml/enterprise_deploy/app_install_ubuntu_on_prem', 'deploying_clearml/enterprise_deploy/app_install_ubuntu_on_prem',
'deploying_clearml/enterprise_deploy/app_install_ex_server', 'deploying_clearml/enterprise_deploy/app_install_ex_server',
'deploying_clearml/enterprise_deploy/app_custom', 'deploying_clearml/enterprise_deploy/app_custom',
] ]
}, },
{ {'User Management': [
'User Management': [
'user_management/user_groups', 'user_management/user_groups',
'user_management/access_rules', 'user_management/access_rules',
'user_management/admin_vaults', 'user_management/admin_vaults',
@ -690,6 +683,8 @@ module.exports = {
}, },
] ]
}, },
]
},
], ],
bestPracticesSidebar: [ bestPracticesSidebar: [
{ {