mirror of
https://github.com/clearml/clearml-docs
synced 2025-04-04 21:33:17 +00:00
Merge
This commit is contained in:
commit
e17f1986ee
@ -27,6 +27,7 @@ but can be overridden by command-line arguments.
|
||||
|**CLEARML_AGENT_DOCKER_ARGS_HIDE_ENV** | Hide Docker environment variables containing secrets when printing out the Docker command. When printed, the variable values will be replaced by `********`. See [`agent.hide_docker_command_env_vars`](../configs/clearml_conf.md#hide_docker) |
|
||||
|**CLEARML_AGENT_DISABLE_SSH_MOUNT** | Disables the auto `.ssh` mount into the docker |
|
||||
|**CLEARML_AGENT_FORCE_CODE_DIR**| Allows overriding the remote execution code directory to bypass repository cloning and use a repo already available where the remote agent is running. |
|
||||
|**CLEARML_AGENT_FORCE_UV**| If set to `1`, force the agent to use UV as the package manager. Overrides the default manager set in the [clearml.conf](../configs/clearml_conf.md) under `agent.package_manager.type` |
|
||||
|**CLEARML_AGENT_FORCE_EXEC_SCRIPT**| Allows overriding the remote execution script to bypass repository cloning and execute code already available where the remote agent is running. Use `module:file.py` format to specify a module and a script to execute (e.g. `.:main.py` to run `main.py` from the working dir)|
|
||||
|**CLEARML_AGENT_FORCE_TASK_INIT**| If set to `1`, ClearML Agent adds `Task.init()` to scripts that do not have the call, creating a Task to capture code execution information and output, which is then sent to the ClearML Server. If set to `0` and the script does not include `Task.init()`, the agent will capture only the output streams and console output, without tracking code execution details, metrics, or models. |
|
||||
|**CLEARML_AGENT_FORCE_SYSTEM_SITE_PACKAGES** | If set to `1`, overrides default [`agent.package_manager.system_site_packages: true`](../configs/clearml_conf.md#system_site_packages) behavior when running tasks in containers (docker mode and k8s-glue)|
|
||||
|
@ -13,6 +13,7 @@ multiple tasks (see [Virtual Environment Reuse](clearml_agent_env_caching.md#vir
|
||||
ClearML Agent supports working with one of the following package managers:
|
||||
* [`pip`](https://en.wikipedia.org/wiki/Pip_(package_manager)) (default)
|
||||
* [`conda`](https://docs.conda.io/en/latest/)
|
||||
* [`uv`](https://docs.astral.sh/uv/)
|
||||
* [`poetry`](https://python-poetry.org/)
|
||||
|
||||
To change the package manager used by the agent, edit the [`package_manager.type`](../configs/clearml_conf.md#agentpackage_manager)
|
||||
|
@ -80,7 +80,7 @@ For either setup, you can set up in your Enterprise ClearML Agent Helm chart the
|
||||
each queue. When a task is enqueued in ClearML, it translates into a Kubernetes pod running on the designated device
|
||||
with the specified fractional resource as defined in the Agent Helm chart.
|
||||
|
||||
#### MIG-enabled GPUs
|
||||
#### MIG-enabled GPUs
|
||||
The **ClearML Dynamic MIG Operator** (CDMO) chart enables running AI workloads on K8s with optimized hardware utilization
|
||||
and workload performance by facilitating MIG GPU partitioning. Make sure you have a [MIG capable GPU](https://docs.nvidia.com/datacenter/tesla/mig-user-guide/index.html#supported-gpus).
|
||||
|
||||
|
@ -2,7 +2,7 @@
|
||||
title: ClearML Python Package
|
||||
---
|
||||
|
||||
This is step-by-step guide for installing the `clearml` Python package and connecting it to the ClearML Server. Once done,
|
||||
This is a step-by-step guide for installing the `clearml` Python package and connecting it to the ClearML Server. Once done,
|
||||
you can integrate `clearml` into your code.
|
||||
|
||||
## Install ClearML
|
||||
|
@ -74,6 +74,7 @@ After invoking `Task.init` in a script, ClearML starts its automagical logging,
|
||||
* [AutoKeras](../integrations/autokeras.md)
|
||||
* [CatBoost](../integrations/catboost.md)
|
||||
* [Fast.ai](../integrations/fastai.md)
|
||||
* [Hugging Face Transformers](../integrations/transformers.md)
|
||||
* [LightGBM](../integrations/lightgbm.md)
|
||||
* [MegEngine](../integrations/megengine.md)
|
||||
* [MONAI](../integrations/monai.md)
|
||||
|
@ -15,7 +15,7 @@ The following page goes over how to set up and upgrade `clearml-serving`.
|
||||
[free hosted service](https://app.clear.ml)
|
||||
1. Connect `clearml` SDK to the server, see instructions [here](../clearml_sdk/clearml_sdk_setup#install-clearml)
|
||||
|
||||
1. Install clearml-serving CLI:
|
||||
1. Install the `clearml-serving` CLI:
|
||||
|
||||
```bash
|
||||
pip3 install clearml-serving
|
||||
@ -27,21 +27,22 @@ The following page goes over how to set up and upgrade `clearml-serving`.
|
||||
clearml-serving create --name "serving example"
|
||||
```
|
||||
|
||||
The new serving service UID should be printed
|
||||
This command prints the Serving Service UID:
|
||||
|
||||
```console
|
||||
New Serving Service created: id=aa11bb22aa11bb22
|
||||
```
|
||||
|
||||
Write down the Serving Service UID
|
||||
Copy the Serving Service UID (e.g., `aa11bb22aa11bb22`), as you will need it in the next steps.
|
||||
|
||||
1. Clone the `clearml-serving` repository:
|
||||
```bash
|
||||
git clone https://github.com/clearml/clearml-serving.git
|
||||
```
|
||||
|
||||
1. Edit the environment variables file (docker/example.env) with your clearml-server credentials and Serving Service UID.
|
||||
For example, you should have something like
|
||||
1. Edit the environment variables file (`docker/example.env`) with your `clearml-server` API credentials and Serving Service UID.
|
||||
For example:
|
||||
|
||||
```bash
|
||||
cat docker/example.env
|
||||
```
|
||||
@ -55,31 +56,30 @@ The following page goes over how to set up and upgrade `clearml-serving`.
|
||||
CLEARML_SERVING_TASK_ID="<serving_service_id_here>"
|
||||
```
|
||||
|
||||
1. Spin up the `clearml-serving` containers with `docker-compose` (or if running on Kubernetes, use the helm chart)
|
||||
1. Spin up the `clearml-serving` containers with `docker-compose` (or if running on Kubernetes, use the helm chart):
|
||||
|
||||
```bash
|
||||
cd docker && docker-compose --env-file example.env -f docker-compose.yml up
|
||||
```
|
||||
|
||||
If you need Triton support (keras/pytorch/onnx etc.), use the triton docker-compose file
|
||||
If you need Triton support (Keras/PyTorch/ONNX etc.), use the triton `docker-compose` file:
|
||||
```bash
|
||||
cd docker && docker-compose --env-file example.env -f docker-compose-triton.yml up
|
||||
```
|
||||
|
||||
If running on a GPU instance with Triton support (keras/pytorch/onnx etc.), use the triton gpu docker-compose file:
|
||||
If running on a GPU instance with Triton support (Keras/PyTorch/ONNX etc.), use the triton gpu docker-compose file:
|
||||
```bash
|
||||
cd docker && docker-compose --env-file example.env -f docker-compose-triton-gpu.yml up
|
||||
```
|
||||
|
||||
:::note
|
||||
Any model that registers with Triton engine will run the pre/post-processing code on the Inference service container,
|
||||
Any model that registers with Triton engine will run the pre/post-processing code in the Inference service container,
|
||||
and the model inference itself will be executed on the Triton Engine container.
|
||||
:::
|
||||
|
||||
## Advanced Setup - S3/GS/Azure Access (Optional)
|
||||
To add access credentials and allow the inference containers to download models from your S3/GS/Azure object-storage,
|
||||
add the respective environment variables to your env files (example.env). For further details, see
|
||||
[Configuring Storage](../integrations/storage.md#configuring-storage).
|
||||
To enable inference containers to download models from S3, Google Cloud Storage (GS), or Azure,
|
||||
add access credentials in the respective environment variables to your env files (`example.env`):
|
||||
|
||||
```
|
||||
AWS_ACCESS_KEY_ID
|
||||
@ -92,14 +92,21 @@ AZURE_STORAGE_ACCOUNT
|
||||
AZURE_STORAGE_KEY
|
||||
```
|
||||
|
||||
For further details, see [Configuring Storage](../integrations/storage.md#configuring-storage).
|
||||
|
||||
## Upgrading ClearML Serving
|
||||
|
||||
**Upgrading to v1.1**
|
||||
|
||||
1. Take down the serving containers (`docker-compose` or k8s)
|
||||
1. Update the `clearml-serving` CLI `pip3 install -U clearml-serving`
|
||||
1. Shut down the serving containers (`docker-compose` or k8s)
|
||||
1. Update the `clearml-serving` CLI:
|
||||
|
||||
```
|
||||
pip3 install -U clearml-serving
|
||||
```
|
||||
|
||||
1. Re-add a single existing endpoint with `clearml-serving model add ...` (press yes when asked). It will upgrade the
|
||||
`clearml-serving` session definitions
|
||||
`clearml-serving` session definitions.
|
||||
1. Pull the latest serving containers (`docker-compose pull ...` or k8s)
|
||||
1. Re-spin serving containers (`docker-compose` or k8s)
|
||||
|
||||
|
@ -515,8 +515,12 @@ These settings define which Docker image and arguments should be used unless [ex
|
||||
|
||||
**`agent.package_manager`** (*dict*)
|
||||
|
||||
* Dictionary containing the options for the Python package manager. The currently supported package managers are pip, conda,
|
||||
and, if the repository contains a `poetry.lock` file, poetry.
|
||||
* Dictionary containing the options for the Python package manager.
|
||||
* The currently supported package managers are
|
||||
* pip
|
||||
* conda
|
||||
* uv, if the root repository contains a `uv.lock` or `pyproject.toml` file
|
||||
* poetry, if the repository contains a `poetry.lock` or `pyproject.toml` file
|
||||
|
||||
---
|
||||
|
||||
@ -661,13 +665,38 @@ Torch Nightly builds are ephemeral and are deleted from time to time.
|
||||
* `pip`
|
||||
* `conda`
|
||||
* `poetry`
|
||||
* `uv`
|
||||
|
||||
* If `pip` or `conda` are used, the agent installs the required packages based on the "Python Packages" section of the
|
||||
Task. If the "Python Packages" section is empty, it will revert to using `requirements.txt` from the repository's root
|
||||
directory. If `poetry` is selected, and the root repository contains `poetry.lock` or `pyproject.toml`, the "Python
|
||||
directory.
|
||||
* If `poetry` is selected, and the root repository contains `poetry.lock` or `pyproject.toml`, the "Python
|
||||
Packages" section is ignored, and `poetry` is used. If `poetry` is selected and no lock file is found, it reverts to
|
||||
`pip` package manager behaviour.
|
||||
|
||||
* If `uv` is selected, and the root repository contains `uv.lock` or `pyproject.toml`, the "Python
|
||||
Packages" section is ignored, and `uv` is used. If `uv` is selected and no lock file is found, it reverts to
|
||||
`pip` package manager behaviour.
|
||||
|
||||
---
|
||||
|
||||
**`agent.package_manager.uv_files_from_repo_working_dir`** (*bool*)
|
||||
|
||||
* If set to `true`, the agent will look for the `uv.lock` or `pyproject.toml` file in the provided directory path instead of
|
||||
the repository's root directory.
|
||||
|
||||
---
|
||||
|
||||
**`agent.package_manager.uv_sync_extra_args`** (*list*)
|
||||
|
||||
* List extra command-line arguments to pass when using `uv`.
|
||||
|
||||
---
|
||||
|
||||
**`agent.package_manager.uv_version`** (*string*)
|
||||
|
||||
* The `uv` version requirements. For example, `">0.4"`, `"==0.4"`, `""` (empty string will install the latest version).
|
||||
|
||||
|
||||
<br/>
|
||||
|
||||
#### agent.pip_download_cache
|
||||
|
@ -4,7 +4,7 @@ title: ClearML Server
|
||||
|
||||
## What is ClearML Server?
|
||||
The ClearML Server is the backend service infrastructure for ClearML. It allows multiple users to collaborate and
|
||||
manage their tasks by working seamlessly with the ClearML Python package and [ClearML Agent](../clearml_agent.md).
|
||||
manage their tasks by working seamlessly with the [ClearML Python package](../clearml_sdk/clearml_sdk_setup.md) and [ClearML Agent](../clearml_agent.md).
|
||||
|
||||
ClearML Server is composed of the following:
|
||||
* Web server including the [ClearML Web UI](../webapp/webapp_overview.md), which is the user interface for tracking, comparing, and managing tasks.
|
||||
|
@ -233,7 +233,7 @@ The following example, which is based on AWS load balancing, demonstrates the co
|
||||
|
||||
|
||||
|
||||
### Opening Elasticsearch, MongoDB, and Redis for External Access
|
||||
### Opening Elasticsearch, MongoDB, and Redis for External Access
|
||||
|
||||
For improved security, the ports for ClearML Server Elasticsearch, MongoDB, and Redis servers are not exposed by default;
|
||||
they are only open internally in the docker network. If external access is needed, open these ports (but make sure to
|
||||
|
@ -29,12 +29,12 @@ The `General` section is the root-level section of the configuration file, and c
|
||||
* `id` - A unique id for the application
|
||||
* `name` - The name to display in the web application
|
||||
* `version` - The version of the application implementation. Recommended to have three numbers and to bump up when updating applications, so that older running instances can still be displayed
|
||||
* `provider` - The person/team/group who is the owner of the application. This will appears in the UI
|
||||
* `provider` - The person/team/group who is the owner of the application. This will appear in the UI
|
||||
* `description` - Short description of the application to be displayed in the ClearML Web UI
|
||||
* `icon` (*Optional*) - Small image to display in the ClearML web UI as an icon for the application. Can be a public web url or an image in the application’s assets directory (described below)
|
||||
* `no_info_html` (*Optional*) - HTML content to display as a placeholder for the dashboard when no instance is available. Can be a public web url or a file in the application’s assets directory (described below)
|
||||
* `default-queue` - The queue to which application instance will be sent when launching a new instance. This queue should have an appropriate agent servicing it. See details in the Custom Apps Agent section below.
|
||||
* `badges` (*Optional*) - List of strings to display as a bacge/label in the UI
|
||||
* `badges` (*Optional*) - List of strings to display as a badge/label in the UI
|
||||
* `resumable` - Boolean indication whether a running application instance can be restarted if required. Default is false.
|
||||
* `category` (*Optional*) - Way to separate apps into different tabs in the ClearML web UI
|
||||
* `featured` (*Optional*) - Value affecting the order of applications. Lower values are displayed first. Defaults to 500
|
||||
@ -264,7 +264,7 @@ The dashboard elements are organized into lines.
|
||||
|
||||
The section contains the following information:
|
||||
* `lines` - The array of line elements, each containing:
|
||||
* `style` - CSS definitions for the line e.g setting the line height
|
||||
* `style` - CSS definitions for the line e.g. setting the line height
|
||||
* `contents` - An array of dashboard elements to display in a given line. Each element may have several fields:
|
||||
* `title` - Text to display at the top of the field
|
||||
* `type` - one of the following:
|
||||
|
@ -30,12 +30,12 @@ their instances:
|
||||
* [Embedding Model Deployment](../../webapp/applications/apps_embed_model_deployment.md)
|
||||
* [Llama.cpp Model Deployment](../../webapp/applications/apps_llama_deployment.md)
|
||||
|
||||
The AI Application Gateway is provided through an additional component to the ClearML Server deployment: The ClearML Task Traffic Router.
|
||||
If your ClearML Deployment does not have the Task Traffic Router properly installed, these application instances may not be accessible.
|
||||
The AI Application Gateway requires an additional component to the ClearML Server deployment: the **ClearML App Gateway Router**.
|
||||
If your ClearML Deployment does not have the App Gateway Router properly installed, these application instances may not be accessible.
|
||||
|
||||
#### Installation
|
||||
|
||||
The Task Traffic Router supports two deployment options:
|
||||
The App Gateway Router supports two deployment options:
|
||||
|
||||
* [Docker Compose](appgw_install_compose.md)
|
||||
* [Kubernetes](appgw_install_k8s.md)
|
||||
|
@ -40,77 +40,72 @@ This is an example of the `docker-compose` file you will need:
|
||||
```
|
||||
version: '3.5'
|
||||
services:
|
||||
task_traffic_webserver:
|
||||
image: allegroai/task-traffic-router-webserver:${TASK-TRAFFIC-ROUTER-WEBSERVER-TAG}
|
||||
ports:
|
||||
- "80:8080"
|
||||
restart: unless-stopped
|
||||
container_name: task_traffic_webserver
|
||||
volumes:
|
||||
- ./task_traffic_router/config/nginx:/etc/nginx/conf.d:ro
|
||||
- ./task_traffic_router/config/lua:/usr/local/openresty/nginx/lua:ro
|
||||
task_traffic_router:
|
||||
image: allegroai/task-traffic-router:${TASK-TRAFFIC-ROUTER-TAG}
|
||||
restart: unless-stopped
|
||||
container_name: task_traffic_router
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock
|
||||
- ./task_traffic_router/config/nginx:/etc/nginx/conf.d:rw
|
||||
- ./task_traffic_router/config/lua:/usr/local/openresty/nginx/lua:rw
|
||||
environment:
|
||||
- LOGGER_LEVEL=INFO
|
||||
- CLEARML_API_HOST=${CLEARML_API_HOST:?err}
|
||||
- CLEARML_API_ACCESS_KEY=${CLEARML_API_ACCESS_KEY:?err}
|
||||
- CLEARML_API_SECRET_KEY=${CLEARML_API_SECRET_KEY:?err}
|
||||
- ROUTER_URL=${ROUTER_URL:?err}
|
||||
- ROUTER_NAME=${ROUTER_NAME:?err}
|
||||
- AUTH_ENABLED=${AUTH_ENABLED:?err}
|
||||
- SSL_VERIFY=${SSL_VERIFY:?err}
|
||||
- AUTH_COOKIE_NAME=${AUTH_COOKIE_NAME:?err}
|
||||
- AUTH_BASE64_JWKS_KEY=${AUTH_BASE64_JWKS_KEY:?err}
|
||||
- LISTEN_QUEUE_NAME=${LISTEN_QUEUE_NAME}
|
||||
- EXTRA_BASH_COMMAND=${EXTRA_BASH_COMMAND}
|
||||
- TCP_ROUTER_ADDRESS=${TCP_ROUTER_ADDRESS}
|
||||
- TCP_PORT_START=${TCP_PORT_START}
|
||||
- TCP_PORT_END=${TCP_PORT_END}
|
||||
|
||||
task_traffic_webserver:
|
||||
image: clearml/ai-gateway-proxy:${PROXY_TAG:?err}
|
||||
network_mode: "host"
|
||||
restart: unless-stopped
|
||||
container_name: task_traffic_webserver
|
||||
volumes:
|
||||
- ./task_traffic_router/config/nginx:/etc/nginx/conf.d:ro
|
||||
- ./task_traffic_router/config/lua:/usr/local/openresty/nginx/lua:ro
|
||||
task_traffic_router:
|
||||
image: clearml/ai-gateway-router:${ROUTER_TAG:?err}
|
||||
restart: unless-stopped
|
||||
container_name: task_traffic_router
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock
|
||||
- ./task_traffic_router/config/nginx:/etc/nginx/conf.d:rw
|
||||
- ./task_traffic_router/config/lua:/usr/local/openresty/nginx/lua:rw
|
||||
environment:
|
||||
- ROUTER_NAME=${ROUTER_NAME:?err}
|
||||
- ROUTER__WEBSERVER__SERVER_PORT=${ROUTER__WEBSERVER__SERVER_PORT:?err}
|
||||
- ROUTER_URL=${ROUTER_URL:?err}
|
||||
- CLEARML_API_HOST=${CLEARML_API_HOST:?err}
|
||||
- CLEARML_API_ACCESS_KEY=${CLEARML_API_ACCESS_KEY:?err}
|
||||
- CLEARML_API_SECRET_KEY=${CLEARML_API_SECRET_KEY:?err}
|
||||
- AUTH_COOKIE_NAME=${AUTH_COOKIE_NAME:?err}
|
||||
- AUTH_SECURE_ENABLED=${AUTH_SECURE_ENABLED}
|
||||
- TCP_ROUTER_ADDRESS=${TCP_ROUTER_ADDRESS}
|
||||
- TCP_PORT_START=${TCP_PORT_START}
|
||||
- TCP_PORT_END=${TCP_PORT_END}
|
||||
```
|
||||
|
||||
Create a *runtime.env* file containing the following entries:
|
||||
Create a `runtime.env` file containing the following entries:
|
||||
|
||||
```
|
||||
TASK-TRAFFIC-ROUTER-WEBSERVER-TAG=
|
||||
TASK-TRAFFIC-ROUTER-TAG=
|
||||
CLEARML_API_HOST=https://api.
|
||||
PROXY_TAG=
|
||||
ROUTER_TAG=
|
||||
ROUTER_NAME=main-router
|
||||
ROUTER__WEBSERVER__SERVER_PORT=8010
|
||||
ROUTER_URL=
|
||||
CLEARML_API_HOST=
|
||||
CLEARML_API_ACCESS_KEY=
|
||||
CLEARML_API_SECRET_KEY=
|
||||
ROUTER_URL=
|
||||
ROUTER_NAME=main-router
|
||||
AUTH_ENABLED=true
|
||||
SSL_VERIFY=true
|
||||
AUTH_COOKIE_NAME=
|
||||
AUTH_BASE64_JWKS_KEY=
|
||||
LISTEN_QUEUE_NAME=
|
||||
EXTRA_BASH_COMMAND=
|
||||
AUTH_SECURE_ENABLED=true
|
||||
TCP_ROUTER_ADDRESS=
|
||||
TCP_PORT_START=
|
||||
TCP_PORT_END=
|
||||
```
|
||||
|
||||
Edit it according to the following guidelines:
|
||||
|
||||
* `CLEARML_API_HOST`: URL usually starting with `https://api.`
|
||||
* `CLEARML_API_ACCESS_KEY`: ClearML server api key
|
||||
* `CLEARML_API_SECRET_KEY`: ClearML server secret key
|
||||
* `ROUTER_URL`: URL for this router that was previously configured in the load balancer starting with `https://`
|
||||
* `ROUTER_NAME`: Unique name for this router
|
||||
* `AUTH_ENABLED`: Enable or disable http calls authentication when the router is communicating with the ClearML server
|
||||
* `SSL_VERIFY`: Enable or disable SSL certificate validation when the router is communicating with the ClearML server
|
||||
* `AUTH_COOKIE_NAME`: Cookie name used by the ClearML server to store the ClearML authentication cookie. This can usually be found in the `value_prefix` key starting with `allegro_token` in `envoy.yaml` file in the ClearML server installation (`/opt/allegro/config/envoy/envoy.yaml`) (see below)
|
||||
* `AUTH_SECURE_ENABLED`: Enable the Set-Cookie `secure` parameter
|
||||
* `AUTH_BASE64_JWKS_KEY`: Value form `k` key in the `jwks.json` file in the ClearML server installation
|
||||
* `LISTEN_QUEUE_NAME`: (*optional*) Name of queue to check for tasks (if none, every task is checked)
|
||||
* `EXTRA_BASH_COMMAND`: Command to be launched before starting the router
|
||||
* `PROXY_TAG`: AI Application Gateway proxy tag. The Docker image tag for the proxy component, which needs to be
|
||||
specified during installation. This tag is provided by ClearML to ensure compatibility with the recommended version.
|
||||
* `ROUTER_TAG`: App Gateway Router tag. The Docker image tag for the router component. It defines the specific version
|
||||
to be installed and is provided by ClearML as part of the setup process.
|
||||
* `ROUTER_NAME`: In the case of [multiple routers on the same tenant](#multiple-router-in-the-same-tenant), each router
|
||||
needs to have a unique name.
|
||||
* `ROUTER__WEBSERVER__SERVER_PORT`: Webserver port. The default port is 8080, but it can be adjusted to meet specific network requirements.
|
||||
* `ROUTER_URL`: External address to access the router. This can be the IP address or DNS of the node where the router
|
||||
is running, or the address of a load balancer if the router operates behind a proxy/load balancer. This URL is used
|
||||
to access AI workload applications (e.g. remote IDE, model deployment, etc.), so it must be reachable and resolvable for them.
|
||||
* `CLEARML_API_HOST`: ClearML API server URL starting with `https://api.`
|
||||
* `CLEARML_API_ACCESS_KEY`: ClearML server API key.
|
||||
* `CLEARML_API_SECRET_KEY`: ClearML server secret key.
|
||||
* `AUTH_COOKIE_NAME`: Cookie used by the ClearML server to store the ClearML authentication cookie. This can usually be
|
||||
found in the `envoy.yaml` file in the ClearML server installation (`/opt/allegro/config/envoy/envoy.yaml`), under the
|
||||
`value_prefix` key starting with `allegro_token`
|
||||
* `AUTH_SECURE_ENABLED`: Enable the Set-Cookie `secure` parameter. Set to `false` in case services are exposed with `http`.
|
||||
* `TCP_ROUTER_ADDRESS`: Router external address, can be an IP or the host machine or a load balancer hostname, depends on network configuration
|
||||
* `TCP_PORT_START`: Start port for the TCP Session feature
|
||||
* `TCP_PORT_END`: End port for the TCP Session feature
|
||||
@ -121,12 +116,42 @@ Run the following command to start the router:
|
||||
sudo docker compose --env-file runtime.env up -d
|
||||
```
|
||||
|
||||
:::note How to find my jwkskey
|
||||
### Advanced Configuration
|
||||
|
||||
The *JSON Web Key Set* (*JWKS*) is a set of keys containing the public keys used to verify any JSON Web Token (JWT).
|
||||
#### Using Open HTTP
|
||||
|
||||
In a `docker-compose` server installation, this can be found in the `CLEARML__secure__auth__token_secret` env var in the apiserver server component.
|
||||
To deploy the App Gateway Router on open HTTP (without a certificate), set the `AUTH_SECURE_ENABLED` entry
|
||||
to `false` in the `runtime.env` file.
|
||||
|
||||
:::
|
||||
#### Multiple Router in the Same Tenant
|
||||
|
||||
If you have workloads running in separate networks that cannot communicate with each other, you need to deploy multiple
|
||||
routers, one for each isolated environment. Each router will only process tasks from designated queues, ensuring that
|
||||
tasks are correctly routed to agents within the same network.
|
||||
|
||||
For example:
|
||||
* If Agent A and Agent B are in separate networks, each must have its own router to receive tasks.
|
||||
* Router A will handle tasks from Agent A’s queues. Router B will handle tasks from Agent B’s queues.
|
||||
|
||||
To achieve this, each router must be configured with:
|
||||
* A unique `ROUTER_NAME`
|
||||
* A distinct set of queues defined in `LISTEN_QUEUE_NAME`.
|
||||
|
||||
##### Example Configuration
|
||||
Each router's `runtime.env` file should include:
|
||||
|
||||
* Router A:
|
||||
|
||||
```
|
||||
ROUTER_NAME=router-a
|
||||
LISTEN_QUEUE_NAME=queue1,queue2
|
||||
```
|
||||
|
||||
* Router B:
|
||||
|
||||
```
|
||||
ROUTER_NAME=router-b
|
||||
LISTEN_QUEUE_NAME=queue3,queue4
|
||||
```
|
||||
|
||||
Make sure `LISTEN_QUEUE_NAME` is set in the [`docker-compose` environment variables](#docker-compose-file) for each router instance.
|
||||
|
@ -3,17 +3,26 @@ title: Kubernetes Deployment
|
||||
---
|
||||
|
||||
:::important Enterprise Feature
|
||||
The Application Gateway is available under the ClearML Enterprise plan.
|
||||
The AI Application Gateway is available under the ClearML Enterprise plan.
|
||||
:::
|
||||
|
||||
This guide details the installation of the ClearML App Gateway Router.
|
||||
The App Gateway Router enables access to your AI workload applications (e.g. remote IDEs like VSCode and Jupyter, model API interface, etc.).
|
||||
It acts as a proxy, identifying ClearML Tasks running within its [K8s namespace](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/)
|
||||
and making them available for network access.
|
||||
|
||||
:::important
|
||||
The App Gateway Router must be installed in the same K8s namespace as a dedicated ClearML Agent.
|
||||
It can only configure access for ClearML Tasks within its own namespace.
|
||||
:::
|
||||
|
||||
This guide details the installation of the ClearML AI Application Gateway, specifically the ClearML Task Router Component.
|
||||
|
||||
## Requirements
|
||||
|
||||
* Kubernetes cluster: `>= 1.21.0-0 < 1.32.0-0`
|
||||
* Helm installed and configured
|
||||
* Helm token to access `allegroai` helm-chart repo
|
||||
* Credentials for `allegroai` docker repo
|
||||
* Helm token to access `clearml` helm-chart repo
|
||||
* Credentials for `clearml` docker repo
|
||||
* A valid ClearML Server installation
|
||||
|
||||
## Optional for HTTPS
|
||||
@ -26,62 +35,55 @@ This guide details the installation of the ClearML AI Application Gateway, speci
|
||||
### Login
|
||||
|
||||
```
|
||||
helm repo add allegroai-enterprise \
|
||||
helm repo add clearml-enterprise \
|
||||
https://raw.githubusercontent.com/clearml/clearml-enterprise-helm-charts/gh-pages \
|
||||
--username <GITHUB_TOKEN> \
|
||||
--password <GITHUB_TOKEN>
|
||||
```
|
||||
|
||||
Replace `<GITHUB_TOKEN>` with your valid GitHub token that has access to the ClearML Enterprise Helm charts repository.
|
||||
|
||||
### Prepare Values
|
||||
|
||||
Before installing the TTR, create a `helm-override` files named `task-traffic-router.values-override.yaml`:
|
||||
Before installing the App Gateway Router, create a Helm override file:
|
||||
|
||||
```
|
||||
imageCredentials:
|
||||
password: "<DOCKERHUB_TOKEN>"
|
||||
password: ""
|
||||
clearml:
|
||||
apiServerKey: ""
|
||||
apiServerSecret: ""
|
||||
apiServerUrlReference: "https://api."
|
||||
jwksKey: ""
|
||||
authCookieName: ""
|
||||
apiServerKey: ""
|
||||
apiServerSecret: ""
|
||||
apiServerUrlReference: ""
|
||||
authCookieName: ""
|
||||
sslVerify: true
|
||||
ingress:
|
||||
enabled: true
|
||||
hostName: "task-router.dev"
|
||||
enabled: true
|
||||
hostName: ""
|
||||
tcpSession:
|
||||
routerAddress: ""
|
||||
portRange:
|
||||
start:
|
||||
end:
|
||||
routerAddress: ""
|
||||
service:
|
||||
type: LoadBalancer
|
||||
portRange:
|
||||
start:
|
||||
end:
|
||||
```
|
||||
|
||||
Edit it accordingly to these guidelines:
|
||||
Configuration options:
|
||||
|
||||
* `clearml.apiServerUrlReference`: URL usually starting with `https://api.`
|
||||
* `clearml.apiServerKey`: ClearML server api key
|
||||
* `clearml.apiServerSecret`: ClearML server secret key
|
||||
* `ingress.hostName`: URL of router we configured previously for load balancer starting with `https://`
|
||||
* `clearml.sslVerify`: Enable or disable SSL certificate validation on apiserver calls check
|
||||
* `clearml.authCookieName`: Value from `value_prefix` key starting with `allegro_token` in `envoy.yaml` file in ClearML server installation.
|
||||
* `clearml.jwksKey`: Value form `k` key in `jwks.json` file in ClearML server installation (see below)
|
||||
* `tcpSession.routerAddress`: Router external address can be an IP or the host machine or a load balancer hostname, depends on the network configuration
|
||||
* `tcpSession.portRange.start`: Start port for the TCP Session feature
|
||||
* `tcpSession.portRange.end`: End port for the TCP Session feature
|
||||
|
||||
:::note How to find my jwkskey
|
||||
|
||||
The *JSON Web Key Set* (*JWKS*) is a set of keys containing the public keys used to verify any JSON Web Token (JWT).
|
||||
|
||||
```
|
||||
kubectl -n clearml get secret clearml-conf \
|
||||
-o jsonpath='{.data.secure_auth_token_secret}' \
|
||||
| base64 -d && echo
|
||||
```
|
||||
|
||||
:::
|
||||
* `imageCredentials.password`: ClearML DockerHub Access Token.
|
||||
* `clearml.apiServerKey`: ClearML server API key.
|
||||
* `clearml.apiServerSecret`: ClearML server secret key.
|
||||
* `clearml.apiServerUrlReference`: ClearML API server URL starting with `https://api.`.
|
||||
* `clearml.authCookieName`: Cookie used by the ClearML server to store the ClearML authentication cookie.
|
||||
* `clearml.sslVerify`: Enable or disable SSL certificate validation on `apiserver` calls check.
|
||||
* `ingress.hostName`: Hostname of router used by the ingress controller to access it.
|
||||
* `tcpSession.routerAddress`: The external router address (can be an IP, hostname, or load balancer address) depending on your network setup. Ensure this address is accessible for TCP connections.
|
||||
* `tcpSession.service.type`: Service type used to expose TCP functionality, default is `NodePort`.
|
||||
* `tcpSession.portRange.start`: Start port for the TCP Session feature.
|
||||
* `tcpSession.portRange.end`: End port for the TCP Session feature.
|
||||
|
||||
|
||||
The whole list of supported configuration is available with the command:
|
||||
The full list of supported configuration is available with the command:
|
||||
|
||||
```
|
||||
helm show readme allegroai-enterprise/clearml-enterprise-task-traffic-router
|
||||
@ -94,9 +96,22 @@ To install the TTR component via Helm use the following command:
|
||||
```
|
||||
helm upgrade --install \
|
||||
<RELEASE_NAME> \
|
||||
-n <NAME_SPACE> \
|
||||
-n <WORKLOAD_NAMESPACE> \
|
||||
allegroai-enterprise/clearml-enterprise-task-traffic-router \
|
||||
--version <CURRENT CHART VERSION> \
|
||||
-f task-traffic-router.values-override.yaml
|
||||
--version <CHART_VERSION> \
|
||||
-f override.yaml
|
||||
```
|
||||
|
||||
Replace the placeholders with the following values:
|
||||
|
||||
* `<RELEASE_NAME>` - Unique name for the App Gateway Router within the K8s namespace. This is a required parameter in
|
||||
Helm, which identifies a specific installation of the chart. The release name also defines the router’s name and
|
||||
appears in the UI within AI workload application URLs (e.g. Remote IDE URLs). This can be customized to support multiple installations within the same
|
||||
namespace by assigning different release names.
|
||||
* `<WORKLOAD_NAMESPACE>` - [Kubernetes Namespace](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/)
|
||||
where workloads will be executed. This namespace must be shared between a dedicated ClearML Agent and an App
|
||||
Gateway Router. The agent is responsible for monitoring its assigned task queues and spawning workloads within this
|
||||
namespace. The router monitors the same namespace for AI workloads (e.g. remote IDE applications). The router has a
|
||||
namespace-limited scope, meaning it can only detect and manage tasks within its
|
||||
assigned namespace.
|
||||
* `<CHART_VERSION>` - Version recommended by the ClearML Support Team.
|
@ -513,31 +513,30 @@ Create a `NetworkPolicy` in the tenant namespace with the following configuratio
|
||||
- podSelector: {}
|
||||
```
|
||||
|
||||
### Install Task Traffic Router Chart
|
||||
### Install the App Gateway Router Chart
|
||||
|
||||
Install the [Task Traffic Router](appgw.md) in your Kubernetes cluster, allowing it to manage and route tasks:
|
||||
Install the App Gateway Router in your Kubernetes cluster, allowing it to manage and route tasks:
|
||||
|
||||
1. Prepare the `overrides.yaml` file with the following content:
|
||||
|
||||
```
|
||||
imageCredentials:
|
||||
password: "<allegroaienterprise_DockerHub_TOKEN>"
|
||||
password: "<clearmlenterprise_DockerHub_TOKEN>"
|
||||
clearml:
|
||||
apiServerUrlReference: "<http://clearml-enterprise-apiserver.clearml:8008>"
|
||||
apiserverKey: "<TENANT_KEY>"
|
||||
apiserverSecret: "<TENANT_SECRET>"
|
||||
jwksKey: "<JWKS_KEY>"
|
||||
ingress:
|
||||
enabled: true
|
||||
hostName: "<unique url in same domain as apiserver/webserver>"
|
||||
```
|
||||
|
||||
2. Install Task Traffic Router in the specified tenant namespace:
|
||||
2. Install App Gateway Router in the specified tenant namespace:
|
||||
|
||||
```
|
||||
helm install -n <TENANT_NAMESPACE> \\
|
||||
clearml-ttr \\
|
||||
allegroai-enterprise/clearml-task-traffic-router \\
|
||||
clearml-enterprise/clearml-task-traffic-router \\
|
||||
--create-namespace \\
|
||||
-f overrides.yaml
|
||||
```
|
||||
|
@ -82,7 +82,7 @@ Currently, these runtime properties can only be set using an ClearML REST API ca
|
||||
endpoint, as follows:
|
||||
|
||||
* The body of the request must contain the `worker-id`, and the runtime property to add.
|
||||
* An expiry date is optional. Use the format `"expiry":<time>`. For example, `"expiry":86400` will set an expiry of 24 hours.
|
||||
* An expiry date is optional. Use the format `"expiry":<time>`. For example, `"expiry":86400` will set an expiry of 24 hours.
|
||||
* To delete the property, set the expiry date to zero, `"expiry":0`.
|
||||
|
||||
For example, to force a worker on for 24 hours:
|
||||
|
@ -6,9 +6,22 @@ ClearML provides a comprehensive set of monitoring tools to help effectively tra
|
||||
These tools offer both high-level overviews and detailed insights into task execution, resource
|
||||
utilization, and project performance.
|
||||
|
||||
## Offerings
|
||||
|
||||
### Project Dashboard
|
||||
## Project Overview
|
||||
|
||||
A project's **OVERVIEW** tab in the UI presents a general picture of a project:
|
||||
* Metric Snapshot – A graphical representation of selected metric values across project tasks, offering a quick assessment of progress.
|
||||
* Task Status Tracking – When a single metric variant is selected for the snapshot, task status is color-coded (e.g.,
|
||||
Completed, Aborted, Published, Failed) for better visibility.
|
||||
|
||||
Use the Metric Snapshot to track project progress and identify trends in task performance.
|
||||
|
||||
For more information, see [Project Overview](../webapp/webapp_project_overview.md).
|
||||
|
||||

|
||||

|
||||
|
||||
## Project Dashboard
|
||||
|
||||
:::info Pro Plan Offering
|
||||
The Project Dashboard app is available under the ClearML Pro plan.
|
||||
@ -28,16 +41,22 @@ For more information, see [Project Dashboard](../webapp/applications/apps_dashbo
|
||||

|
||||

|
||||
|
||||
### Project Overview
|
||||
## Task Monitoring
|
||||
|
||||
A project's **OVERVIEW** tab in the UI presents a general picture of a project:
|
||||
* Metric Snapshot – A graphical representation of selected metric values across project tasks, offering a quick assessment of progress.
|
||||
* Task Status Tracking – When a single metric variant is selected for the snapshot, task status is color-coded (e.g.,
|
||||
Completed, Aborted, Published, Failed) for better visibility.
|
||||
ClearML provides task monitoring capabilities through the [`clearml.automation.Monitor`](https://github.com/clearml/clearml/blob/master/clearml/automation/monitor.py)
|
||||
class. With this class you can implement monitoring workflows such as:
|
||||
|
||||
Use the Metric Snapshot to track project progress and identify trends in task performance.
|
||||
* Send notifications via Slack or other channels
|
||||
* Trigger automated responses based on specific task conditions
|
||||
|
||||
For more information, see [Project Overview](../webapp/webapp_project_overview.md).
|
||||
For a practical example, see the [Slack Alerts Example](../guides/services/slack_alerts.md), which demonstrates how to:
|
||||
|
||||
* Track task status (completion, failure, etc.)
|
||||
* Send notifications to a specified Slack channel
|
||||
* Retrieve task details such as status, console logs, and links to the ClearML Web UI
|
||||
|
||||
You can also configure filters for task types and projects to reduce unnecessary notifications.
|
||||
|
||||

|
||||

|
||||
|
||||

|
||||

|
||||
|
@ -6,11 +6,12 @@ The [Slack alerts example](https://github.com/clearml/clearml/blob/master/exampl
|
||||
demonstrates how to use the `clearml.automation.monitor` class to implement a service that monitors the completion and
|
||||
failure of tasks, and posts alert messages on a Slack channel.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Creating a Slack Bot
|
||||
## Creating a Slackbot
|
||||
|
||||
Before configuring and running the Slack alert service, create a Slack Bot (**ClearML Bot**).
|
||||
Before configuring and running the Slack alert service, create a Slackbot (**ClearML Bot**).
|
||||
|
||||
:::important
|
||||
The Slack API token and channel you create are required to configure the Slack alert service.
|
||||
|
@ -16,6 +16,10 @@ Use annotation tasks to efficiently organize the annotation of frames in Dataset
|
||||
|
||||
Click on an annotation task card to open the frame viewer, where you can view the task's frames and annotate them.
|
||||
|
||||
Use the search bar <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-sm" />
|
||||
to find specific annotation tasks. You can query by the task’s name, hyper-dataset version, and ID.
|
||||
To search using regex, click the `.*` icon on the search bar.
|
||||
|
||||
## Annotation Task Actions
|
||||
Click <img src="/docs/latest/icons/ico-bars-menu.svg" alt="Menu" className="icon size-md space-sm" /> on the top right
|
||||
of an annotation task card to open its context menu and access annotation task actions.
|
||||
|
@ -18,6 +18,10 @@ using the buttons on the top left of the page. Use the table view for a comparat
|
||||
columns of interest. Use the details view to access a selected Dataview's details, while keeping the Dataview list in view.
|
||||
Details view can also be accessed by double-clicking a specific Dataview in the table view to open its details view.
|
||||
|
||||
Use the search bar <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-sm" />
|
||||
to find specific dataviews. You can query by the dataview name, ID, description, hyper-datasets, and versions.
|
||||
To search using regex, click the `.*` icon on the search bar.
|
||||
|
||||
You can archive Dataviews so the Dataview table doesn't get too cluttered. Click **OPEN ARCHIVE** on the top of the
|
||||
table to open the archive and view all archived Dataviews. From the archive, you can restore
|
||||
Dataviews to remove them from the archive. You can also permanently delete Dataviews.
|
||||
|
Binary file not shown.
Before Width: | Height: | Size: 231 KiB After Width: | Height: | Size: 459 KiB |
BIN
docs/img/examples_slack_alerts_dark.png
Normal file
BIN
docs/img/examples_slack_alerts_dark.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 458 KiB |
@ -8,6 +8,7 @@ ClearML seamlessly integrates with a wide range of popular machine learning fram
|
||||
* [Keras](keras.md)
|
||||
* [YOLO v5](yolov5.md)
|
||||
* [YOLO v8](yolov8.md)
|
||||
* [Hugging Face Accelerate](accelerate.md)
|
||||
* [Hugging Face Transformers](transformers.md)
|
||||
* [MMEngine](mmengine.md)
|
||||
* [MMCV](mmcv.md)
|
||||
|
@ -77,12 +77,12 @@ cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents:
|
||||
and shuts down instances as needed, according to a resource budget that you set.
|
||||
|
||||
|
||||
### Reproducing Tasks
|
||||
### Reproducing Task Runs
|
||||
|
||||

|
||||

|
||||
|
||||
Use ClearML's web interface to reproduce tasks and edit their details, like hyperparameters or input models, then execute the tasks
|
||||
Use ClearML's web interface to reproduce task runs and edit their details, like hyperparameters or input models, then execute the tasks
|
||||
with the new configuration on a remote machine.
|
||||
|
||||
When ClearML is integrated into a script, it captures and stores configurations, such as hyperparameters
|
||||
|
@ -3,6 +3,33 @@ title: Version 3.24
|
||||
---
|
||||
|
||||
|
||||
### Enterprise Server 3.24.2
|
||||
|
||||
**New Features**
|
||||
* Add support for additional billing event formats
|
||||
* Improve login to multi-tenant service
|
||||
|
||||
**Bug Fixes**
|
||||
* Security fixes when exporting data to CSV
|
||||
* Fix access permissions to UI Applications
|
||||
|
||||
### Enterprise Server 3.24.1
|
||||
|
||||
**New Features**
|
||||
* Add service for presigning AWS S3 URLs
|
||||
* Add support for additional billing event formats
|
||||
* Add support for user-customized UI themes
|
||||
* Add configuration to disable adding users to the workspace via the UI
|
||||
* Add on-demand refresh to UI applications
|
||||
* Add grouped same-event view to UI "Latest Task Events"
|
||||
|
||||
**Bug Fixes**
|
||||
* Fix downloaded CSV file of UI “Latest Task Events” missing some events
|
||||
* Fix access permissions to UI Reports
|
||||
* Fix configuration modal of UI application instance displays incorrect values
|
||||
* Fix UI Hyper-Dataset frame viewer navigation controls not displaying
|
||||
|
||||
|
||||
### Enterprise Server 3.24.0
|
||||
|
||||
**Default Behavior Change: Access Rules Enabled**
|
||||
@ -20,7 +47,7 @@ title: Version 3.24
|
||||
* Add UI pipeline DAG presentation with pipeline steps grouped into stages
|
||||
* Add Hyper-Dataset version sort by update time in ascending and descending order
|
||||
* Add opacity control to Hyper-Dataset frame ROIs
|
||||
* Add search bar for UI Settings's Users and Groups
|
||||
* Add search bar for UI Settings' Users and Groups
|
||||
* Add number of frames display to UI DataView preview
|
||||
* Remove legacy "Augmentation" sections from UI Dataview pages
|
||||
* Add control to collapse and expand UI Hyper-Dataset version list
|
||||
|
@ -13,7 +13,7 @@ running, it serves your embedding model through a secure, publicly accessible ne
|
||||
endpoint activity and shuts down if the model remains inactive for a specified maximum idle time.
|
||||
|
||||
:::info AI Application Gateway
|
||||
The Embedding Model Deployment app makes use of the ClearML Traffic Router which implements a secure, authenticated
|
||||
The Embedding Model Deployment app makes use of the App Gateway Router which implements a secure, authenticated
|
||||
network endpoint for the model.
|
||||
|
||||
If the ClearML AI application Gateway is not available, the model endpoint might not be accessible.
|
||||
|
@ -16,7 +16,7 @@ The Gradio launcher monitors the Gradio app activity and shuts down if it is ina
|
||||
<a id="traffic_router"/>
|
||||
|
||||
:::important AI Application Gateway
|
||||
The Gradio Launcher relies on the ClearML Traffic Router which implements user authentication, and redirects requests
|
||||
The Gradio Launcher relies on the ClearML App Gateway Router which implements user authentication, and redirects requests
|
||||
to the IP/port served by the Gradio app.
|
||||
|
||||
If the ClearML AI application Gateway is not available, the Gradio app might not be accessible.
|
||||
|
@ -12,7 +12,7 @@ running, it serves your model through a secure, publicly accessible network endp
|
||||
and shuts down if the model remains inactive for a specified maximum idle time.
|
||||
|
||||
:::important AI Application Gateway
|
||||
The llama.cpp Model Deployment app makes use of the ClearML Traffic Router which implements a secure, authenticated
|
||||
The llama.cpp Model Deployment app makes use of the App Gateway Router which implements a secure, authenticated
|
||||
network endpoint for the model.
|
||||
|
||||
If the ClearML AI application Gateway is not available, the model endpoint might not be accessible.
|
||||
|
@ -13,7 +13,7 @@ it serves your model through a secure, publicly accessible network endpoint. The
|
||||
shuts down if the model remains inactive for a specified maximum idle time.
|
||||
|
||||
:::info AI Application Gateway
|
||||
The vLLM Model Deployment app makes use of the ClearML Traffic Router which implements a secure, authenticated
|
||||
The vLLM Model Deployment app makes use of the App Gateway Router which implements a secure, authenticated
|
||||
network endpoint for the model.
|
||||
|
||||
If the ClearML AI application Gateway is not available, the model endpoint might not be accessible.
|
||||
|
@ -17,7 +17,7 @@ time.
|
||||
<a id="traffic_router"/>
|
||||
|
||||
:::important AI Application Gateway
|
||||
The Streamlit Launcher relies on the ClearML Traffic Router which implements user authentication, and redirects requests
|
||||
The Streamlit Launcher relies on the ClearML App Gateway Router which implements user authentication, and redirects requests
|
||||
to the IP/port served by the Streamlit app.
|
||||
|
||||
If the ClearML AI application Gateway is not available, the Streamlit app might not be accessible.
|
||||
|
@ -25,6 +25,10 @@ Filter the datasets to find the one you're looking for more easily. These filter
|
||||
respectively. These options appear on the top of the tag list.
|
||||
* Filter by the absence of a tag (logical "NOT") by clicking its checkbox twice. An X will appear in the tag's checkbox.
|
||||
|
||||
Use the search bar <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-sm" />
|
||||
to find specific datasets. You can query by the dataset name, ID, or description. To search using regex, click the `.*`
|
||||
icon on the search bar.
|
||||
|
||||

|
||||

|
||||
|
||||
|
@ -12,9 +12,13 @@ or comparison view <img src="/docs/latest/icons/ico-charts-view.svg" alt="Compar
|
||||
using the buttons on the top left of the page. Use the details view to access a selected run's details, while keeping
|
||||
the run list in view. Details view can also be accessed by double-clicking a specific pipeline run in the table view to
|
||||
open its details view. Use the [comparison view](#comparing-runs) to compare your pipeline run's scalar and plot results.
|
||||
This view compares the scalars/plots of currently selected pipeline runs. If no runs are selected, The first 100 visible
|
||||
This view compares the scalars/plots of currently selected pipeline runs. If no runs are selected, the first 100 visible
|
||||
runs in the table are compared.
|
||||
|
||||
Use the search bar <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-sm" />
|
||||
to find specific pipeline runs. You can query by the run name, ID, and description.
|
||||
To search using regex, click the `.*` icon on the search bar.
|
||||
|
||||
You can archive pipeline runs so the runs table doesn't get too cluttered. Click **OPEN ARCHIVE** on the top of the
|
||||
table to open the archive and view all archived runs. From the archive, you can restore
|
||||
runs to remove them from the archive. You can also permanently delete runs.
|
||||
|
@ -17,6 +17,11 @@ depth comparison, see [Comparing Tasks](webapp_exp_comparing.md)). This view com
|
||||
the scalars/plots of currently selected tasks. If no tasks are selected, the first 100
|
||||
visible tasks in the table are compared.
|
||||
|
||||
Use the search bar <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-sm" />
|
||||
to find specific tasks. You can query by the task name, ID, description and input and output models. In the Enterprise version,
|
||||
you can also query by the task’s dataviews' hyper-datasets and versions. To search using regex, click the `.*`
|
||||
icon on the search bar.
|
||||
|
||||
You can archive tasks so the table doesn't get too cluttered. Click **OPEN ARCHIVE** on the top of the
|
||||
table to open the archive and view all archived tasks. From the archive, you can restore
|
||||
tasks to remove them from the archive. You can also permanently delete tasks.
|
||||
|
@ -6,9 +6,9 @@ Tune tasks and edit their execution details, then execute the tuned tasks on loc
|
||||
|
||||
## To Tune a Task and Execute it Remotely:
|
||||
|
||||
1. Locate the task. Open the task's Project page from the Dashboard or the main Projects page.
|
||||
1. Locate the task. Open the task's Project page from the Project Dashboard or the main Projects page.
|
||||
|
||||
* On the Dashboard,
|
||||
* On the Project Dashboard,
|
||||
* Click on a task from RECENT TASKS
|
||||
* In RECENT PROJECTS **>** click on a project card **>** click task
|
||||
* In RECENT PROJECTS **>** click **VIEW ALL** **>** click the project card **>** click task
|
||||
|
@ -1,36 +1,17 @@
|
||||
---
|
||||
title: Dashboard
|
||||
title: Project Dashboard
|
||||
---
|
||||
|
||||
|
||||
The **Dashboard** provides the following options:
|
||||
* Quickly access the summarized monitoring of recently updated projects and their experiments
|
||||
* Create new projects
|
||||
* Open the [**Orchestration**](webapp_workers_queues.md) tab to autoscale, monitor, and manage your resource usage and
|
||||
worker queues.
|
||||
The **project dashboard** provides a summary of your most recent projects, reports, and tasks. Click a project, report
|
||||
or task to quickly access it.
|
||||
|
||||

|
||||

|
||||
|
||||
**To select a project, experiment, or model:**
|
||||
To access the [projects page](webapp_projects_page.md), click the `View All` button next to the recent projects summary.
|
||||
Similarly, to access the [reports page](webapp_reports.md), click the `View All` button next to the recent reports summary.
|
||||
|
||||
* A project or all projects - to view activity for all experiments in a project.
|
||||
* In **RECENT PROJECTS**, click a specific project's card or **VIEW ALL**
|
||||
|
||||
* An experiment - to view experiment results, edit an experiment, enqueue an experiment to execute, etc.
|
||||
* In **RECENT EXPERIMENTS**, click the experiment.
|
||||
* In **RECENT PROJECTS**, click a project or **VIEW ALL** **>** Click the experiment.
|
||||
To create a new project, click the `+ New Project` button. Similarly, to create a new report, click the `+ New Report`.
|
||||
|
||||
* A model - to view a model's configuration, label enumeration, or other details.
|
||||
* From an experiment, click **ARTIFACTS** **>** In **Input Model** or **Output Model**, click the model.
|
||||
* In **RECENT PROJECTS**, click a project or **VIEW ALL** **>** **MODELS** tab **>** Click the model.
|
||||
|
||||
**To create a new project:**
|
||||
|
||||
1. Click **+ NEW PROJECT**
|
||||
1. Enter the project name, and, optionally, the description and default output destination
|
||||
1. Click **CREATE PROJECT**
|
||||
|
||||
**To autoscale, monitor, and manage your resource usage and workers queues:**
|
||||
|
||||
* Click **MANAGE WORKERS AND QUEUES** to go to the [**Orchestration**](webapp_workers_queues.md) page.
|
||||
To access the [orchestration page](webapp_workers_queues.md), click `Manage Workers and Queues` (Open Source)/`Orchestration Control Center`.
|
@ -11,6 +11,10 @@ using the buttons on the top left of the page. Use the table view for a comparat
|
||||
columns of interest. Use the details view to access a selected model's details, while keeping the model list in view.
|
||||
Details view can also be accessed by double-clicking a specific model in the table view to open its details view.
|
||||
|
||||
Use the search bar <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-sm" />
|
||||
to find specific models. You can query by the model name, ID and description. To search using regex, click the `.*`
|
||||
icon on the search bar.
|
||||
|
||||
You can archive models so the model table doesn't get too cluttered. Click **OPEN ARCHIVE** on the top of the
|
||||
table to open the archive and view all archived models. From the archive, you can restore
|
||||
models to remove them from the archive, and permanently delete models.
|
||||
|
@ -3,11 +3,13 @@ title: WebApp
|
||||
---
|
||||
|
||||
The **ClearML Web UI** is the graphical user interface for the ClearML platform, which includes:
|
||||
* Task management
|
||||
* Browsing
|
||||
* Resource utilization monitoring
|
||||
* Profile management
|
||||
* Direct access to the ClearML community (Slack channel, YouTube, and GitHub).
|
||||
* ML workload automation
|
||||
* Resource utilization monitoring and management
|
||||
* Live model endpoint monitoring
|
||||
* ML experiment management and visualization
|
||||
* Model and Dataset viewing and management
|
||||
* Pipeline creation and monitoring
|
||||
* User and administrator settings
|
||||
|
||||

|
||||

|
||||
@ -15,43 +17,46 @@ The **ClearML Web UI** is the graphical user interface for the ClearML platform,
|
||||
## UI Modules
|
||||
The WebApp's sidebar provides access to the following modules:
|
||||
|
||||
* <img src="/docs/latest/icons/ico-homepage.svg" alt="Homepage" className="icon size-md space-md" />[Dashboard](webapp_home.md) - The dashboard for recent activity and quick access to tasks and projects.
|
||||
|
||||
* <img src="/docs/latest/icons/ico-projects.svg" alt="Projects" className="icon size-md space-md" />[Projects](webapp_projects_page.md) - The main experimentation page. Access your tasks and models as they are organized into projects. The tasks and models are displayed in tables which let you:
|
||||
* Track ongoing tasks and visualize their results
|
||||
* Reproduce previously run tasks
|
||||
* Tune tasks with no code change
|
||||
* Compare tasks
|
||||
* Share tasks and their models with other ClearML hosted service users
|
||||
* <img src="/docs/latest/icons/ico-side-bar-datasets.svg" alt="Datasets" className="icon size-md space-md" />[Datasets](datasets/webapp_dataset_page.md) - View and manage your datasets.
|
||||
* <img src="/docs/latest/icons/ico-pipelines.svg" alt="Pipelines" className="icon size-md space-md" />[Pipelines](pipelines/webapp_pipeline_page.md) - View and manage your pipelines.
|
||||
* <img src="/docs/latest/icons/ico-model-endpoints.svg" alt="Model endpoints" className="icon size-md space-md" />[Model Endpoints](webapp_model_endpoints.md) - Monitor your live model endpoints.
|
||||
* <img src="/docs/latest/icons/ico-reports.svg" alt="Reports" className="icon size-md space-md" />[Reports](webapp_reports.md) - View and manage your reports.
|
||||
* <img src="/docs/latest/icons/ico-workers.svg" alt="Workers and Queues" className="icon size-md space-md" />[Orchestration](webapp_workers_queues.md) - Autoscale, monitor, and manage your resource usage and workers queues.
|
||||
* <img src="/docs/latest/icons/ico-applications.svg" alt="ClearML Apps" className="icon size-md space-md" />[Applications](applications/apps_overview.md) - ClearML's GUI applications for no-code workflow execution (available in the ClearML Pro and Enterprise plans).
|
||||
* <img src="/docs/latest/icons/ico-workers.svg" alt="Workers and Queues" className="icon size-md space-md" />[Orchestration](webapp_workers_queues.md) - Autoscaling, resource usage monitoring and allocation management.
|
||||
* <img src="/docs/latest/icons/ico-model-endpoints.svg" alt="Model endpoints" className="icon size-md space-md" />[Model Endpoints](webapp_model_endpoints.md) - Monitor your live model endpoints.
|
||||
* <img src="/docs/latest/icons/ico-side-bar-datasets.svg" alt="Datasets" className="icon size-md space-md" />[Datasets](datasets/webapp_dataset_page.md) - View and manage your datasets.
|
||||
* <img src="/docs/latest/icons/ico-projects.svg" alt="Projects" className="icon size-md space-md" />[Projects](webapp_projects_page.md) - The main experimentation page. Access your tasks and models as they are organized into projects. The tasks and models are displayed in tables which let you:
|
||||
* Track ongoing tasks and visualize their results
|
||||
* Reproduce previous task runs
|
||||
* Tune task parameter values with no code change
|
||||
* Compare tasks and models
|
||||
* Share tasks and models with other ClearML hosted service users
|
||||
* Create and share rich content [Reports](webapp_reports.md)
|
||||
* <img src="/docs/latest/icons/ico-pipelines.svg" alt="Pipelines" className="icon size-md space-md" />[Pipelines](pipelines/webapp_pipeline_page.md) - View and manage your pipelines.
|
||||
|
||||
## UI Top Bar
|
||||
### Settings Menu
|
||||
|
||||
Click the profile menu button <img src="/docs/latest/icons/ico-me.svg" alt="Profile button" className="icon size-lg space-sm" />
|
||||
in the top right corner of the web UI screen to access the following:
|
||||
* **Settings** - Navigate to ClearML's user [Settings](settings/webapp_settings_profile.md) page:
|
||||
* Set [WebApp preferences](settings/webapp_settings_profile.md)
|
||||
* Manage [workspace API credentials](settings/webapp_settings_profile.md#clearml-api-credentials)
|
||||
to access the following:
|
||||
* **Settings** - Navigate to ClearML's [Settings](settings/webapp_settings_profile.md) page:
|
||||
* Set personal [WebApp preferences](settings/webapp_settings_profile.md)
|
||||
* Manage [workspace API credentials](settings/webapp_settings_profile.md#clearml-api-credentials)
|
||||
* Manage [personal configuration vault](settings/webapp_settings_profile.md#configuration-vault) (Enterprise offering)
|
||||
* Configure [cloud storage access credentials](settings/webapp_settings_profile.md#browser-cloud-storage-access) for the ClearML Web UI
|
||||
* ClearML Hosted service specific options
|
||||
* Administrator settings
|
||||
* Manage [users and workspaces](settings/webapp_settings_users.md)
|
||||
* Manage [resource access permissions](settings/webapp_settings_access_rules.md) (available in the ClearML Enterprise plan)
|
||||
* View [usage and billing](settings/webapp_settings_usage_billing.md) information (Free Hosted Service)
|
||||
* Manage [access rules](settings/webapp_settings_access_rules.md) (available in the ClearML Enterprise plan)
|
||||
* Define [configuration vaults](settings/webapp_settings_admin_vaults.md) to apply to designated user groups (available in the ClearML Enterprise plan)
|
||||
* Manage [server identity providers](settings/webapp_settings_id_providers.md) (available in the ClearML Enterprise plan)
|
||||
* Define the [available resources](settings/webapp_settings_resource_configs.md) and the way in which they will be
|
||||
allocated to different workloads (available in the ClearML Enterprise plan)
|
||||
* View [billing and usage](settings/webapp_settings_usage_billing.md) information
|
||||
* **Invite a User** to your workspace (supported in hosted service). Click **Invite a User** > input user's
|
||||
email > click **ADD** > page redirects to the [Users & Groups](settings/webapp_settings_users.md#user-groups) section of
|
||||
* Define the [resource access policies](settings/webapp_settings_resource_configs.md) (available in the ClearML Enterprise plan)
|
||||
* Workspace Control (Free Hosted Service)
|
||||
* **Invite a User** to your workspace (supported in hosted service). Click **Invite a User** > input user's
|
||||
email > click **ADD** > page redirects to the [Users & Groups](settings/webapp_settings_users.md#user-groups) section of
|
||||
the **Settings** page
|
||||
* **Switch to Workspace** - Hosted service users can be members of multiple workspaces. These workspaces are listed here.
|
||||
* **Switch to Workspace** - Hosted service users can be members of multiple workspaces. These workspaces are listed here.
|
||||
Click a workspace to switch to.
|
||||
* Appearance - Select the UI color scheme:
|
||||
* Light: ClearML will be in a light theme.
|
||||
* Dark: ClearML will be in a dark theme.
|
||||
* System: ClearML will follow your device’s theme.
|
||||
* **Logout** of ClearML
|
||||
|
||||
### Finding What You're Looking for
|
||||
@ -60,26 +65,18 @@ to find your ClearML resources.
|
||||
|
||||
To search using regex, click the `.*` icon on the search bar.
|
||||
|
||||
Search results are returned for the different ClearML objects:
|
||||
* Tasks - Searching a task table looks for matches in the tasks' name, ID, description and input and
|
||||
output models. The enterprise version also includes task Dataviews' hyper-datasets and versions.
|
||||
* Models - Searching a Model table looks for matches in the models' name, ID and description.
|
||||
* Dataviews (Enterprise only) - Searching a Dataview table looks for matches in the Dataviews' name, ID, description,
|
||||
hyper-datasets, and versions.
|
||||
* Datasets - Searching the datasets page looks for matches in the datasets' name, ID, and description. In a dataset's
|
||||
version table, a search looks for matches in the versions' name, ID, and description.
|
||||
* Pipelines - Searching the pipelines page looks for matches in the datasets' name, ID, and description. In a pipeline
|
||||
run's table, a search looks for matches in the runs' name and ID.
|
||||
* Reports - Searching the reports page looks for matches in the reports' name, ID, tags, project, description, and
|
||||
report content.
|
||||
The search functionality is tailored to each page, returning results specific to the object type displayed on the page.
|
||||
For example, searching a task table looks for matches in the tasks' name, ID, description and input and output models.
|
||||
On the reports page, it matches reports by name, ID, tags, project, description, and content. Similarly, searches in
|
||||
models, datasets, pipelines, dataviews, and annotations, focus on attributes relevant to their respective objects.
|
||||
|
||||
The search bar in the [Dashboard](webapp_home.md) page searches the whole WebApp for objects that match queries as
|
||||
The search bar in the [Project Dashboard](webapp_home.md) page searches the whole WebApp for objects that match queries as
|
||||
specified above and returns results divided by object type (projects, tasks, models, etc.).
|
||||
|
||||
:::tip Additional filtering
|
||||
ClearML's object tables (e.g. [tasks](webapp_exp_table.md), [models](webapp_model_table.md), [pipelines](pipelines/webapp_pipeline_table.md),
|
||||
and [datasets](datasets/webapp_dataset_page.md)) provide column filters to easily focus your search by object properties
|
||||
(e.g status, creation/update time, metric values, etc.).
|
||||
(e.g. status, creation/update time, metric values, etc.).
|
||||
:::
|
||||
|
||||
### Helpful Resources
|
||||
|
@ -18,6 +18,10 @@ contents (i.e. tasks, models etc.) via the folder with the bracketed (`[ ]`) pro
|
||||
If a project does not contain any subprojects, clicking on its folder will open its task table (or [Project Overview](webapp_project_overview.md)
|
||||
page when relevant).
|
||||
|
||||
Use the search bar <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-sm" />
|
||||
to find specific projects. You can query by the project’s name and ID.
|
||||
To search using regex, click the `.*` icon on the search bar.
|
||||
|
||||
## Project Folders
|
||||
|
||||
Project folders display summarized project information:
|
||||
|
@ -175,6 +175,10 @@ or in List view <img src="/docs/latest/icons/ico-flat-view.svg" alt="List view"
|
||||
view, all reports are shown side-by-side. In Project view, reports are organized according to their projects, and
|
||||
top-level projects are displayed. Click on a project card to view the project's reports.
|
||||
|
||||
Use the search bar <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-sm" />
|
||||
to find specific reports. You can query by the report name, ID, tags, project, description, and report content.
|
||||
To search using regex, click the `.*` icon on the search bar.
|
||||
|
||||

|
||||

|
||||
|
||||
|
34
package-lock.json
generated
34
package-lock.json
generated
@ -737,11 +737,13 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@babel/helpers": {
|
||||
"version": "7.26.7",
|
||||
"version": "7.26.10",
|
||||
"resolved": "https://registry.npmjs.org/@babel/helpers/-/helpers-7.26.10.tgz",
|
||||
"integrity": "sha512-UPYc3SauzZ3JGgj87GgZ89JVdC5dj0AoetR5Bw6wj4niittNyFh6+eOGonYvJ1ao6B8lEa3Q3klS7ADZ53bc5g==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@babel/template": "^7.25.9",
|
||||
"@babel/types": "^7.26.7"
|
||||
"@babel/template": "^7.26.9",
|
||||
"@babel/types": "^7.26.10"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=6.9.0"
|
||||
@ -826,10 +828,12 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@babel/parser": {
|
||||
"version": "7.26.8",
|
||||
"version": "7.26.10",
|
||||
"resolved": "https://registry.npmjs.org/@babel/parser/-/parser-7.26.10.tgz",
|
||||
"integrity": "sha512-6aQR2zGE/QFi8JpDLjUZEPYOs7+mhKXm86VaKFiLP35JQwQb6bwUE+XbvkH0EptsYhbNBSUGaUBLKqxH1xSgsA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@babel/types": "^7.26.8"
|
||||
"@babel/types": "^7.26.10"
|
||||
},
|
||||
"bin": {
|
||||
"parser": "bin/babel-parser.js"
|
||||
@ -1939,7 +1943,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@babel/runtime": {
|
||||
"version": "7.26.7",
|
||||
"version": "7.26.10",
|
||||
"resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.26.10.tgz",
|
||||
"integrity": "sha512-2WJMeRQPHKSPemqk/awGrAiuFfzBmOIPXKizAsVhWH9YJqLZ0H+HS4c8loHGgW6utJ3E/ejXQUsiGaQy2NZ9Fw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"regenerator-runtime": "^0.14.0"
|
||||
@ -1960,12 +1966,14 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@babel/template": {
|
||||
"version": "7.26.8",
|
||||
"version": "7.26.9",
|
||||
"resolved": "https://registry.npmjs.org/@babel/template/-/template-7.26.9.tgz",
|
||||
"integrity": "sha512-qyRplbeIpNZhmzOysF/wFMuP9sctmh2cFzRAZOn1YapxBsE1i9bJIY586R/WBLfLcmcBlM8ROBiQURnnNy+zfA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@babel/code-frame": "^7.26.2",
|
||||
"@babel/parser": "^7.26.8",
|
||||
"@babel/types": "^7.26.8"
|
||||
"@babel/parser": "^7.26.9",
|
||||
"@babel/types": "^7.26.9"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=6.9.0"
|
||||
@ -1988,7 +1996,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@babel/types": {
|
||||
"version": "7.26.8",
|
||||
"version": "7.26.10",
|
||||
"resolved": "https://registry.npmjs.org/@babel/types/-/types-7.26.10.tgz",
|
||||
"integrity": "sha512-emqcG3vHrpxUKTrxcblR36dcrcoRDvKmnL/dCL6ZsHaShW80qxCAcNhzQZrpeM765VzEos+xOi4s+r4IXzTwdQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@babel/helper-string-parser": "^7.25.9",
|
||||
@ -15567,7 +15577,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/prismjs": {
|
||||
"version": "1.29.0",
|
||||
"version": "1.30.0",
|
||||
"resolved": "https://registry.npmjs.org/prismjs/-/prismjs-1.30.0.tgz",
|
||||
"integrity": "sha512-DEvV2ZF2r2/63V+tK8hQvrR2ZGn10srHbXviTlcv7Kpzw8jWiNTqbVgjO3IY8RxrrOUF8VPMQQFysYYYv0YZxw==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=6"
|
||||
|
217
sidebars.js
217
sidebars.js
@ -470,68 +470,6 @@ module.exports = {
|
||||
label: 'WebApp',
|
||||
link: {type: 'doc', id: 'webapp/webapp_overview'},
|
||||
items: [
|
||||
'webapp/webapp_home',
|
||||
{'Projects': [
|
||||
'webapp/webapp_projects_page',
|
||||
'webapp/webapp_project_overview',
|
||||
{'Tasks': [
|
||||
'webapp/webapp_exp_table',
|
||||
'webapp/webapp_exp_track_visual',
|
||||
'webapp/webapp_exp_reproducing',
|
||||
'webapp/webapp_exp_tuning',
|
||||
'webapp/webapp_exp_comparing'
|
||||
]},
|
||||
{'Models': [
|
||||
'webapp/webapp_model_table',
|
||||
'webapp/webapp_model_viewing',
|
||||
'webapp/webapp_model_comparing'
|
||||
]},
|
||||
{'Dataviews': [
|
||||
'hyperdatasets/webapp/webapp_dataviews',
|
||||
'hyperdatasets/webapp/webapp_exp_track_visual',
|
||||
'hyperdatasets/webapp/webapp_exp_modifying',
|
||||
'hyperdatasets/webapp/webapp_exp_comparing'
|
||||
]},
|
||||
'webapp/webapp_exp_sharing'
|
||||
]},
|
||||
{'Datasets': [
|
||||
'webapp/datasets/webapp_dataset_page',
|
||||
'webapp/datasets/webapp_dataset_viewing'
|
||||
]},
|
||||
{'Hyper-Datasets': [
|
||||
'hyperdatasets/webapp/webapp_datasets',
|
||||
'hyperdatasets/webapp/webapp_datasets_versioning',
|
||||
'hyperdatasets/webapp/webapp_datasets_frames',
|
||||
'hyperdatasets/webapp/webapp_annotator'
|
||||
]},
|
||||
{'Pipelines': [
|
||||
'webapp/pipelines/webapp_pipeline_page',
|
||||
'webapp/pipelines/webapp_pipeline_table',
|
||||
'webapp/pipelines/webapp_pipeline_viewing'
|
||||
]},
|
||||
'webapp/webapp_model_endpoints',
|
||||
'webapp/webapp_reports',
|
||||
{
|
||||
type: 'category',
|
||||
collapsible: true,
|
||||
collapsed: true,
|
||||
label: 'Orchestration',
|
||||
link: {type: 'doc', id: 'webapp/webapp_workers_queues'},
|
||||
items: [
|
||||
'webapp/webapp_orchestration_dash',
|
||||
{
|
||||
type: 'category',
|
||||
collapsible: true,
|
||||
collapsed: true,
|
||||
label: 'Autoscalers',
|
||||
items: [
|
||||
'webapp/applications/apps_aws_autoscaler',
|
||||
'webapp/applications/apps_gcp_autoscaler',
|
||||
]
|
||||
},
|
||||
'webapp/resource_policies'
|
||||
]
|
||||
},
|
||||
{
|
||||
type: 'category',
|
||||
collapsible: true,
|
||||
@ -561,6 +499,70 @@ module.exports = {
|
||||
]},
|
||||
]
|
||||
},
|
||||
{
|
||||
type: 'category',
|
||||
collapsible: true,
|
||||
collapsed: true,
|
||||
label: 'Orchestration',
|
||||
link: {type: 'doc', id: 'webapp/webapp_workers_queues'},
|
||||
items: [
|
||||
'webapp/webapp_orchestration_dash',
|
||||
{
|
||||
type: 'category',
|
||||
collapsible: true,
|
||||
collapsed: true,
|
||||
label: 'Autoscalers',
|
||||
items: [
|
||||
'webapp/applications/apps_aws_autoscaler',
|
||||
'webapp/applications/apps_gcp_autoscaler',
|
||||
]
|
||||
},
|
||||
'webapp/resource_policies'
|
||||
]
|
||||
},
|
||||
'webapp/webapp_model_endpoints',
|
||||
{'Datasets': [
|
||||
'webapp/datasets/webapp_dataset_page',
|
||||
'webapp/datasets/webapp_dataset_viewing'
|
||||
]
|
||||
},
|
||||
{'Hyper-Datasets': [
|
||||
'hyperdatasets/webapp/webapp_datasets',
|
||||
'hyperdatasets/webapp/webapp_datasets_versioning',
|
||||
'hyperdatasets/webapp/webapp_datasets_frames',
|
||||
'hyperdatasets/webapp/webapp_annotator'
|
||||
]},
|
||||
{'Projects': [
|
||||
'webapp/webapp_home',
|
||||
'webapp/webapp_projects_page',
|
||||
'webapp/webapp_project_overview',
|
||||
{'Tasks': [
|
||||
'webapp/webapp_exp_table',
|
||||
'webapp/webapp_exp_track_visual',
|
||||
'webapp/webapp_exp_reproducing',
|
||||
'webapp/webapp_exp_tuning',
|
||||
'webapp/webapp_exp_comparing'
|
||||
]},
|
||||
{'Models': [
|
||||
'webapp/webapp_model_table',
|
||||
'webapp/webapp_model_viewing',
|
||||
'webapp/webapp_model_comparing'
|
||||
]},
|
||||
{'Dataviews': [
|
||||
'hyperdatasets/webapp/webapp_dataviews',
|
||||
'hyperdatasets/webapp/webapp_exp_track_visual',
|
||||
'hyperdatasets/webapp/webapp_exp_modifying',
|
||||
'hyperdatasets/webapp/webapp_exp_comparing'
|
||||
]},
|
||||
'webapp/webapp_exp_sharing',
|
||||
'webapp/webapp_reports',
|
||||
]},
|
||||
{'Pipelines': [
|
||||
'webapp/pipelines/webapp_pipeline_page',
|
||||
'webapp/pipelines/webapp_pipeline_table',
|
||||
'webapp/pipelines/webapp_pipeline_viewing'
|
||||
]},
|
||||
|
||||
{
|
||||
type: 'category',
|
||||
collapsible: true,
|
||||
@ -634,61 +636,56 @@ module.exports = {
|
||||
]},
|
||||
]
|
||||
},
|
||||
/* {'Getting Started': [
|
||||
'getting_started/architecture',
|
||||
]},*/
|
||||
{
|
||||
'Enterprise Server': {
|
||||
'Deployment Options': [
|
||||
'deploying_clearml/enterprise_deploy/multi_tenant_k8s',
|
||||
'deploying_clearml/enterprise_deploy/vpc_aws',
|
||||
'deploying_clearml/enterprise_deploy/on_prem_ubuntu',
|
||||
],
|
||||
'Maintenance and Migration': [
|
||||
'deploying_clearml/enterprise_deploy/import_projects',
|
||||
'deploying_clearml/enterprise_deploy/change_artifact_links',
|
||||
'deploying_clearml/enterprise_deploy/delete_tenant',
|
||||
]
|
||||
|
||||
}
|
||||
},
|
||||
{
|
||||
type: 'category',
|
||||
collapsible: true,
|
||||
collapsed: true,
|
||||
label: 'ClearML Application Gateway',
|
||||
label: 'Enterprise Server',
|
||||
items: [
|
||||
'deploying_clearml/enterprise_deploy/appgw_install_compose',
|
||||
'deploying_clearml/enterprise_deploy/appgw_install_compose_hosted',
|
||||
'deploying_clearml/enterprise_deploy/appgw_install_k8s',
|
||||
]
|
||||
},
|
||||
'deploying_clearml/enterprise_deploy/custom_billing',
|
||||
{
|
||||
'UI Applications': [
|
||||
'deploying_clearml/enterprise_deploy/app_install_ubuntu_on_prem',
|
||||
'deploying_clearml/enterprise_deploy/app_install_ex_server',
|
||||
'deploying_clearml/enterprise_deploy/app_custom',
|
||||
]
|
||||
},
|
||||
{
|
||||
'User Management': [
|
||||
'user_management/user_groups',
|
||||
'user_management/access_rules',
|
||||
'user_management/admin_vaults',
|
||||
{
|
||||
type: 'category',
|
||||
collapsible: true,
|
||||
collapsed: true,
|
||||
label: 'Identity Provider Integration',
|
||||
link: {type: 'doc', id: 'user_management/identity_providers'},
|
||||
items: [
|
||||
'deploying_clearml/enterprise_deploy/sso_multi_tenant_login',
|
||||
'deploying_clearml/enterprise_deploy/sso_saml_k8s',
|
||||
'deploying_clearml/enterprise_deploy/sso_keycloak',
|
||||
'deploying_clearml/enterprise_deploy/sso_active_directory'
|
||||
{'Deployment Options': [
|
||||
'deploying_clearml/enterprise_deploy/multi_tenant_k8s',
|
||||
'deploying_clearml/enterprise_deploy/vpc_aws',
|
||||
'deploying_clearml/enterprise_deploy/on_prem_ubuntu',
|
||||
]
|
||||
},
|
||||
{'Maintenance and Migration': [
|
||||
'deploying_clearml/enterprise_deploy/import_projects',
|
||||
'deploying_clearml/enterprise_deploy/change_artifact_links',
|
||||
'deploying_clearml/enterprise_deploy/delete_tenant',
|
||||
]
|
||||
},
|
||||
{'ClearML Application Gateway': [
|
||||
'deploying_clearml/enterprise_deploy/appgw_install_compose',
|
||||
'deploying_clearml/enterprise_deploy/appgw_install_compose_hosted',
|
||||
'deploying_clearml/enterprise_deploy/appgw_install_k8s',
|
||||
]
|
||||
},
|
||||
'deploying_clearml/enterprise_deploy/custom_billing',
|
||||
{'UI Applications': [
|
||||
'deploying_clearml/enterprise_deploy/app_install_ubuntu_on_prem',
|
||||
'deploying_clearml/enterprise_deploy/app_install_ex_server',
|
||||
'deploying_clearml/enterprise_deploy/app_custom',
|
||||
]
|
||||
},
|
||||
{'User Management': [
|
||||
'user_management/user_groups',
|
||||
'user_management/access_rules',
|
||||
'user_management/admin_vaults',
|
||||
{
|
||||
type: 'category',
|
||||
collapsible: true,
|
||||
collapsed: true,
|
||||
label: 'Identity Provider Integration',
|
||||
link: {type: 'doc', id: 'user_management/identity_providers'},
|
||||
items: [
|
||||
'deploying_clearml/enterprise_deploy/sso_multi_tenant_login',
|
||||
'deploying_clearml/enterprise_deploy/sso_saml_k8s',
|
||||
'deploying_clearml/enterprise_deploy/sso_keycloak',
|
||||
'deploying_clearml/enterprise_deploy/sso_active_directory'
|
||||
]
|
||||
},
|
||||
]
|
||||
},
|
||||
]
|
||||
},
|
||||
],
|
||||
|
Loading…
Reference in New Issue
Block a user