ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-06-26 18:17:44 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Update docs (#948)

Browse Source
This commit is contained in:
pollfly
2024-11-03 15:32:30 +02:00
committed by GitHub
parent af5639ac99
commit be102be4c3
51 changed files with 458 additions and 48 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/webapp/applications/apps_overview.md
View File

@@ -50,6 +50,8 @@ Each application's page is split into two sections:
* App Instance List - Launch new app instances and view previously launched instances. Click on an instance to view its
dashboard. Hover over it to access the [app instance actions](#app-instance-actions).
* App Instance Dashboard - The main section of the app page: displays the selected app instance's status and results.
Use the search bar <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-sm" />
to quickly find an app instance by name.
![App format](../../img/apps_format_overview.png)

49
docs/webapp/pipelines/webapp_pipeline_table.md
View File

@@ -6,11 +6,14 @@ The pipeline runs table is a [customizable](#customizing-the-runs-table) list of
view a run's details, and manage runs (create, continue, or abort). The runs table's auto-refresh allows users
to continually monitor run progress.
View the runs table in table view <img src="/docs/latest/icons/ico-table-view.svg" alt="Table view" className="icon size-md space-sm" />
or in details view <img src="/docs/latest/icons/ico-split-view.svg" alt="Details view" className="icon size-md space-sm" />,
using the buttons on the top left of the page. Use the table view for a comparative view of your runs according to
columns of interest. Use the details view to access a selected run's details, while keeping the pipeline runs list in view.
Details view can also be accessed by double-clicking a specific pipeline run in the table view to open its details view.
View the runs table in table view <img src="/docs/latest/icons/ico-table-view.svg" alt="Table view" className="icon size-md space-sm" />,
details view <img src="/docs/latest/icons/ico-split-view.svg" alt="Details view" className="icon size-md space-sm" />,
or comparison view <img src="/docs/latest/icons/ico-charts-view.svg" alt="Comparison view" className="icon size-md space-sm" />
using the buttons on the top left of the page. Use the details view to access a selected run's details, while keeping
the run list in view. Details view can also be accessed by double-clicking a specific pipeline run in the table view to
open its details view. Use the [comparison view](#comparing-runs) to compare your pipeline run's scalar and plot results.
This view compares the scalars/plots of currently selected pipeline runs. If no runs are selected, The first 100 visible
runs in the table are compared.
You can archive pipeline runs so the runs table doesn't get too cluttered. Click **OPEN ARCHIVE** on the top of the
table to open the archive and view all archived runs. From the archive, you can restore
@@ -146,3 +149,39 @@ selecting items beyond the items currently on-screen:
* **None** - Clear selection
* **Filtered** - Select all runs in the project that match the current active table filters
## Comparing Runs
The comparison view compares pipeline run scalar and plot results. When selected, the view presents a comparison of all
[selected runs](#selecting-multiple-runs). If no runs are selected, the first 100 visible runs in the table are compared.
In the dropdown menu, select to view **Scalars** or **Plots**.
**Scalars** shows pipeline run scalar results as time series line graphs.
![Scalar line graphs](../../img/pipelines_comparison_scalars.png)
All single value scalars are plotted into a single clustered bar chart under the "Summary" title, where each cluster
represents a reported metric, and each bar in the cluster represents an experiment.
![Single scalar comparison](../../img/pipelines_comparison_single_scalar.png)
Click <img src="/docs/latest/icons/ico-settings.svg" alt="Setting Gear" className="icon size-md" /> to customize which
metrics to view.
In the **Scalars** view, click <img src="/docs/latest/icons/ico-tune.svg" alt="Tuning" className="icon size-md" /> to access [scalar plot tools](../webapp_exp_track_visual.md#scalar-plot-tools).
**Plots** shows the last reported iteration sample of each metric/variant combination per compared run.
Line, scatter, box, and bar graphs are compared by overlaying each metric/variant from all compared runs' into a single
comparative plot.
![Merged plots](../../img/pipelines_comparison_plots_merged.png)
Other plot types are displayed separately for each run.
![Side-by-side plots](../../img/pipelines_comparison_plots.png)
### Run Details Comparison
For a more in depth comparison of pipeline runs, select the runs to compare and click **Compare** in the batch action
bar. In the run comparison pages, you can compare details, hyperparameters, scalars, plots, and debug samples. For more
information, see [Comparing Experiments](../webapp_exp_comparing.md).

40
docs/webapp/settings/webapp_settings_admin_vaults.md
View File

@@ -7,10 +7,17 @@ This feature is available under the ClearML Enterprise plan.
:::
Administrators can define multiple [configuration vaults](webapp_settings_profile.md#configuration-vault) which will each be applied to designated
[user groups](webapp_settings_users.md). Use configuration vaults to extend and/or override entries in the local ClearML [configuration file](../../configs/clearml_conf.md)
where a ClearML task is executed. Configuration vault values will be applied to tasks run by members of the designated user groups.
[user groups](webapp_settings_users.md). There are two types of vaults:
* [Client configuration (Agent/SDK/CLI)](#client-configuration-agentsdkcli)
* [UI storage credentials](#ui-storage-credentials)
To apply its contents, a vault should be enabled. New entries will extend the configuration in the local ClearML [configuration file](../../configs/clearml_conf.md).
To apply its contents, a vault should be enabled in the [Administrator Vault Table](#administrator-vault-table).
## Client Configuration (Agent/SDK/CLI)
Client configuration vaults extend and/or override entries in the local ClearML [configuration file](../../configs/clearml_conf.md)
where a task is executed. Vault values will be applied to tasks run by members of the designated user groups.
New entries will extend the configuration in the local ClearML [configuration file](../../configs/clearml_conf.md).
Most existing configuration file entries will be overridden by the vault values.
:::info
@@ -23,24 +30,41 @@ The following configuration values are machine and/or agent specific, so they ca
* `agent.debug`
:::
**To create a vault:**
**To create a Client configuration vault:**
1. Click **+ Add Vault**
1. Fill in vault details:
1. Vault name - Name that appears in the Administrator Vaults table
1. User Group - Specify the User Group that the vault affects
1. Format - Specify the configuration format: HOCON / JSON / YAML.
1. Fill in the configuration values (click <img src="/docs/latest/icons/ico-info.svg" alt="Info" className="icon size-md space-sm" />
to view configuration file reference). To import and existing configuration file, click <img src="/docs/latest/icons/ico-import.svg" alt="Import" className="icon size-md space-sm" />.
1. Target - Vault type. Select `Client (Agent/SDK/UI)`
1. Format - Specify the configuration format: HOCON / JSON / YAML.
1. Fill in the configuration values (click <img src="/docs/latest/icons/ico-info.svg" alt="Info" className="icon size-md space-sm" />
to view configuration file reference). To import an existing configuration file, click <img src="/docs/latest/icons/ico-import.svg" alt="Import" className="icon size-md space-sm" />.
1. Click **Save**
## UI Storage Credentials
UI storage credential vaults configure UI access to cloud storage credentials for a designated group of users.
**To create a vault:**
1. Click **+ Add Vault**
1. Fill in vault details:
1. Vault name - Name that appears in the Administrator Vaults table
1. User Group - Specify the User Group that the vault affects
1. Target - Select `UI storage credentials`
1. \+ Add access keys - Enter storage credentials (see [Browser Cloud Storage Access](webapp_settings_profile.md#browser-cloud-storage-access))
1. Click **Save**
## Administrator Vault Table
The **Administrator Vaults** table lists all currently defined vaults, and the following details:
* Active - Toggle to enable / disable the vault
* Name - Vault name
* Target - Type of vault: `Client (Agent/SDK/CLI)` or `UI storage credentials`
* Group - User groups to apply this vault to
* ID - Vault ID (click to copy)
* Vault Content - Vault content summary
* Update - Last update time
Hover over a vault in the table to Download, Edit, or Delete a vault.
Hover over a vault in the table to **Download**, **Edit**, or **Delete** a vault.
![Admin vaults](../../img/settings_admin_vaults.png)

4
docs/webapp/settings/webapp_settings_overview.md
View File

@@ -24,4 +24,6 @@ The Settings page consists of the following sections:
* [Identity Providers](webapp_settings_id_providers.md) (ClearML Enterprise Server) - Manage server identity providers
* [Resource Configuration](webapp_settings_resource_configs.md) (ClearML Enterprise Server) - Define the available resources and the way in which they
will be allocated to different workloads
* [Usage & Billing](webapp_settings_usage_billing.md) (ClearML Hosted Service) - View current usage information and billing details
* [Usage & Billing](webapp_settings_usage_billing.md) (ClearML Hosted Service) - View current usage information and billing details
* [Storage Credentials](webapp_settings_storage_credentials.md) (ClearML Enterprise Server) - Configure storage provider access credentials to
enable ClearML to delete artifacts stored in cloud storage when tasks and models are deleted

16
docs/webapp/settings/webapp_settings_profile.md
View File

@@ -62,7 +62,7 @@ to switch to.
![Workspace configuration page](../../img/settings_workspace_configuration.png)
### ClearML Credentials
### ClearML App Credentials
Generate ClearML credentials, made up of an access and secret key pair, and insert them into your [configuration file](../../configs/clearml_conf.md)
or Jupyter Notebook to grant the ClearML SDK and the ClearML Agent API access to the server.
@@ -91,6 +91,20 @@ these credentials cannot be recovered.
**To revoke ClearML credentials:** hover over the desired credentials, and click <img src="/docs/latest/icons/ico-trash.svg" alt="Trash can" className="icon size-md" />
## AI Application Gateway Tokens
:::important Enterprise Feature
This feature is available under the ClearML Enterprise plan.
:::
The AI Application Gateway enables external access to ClearML tasks and applications. The gateway is configured with an
endpoint or external address (ingress), accessible from outside ClearML.
Generate tokens providing API access to the AI Application Gateway endpoints:
1. Click **Generate a Token**
1. Under `Expiration`, enter the number of days the token should remain valid
1. Click `Generate`, which creates a token and copies it to your clipboard
### Changing Your Workspace Name
To change the name of your own workspace, click **Edit workspace name** <img src="/docs/latest/icons/ico-edit.svg" alt="Edit Pencil" className="icon size-md" />

55
docs/webapp/settings/webapp_settings_storage_credentials.md Normal file
View File

@@ -0,0 +1,55 @@
---
title: Storage Credentials
---
:::important Enterprise Feature
This feature is available under the ClearML Enterprise plan.
:::
To enable ClearML to delete task artifacts stored in cloud storage when a task is deleted, configure access credentials for your storage provider:
* [Google Cloud Storage](#google-cloud-storage)
* [AWS S3 Storage](#aws-s3-storage)
* [Azure](#azure)
![Storage Credentials page](../../img/webapp_settings_storage_credentials.png)
## Google Cloud Storage
Set up credentials for Google Cloud buckets:
* Default credentials - These credentials apply to all GCS buckets unless bucket-specific credentials are set.
* Project - Default Google Cloud Storage project
* Credentials JSON
* Bucket specific credentials:
* Bucket
* Project
* Credentials JSON
## AWS S3 Storage
Set up credentials for S3 protocol storage (i.e. AWS S3, MinIO, etc.):
* Default credentials - These credentials apply to all buckets unless bucket-specific credentials are set:
* Access Key - Default access key for the storage service.
* Secret - Default secret access key.
* Access token - Session key for temporary credentials (if applicable).
* Region - Default region for all unspecified buckets.
* Credentials Chain - If selected, use boto3 default [credentials search](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html#configuring-credentials)
(i.e. look for credentials in environment variables, credential files, and instance metadata services).
* Bucket Specific Credentials:
* Bucket - Name of the specific bucket.
* Region - Region for the bucket.
* Host - For non-AWS endpoints, the host URL and port number of the specific bucket. Note that port specification
is *always* needed (e.g. `my-minio-host:9000`), even for standard ports like 433 for HTTPS (e.g. `my-minio-host:433`)
* Secure Host - Select in order to enable TLS.
* Verify SSL certificate - Select to enable SSL verification for secure hosts.
* Access key - Access key for the bucket.
* Secret - Secret key for the bucket.
* Access token - The session key for your bucket. This is only needed when you are using temporary credentials.
* Use Credentials chain - If selected, use boto3 default [credentials search](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html#configuring-credentials)
(i.e. looks for credentials in environment variables, credential files, and instance metadata services).
## Azure
Set up credentials for Azure storage containers:
* Account name - Azure storage account name.
* Account key - Azure storage account key.
* Container name - Name of the specific container.

26
docs/webapp/settings/webapp_settings_users.md
View File

@@ -109,6 +109,32 @@ To revoke a set of credentials:
1. In the editing panel, hover of the relevant credential's row
2. Click the <img src="/docs/latest/icons/ico-trash.svg" alt="Trash can" className="icon size-md" /> button
### Service Account Configuration Vault
Use a service account’s configuration vault to store ClearML configuration entries that can extend the ClearML
[configuration file](../../configs/clearml_conf.md) of any ClearML Agents or ClearML SDK running with the service account's
credentials.
Vault entries will extend the configuration in the ClearML [configuration file](../../configs/clearml_conf.md) if they
don't yet exist, and override values for those already present in the file.
Fill in values using any of ClearML supported configuration formats: HOCON / JSON / YAML.
To edit vault contents:
1. Click on the relevant service account to open its details panel
2. Click **EDIT** on the configuration vault
3. Insert / edit the configurations in the vault
4. Press **OK**
To apply vault contents:
* Click the toggle atop the vault to enable / disable the configurations
* Once enabled, the configurations will be merged to the configuration file during ClearML and ClearML Agent usage
In addition to the service account-specific configuration vault, [administrator vaults](#administrator-vaults) can also
be applied to service accounts. See all the vaults applied to the account in the **Applied administrator vaults** below
the configuration vault.
![Service Account Config Vault](../../img/settings_service_account_config_vault.png)
### Deleting Service Account
Deleting a service account will revoke its credentials, causing agents using the account's credentials to fail.
Tasks and associated artifacts logged to your workspace by a service account will remain in your workspace.

2
docs/webapp/webapp_exp_comparing.md
View File

@@ -145,7 +145,7 @@ The **Scalars** (Graph view) and **Plots** tabs compare experiments' plots.
The **Scalars** tab displays scalar values as time series line charts. The **Plots** tab compares the last reported
iteration sample of each metric/variant combination per compared experiment.
Line, scatter, and bar graphs are compared by overlaying each metric/variant from all compared experiments' into a single
Line, scatter, box, and bar graphs are compared by overlaying each metric/variant from all compared experiments' into a single
comparative plot.
Use **Group by** to select how to group plots:

5
docs/webapp/webapp_exp_table.md
View File

@@ -223,11 +223,12 @@ to open the context menu
| Archive | Move experiment to the project's archive. If it is shared (ClearML Hosted Service only), the experiment becomes private. | Any state | *Pending* to *Draft* |
| Restore |Action available in the archive. Restore an experiment to the active experiments table.| Any State | None |
| Delete | Action available in the archive. Delete an experiment, which will also remove all their logs, results, artifacts and debug samples. | Any State | N/A |
| Enqueue | Add an experiment to a queue for a worker or workers (listening to the queue) to execute. | *Draft* | *Pending* |
| Enqueue | Add an experiment to a queue for a worker or workers (listening to the queue) to execute. | *Draft*, *Aborted* | *Pending* |
| Dequeue | Remove an experiment from a queue. | *Pending* | *Draft* |
| Reset | Delete the log and output from a previous run of an experiment (for example, before rerunning it). | *Completed*, *Aborted*, or *Failed* | *Draft* |
| Abort | Manually terminate a *Running* experiment. | *Running* | *Aborted* |
| Abort All Children | Manually terminate all *Running* experiments which have this task as a parent | *Running* or *Aborted* | None for parent experiment, *Aborted* for child experiments |
| Retry | Enqueue a failed experiment in order to rerun it. Make sure you have resolved the external problem which previously prevented the experiment’s completion. | *Failed* | *Pending* |
| Publish | Publish an experiment to prevent changes to its tracking data, inputs, and outputs. Published experiments and their models are read-only. *Published* experiments cannot be enqueued, but they can be cloned, and their clones can be edited, tuned, and enqueued. | *Completed*, *Aborted*, or *Failed*. | *Published* |
| Add Tag | Tag experiments with color-coded labels to assist you in organizing your work. See [tagging experiments](webapp_exp_track_visual.md#tagging-experiments). | Any state | None |
| Clone | Make an exact, editable copy of an experiment (for example, to reproduce an experiment, but keep the original). | *Draft* | Newly Cloned Experiment is *Draft* |
@@ -289,7 +290,7 @@ Click <img src="/docs/latest/icons/ico-tune.svg" alt="Tune" className="icon size
**Plots** shows the last reported iteration sample of each metric/variant combination per compared
experiment.
Line, scatter, and bar graphs are compared by overlaying each metric/variant from all compared experiments' into a
Line, scatter, box, and bar graphs are compared by overlaying each metric/variant from all compared experiments' into a
single comparative plot.
![Merged comparison plots](../img/webapp_compare_view_4.png)

78
docs/webapp/webapp_exp_track_visual.md
View File

@@ -8,7 +8,7 @@ including:
* [Execution details](#execution) - Code, the base Docker image used for [ClearML Agent](../clearml_agent.md), output destination for artifacts, and the logging level.
* [Configuration](#configuration) - Hyperparameters, user properties, and configuration objects.
* [Artifacts](#artifacts) - Input model, output model, model snapshot locations, other artifacts.
* [General information](#general-information) - Information about the experiment, for example: the experiment start, create, and last update times and dates, user creating the experiment, and its description.
* [Info](#info) - Extended experiment information, such as the experiment start, create, and last update times and dates, user creating the experiment, and its description.
* [Console](#console) - stdout, stderr, output to the console from libraries, and ClearML explicit reporting.
* [Scalars](#scalars) - Metric plots.
* [Plots](#plots) - Other plots and data, for example: Matplotlib, Plotly, and ClearML explicit reporting.
@@ -216,10 +216,41 @@ In the **ARTIFACTS** tab **>** **OTHER** **>** Select an artifact **>** Either:
![Other artifacts section](../img/webapp_tracking_30.png)
## General Information
## Info
The **INFO** tab shows extended experiment information:
* [Latest experiment events log](#latest-events-log)
* [Experiment description](#description)
* [Experiment details](#experiment-details)
### Latest Events Log
:::important Enterprise Feature
This feature is available under the ClearML Enterprise plan
:::
The Enterprise Server also displays a detailed history of task activity:
* Task action (e.g. status changes, project move, etc.)
* Action time
* Acting user
* Action source (i.e. ClearML Agent, SDK, or UI)
* Action source version
To download the task history as a CSV file, hover over the log and click <img src="/docs/latest/icons/ico-download.svg" alt="Download" className="icon size-md space-sm" />.
![Task audit log](../img/webapp_info_audit_log.png)
:::note Limited persistency
ClearML maintains a system-wide, large but strict limit for task history items. Once the limit is reached, the oldest entries are purged to make room for fresh entries.
:::
### Description
Add descriptive text to the experiment in the **Description** section. To modify the description, hover over the
description box and click **Edit**.
### Experiment Details
The **Experiment Details** section lists information describing the experiment:
General experiment details appear in the **INFO** tab. This includes information describing the stored experiment:
* Experiment description (editable, hover over the description box and click **Edit**)
* The parent experiment
* Project name
* Creation, start, and last update dates and times
@@ -262,19 +293,21 @@ is downloadable. To view the end of the log, click **Jump to end**.
All scalars that ClearML automatically logs, as well as those explicitly reported in code, appear in
**SCALARS**.
Scalar series are presented in a line chart. To see the series for a metric in high resolution,
view it in full screen mode by hovering over the graph and clicking <img src="/docs/latest/icons/ico-maximize.svg" alt="Maximize plot icon" className="icon size-sm space-sm" />.
Scalar series can be displayed in [graph view](#graph-view) (default) or in [metric values view](#metric-values-view):
#### Graph View
Scalar graph view (<img src="/docs/latest/icons/ico-charts-view.svg" alt="Graph view" className="icon size-md space-sm" />)
shows scalar series plotted as a time series line chart. The series are sub-sampled for
display efficiency. For high resolution, view a series in full screen mode by hovering over the graph and clicking <img src="/docs/latest/icons/ico-maximize.svg" alt="Maximize plot icon" className="icon size-sm space-sm" />.
Single value scalars (see [`Logger.report_single_value`](../references/sdk/logger.md#report_single_value)) are shown in
a `Summary` table.
:::info Full Screen Refresh
Scalar graphs in full screen mode do not auto-refresh. Click <img src="/docs/latest/icons/ico-reset.svg" alt="Refresh" className="icon size-md space-sm medium-zoom-image" />
to update the graph.
:::
Reported single value scalars are aggregated into a table plot displaying scalar names and values (see [`Logger.report_single_value`](../references/sdk/logger.md#report_single_value)).
#### Scalar Plot Tools
Use the scalar tools to improve analysis of scalar metrics. In the info panel, click <img src="/docs/latest/icons/ico-settings.svg" alt="Settings gear" className="icon size-md space-sm" /> to use the tools. In the full screen details view, the tools
are on the left side of the window. The tools include:
* **Group by** - Select one of the following:
@@ -310,6 +343,29 @@ are retrieved when the report is displayed allowing your reports to show the lat
See additional [plot controls](#plot-controls) below.
#### Metric Values View
The metric values view (<img src="/docs/latest/icons/ico-table-view.svg" alt="Table view" className="icon size-md space-sm" />)
shows a table summary of your metrics with a row per metric/variant:
* First - The metric/variant series' initial value
* Last - The metric/variant series' last value
* Min - The metric/variant series' minimum value
* Max - The metric/variant series' maximum value
* Mean - The metric/variant series' mean value
If all the values of a specific metric/variant are the same, the row will display a <img src="/docs/latest/icons/ico-equal-outline.svg" alt="Same values" className="icon size-md space-sm" /> sign.
![Plots tab](../img/webapp_tracking_34a.png)
Choose which metrics to view using one of the following out the options:
* Use the quick filter bar (<img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-sm" />)
to only show metrics/variants whose name fit a partial-string match
* Use the filter menu (<img src="/docs/latest/icons/ico-filter-off.svg" alt="Filter" className="icon size-md" />) to
select which metrics to view by clicking their show/hide button (<img src="/docs/latest/icons/ico-show.svg" alt="Eye Show" className="icon size-md space-sm" />).
Click **Hide/Show all**, to quickly hide/show all metrics.
### Plots
Non-time-series plots appear in **PLOTS**. These include data generated by libraries, visualization tools, and
explicitly reported using the ClearML Logger. These may include 2D and 3D plots, tables (Pandas and CSV files), and

18
docs/webapp/webapp_exp_tuning.md
View File

@@ -56,16 +56,18 @@ User parameters are editable in any experiment, except experiments whose status
#### Source Code
Select source code by changing any of the following:
Modify code execution by changing any of the following:
* Repository, commit (select by ID, tag name, or choose the last commit in the branch), script, and/or working directory.
* Installed Python packages and/or versions - Edit or clear (remove) them all.
* Uncommitted changes - Edit or clear (remove) them all.
* Repository, commit (select by ID, tag name, or choose the last commit in the branch), script, working directory,
and/or binary.
* The Python packages to be installed and/or their versions - Edit the package list, or clear it to have the ClearML
Agent either not install any packages or use an existing repo `requirements.txt` file. If the experiment is based on a
run in which the packages used were eventually different to the ones originally specified, you can easily or reset the
packages to originally recorded values ("Original Pip").
* Uncommitted changes - Edit or clear all.
**To select different source code:**
* In the **EXECUTION** tab, hover over a section **>** **EDIT** or (**DISCARD DIFFS** for **UNCOMMITTED CHANGES**) **>**
edit **>** **SAVE**.
**To modify the source code**, hover over the relevant sections in the **EXECUTION** tab to access Edit, Clear/Discard,
and/or Reset functions.

2
docs/webapp/webapp_model_comparing.md
View File

@@ -83,7 +83,7 @@ The **Scalars** (Graph view) and **Plots** tabs display plots attached to the mo
scalar values as time series line charts. The **Plots** tab compares the last reported iteration sample of each
metric/variant combination per compared model.
Line, scatter, and bar graphs are compared by a single plot per metric/variant into which the plots of all compared
Line, scatter, box, and bar graphs are compared by a single plot per metric/variant into which the plots of all compared
models are combined.
![Merged plots](../img/webapp_compare_models_merge_plots.png)

93
docs/webapp/webapp_model_endpoints.md Normal file
View File

@@ -0,0 +1,93 @@
---
title: Model Endpoints
---
:::important Enterprise Feature
This feature is available under the ClearML Enterprise plan.
:::
The Model Endpoints table lists all currently live (active, and being brought up) model endpoints, allowing you to view
endpoint details and monitor status over time. Whenever you deploy a model through the [ClearML Deploy UI applications](applications/apps_overview.md#deploy),
it will be listed in the table.
View the model endpoints in table view <img src="/docs/latest/icons/ico-table-view.svg" alt="Table view" className="icon size-md space-sm" />
or in details view <img src="/docs/latest/icons/ico-split-view.svg" alt="Details view" className="icon size-md space-sm" />
using the buttons on the top left of the page. Use the table view for a comparative view of your endpoints according to
columns of interest. Use the details view to access a selected endpoint's details. Details view can also be accessed by
double-clicking a specific endpoint in the table view to open its details view.
Download the model endpoint table as a CSV file by clicking **Download** <img src="/docs/latest/icons/ico-download.svg" alt="Download" className="icon size-sm space-sm" />.
## Endpoint Tables
Active Endpoints are displayed in the **Active** tab. The table provides the following information:
* Endpoint - Endpoint name
* Model - Model Name
* URL - Endpoint URL
* \# Instances - Number of model instances
* Uptime - Longest duration that any of the model instances has been running
* \# Requests - Total number of requests to the endpoint
* Requests / MIN (avg) - Average request rate in the last minute
* Latency (avg) - Average endpoint response latency
![Active endpoints](../img/webapp_model_endpoints_active_table.png)
The **Loading** tab shows endpoints that are being set up, but are not yet active. The table provides the following
information:
* Instance ID - The model deployment application instance ID
* Model - Model Name
* Uptime - Time since this endpoint has started setting up
* Preprocess artifact - Preprocessing code used for the endpoint
* Input type - Model matrix input type (e.g. uint8, float32, int16, float16)
* Input size - Model matrix input size
![Loading endpoints](../img/webapp_model_endpoints_loading_table.png)
You can apply column filters by clicking <img src="/docs/latest/icons/ico-filter-off.svg" alt="Filter" className="icon size-md" />,
and sort endpoints by clicking <img src="/docs/latest/icons/ico-sort-off.svg" alt="Sort order" className="icon size-md space-sm" /> on the relevant column.
## Active Endpoint Details
Clicking on a model endpoint opens it in detailed view.
### Details
The **Details** tab displays the model endpoint information:
* Endpoint Name
* Endpoint URL
* Model name (click to go to ClearML or HuggingFace model page)
* Uptime - Duration of longest running endpoint instance
* Preprocess artifact - Preprocessing code used for the endpoint
* Input type - Model matrix input type (e.g. uint8, float32, int16, float16)
* Input size - Model matrix input size
* Model instances list - Model instances servicing the endpoint. The following information is displayed for each instance:
* Instance ID - Click instance ID to go to the model endpoint's ClearML Application instance dashboard .
* Uptime
* Number of requests - Total count
* Request rate - Average requests per minute
* CPU Count
* GPU Count
* Latency - Average request latency in the last minute
![Endpoints details](../img/webapp_model_endpoints_details.png)
### Monitor
The Monitor tab displays the endpoint's operational metrics and resource usage over time graphs:
* Total number of requests
* Average requests/min
* Average latency
* Machine utilization metrics:
* CPU and GPU Usage
* Memory Usage
* Video Memory Usage
* Network Usage
![Endpoints monitor](../img/webapp_model_endpoints_monitor.png)
The graphs' time span can be controlled through the menu at its top right corner.
Click <img src="/docs/latest/icons/ico-show.svg" alt="Eye Show" className="icon size-md space-sm" /> to control which
plots to display.
For example, to display specific plots, click **HIDE ALL**, and then click <img src="/docs/latest/icons/ico-show.svg" alt="Eye Show" className="icon size-md space-sm" />
on each plot you want to view.

5
docs/webapp/webapp_overview.md
View File

@@ -35,6 +35,9 @@ The WebApp's sidebar provides access to the following modules:
<li>
<img src="/docs/latest/icons/ico-pipelines.svg" alt="Pipelines" className="icon size-md space-sm" /> <a href="pipelines/webapp_pipeline_page">Pipelines</a> - View and manage your pipelines.
</li>
<li>
<img src="/docs/latest/icons/ico-model-endpoints.svg" alt="Model endpoints" className="icon size-md space-sm" /> <a href="pipelines/webapp_pipeline_page">Model Endpoints</a> - Monitor your live model endpoints (available in the ClearML Enterprise plan).
</li>
<li>
<img src="/docs/latest/icons/ico-reports.svg" alt="Reports" className="icon size-md space-sm" /> <a href="webapp_reports">Reports</a> - View and manage your reports.
</li>
@@ -42,7 +45,7 @@ The WebApp's sidebar provides access to the following modules:
<img src="/docs/latest/icons/ico-workers.svg" alt="Workers and Queues" className="icon size-md space-sm" /> <a href="webapp_workers_queues">Orchestration</a> - Autoscale, monitor, and manage your resource usage and workers queues.
</li>
<li>
<img src="/docs/latest/icons/ico-applications.svg" alt="ClearML Apps" className="icon size-md space-sm" /> <a href="applications/apps_overview">Applications</a> - ClearML's GUI applications for no-code workflow execution.
<img src="/docs/latest/icons/ico-applications.svg" alt="ClearML Apps" className="icon size-md space-sm" /> <a href="applications/apps_overview">Applications</a> - ClearML's GUI applications for no-code workflow execution (available in the ClearML Pro and Enterprise plans).
</li>
</ul>

28
docs/webapp/webapp_projects_page.md
View File

@@ -21,12 +21,23 @@ page when relevant).
Project folders display summarized project information:
<div class="max-w-50">
![Project card](../img/webapp_project_card.png)
When relevant, the folder has a tab which displays the number of subprojects in the project. Click the tab to view a list of
subprojects. Click on a subproject's name to navigate to it.
</div>
![Subproject tab](../img/webapp_sub_project_card.png)
* When the project contains subprojects, the folder shows the number of subprojects within the project as an additional
tab. Click the tab to view a list of subprojects, and click on a subproject's name to navigate to it.
<div class="max-w-50">
![Subproject tab](../img/webapp_sub_project_card.png)
</div>
* When a project's `default_output_destination` is set, the folder displays the <img src="/docs/latest/icons/ico-info.svg" alt="Info" className="icon size-md space-sm" />
indicator. Hover over the indicator to view the default output destination.
<Collapsible title="Hidden Projects" type="configuration">
@@ -47,7 +58,7 @@ of a project folder to open its context menu and access the following project ac
![Project context menu](../img/webapp_projects_context_menu.png)
* **Rename** - Rename the project.
* **Edit** - Rename the project and/or modify its default output destination.
* **New Project** - Create a new project (by default a subproject).
* **Move to** - Move the project into another project. If the target project does not exist, it is created on-the-fly.
* **Delete** - Delete the project. To delete a project, all of its contents (i.e. any pipelines/reports/datasets) must
@@ -64,4 +75,11 @@ action. For more information see [Custom UI Context Menu Actions](../deploying_c
To create a project, click the **+ NEW PROJECT** button in the top right of the page or in a project's context menu,
which will open a **New Project** modal.
![New project modal](../img/webapp_projects_new_project.png)
![New project modal](../img/webapp_projects_new_project.png)
* Project name
* Create in - Where the project should be created, either as a top-level project (create in `Projects root`) or as a
subproject of another project
* Description
* Default output destination - The storage location where model checkpoints (snapshots) and artifacts will be uploaded
when tasks in this project that do not have an explicit output destination specified are executed