mirror of
https://github.com/clearml/clearml-docs
synced 2025-06-26 18:17:44 +00:00
This commit is contained in:
pollfly
GitHub
@@ -38,7 +38,8 @@ of the chosen metric over time.
|
||||
* **Export Configuration** - Export the app instance configuration as a JSON file, which you can later import to create
|
||||
a new instance with the same configuration.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Dashboard
|
||||
|
||||
@@ -62,4 +63,5 @@ external tools (e.g. Notion). Hover over the plot and click <img src="/docs/late
|
||||
to copy the embed code, and navigate to a report to paste the embed code.
|
||||
:::
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -44,7 +44,8 @@ After starting an Embedding Model Deployment instance, you can view the followin
|
||||
* GPU temperature
|
||||
* Console log - The console log shows the app instance's console output: setup progress, status changes, error messages, etc.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Embedding Model Deployment Instance Configuration
|
||||
|
||||
@@ -90,4 +91,5 @@ Deployment app instance task will be enqueued (make sure an agent is assigned to
|
||||
* **Export Configuration** - Export the app instance configuration as a JSON file, which you can later import to create a
|
||||
new instance with the same configuration
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -37,7 +37,9 @@ Once you start a Gradio launcher instance, you can view the following informatio
|
||||
* Live preview of the Gradio app
|
||||
* Console Log - The console log shows the launcher instance's activity, including server setup progress, server status
|
||||
changes
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Gradio Launcher Instance Configuration
|
||||
When configuring a new Gradio launcher instance, you can fill in the required parameters or reuse the configuration of
|
||||
|
||||
@@ -74,12 +74,14 @@ limits.
|
||||
* **Export Configuration** - Export the app instance configuration as a JSON file, which you can later import to create
|
||||
a new instance with the same configuration
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Dashboard
|
||||
Once an HPO instance is launched, the dashboard displays a summary of the optimization process.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The HPO dashboard shows:
|
||||
* Optimization Metric - Last reported and maximum / minimum values of objective metric over time
|
||||
|
||||
@@ -25,7 +25,8 @@ Once you have launched an app instance, you can view the following information i
|
||||
* Server's resources monitoring (CPU / GPU / vMem utilization)
|
||||
* Console - The console log shows the instance's activity, including server setup progress, server status changes
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Jupyter Lab Instance Configuration
|
||||
When configuring a new Jupyter Lab instance, you can fill in the required parameters or reuse the configuration of
|
||||
|
||||
@@ -45,7 +45,8 @@ Once you start a Model Deployment instance, you can view the following informati
|
||||
* Console log - The console log shows the app instance's console output: setup progress, status changes, error messages,
|
||||
etc.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Model Deployment Instance Configuration
|
||||
|
||||
@@ -144,4 +145,5 @@ instance task will be enqueued (make sure an agent is assigned to that queue)
|
||||
* **Export Configuration** - Export the app instance configuration as a JSON file, which you can later import to create a
|
||||
new instance with the same configuration
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -8,7 +8,8 @@ ClearML Applications are available under the ClearML Pro plan.
|
||||
|
||||
Use ClearML's GUI Applications to manage ML workloads and automatically run your recurring workflows without any coding.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Configure and launch app instances, then track their execution from the app dashboard.
|
||||
|
||||
@@ -54,7 +55,8 @@ Each application's page is split into two sections:
|
||||
Use the search bar <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-sm" />
|
||||
to quickly find an app instance by name.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Launching an App Instance
|
||||
|
||||
@@ -76,7 +78,12 @@ The prefilled instance launch form can be edited before starting the new app ins
|
||||
## App Instance Actions
|
||||
Access app instance actions, by right-clicking an instance, or through the menu button <img src="/docs/latest/icons/ico-dots-v-menu.svg" alt="Dot menu" className="icon size-md space-sm" /> (available on hover).
|
||||
|
||||
|
||||
<div class="max-w-75">
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
* **Rename** - Rename the instance
|
||||
* **Configuration** - View an instance's configuration
|
||||
@@ -91,7 +98,12 @@ Access app instance actions, by right-clicking an instance, or through the menu
|
||||
Access the instance list actions by clicking the action menu (<img src="/docs/latest/icons/ico-dots-v-menu.svg" alt="Dot menu" className="icon size-md space-sm" />)
|
||||
on the instance list header:
|
||||
|
||||
|
||||
<div class="max-w-75">
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
* **Import Configuration** - Import an app instance's configuration file. This opens the app instance launch form
|
||||
prefilled according to the imported file. You can modify the configuration before launching the instance.
|
||||
|
||||
@@ -41,7 +41,8 @@ Once you have launched an app instance, you can view the following information i
|
||||
* GPU memory usage
|
||||
* Console - The console log shows the instance's activity, including session setup progress and SSH server status changes
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## SSH Session Instance Configuration
|
||||
|
||||
@@ -92,4 +93,5 @@ closed and restored into a new instance when the session app instance is cloned
|
||||
* **Export Configuration** - Export the app instance configuration as a JSON file, which you can later import to create a
|
||||
new instance with the same configuration
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -33,14 +33,16 @@ times.
|
||||
* **Export Configuration** - Export the app instance configuration as a JSON file, which you can later import to create
|
||||
a new instance with the same configuration
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Dashboard
|
||||
|
||||
Once a Task Scheduler instance is launched, the dashboard displays a summary of the scheduled tasks.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The Task Scheduler dashboard shows:
|
||||
* Scheduled Tasks - Table of tasks scheduled for execution. The table displays the ID of the task scheduled for execution,
|
||||
@@ -49,12 +51,15 @@ The Task Scheduler dashboard shows:
|
||||
scheduled to be launched daily (`day=1`) at 06:20 UTC (`minute=20, hour=6`).
|
||||
The task in the third row is scheduled to be launched every month (`month=1`) on the 15th at 12:00 UTC (`day=15, hour=12`).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
* Executed Tasks - Table of tasks that have been executed. The table displays the `started` time, which is the time
|
||||
the task was enqueued, and its `finished` time, which is the time the task's execution was completed. If it says `None`,
|
||||
under the `finished` column, the task has not yet completed its execution.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
* Scheduler Log - Application console log containing everything printed to stdout and stderr. The log
|
||||
includes when the scheduler syncs, and when it launches tasks for execution.
|
||||
|
||||
@@ -52,11 +52,13 @@ The app monitors your workspace for trigger events and will launch copies of the
|
||||
* **Export Configuration** - Export the app instance configuration as a JSON file, which you can later import to create
|
||||
a new instance with the same configuration
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Dashboard
|
||||
|
||||
The Trigger Manager app instance's dashboard displays its console log. The log shows the instance's activity: periodic
|
||||
polling, and events triggered
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -25,7 +25,8 @@ Once you have launched an app instance, you can view the following information i
|
||||
* Server's resources monitoring (CPU / GPU / vMem utilization)
|
||||
* Console - The console log shows the instance's activity, including server setup progress, server status changes
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## VS Code App Instance Configuration
|
||||
When configuring a new VS Code instance, you can fill in the required parameters or reuse the configuration of
|
||||
|
||||
@@ -25,7 +25,8 @@ Filter the datasets to find the one you're looking for more easily. These filter
|
||||
respectively. These options appear on the top of the tag list.
|
||||
* Filter by the absence of a tag (logical "NOT") by clicking its checkbox twice. An X will appear in the tag's checkbox.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Project Cards
|
||||
|
||||
@@ -33,7 +34,8 @@ In Project view, project cards display a project's summarized dataset informatio
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
@@ -51,7 +53,8 @@ In List view, the dataset cards display summarized dataset information:
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
@@ -70,7 +73,8 @@ of a dataset card to open its context menu and access dataset actions.
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
@@ -10,13 +10,15 @@ Datasets created with earlier versions of `clearml` are available in their origi
|
||||
The dataset page lists the dataset's versions. For a selected version, the **Dataset Version Panel** shows its lineage
|
||||
in graph form.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Each node in the graph represents a dataset version, and shows the following details:
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
@@ -61,13 +63,15 @@ On the right side of the dataset version panel, view the **VERSION INFO** which
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
To view a version's detailed information, click **Full details**, which will open the dataset version's [task page](../webapp_exp_track_visual.md).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
To view the information for any version in the lineage graph, click its node, and the **VERSION INFO** panel displays
|
||||
that version's details.
|
||||
@@ -78,15 +82,18 @@ Click on **DETAILS** on the top left of the info panel or hover over a version n
|
||||
to view the version's details panel. The panel includes three tabs:
|
||||
* **CONTENT** - Table summarizing version contents, including file names, file sizes, and hashes
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
* **PREVIEW** - A preview of the dataset version's contents.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
* **CONSOLE** - The dataset version's console output
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Click <img src="/docs/latest/icons/ico-max-panel.svg" alt="Expand" className="icon size-md space-sm" /> on the content panel header to view the panel in full screen.
|
||||
@@ -103,7 +110,8 @@ Access these actions with the context menu by right-clicking a version on the da
|
||||
|Restore|Action available in the archive. Restore a version to the active dataset versions table.|
|
||||
|Delete| Delete an archived version and its artifacts. This action is available only from the dataset's archive. |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The actions mentioned in the chart above can be performed on multiple versions at once. [Select multiple versions](#selecting-multiple-versions),
|
||||
then use either the context menu, or the bar that appears at the bottom of the page, to perform operations on the
|
||||
|
||||
@@ -21,14 +21,16 @@ Filter the pipelines to find the one you're looking for more easily. These filte
|
||||
* Filter by the absence of a tag (logical "NOT") by clicking its checkbox twice. An X will appear in the tag's checkbox.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Project Cards
|
||||
In Project view, project cards display a project's summarized pipeline information:
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
@@ -41,7 +43,8 @@ In List view, the pipeline cards display summarized pipeline information:
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
@@ -58,7 +61,8 @@ of a pipeline card to open its context menu and access pipeline actions.
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
@@ -26,7 +26,8 @@ and choosing one of these options:
|
||||
|
||||
The downloaded data consists of the currently displayed table columns.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Run Table Columns
|
||||
|
||||
@@ -98,7 +99,8 @@ you can set the run's parameters. By default, the fields are pre-filled with the
|
||||
|
||||
Click **Advanced configurations** to change the run's execution queue.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
After clicking **RUN**, the new pipeline run is enqueued in the specified queue, and the run is added to the pipeline run table.
|
||||
|
||||
@@ -125,7 +127,8 @@ Access these actions with the context menu in any of the following ways:
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
@@ -157,12 +160,14 @@ In the dropdown menu, select to view **Scalars** or **Plots**.
|
||||
|
||||
**Scalars** shows pipeline run scalar results as time series line graphs.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
All single value scalars are plotted into a single clustered bar chart under the "Summary" title, where each cluster
|
||||
represents a reported metric, and each bar in the cluster represents an experiment.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Click <img src="/docs/latest/icons/ico-settings.svg" alt="Setting Gear" className="icon size-md" /> to customize which
|
||||
metrics to view.
|
||||
@@ -174,11 +179,13 @@ In the **Scalars** view, click <img src="/docs/latest/icons/ico-tune.svg" alt="T
|
||||
Line, scatter, box, and bar graphs are compared by overlaying each metric/variant from all compared runs' into a single
|
||||
comparative plot.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Other plot types are displayed separately for each run.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Run Details Comparison
|
||||
|
||||
@@ -5,11 +5,13 @@ title: Pipeline Run Details
|
||||
The run details panel shows the pipeline's structure and the execution status of every step, as well as the run's
|
||||
configuration parameters and output.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Each step shows:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
* Step name
|
||||
* Step status
|
||||
@@ -25,15 +27,15 @@ Steps that were set into the same stage when created, are grouped into stages in
|
||||
In stages view, click`Expand Stages` to view the complete pipeline DAG with all steps visible.
|
||||
In the detailed view, click `Collapse Stages` to view the pipeline DAG outline with the same stage steps grouped into a single stage.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
In the collapsed view, each stage displays:
|
||||
|
||||
<div class="max-w-25">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
@@ -47,7 +49,8 @@ Click on a stage to view its **STAGE INFO**:
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
@@ -64,7 +67,8 @@ On the pipeline run panel, view the **RUN INFO** which shows:
|
||||
* Produced Artifacts
|
||||
* Output Models
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
To view a run's complete information, click **Full details**, which will open the pipeline's controller [task page](../webapp_exp_track_visual.md).
|
||||
View each list's complete details in the pipeline task's corresponding tabs:
|
||||
@@ -72,12 +76,14 @@ View each list's complete details in the pipeline task's corresponding tabs:
|
||||
* **METRICS** list > **SCALARS** tab
|
||||
* **ARTIFACTS** and **MODELS** lists > **ARTIFACTS** tab
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
To view a specific step's information, click the step on the execution graph, and the info panel displays its **STEP INFO**.
|
||||
The panel displays the step's name, task type, and status, as well as its parameters, metrics, artifacts, and models.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
To return to viewing the run's information, click the pipeline graph, outside any of the steps.
|
||||
|
||||
@@ -89,16 +95,19 @@ details panel, click **DETAILS** and then click on a step node, or hover over a
|
||||
The details panel includes three tabs:
|
||||
* **Preview** - View debug samples and plots attached to the pipeline controller or step
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
* **Console** - The console log for the pipeline controller or steps: contains everything printed to stdout and stderr.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
* **Code** - For pipeline steps generated from functions using either [`PipelineController.add_function_step`](../../references/sdk/automation_controller_pipelinecontroller.md#add_function_step)
|
||||
or [`PipelineDecorator.component`](../../references/sdk/automation_controller_pipelinecontroller.md#pipelinedecoratorcomponent),
|
||||
you can view the selected step's code.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Click <img src="/docs/latest/icons/ico-max-panel.svg" alt="Expand" className="icon size-md space-sm" /> on the details panel header to view the panel in full screen.
|
||||
|
||||
@@ -19,7 +19,8 @@ Set resource reservation and limits for user groups
|
||||
* Connect resource profiles to a policy, making them available to its user group via ClearML queues
|
||||
* Non-administrator users can see the resource policies currently applied to them.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Create a Policy
|
||||
|
||||
@@ -70,7 +71,8 @@ The top card displays the policy information:
|
||||
* Resource limit
|
||||
* User group that the policy applies to - click to show list of users in the group
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The cards below the policy card display the profiles that are connected to the policy:
|
||||
* Resource profile name
|
||||
@@ -79,11 +81,13 @@ of resources consumed by each job enqueued through this profile's queue
|
||||
* <img src="/docs/latest/icons/ico-queued-jobs.svg" alt="Queued jobs" className="icon size-md space-sm" /> - Currently queued jobs
|
||||
* <img src="/docs/latest/icons/ico-running-jobs.svg" alt="Running jobs" className="icon size-md space-sm" /> - Currently running jobs
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Administrators can also see each resource profile’s resource pool links listed in order of routing priority.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The arrow connecting the policy card with a profile card is labeled with the name of the queue the policy’s users should
|
||||
use to run tasks through that resource profile.
|
||||
@@ -112,7 +116,8 @@ and description
|
||||
|
||||
**To remove a resource profile:** On the relevant resource profile box, click `X`.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Removing a profile from a policy will also delete the queue which made this profile available to the policy’s users.
|
||||
Any tasks enqueued on this queue will be set to `draft` status.
|
||||
|
||||
@@ -34,7 +34,12 @@ Access privileges can be viewed, defined, and edited in the **Access Rules** tab
|
||||
button
|
||||
1. Click **SAVE**
|
||||
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
Access is inherited according to resource hierarchy. For example, if a user is given access to a project, the user will
|
||||
also have access to the project's contents (tasks, models, etc.). A user who is granted access to a specific task will
|
||||
|
||||
@@ -67,4 +67,5 @@ The **Administrator Vaults** table lists all currently defined vaults, and the f
|
||||
|
||||
Hover over a vault in the table to **Download**, **Edit**, or **Delete** a vault.
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -145,4 +145,5 @@ The **Provider Connections** table lists all currently defined connections, and
|
||||
|
||||
Hover over a connection in the table to **Edit** or **Delete** it.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -60,7 +60,8 @@ ClearML Hosted Service users can be members of multiple workspaces, which are li
|
||||
To switch to another workspace, click on the **SWITCH TO WORKSPACE** button next to the name of the workspace you want
|
||||
to switch to.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### ClearML App Credentials
|
||||
|
||||
@@ -80,7 +81,12 @@ You can create credentials for any workspace that you are a member of.
|
||||
The dialog displays new credentials, formatted as a ready-to-copy configuration file section (including server configuration
|
||||
information).
|
||||
|
||||
|
||||
<div class="max-w-75">
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
You can edit the labels of credentials in your own workspace, or credentials that you created in other workspaces.
|
||||
|
||||
@@ -161,4 +167,5 @@ Fill in values using any of ClearML supported configuration formats: HOCON / JSO
|
||||
* Click the toggle atop the vault to enable / disable the configurations
|
||||
* Once enabled, the configurations will be merged to the configuration file during ClearML and ClearML Agent usage
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -8,7 +8,8 @@ reservations for different user groups to prioritize workload usage across avail
|
||||
Under the **Resource Configuration** section, administrators define the available resources and the way in which they
|
||||
will be allocated to different workloads.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The Resource Configuration settings page shows the [currently provisioned](#applying-resource-configuration) configuration:
|
||||
the defined resource pools, resource profiles, and the resource allocation architecture.
|
||||
@@ -25,7 +26,12 @@ versa).
|
||||
The resource pool cards are displayed on the top of the Resource Configuration settings page. Each card displays the
|
||||
following information:
|
||||
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
* Pool name
|
||||
* Number of resources currently in use out of the total available resources
|
||||
@@ -45,7 +51,17 @@ R&D team and DevOps team both have pending jobs - run the R&D team's jobs first
|
||||
The resource profile cards are displayed on the bottom of the Resource Configuration settings page. Each card displays
|
||||
the following information:
|
||||
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
* Profile name
|
||||
* <img src="/docs/latest/icons/ico-resource-number.svg" alt="Number of resources" className="icon size-md space-sm" /> - Number
|
||||
@@ -61,18 +77,21 @@ You have GPUs spread across a local H100 and additional bare metal servers, as w
|
||||
by an autoscaler). Assume that currently most of your resources are already assigned to jobs, and only 16 resources are available: 8 in the
|
||||
H100 resource pool and 8 in the Bare Metal pool:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Teams' jobs have varying resource requirements of 0.5, 2, 4, and 8 GPUs. Resource profiles are defined to reflect these:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The different jobs will be routed to different resource pools by connecting the profiles to the resource pools. Jobs
|
||||
enqueued through the profiles will be run in the pools where there are available resources in order of their priority.
|
||||
For example, the H100 pool will run jobs with the following precedence: 2 GPU jobs first, then 4 GPU ones, then 8 GPU,
|
||||
and lastly 0.5 GPU.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Resource policies are implemented for two teams:
|
||||
* Dev team
|
||||
@@ -81,21 +100,33 @@ Resource policies are implemented for two teams:
|
||||
Each team has a resource policy configured with 8 reserved resources and a 16 resource limit. Both teams make use of the
|
||||
4xGPU profile (i.e. each job running through this profile requires 4 resources).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The Dev team is prioritized over the Research team by placing it higher in the Resource Profile's Policies Priority list:
|
||||
|
||||
|
||||
<div class="max-w-75">
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
Both the Dev team and the Research team enqueue four 4-resource jobs each: Dev team jobs will be allocated resources
|
||||
first. The `4xGPU` resource profile is connected to two resource pools: `Bare Metal Low END GPUs` (with the
|
||||
`4 GPU Low End` link) and `H100 Half a Superpod` (with the `4 GPU H100 link`).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Resources are assigned from the `Bare Metal` pool first (precedence set on the resource profile card):
|
||||
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
If the first pool cannot currently satisfy the profile’s resource requirements, resources are assigned from the next
|
||||
listed pool. Let's look at the first pool in the image below. Notice that the pool has 8 available resources, therefore
|
||||
@@ -103,7 +134,8 @@ it can run two 4-resource jobs.
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
@@ -7,7 +7,8 @@ To enable ClearML to delete task artifacts stored in cloud storage when a task i
|
||||
* [AWS S3 Storage](#aws-s3-storage)
|
||||
* [Azure](#azure)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Google Cloud Storage
|
||||
Set up credentials for Google Cloud buckets:
|
||||
|
||||
@@ -7,7 +7,8 @@ The **Billing & Usage** section displays your ClearML workspace usage informatio
|
||||
* Available storage
|
||||
* Number of monthly API calls
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
To add users to your workspace, click **INVITE USERS** in the **USERS** section. This will redirect you to the
|
||||
**USER MANAGEMENT** page, where you can invite users (see details [here](webapp_settings_users.md#inviting-new-teammates)).
|
||||
@@ -20,7 +21,8 @@ for additional plans information.
|
||||
With the ClearML Pro plan, the **Billing & Usage** page additionally shows estimated charges for the current billing
|
||||
period and provides access to billing information.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
**To Upgrade your account to ClearML Pro**, click **UPGRADE** in the top right corner of the page. This will open a
|
||||
wizard where you will be able to read the terms and conditions and input your billing information.
|
||||
|
||||
@@ -19,7 +19,8 @@ Each row of the table includes:
|
||||
* If the user's invitation is pending, the date the user was added
|
||||
* [User groups](#user-groups) (ClearML Enterprise feature)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Inviting New Teammates
|
||||
|
||||
@@ -69,7 +70,8 @@ Each row of the table includes:
|
||||
|
||||
Hover over a service account in the table to **Edit** or **Delete** it.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Creating a Service Account
|
||||
|
||||
@@ -133,7 +135,8 @@ In addition to the service account-specific configuration vault, [administrator
|
||||
be applied to service accounts. See all the vaults applied to the account in the **Applied administrator vaults** below
|
||||
the configuration vault.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Deleting Service Account
|
||||
Deleting a service account will revoke its credentials, causing agents using the account's credentials to fail.
|
||||
@@ -161,7 +164,8 @@ The system includes three pre-configured groups that can't be removed:
|
||||
|
||||
The user group table lists all the active user groups. Each row includes a group's name, description, member list, and ID.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Creating a User Group
|
||||
|
||||
@@ -173,7 +177,12 @@ The user group table lists all the active user groups. Each row includes a group
|
||||
the <img src="/docs/latest/icons/ico-trash.svg" alt="Trash can" className="icon size-md" /> button
|
||||
1. Click **SAVE**
|
||||
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
### Editing a User Group
|
||||
|
||||
|
||||
@@ -34,12 +34,14 @@ The comparison page opens in the **DETAILS** tab with the experiments [compared
|
||||
Click the `EXPERIMENTS` button to view your currently compared experiments. Click `X` on a listed experiment to remove
|
||||
it from the comparison.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
You can add/remove experiments to your comparison:
|
||||
1. Click the `+` button in any of the comparison tabs. This opens up a window with an experiment table with the currently
|
||||
compared experiments at the top.
|
||||
|
||||
|
||||
|
||||
1. Find the experiments to add by sorting and [filtering](webapp_exp_table.md#filtering-columns) the experiments with the
|
||||
appropriate column header controls. Alternatively, use the search bar to find experiments by name.
|
||||
1. Select experiments to include in the comparison (and/or clear the selection of any experiments you wish to remove).
|
||||
@@ -85,8 +87,8 @@ in the tab header. Obscure identical fields by switching on the **Hide Identical
|
||||
Use the search bar to find any field names or values. Lines that match the search query are highlighted, and you can
|
||||
navigate between search results.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Tabular Scalar Comparison
|
||||
The **Scalars** tab **Values** view lays out the experiments' metric values in a table: a row per metric/variant and a
|
||||
@@ -99,7 +101,8 @@ You can download the scalar comparison table as a CSV file by clicking <img src=
|
||||
|
||||
Switch on the **Show row extremes** toggle to highlight each variant's maximum and minimum values.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Parallel Coordinates Mode
|
||||
|
||||
@@ -116,7 +119,8 @@ metrics:
|
||||
For example, plot the metric/variant `accuracy`/`total` against the hyperparameters
|
||||
`base_lr`, `dropout`, and `number_of_epochs`.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
To focus on a specific experiment, hover over its name in the graph legend.
|
||||
|
||||
@@ -137,7 +141,8 @@ To show the value distribution:
|
||||
Hovering over each datapoint in the resulting plot will show the experiment name and the metric and parameter value for that
|
||||
point. You can add additional metrics and hyperparameters values to the datapoint tooltip through **ADDITIONAL DATA POINT INFORMATION**.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Plot Comparison
|
||||
The **Scalars** (Graph view) and **Plots** tabs compare experiments' plots.
|
||||
@@ -151,20 +156,24 @@ comparative plot.
|
||||
For overlaid plots, use **Group by** to select how to group plots:
|
||||
* **Metric** - All variants for a metric appear on the same plot.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
* **Metric+Variant** (default) - Every variant appears on its own plot.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Other plot types that are not overlaid are displayed separately for each experiment.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
All single value scalars are plotted into a single clustered bar chart under the "Summary" title, where each cluster
|
||||
represents a reported metric, and each bar in the cluster represents an experiment.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Hover over plots to access plot controls (see [Scalar Plot Tools](webapp_exp_track_visual.md#scalar-plot-tools)).
|
||||
|
||||
@@ -182,11 +191,13 @@ to synchronize iteration and metric selection across experiments. For example, i
|
||||
experiment's debug samples, the same metric will be automatically selected for the rest of the experiments in the
|
||||
comparison.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Open a debug sample (image, audio, or video) in the viewer or player, by clicking the thumbnail.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
To move to the same sample in another iteration, click <img src="/docs/latest/icons/ico-previous.svg" alt="Left arrow" className="icon size-md space-sm" />
|
||||
(previous), <img src="/docs/latest/icons/ico-next.svg" alt="Right arrow" className="icon size-md space-sm" /> (next), or move the slider.
|
||||
|
||||
@@ -25,7 +25,8 @@ have a parent, in which case the original task is set as the parent. Select `Set
|
||||
the original experiment to become the clone's parent.
|
||||
:::
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Resetting
|
||||
@@ -36,7 +37,8 @@ To reset an experiment:
|
||||
ClearML file server, click the checkbox
|
||||
1. Click **Reset**
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Final Steps
|
||||
|
||||
|
||||
@@ -28,7 +28,8 @@ and choosing one of these options:
|
||||
|
||||
The downloaded data consists of the currently displayed table columns.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Creating Tasks
|
||||
|
||||
@@ -162,7 +163,8 @@ all the tasks in the project. The customizations of these two views are saved se
|
||||
|
||||
### Adding Metrics and/or Hyperparameters
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Add metrics and/or hyperparameters columns to the task table. The metrics and hyperparameters depend upon the
|
||||
tasks in the table.
|
||||
@@ -184,7 +186,8 @@ tooltip that appears. To view all precise values in a column, hover over a float
|
||||
|
||||
### Filtering Columns
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Filters can be applied by clicking <img src="/docs/latest/icons/ico-filter-off.svg" alt="Filter" className="icon size-md" />
|
||||
on a column, and the relevant filter appears.
|
||||
@@ -252,7 +255,8 @@ operations on the selected tasks. Actions can be performed only on the tasks tha
|
||||
of tasks that can be affected by each action. The same information can be found in the batch action bar, in a tooltip that
|
||||
appears when hovering over an action icon.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Selecting Multiple Tasks
|
||||
|
||||
@@ -277,12 +281,14 @@ In the dropdown menu, select to view **Scalars** or **Plots**.
|
||||
|
||||
**Scalars** shows task scalar results as time series line graphs.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
All single value scalars are plotted into a single clustered bar chart under the "Summary" title, where each cluster
|
||||
represents a reported metric, and each bar in the cluster represents a task.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Click <img src="/docs/latest/icons/ico-settings.svg" alt="Setting Gear" className="icon size-md" /> to customize which
|
||||
metrics to view.
|
||||
@@ -296,11 +302,13 @@ task.
|
||||
Line, scatter, box, and bar graphs are compared by overlaying each metric/variant from all compared tasks' into a
|
||||
single comparative plot.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Other plot types are displayed separately for each task.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Creating a Task Leaderboard
|
||||
|
||||
@@ -321,4 +329,5 @@ based on their name by using the search bar.
|
||||
The final dashboard can be shared by copying the URL from the address bar, this address will replicate the exact same dashboard on any browser.
|
||||
The dashboard can also be bookmarked for later use.
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -31,20 +31,22 @@ table / full screen**.
|
||||
The info panel keeps the task table in view so that [task actions](webapp_exp_table.md#task-actions)
|
||||
can be performed from the table (as well as the menu in the info panel).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Click <img src="/docs/latest/icons/ico-compact-view.svg" alt="Compressed view" className="icon size-md space-sm" /> to
|
||||
hide details in the task table, so only the task names and statuses are displayed
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Full Screen Details View
|
||||
|
||||
The full screen details view allows for easier viewing and working with task tracking and results. The task
|
||||
table is not visible when the full screen details view is open. Perform task actions from the menu.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Execution
|
||||
A task's **EXECUTION** tab of lists the following:
|
||||
@@ -65,25 +67,29 @@ The Source Code section of a task's **EXECUTION** tab includes:
|
||||
* Working directory
|
||||
* Binary (Python executable)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Uncommitted Changes
|
||||
|
||||
ClearML displays the git diff of the task in the Uncommitted Changes section.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Python Packages
|
||||
|
||||
The Python Packages section lists the task's installed Python packages and their versions.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
When a ClearML agent executing a task ends up using a different set of Python packages than was originally
|
||||
specified, both the original specification (`original pip` or `original conda`), and the packages the agent ended up
|
||||
using to set up an environment (`pip` or `conda`) are available. Select which requirements to view in the dropdown menu.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Container
|
||||
The Container section list the following information:
|
||||
@@ -102,14 +108,16 @@ clearml-agent daemon --queue <execution_queue_to_pull_from> --docker [optional d
|
||||
For more information, see [Docker Mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode).
|
||||
:::
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Output
|
||||
The Output details include:
|
||||
* The output destination used for storing model checkpoints (snapshots) and artifacts (see also, [default_output_uri](../configs/clearml_conf.md#config_default_output_uri)
|
||||
in the configuration file, and `output_uri` in [`Task.init`](../references/sdk/task.md#taskinit) parameters).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Configuration
|
||||
|
||||
@@ -127,7 +135,8 @@ The **Args** group shows automatically logged argument parser parameters (e.g. `
|
||||
Hover over <img src="/docs/latest/icons/ico-description.svg" alt="Description" className="icon size-md space-sm" /> (menu) on a
|
||||
parameter's line, and the type, description, and default value appear, if they were provided.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
#### Environment Variables
|
||||
|
||||
@@ -138,7 +147,8 @@ field of the `clearml.conf` file, the **Environment** group displays the listed
|
||||
The `CLEARML_LOG_ENVIRONMENT` variable always overrides the `clearml.conf` file.
|
||||
:::
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
#### Custom Parameter Groups
|
||||
|
||||
@@ -146,33 +156,37 @@ Custom parameter groups show parameter dictionaries if the parameters were conne
|
||||
[`Task.connect()`](../references/sdk/task.md#connect) with a `name` argument provided. `General` is the default section
|
||||
if a name is not provided.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
#### TensorFlow Definitions
|
||||
|
||||
The **TF_DEFINE** parameter group shows automatic TensorFlow logging.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### User Properties
|
||||
|
||||
User properties allow to store any descriptive information in a key-value pair format. They are editable in any task,
|
||||
except *Published* ones (read-only).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Configuration Objects
|
||||
|
||||
ClearML tracks a task's model configuration objects, which appear in **Configuration Objects** **>** **General**.
|
||||
These objects include those that are automatically tracked, and those connected to a Task in code (see [`Task.connect_configuration`](../references/sdk/task.md#connect_configuration)).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
ClearML supports providing a name for a Task model configuration object (see the `name`
|
||||
parameter in [`Task.connect_configuration`](../references/sdk/task.md#connect_configuration)).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Artifacts
|
||||
|
||||
@@ -190,7 +204,8 @@ to obtain the file path to facilitate local storage access since web application
|
||||
* **Remote Files** (e.g. network-hosted artifacts with `https://`, `s3://`, etc. URIs): Use the download action (<img src="/docs/latest/icons/ico-download-json.svg" alt="Download" className="icon size-md space-sm" />)
|
||||
to retrieve the file.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Models
|
||||
|
||||
@@ -201,7 +216,8 @@ The task's input and output models appear in the **ARTIFACTS** tab. Each model e
|
||||
|
||||
Input models also display their creating task, which on-click navigates you to the task's page.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
To view more model details, including design, label enumeration, and general information, click the model name
|
||||
to navigate to its page in the **MODELS** tab (see [Model Details](webapp_model_viewing.md)).
|
||||
@@ -229,7 +245,8 @@ The Enterprise Server also displays a detailed history of task activity:
|
||||
|
||||
To download the task history as a CSV file, hover over the log and click <img src="/docs/latest/icons/ico-download.svg" alt="Download" className="icon size-md space-sm" />.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
:::note Limited persistency
|
||||
ClearML maintains a system-wide, large but strict limit for task history items. Once the limit is reached, the oldest entries are purged to make room for fresh entries.
|
||||
@@ -260,7 +277,8 @@ The **Task Details** section lists information describing the task:
|
||||
* Python version
|
||||
* Task Progress
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Task Results
|
||||
|
||||
@@ -275,8 +293,8 @@ tools (e.g. Notion). See [Plot Controls](#plot-controls).
|
||||
The complete task log containing everything printed to stdout and stderr appears in the **CONSOLE** tab. The full log
|
||||
is downloadable. To view the end of the log, click **Jump to end**.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Scalars
|
||||
@@ -302,7 +320,8 @@ to update the graph.
|
||||
Single value scalars (see [`Logger.report_single_value`](../references/sdk/logger.md#report_single_value)) are shown in
|
||||
a `Summary` table.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Use the scalar tools to improve analysis of scalar metrics. In the info panel, click <img src="/docs/latest/icons/ico-settings.svg" alt="Settings gear" className="icon size-md space-sm" /> to use the tools. In the full screen details view, the tools
|
||||
are on the left side of the window. The tools include:
|
||||
@@ -313,13 +332,15 @@ are on the left side of the window. The tools include:
|
||||
* **Metric** - Displays all variants for a metric on the same plot. For example, if you have a "Test" metric with
|
||||
"loss" and "accuracy" variants, both variants will appear on the same plot that is titled "Test".
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
* **None** - Displays individual plots for each metric-variant combination, grouped into sections by metric. For
|
||||
example, a "Test" metric with "loss" and "accuracy" variants will have a separate plot for each variant under the
|
||||
"Test" group.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
* Horizontal axis - Select the x-axis units:
|
||||
* Iterations
|
||||
@@ -336,7 +357,8 @@ which will copy to clipboard the embed code to put in your Reports. To quickly g
|
||||
specific metric, click Embed <img src="/docs/latest/icons/ico-plotly-embed-code.svg" alt="Embed code" className="icon size-md space-sm" />
|
||||
on the group section header (available when plots are [grouped by](#group_by) `None`).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
In contrast to static screenshots, embedded resources
|
||||
are retrieved when the report is displayed allowing your reports to show the latest up-to-date data.
|
||||
@@ -356,7 +378,8 @@ shows a table summary of your metrics with a row per metric/variant:
|
||||
|
||||
If all the values of a specific metric/variant are the same, the row will display a <img src="/docs/latest/icons/ico-equal-outline.svg" alt="Same values" className="icon size-md space-sm" /> sign.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Choose which metrics to view using one of the following out the options:
|
||||
* Use the quick filter bar (<img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-sm" />)
|
||||
@@ -371,7 +394,8 @@ Non-time-series plots appear in **PLOTS**. These include data generated by libra
|
||||
explicitly reported using the ClearML Logger. These may include 2D and 3D plots, tables (Pandas and CSV files), and
|
||||
Plotly plots. Individual plots can be shown / hidden or filtered by title.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Plots are grouped into sections by metric. To quickly get the embed codes for all plots of a specific metric, click Embed <img src="/docs/latest/icons/ico-plotly-embed-code.svg" alt="Embed code" className="icon size-md space-sm" />
|
||||
on the group section header.
|
||||
@@ -384,7 +408,8 @@ previous/next plot in the current iteration using the <img src="/docs/latest/ico
|
||||
buttons (or using the left/right arrow keyboard shortcut).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
#### Plot Controls
|
||||
|
||||
@@ -431,7 +456,8 @@ libraries and visualization tools, and explicitly reported using the [ClearML Lo
|
||||
You can view debug samples by metric in the reported iterations. Filter the samples by metric by selecting a metric from the
|
||||
dropdown menu above the samples. The most recent iteration appears first.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
For each metric, the latest reported debug sample is displayed.
|
||||
|
||||
@@ -442,7 +468,8 @@ arrow keyboard shortcut). Go to the previous/next sample in the current iteratio
|
||||
buttons (or using the left/right arrow keyboard shortcut).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Tagging Tasks
|
||||
|
||||
|
||||
@@ -9,7 +9,8 @@ The **Dashboard** provides the following options:
|
||||
* Open the [**Orchestration**](webapp_workers_queues.md) tab to autoscale, monitor, and manage your resource usage and
|
||||
worker queues.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
**To select a project, experiment, or model:**
|
||||
|
||||
|
||||
@@ -17,12 +17,14 @@ The comparison page opens in the **DETAILS** tab, with the models compared [side
|
||||
Click the `MODELS` button to view your currently compared models. Click `X` on a listed model to remove
|
||||
it from the comparison.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
You can add/remove models to your comparison:
|
||||
1. Click the `+` button in any of the comparison tabs. This opens up a window with a model table with the currently
|
||||
compared models at the top.
|
||||
|
||||
|
||||
|
||||
1. Find the models to add by sorting and [filtering](webapp_model_table.md#filtering-columns) the models with the
|
||||
appropriate column header controls. Alternatively, use the search bar to find models by name.
|
||||
1. Select models to include in the comparison (and/or clear the selection of any models you wish to remove).
|
||||
@@ -62,7 +64,8 @@ on the model that will be the new base.
|
||||
The differences between the models are highlighted. You can obscure identical fields by switching on the
|
||||
**Hide Identical Fields** toggle.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Tabular Scalar Comparison
|
||||
The **Scalars** tab (**Values** view) lays out the models' reported metric values in a table: a row per metric/variant and a
|
||||
@@ -75,7 +78,8 @@ You can download the scalar comparison table as a CSV file by clicking <img src=
|
||||
|
||||
Switch on the **Show row extremes** toggle to highlight each variant's maximum and minimum values.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Graphic Comparison
|
||||
@@ -86,15 +90,18 @@ metric/variant combination per compared model.
|
||||
Line, scatter, box, and bar graphs are compared by a single plot per metric/variant into which the plots of all compared
|
||||
models are combined.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Other plot types are displayed separately for each model.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
All single value scalars are plotted into a single clustered bar chart under the "Summary" title, where each cluster
|
||||
represents a reported metric, and each bar in the cluster represents a model.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
For better plot analysis, see [Plot Controls](webapp_exp_track_visual.md#plot-controls).
|
||||
|
||||
@@ -26,7 +26,8 @@ Active Endpoints are displayed in the **Active** tab. The table provides the fol
|
||||
* Requests / MIN (avg) - Average request rate in the last minute
|
||||
* Latency (avg) - Average endpoint response latency
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The **Loading** tab shows endpoints that are being set up, but are not yet active. The table provides the following
|
||||
information:
|
||||
@@ -38,7 +39,8 @@ information:
|
||||
* Input type - Model matrix input type (e.g. uint8, float32, int16, float16)
|
||||
* Input size - Model matrix input size
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
You can apply column filters by clicking <img src="/docs/latest/icons/ico-filter-off.svg" alt="Filter" className="icon size-md" />,
|
||||
and sort endpoints by clicking <img src="/docs/latest/icons/ico-sort-off.svg" alt="Sort order" className="icon size-md space-sm" /> on the relevant column.
|
||||
@@ -65,7 +67,8 @@ The **Details** tab displays the model endpoint information:
|
||||
* GPU Count
|
||||
* Latency - Average request latency in the last minute
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Monitor
|
||||
The Monitor tab displays the endpoint's operational metrics and resource usage over time graphs:
|
||||
@@ -78,7 +81,8 @@ The Monitor tab displays the endpoint's operational metrics and resource usage o
|
||||
* Video Memory Usage
|
||||
* Network Usage
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The graphs' time span can be controlled through the menu at its top right corner.
|
||||
Click <img src="/docs/latest/icons/ico-show.svg" alt="Eye Show" className="icon size-md space-sm" /> to control which
|
||||
|
||||
@@ -22,7 +22,8 @@ and choosing one of these options:
|
||||
|
||||
The downloaded data consists of the currently displayed table columns.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Models Table Columns
|
||||
|
||||
@@ -108,7 +109,8 @@ Select multiple models, then use either the context menu, or the bar that appear
|
||||
operations on the selected models. The context menu shows the number of models that can be affected by each action.
|
||||
The same information can be found in the bottom menu, in a tooltip that appears when hovering over an action icon.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Tagging Models
|
||||
|
||||
|
||||
@@ -24,18 +24,21 @@ The **GENERAL** tab lists the model's General information including:
|
||||
If the model is stored in a network location, it can be downloaded by clicking the model URL. If the model was stored on
|
||||
the local machine you can copy its URL to manually access it.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Model Configuration
|
||||
|
||||
The **NETWORK** tab displays the model's configuration.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Hover over the model configuration area to access the following actions:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
* <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-sm" /> Search
|
||||
* <img src="/docs/latest/icons/ico-copy-to-clipboard.svg" alt="Copy" className="icon size-md space-sm" /> Copy configuration
|
||||
@@ -46,24 +49,28 @@ Hover over the model configuration area to access the following actions:
|
||||
|
||||
The **LABELS** tab displays for each class label (`Label`) its name and enumerated value (`ID`).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
To modify / add / delete class labels (for *Draft* models), hover over the label table and click **EDIT**. This opens the
|
||||
label editing window.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Metadata
|
||||
|
||||
The **METADATA** tab lists the model's metadata entries, which consist of a key, type, and value.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
To modify / add / delete model metadata items, hover over the metadata table and click **EDIT**. This opens the metadata editing
|
||||
window.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Lineage
|
||||
|
||||
@@ -75,7 +82,8 @@ You can filter the task list by tags and task status.
|
||||
Use the search bar to look for experiments based on their name, ID, or description.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Scalars
|
||||
|
||||
@@ -91,7 +99,8 @@ are retrieved when the report is displayed allowing your reports to show the lat
|
||||
For better plot analysis, see [Plot Controls](webapp_exp_track_visual.md#plot-controls).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Plots
|
||||
|
||||
@@ -104,5 +113,6 @@ are retrieved when the report is displayed allowing your reports to show the lat
|
||||
|
||||
For better plot analysis, see [Plot Controls](webapp_exp_track_visual.md#plot-controls).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -12,7 +12,8 @@ Use the orchestration dashboard to monitor all of your available and in-use comp
|
||||
* Resource history event log
|
||||
* Detailed resource performance metrics
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Resource Categories and Groups
|
||||
The orchestration dashboard shows your workers by groups and categories, specified by the following naming
|
||||
@@ -82,7 +83,8 @@ to access the worker's information panel.
|
||||
|
||||
The table highlights values that cross user configured thresholds.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Click <img src="/docs/latest/icons/ico-tune.svg" alt="Tune" className="icon size-md" /> to define the threshold values.
|
||||
|
||||
@@ -90,12 +92,18 @@ Click <img src="/docs/latest/icons/ico-tune.svg" alt="Tune" className="icon size
|
||||
The threshold values applied to the dashboard table affect all workspace users who view the page
|
||||
:::
|
||||
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
Clicking on a resource group opens the group's info panel and replace the **Overview** graph with that resource's usage
|
||||
history.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The info panel displays the group's:
|
||||
* Total GPU count
|
||||
|
||||
@@ -9,7 +9,8 @@ The **ClearML Web UI** is the graphical user interface for the ClearML platform,
|
||||
* Profile management
|
||||
* Direct access to the ClearML community (Slack channel, YouTube, and GitHub).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## UI Modules
|
||||
The WebApp's sidebar provides access to the following modules:
|
||||
|
||||
@@ -7,6 +7,7 @@ that can show a snapshot of specified metrics' values across the project's exper
|
||||
project's description. When either overview option is utilized, the **OVERVIEW** tab becomes the project's landing page,
|
||||
meaning that it's the first thing that is seen when opening the project.
|
||||
|
||||
|
||||
|
||||
|
||||
## Metric Snapshot
|
||||
@@ -23,7 +24,8 @@ value to show
|
||||
|
||||
<div class="max-w-75">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
@@ -43,7 +45,8 @@ metric/variant combination.
|
||||
Hover over a point in the snapshot, and a box will appear with the details of the experiment associated with the metric
|
||||
value. Click the point to go to the experiment's details page.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Project Description
|
||||
|
||||
|
||||
@@ -9,7 +9,8 @@ projects in your workspace. Projects that contain nested subprojects are identif
|
||||
An exception is the **All Experiments** folder, which shows all projects' and subprojects' contents in a single, flat
|
||||
list.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
If a project has any subprojects, clicking its folder will open its own project page. Access the projects' top-level
|
||||
contents (i.e. experiments, models etc.) via the folder with the bracketed (`[ ]`) project name.
|
||||
@@ -23,7 +24,8 @@ Project folders display summarized project information:
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
@@ -32,7 +34,8 @@ Project folders display summarized project information:
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
@@ -47,7 +50,8 @@ are labeled with <img src="/docs/latest/icons/ico-ghost.svg" alt="Hidden project
|
||||
|
||||
<br/>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</Collapsible>
|
||||
|
||||
@@ -56,7 +60,12 @@ are labeled with <img src="/docs/latest/icons/ico-ghost.svg" alt="Hidden project
|
||||
Click <img src="/docs/latest/icons/ico-bars-menu.svg" alt="Menu" className="icon size-md space-sm" /> on the top right
|
||||
of a project folder to open its context menu and access the following project actions:
|
||||
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
* **Edit** - Rename the project and/or modify its default output destination.
|
||||
* **New Project** - Create a new project (by default a subproject).
|
||||
@@ -75,7 +84,12 @@ action. For more information see [Custom UI Context Menu Actions](../deploying_c
|
||||
To create a project, click the **+ NEW PROJECT** button in the top right of the page or in a project's context menu,
|
||||
which will open a **New Project** modal.
|
||||
|
||||
|
||||
<div class="max-w-75">
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
* Project name
|
||||
* Create in - Where the project should be created, either as a top-level project (create in `Projects root`) or as a
|
||||
|
||||
@@ -31,7 +31,8 @@ Reports are editable Markdown documents, supporting:
|
||||
* Embedded images uploaded from your computer
|
||||
* Embedded ClearML task, model, and [app](applications/apps_overview.md) content
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Publishing a report locks it for future editing, so you can preserve its contents. You can also share your reports,
|
||||
download a PDF copy, or simply copy the MarkDown content and reuse in your editor of choice.
|
||||
@@ -49,16 +50,19 @@ To add a graphic resource:
|
||||
object comparison)
|
||||
2. Hover over the resource and click <img src="/docs/latest/icons/ico-plotly-embed-code.svg" alt="Generate embed code" className="icon size-md space-sm" />.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Click `Embed in ClearML report`. This generates the embed code for accessing the resource, and copies
|
||||
it to your clipboard.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
3. Return to your report page and paste the code snippet
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Once embedded in the report, you can return to the resource's original location (e.g. comparison page, experiment/model/app page)
|
||||
by clicking <img src="/docs/latest/icons/ico-resource-return.svg" alt="Return to resource" className="icon size-md" />.
|
||||
@@ -171,14 +175,16 @@ or in List view <img src="/docs/latest/icons/ico-flat-view.svg" alt="List view"
|
||||
view, all reports are shown side-by-side. In Project view, reports are organized according to their projects, and
|
||||
top-level projects are displayed. Click on a project card to view the project's reports.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Project Cards
|
||||
In Project view, project cards display a project's summarized report information:
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
@@ -190,7 +196,8 @@ In List view, the report cards display summarized report information:
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
@@ -209,7 +216,8 @@ of a report card to open its context menu and access report actions:
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
@@ -225,7 +233,8 @@ of a report card to open its context menu and access report actions:
|
||||
To create a report, click the **+ NEW REPORT** button in the top right of the page,
|
||||
which will open a **New Report** modal.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## MarkDown Formatting Quick Guide
|
||||
|
||||
@@ -236,9 +245,9 @@ The following is a quick reference for the MarkDown syntax that can be used in C
|
||||
To create headings, add `#` in front of the phrases that you want to turn into
|
||||
headings. The number of `#` signs correspond to the heading level (i.e. `#` for level-1 heading, `##` for level-2, etc.):
|
||||
|
||||
| MarkDown | Rendered Output |
|
||||
|---|---|
|
||||
| <code># H1</code><br/><code>## H2</code><br/><code>### H3</code><br/><code>#### H4</code><br/><code>##### H5</code><br/><code>###### H6</code>||
|
||||
| MarkDown | Rendered Output |
|
||||
|---|-------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| <code># H1</code><br/><code>## H2</code><br/><code>### H3</code><br/><code>#### H4</code><br/><code>##### H5</code><br/><code>###### H6</code>|   |
|
||||
|
||||
### Text Emphasis
|
||||
|
||||
@@ -256,9 +265,9 @@ The following table presents the text format options:
|
||||
To create a blockquote, add a `>` before each line of the quote. Nest blockquotes by adding additional
|
||||
`>` signs before each line of the nested blockquote.
|
||||
|
||||
| MarkDown | Rendered Output |
|
||||
|---|---|
|
||||
| <code>\> Blockquote<br/>\>\> Nested quote 1<br/>\>\>\> Nested quote 2</code>||
|
||||
| MarkDown | Rendered Output |
|
||||
|---|-------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| <code>\> Blockquote<br/>\>\> Nested quote 1<br/>\>\>\> Nested quote 2</code>|   |
|
||||
|
||||
### Lists
|
||||
|
||||
@@ -267,9 +276,9 @@ To create a blockquote, add a `>` before each line of the quote. Nest blockquote
|
||||
Create an ordered list by numbering the list items with numbers followed by periods. The list items do not have to be numbered
|
||||
correctly, but the list will be rendered numerically starting with `1.`.
|
||||
|
||||
| MarkDown | Rendered Output |
|
||||
|---|---|
|
||||
| <code>1. Item 1<br/>2. Item 2<br/>1. Item 3<br/>1. Item 4</code>||
|
||||
| MarkDown | Rendered Output |
|
||||
|---|-----------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| <code>1. Item 1<br/>2. Item 2<br/>1. Item 3<br/>1. Item 4</code>|   |
|
||||
|
||||
#### Unordered List
|
||||
|
||||
@@ -280,9 +289,9 @@ You can also use checkmarks (`* [x]`), following any of the bullet signs.
|
||||
|
||||
To nest lists, indent nested items 2 spaces more than their parent list item.
|
||||
|
||||
| MarkDown | Rendered Output |
|
||||
|---|---|
|
||||
| <code>+ Item 1<br/>+ Item 2<br/> - Sub-item a:<br/> \* Sub-sub-item x<br/> + Sub-sub-item y<br/> - Sub-sub-item z<br/>\* [x] A checkmark </code>||
|
||||
| MarkDown | Rendered Output |
|
||||
|---|-------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| <code>+ Item 1<br/>+ Item 2<br/> - Sub-item a:<br/> \* Sub-sub-item x<br/> + Sub-sub-item y<br/> - Sub-sub-item z<br/>\* [x] A checkmark </code>|   |
|
||||
|
||||
|
||||
### Tables
|
||||
@@ -298,7 +307,8 @@ MarkDown code for a table looks like this:
|
||||
|
||||
The rendered output should look like this:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Add the table column names in the first row; each name is preceded and followed by a pipe (`|`).
|
||||
In the second row, add sets of at least three hyphens (`---`) for each column, and add a pipe before and after each set
|
||||
@@ -403,9 +413,9 @@ It is rendered like this:
|
||||
|
||||
Create horizontal lines using three hyphens (`---`), underscores (`___`), or asterisks (`***`):
|
||||
|
||||
| MarkDown | Rendered Output |
|
||||
|---|---|
|
||||
| <code>---<br/><br/>___<br/><br/>***</code>||
|
||||
| MarkDown | Rendered Output |
|
||||
|---|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| <code>---<br/><br/>___<br/><br/>***</code>|   |
|
||||
|
||||
|
||||
### Images
|
||||
|
||||
@@ -42,7 +42,8 @@ For more information about how autoscalers work, see the [Cloud Autoscaling Over
|
||||
For more information about a specific autoscaler, see [AWS Autoscaler](applications/apps_aws_autoscaler.md)
|
||||
and/or [GCP Autoscaler](applications/apps_gcp_autoscaler.md).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Workers
|
||||
|
||||
@@ -76,7 +77,8 @@ The worker's details panel includes the following two tabs:
|
||||
* Next experiment - The next experiment available in this queue
|
||||
* In Queue - The number of experiments currently enqueued
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -98,12 +100,14 @@ To create a new queue - Click **+ NEW QUEUE** (top left).
|
||||
Hover over a queue and click <img src="/docs/latest/icons/ico-copy-to-clipboard.svg" alt="Copy" className="icon size-md space-sm" />
|
||||
to copy the queue's ID.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Right-click on a queue or hover and click its action button <img src="/docs/latest/icons/ico-dots-v-menu.svg" alt="Dot menu" className="icon size-md space-sm" />
|
||||
to access queue actions:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
* Delete - Delete the queue. Any pending tasks will be dequeued.
|
||||
* Rename - Change the queue's name
|
||||
@@ -130,7 +134,8 @@ The queue's details panel includes the following two tabs:
|
||||
Click on an experiment's menu button <img src="/docs/latest/icons/ico-dots-v-menu.svg" alt="Dot menu" className="icon size-md space-sm" />
|
||||
in the **EXPERIMENTS** tab to reorganize your queue:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
* Move a task to the top or bottom of the queue
|
||||
* Move the task to a different queue
|
||||
|
||||
Reference in New Issue
Block a user