ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-06-26 18:17:44 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge branch 'main' of https://github.com/allegroai/clearml-docs

Browse Source
This commit is contained in:
revital
2025-02-09 08:06:47 +02:00
parent c41e83a61a f50fc93a7b
commit 1db4e0b2bd
159 changed files with 909 additions and 857 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
docs/apps/clearml_param_search.md
View File

@@ -3,7 +3,7 @@ title: ClearML Parameter Search CLI (HPO)
---
Use the `clearml-param-search` CLI tool to launch ClearML's automated hyperparameter optimization (HPO). This process finds
the optimal values for your experiments' hyperparameters that yield the best performing models.
the optimal values for your tasks' hyperparameters that yield the best performing models.
## How Does `clearml-param-search` Work?
@@ -25,11 +25,11 @@ of the optimization results in table and graph forms.
|Name | Description| Mandatory |
|---|----|---|
|`--args`| List of `<argument>=<value>` strings to pass to the remote execution. Currently only argparse/click/hydra/fire arguments are supported. Example: `--args lr=0.003 batch_size=64`|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|`--compute-time-limit`|The maximum compute time in minutes that experiment can consume. If this time limit is exceeded, all jobs are aborted.|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|`--compute-time-limit`|The maximum compute time in minutes that a task can consume. If this time limit is exceeded, all jobs are aborted.|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|`--max-iteration-per-job`|The maximum iterations (of the objective metric) per single job. When iteration maximum is exceeded, the job is aborted.|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|`--max-number-of-concurrent-tasks`|The maximum number of concurrent Tasks (experiments) running at the same time|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|`--max-number-of-concurrent-tasks`|The maximum number of concurrent Tasks running at the same time|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|`--min-iteration-per-job`|The minimum iterations (of the objective metric) per single job.|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|`--local`| If set, run the experiments locally. Notice that no new python environment will be created. The `--script` parameter must point to a local file entry point and all arguments must be passed with `--args`| <img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|`--local`| If set, run the tasks locally. Notice that no new python environment will be created. The `--script` parameter must point to a local file entry point and all arguments must be passed with `--args`| <img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|`--objective-metric-series`| Objective metric series to maximize/minimize (e.g. 'loss').|<img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" />|
|`--objective-metric-sign`| Optimization target, whether to maximize or minimize the value of the objective metric specified. Possible values: "min", "max", "min_global", "max_global". For more information, see [Optimization Objective](#optimization-objective). |<img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" />|
|`--objective-metric-title`| Objective metric title to maximize/minimize (e.g. 'validation').|<img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" />|
@@ -39,8 +39,8 @@ of the optimization results in table and graph forms.
|`--params-override`|Additional parameters of the base task to override for this parameter search. Use the following JSON format for each parameter: `{"name": "param_name", "value": <new_value>}`. Windows users, see [JSON format note](#json_note).|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|`--pool-period-min`|The time between two consecutive polls (minutes).|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|`--project-name`|Name of the project in which the optimization task will be created. If the project does not exist, it is created. If unspecified, the repository name is used.|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|`--queue`|Queue to enqueue the experiments on.|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|`--save-top-k-tasks-only`| Keep only the top \<k\> performing tasks, and archive the rest of the experiments. Input `-1` to keep all tasks. Default: `10`.|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|`--queue`|Queue to enqueue the tasks on.|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|`--save-top-k-tasks-only`| Keep only the top \<k\> performing tasks, and archive the rest of them. Input `-1` to keep all tasks. Default: `10`.|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|`--script`|Script to run the parameter search on. Required unless `--task-id` is specified.|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|`--task-id`|ID of a ClearML task whose hyperparameters will be optimized. Required unless `--script` is specified.|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|`--task-name`|Name of the optimization task. If unspecified, the base Python script's file name is used.|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
@@ -109,8 +109,8 @@ clearml-param-search --script base_template_keras_simple.py --params-search "{\"
### Optimization Objective
Use the `--objective-metric-sign` to specify which optimum your optimization process should use. The options are:
* `min` - Least value of the specified objective metric reported at the end of the experiment
* `max` - Greatest value of the specified objective metric reported at the end of the experiment
* `min_global` - Least value of the specified objective metric reported at any time in the experiment
* `max_global` - Greatest value of the specified objective metric reported at any time in the experiment
* `min` - Least value of the specified objective metric reported at the end of the task
* `max` - Greatest value of the specified objective metric reported at the end of the task
* `min_global` - Least value of the specified objective metric reported at any time in the task
* `max_global` - Greatest value of the specified objective metric reported at any time in the task

12
docs/apps/clearml_session.md
View File

@@ -170,19 +170,19 @@ You can upload your files in conjunction with the `--store-workspace` option to
development machines and remote machines with persistent workspace synchronization. See [Storing and Synchronizing Workspace](#storing-and-synchronizing-workspace).
### Starting a Debugging Session
You can debug previously executed experiments registered in the ClearML system on a remote interactive session.
Input into `clearml-session` the ID of a Task to debug, then `clearml-session` clones the experiment's git repository and
You can debug previously executed tasks registered in the ClearML system on a remote interactive session.
Input into `clearml-session` the ID of a Task to debug, then `clearml-session` clones the task's git repository and
replicates the environment on a remote machine. Then the code can be interactively executed and debugged on JupyterLab / VS Code.
:::note
The Task must be connected to a git repository, since currently single script debugging is not supported.
:::
1. In the **ClearML web UI**, find the experiment (Task) that needs debugging.
1. In the **ClearML web UI**, find the task that needs debugging.
1. Click the `ID` button next to the Task name, and copy the unique ID.
1. Enter the following command: `clearml-session --debugging-session <experiment_id_here>`
1. Enter the following command: `clearml-session --debugging-session <task_id>`
1. Click on the JupyterLab / VS Code link, or connect directly to the SSH session.
1. In JupyterLab / VS Code, access the experiment's repository in the `environment/task_repository` folder.
1. In JupyterLab / VS Code, access the task's repository in the `environment/task_repository` folder.
### Storing and Synchronizing Workspace
You can store and sync your interactive session workspace with the `--store-workspace` option. `clearml-session`
@@ -220,7 +220,7 @@ clearml-session --continue-session <session_id> --store-workspace ~/workspace
| `--base-task-id` | Pass the ID of a task that will become the base task, so the session will use its configurations | `none` or the previously entered base task |
| `--config-file` | Specify a path to another configuration file for `clearml-session` to store its previous state | `.clearml_session.json` or previously entered configuration file |
| `--continue-session` | Pass the session of a previous session to continue, restoring your workspace (see `--store-workspace`) | `none` |
| `--debugging-session` | Pass existing Task ID, create a copy of the experiment on a remote machine, and launch Jupyter/SSH for interactive access. Example `--debugging-session <task_id>`| `none`|
| `--debugging-session` | Pass existing Task ID, create a copy of the task on a remote machine, and launch Jupyter/SSH for interactive access. Example `--debugging-session <task_id>`| `none`|
| `--disable-fingerprint-check` | If set, bypass the remote SSH server fingerprint verification process | `none` |
| `--disable-session-cleanup` | If `True`, previous interactive sessions are not deleted | `false`|
| `--disable-store-defaults` | If set, do not store current setup as new default configuration| `none`|

10
docs/clearml_agent.md
View File

@@ -17,10 +17,10 @@ title: ClearML Agent
**ClearML Agent** is a virtual environment and execution manager for DL / ML solutions on GPU machines. It integrates with the **ClearML Python Package** and ClearML Server to provide a full AI cluster solution. <br/>
Its main focus is around:
- Reproducing experiments, including their complete environments.
- Reproducing tasks, including their complete environments.
- Scaling workflows on multiple target machines.
ClearML Agent executes an experiment or other workflow by reproducing the state of the code from the original machine
ClearML Agent executes a task or other workflow by reproducing the state of the code from the original machine
to a remote machine.
![ClearML Agent flow diagram](img/clearml_agent_flow_diagram.png)
@@ -46,16 +46,16 @@ install Python, so make sure to use a docker or environment with the version you
While the agent is running, it continuously reports system metrics to the ClearML Server (these can be monitored in the
[**Orchestration**](webapp/webapp_workers_queues.md) page).
Continue using ClearML Agent once it is running on a target machine. Reproduce experiments and execute
Continue using ClearML Agent once it is running on a target machine. Reproduce tasks and execute
automated workflows in one (or both) of the following ways:
* Programmatically (using [`Task.enqueue()`](references/sdk/task.md#taskenqueue) or [`Task.execute_remotely()`](references/sdk/task.md#execute_remotely))
* Through the ClearML Web UI (without working directly with code), by cloning experiments and enqueuing them to the
* Through the ClearML Web UI (without working directly with code), by cloning tasks and enqueuing them to the
queue that a ClearML Agent is servicing.
The agent facilitates [overriding task execution detail](webapp/webapp_exp_tuning.md) values through the UI without
code modification. Modifying a task clone’s configuration will have the ClearML agent executing it override the
original values:
* Modified package requirements will have the experiment script run with updated packages
* Modified package requirements will have the task script run with updated packages
* Modified recorded command line arguments will have the ClearML agent inject the new values in their stead
* Code-level configuration instrumented with [`Task.connect()`](references/sdk/task.md#connect) will be overridden by modified hyperparameters

6
docs/clearml_agent/clearml_agent_deployment.md
View File

@@ -8,7 +8,7 @@ service a queue(s). Utilize the machine by enqueuing tasks to the queue that the
pull and execute the tasks.
:::tip cross-platform execution
ClearML Agent is platform agnostic. When using the ClearML Agent to execute experiments cross-platform, set platform
ClearML Agent is platform-agnostic. When using the ClearML Agent to execute tasks cross-platform, set platform
specific environment variables before launching the agent.
For example, to run an agent on an ARM device, set the core type environment variable before spinning up the agent:
@@ -118,7 +118,7 @@ The ClearML Kubernetes-Glue performs the following:
- Prepares a Kubernetes job based on a provided YAML template.
- Inside each job pod, the `clearml-agent`:
- Installs the required environment for the task.
- Executes and monitors the experiment process.
- Executes and monitors the task process.
:::important Enterprise Features
ClearML Enterprise adds advanced Kubernetes features:
@@ -256,7 +256,7 @@ If you are running Slurm with Singularity containers support, set the following:
## Google Colab
ClearML Agent can run on a [Google Colab](https://colab.research.google.com/) instance. This helps users to leverage
compute resources provided by Google Colab and send experiments for execution on it.
compute resources provided by Google Colab and send tasks for execution on it.
Check out [this tutorial](../guides/ide/google_colab.md) on how to run a ClearML Agent on Google Colab!

8
docs/clearml_agent/clearml_agent_docker.md
View File

@@ -6,7 +6,7 @@ title: Building Docker Containers
### Task Container
Build a Docker container that when launched executes a specific experiment, or a clone (copy) of that experiment.
Build a Docker container that when launched executes a specific task, or a clone (copy) of that task.
- Build a Docker container that at launch will execute a specific Task:
@@ -26,7 +26,7 @@ Build a Docker container that when launched executes a specific experiment, or a
docker run <new-docker-name>
```
Check out [this tutorial](../guides/clearml_agent/executable_exp_containers.md) for building executable experiment
Check out [this tutorial](../guides/clearml_agent/executable_exp_containers.md) for building executable task
containers.
### Base Docker Container
@@ -37,10 +37,10 @@ Build a Docker container according to the execution environment of a specific ta
clearml-agent build --id <task-id> --docker --target <new-docker-name>
```
You can add the Docker container as the base Docker image to a task (experiment), using one of the following methods:
You can add the Docker container as the base Docker image to a task, using one of the following methods:
- Using the **ClearML Web UI** - See [Base Docker image](../webapp/webapp_exp_tuning.md#base-docker-image) on the "Tuning
Experiments" page.
Tasks" page.
- In the ClearML configuration file - Use the ClearML configuration file [`agent.default_docker`](../configs/clearml_conf.md#agentdefault_docker)
options.

6
docs/clearml_agent/clearml_agent_env_caching.md
View File

@@ -2,18 +2,18 @@
title: Environment Caching
---
ClearML Agent caches virtual environments so when running experiments multiple times, there's no need to spend time reinstalling
ClearML Agent caches virtual environments so when running tasks multiple times, there's no need to spend time reinstalling
pre-installed packages. To make use of the cached virtual environments, enable the virtual environment reuse mechanism.
## Virtual Environment Reuse
The virtual environment reuse feature may reduce experiment startup time dramatically.
The virtual environment reuse feature may reduce task startup time dramatically.
By default, ClearML uses the package manager's environment caching. This means that even if no
new packages need to be installed, checking the list of packages can take a long time.
ClearML has a virtual environment reuse mechanism which, when enabled, allows using environments as-is without resolving
installed packages. This means that when executing multiple experiments with the same package dependencies,
installed packages. This means that when executing multiple tasks with the same package dependencies,
the same environment will be used.
:::note

4
docs/clearml_agent/clearml_agent_execution_env.md
View File

@@ -5,7 +5,7 @@ ClearML Agent has two primary execution modes: [Virtual Environment Mode](#virtu
## Virtual Environment Mode
In Virtual Environment Mode, the agent creates a virtual environment for the experiment, installs the required Python
In Virtual Environment Mode, the agent creates a virtual environment for the task, installs the required Python
packages based on the task specification, clones the code repository, applies the uncommitted changes and finally
executes the code while monitoring it. This mode uses smart caching so packages and environments can be reused over
multiple tasks (see [Virtual Environment Reuse](clearml_agent_env_caching.md#virtual-environment-reuse)).
@@ -42,7 +42,7 @@ ClearML Agent uses the provided default Docker container, which can be overridde
:::tip Setting Docker Container via UI
You can set the docker container via the UI:
1. Clone the experiment
1. Clone the task
2. Set the Docker in the cloned task's **Execution** tab **> Container** section
![Container section](../img/webapp_exp_container.png#light-mode-only)

8
docs/clearml_agent/clearml_agent_ref.md
View File

@@ -3,10 +3,10 @@ title: ClearML Agent CLI
---
The following page provides a reference to `clearml-agent`'s CLI commands:
* [build](#build) - Create a worker environment without executing an experiment.
* [build](#build) - Create a worker environment without executing a task.
* [config](#config) - List your ClearML Agent configuration data.
* [daemon](#daemon) - Run a worker daemon listening to a queue for Tasks (experiments) to execute.
* [execute](#execute) - Execute an experiment, locally without a queue.
* [daemon](#daemon) - Run a worker daemon listening to a queue for tasks to execute.
* [execute](#execute) - Execute a task, locally without a queue.
* [list](#list) - List the current workers.
@@ -59,7 +59,7 @@ clearml-agent config [-h]
## daemon
Use the `daemon` command to spin up an agent on any machine: on-prem and/or cloud instance. When spinning up an agent,
assign it a queue(s) to service, and when experiments are added to its queues, the agent will pull and execute them.
assign it a queue(s) to service, and when tasks are added to its queues, the agent will pull and execute them.
With the `daemon` command you can configure your agent's behavior: allocate resources, prioritize queues, set it to run
in a Docker, and more.

2
docs/clearml_data/best_practices.md
View File

@@ -14,7 +14,7 @@ The following are some recommendations for using ClearML Data.
## Versioning Datasets
Use ClearML Data to version your datasets. Once a dataset is finalized, it can no longer be modified. This makes clear
which version of the dataset was used with which task, enabling the accurate reproduction of your experiments.
which version of the dataset was used with which task, enabling the accurate reproduction of your tasks.
Once you need to change the dataset's contents, you can create a new version of the dataset by specifying the previous
dataset as a parent. This makes the new dataset version inherit the previous version's contents, with the dataset's new

2
docs/clearml_data/clearml_data.md
View File

@@ -12,7 +12,7 @@ which you then need to be able to share, reproduce, and track.
ClearML Data Management solves two important challenges:
- Accessibility - Making data easily accessible from every machine,
- Versioning - Linking data and experiments for better **traceability**.
- Versioning - Linking data and tasks for better **traceability**.
![Dataset lineage and preview](../img/webapp_dataset_lineage_preview.png)

2
docs/clearml_data/data_management_examples/data_man_cifar_classification.md
View File

@@ -99,7 +99,7 @@ trainset = datasets.CIFAR10(
In cases like this, where you use a dataset in a task, you can have the dataset's ID stored in the task's
hyperparameters. Passing `alias=<dataset_alias_string>` stores the dataset's ID in the
`dataset_alias_string` parameter in the experiment's **CONFIGURATION > HYPERPARAMETERS > Datasets** section. This way
`dataset_alias_string` parameter in the task's **CONFIGURATION > HYPERPARAMETERS > Datasets** section. This way
you can easily track which dataset the task is using.
[`Dataset.get_local_copy()`](../../references/sdk/dataset.md#get_local_copy) returns a path to the cached,

4
docs/clearml_sdk/clearml_sdk.md
View File

@@ -3,7 +3,7 @@ title: ClearML SDK
---
The ClearML Python Package supports the [automatic logging](../fundamentals/logger.md#automatic-reporting) that documents
experiments for you, and an extensive set of powerful features and functionality you can use to improve experimentation
tasks for you, and an extensive set of powerful features and functionality you can use to improve experimentation
and other workflows.
:::tip Installation
@@ -36,7 +36,7 @@ See an [overview](task_sdk.md) of `Task`'s pythonic methods or the [Task SDK ref
The `model` module contains three classes that provide support for working with models in ClearML:
* `Model` - represents an existing model in ClearML that can be loaded and connected to a Task
* `InputModel` - represents an existing model that you can load into ClearML
* `OutputModel` - represents the experiment output model that is always connected to the Task
* `OutputModel` - represents the task output model that is always connected to the Task
See an [overview](../clearml_sdk/model_sdk.md) of the Model classes' pythonic methods, or the SDK reference pages for [`Model`](../references/sdk/model_model.md),
[`InputModel`](../references/sdk/model_inputmodel.md), and [`OutputModel`](../references/sdk/model_outputmodel.md).

10
docs/clearml_sdk/model_sdk.md
View File

@@ -6,10 +6,10 @@ The following page provides an overview of the basic Pythonic interface to Clear
ClearML provides the following classes to work with models:
* `Model` - Represents a ClearML model, regardless of any task connection. Use this class to programmatically access and manage the ClearML model store.
* `InputModel` - Represents a ClearML model to be used in an experiment. Use this class to load a model from ClearML's model store or to import a pre-trained
model from an external resource to use as an experiment's initial starting point.
* `OutputModel` - Represents an experiment's output model (training results). An OutputModel is always connected to a [task](../fundamentals/task.md),
so the models are traceable to experiments.
* `InputModel` - Represents a ClearML model to be used in a task. Use this class to load a model from ClearML's model store or to import a pre-trained
model from an external resource to use as a task's initial starting point.
* `OutputModel` - Represents a task's output model (training results). An OutputModel is always connected to a [task](../fundamentals/task.md),
so the models are traceable to tasks.
## Output Models
@@ -40,7 +40,7 @@ output_model.update_labels({'background': 0, 'label': 255})
```
### Updating Models
ClearML doesn't automatically log the snapshots of manually logged models. To update an experiment's model use the
ClearML doesn't automatically log the snapshots of manually logged models. To update a task's model use the
[OutputModel.update_weights](../references/sdk/model_outputmodel.md#update_weights) method.
```python

18
docs/clearml_sdk/task_sdk.md
View File

@@ -97,7 +97,7 @@ After invoking `Task.init` in a script, ClearML starts its automagical logging,
### Control Automatic Logging
By default, when ClearML is integrated into your script, it automatically captures information from supported frameworks,
and parameters from supported argument parsers. But, you may want to have more control over what your experiment logs.
and parameters from supported argument parsers. But, you may want to have more control over what your task logs.
#### Frameworks
To control a task's framework logging, use the `auto_connect_frameworks` parameter of [`Task.init()`](../references/sdk/task.md#taskinit).
@@ -239,7 +239,7 @@ Set a task's progress to a numeric value between 0 - 100. Access the task's curr
[`Task.get_progress()`](../references/sdk/task.md#get_progress).
```python
task = Task.init(project_name="examples", task_name="Track experiment progress")
task = Task.init(project_name="examples", task_name="Track task progress")
task.set_progress(0)
# task doing stuff
task.set_progress(50)
@@ -248,12 +248,12 @@ print(task.get_progress())
task.set_progress(100)
```
While the task is running, the WebApp will show the task's progress indication in the experiment table, next to the
While the task is running, the WebApp will show the task's progress indication in the task table, next to the
task's status. If a task failed or was aborted, you can view how much progress it had made.
<div class="max-w-50">
![Experiment table progress indication](../img/fundamentals_task_progress.png)
![Task table progress indication](../img/fundamentals_task_progress.png)
</div>
@@ -263,7 +263,7 @@ in the WebApp.
## Accessing Tasks
A task can be identified by its project and name, and by a unique identifier (UUID string). The name and project of
a task can be changed after an experiment has been executed, but its ID can't be changed.
a task can be changed after its execution, but its ID can't be changed.
Programmatically, task objects can be retrieved by querying the system based on either the task ID or a project and name
combination using the [`Task.get_task()`](../references/sdk/task.md#taskget_task) class method. If a project / name
@@ -484,7 +484,7 @@ a_func_task = task.create_function_task(
)
```
Arguments passed to the function will be automatically logged in the
experiment's **CONFIGURATION** tab under the **HYPERPARAMETERS > Function** section.
task's **CONFIGURATION** tab under the **HYPERPARAMETERS > Function** section.
Like any other arguments, they can be changed from the UI or programmatically.
:::note Function Task Creation
@@ -779,14 +779,14 @@ Notice that if one of the frameworks loads an existing weights file, the running
"Input Model", pointing directly to the original training task's model. This makes it easy to get the full lineage of
every trained and used model in your system!
Models loaded by the ML framework appear in an experiment's **Artifacts** tab under the "Input Models" section in the ClearML UI.
Models loaded by the ML framework appear in a task's **Artifacts** tab under the "Input Models" section in the ClearML UI.
### Setting Upload Destination
ClearML automatically captures the storage location of Models created by frameworks such as TensorFlow, PyTorch, and scikit-learn.
By default, it stores the local path they are saved at.
To automatically store all created models by a specific experiment, modify the `Task.init` function as such:
To automatically store all created models by a specific task, modify the `Task.init` function as such:
```python
task = Task.init(
@@ -805,7 +805,7 @@ Specify the model storage URI location using the relevant format:
* Azure Storage: `azure://<account name>.blob.core.windows.net/path/to/file`
:::
To automatically store all models created by any experiment at a specific location, edit the `clearml.conf` (see
To automatically store all models created by any task at a specific location, edit the `clearml.conf` (see
[ClearML Configuration Reference](../configs/clearml_conf.md#sdkdevelopment)) and set `sdk.developmenmt.default_output_uri`
to the desired storage (see [Storage](../integrations/storage.md)). This is especially helpful when
using [clearml-agent](../clearml_agent.md) to execute code.

22
docs/configs/clearml_conf.md
View File

@@ -166,7 +166,7 @@ Compatible with Docker versions 0.6.5 and above
**`agent.docker_force_pull`** (*bool*)
* Always update the Docker image by forcing a Docker `pull` before running an experiment
* Always update the Docker image by forcing a Docker `pull` before running a task
The values are:
@@ -242,14 +242,14 @@ from `system_site_packages`
**`agent.extra_docker_arguments`** (*[string]*)
* Optional arguments to pass to the Docker image when ClearML Agent is running in [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode). These are local for this agent, and will not be updated in the experiment's `docker_cmd` section. For example, `["--ipc=host", ]`.
* Optional arguments to pass to the Docker image when ClearML Agent is running in [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode). These are local for this agent, and will not be updated in the task's `docker_cmd` section. For example, `["--ipc=host", ]`.
---
**`agent.extra_docker_shell_script`** (*[string]*)
* When ClearML Agent is running in [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode), this
optional shell script executes inside the Docker on startup, before the experiment starts. For example, `["apt-get install -y bindfs", ]`.
optional shell script executes inside the Docker on startup, before the task starts. For example, `["apt-get install -y bindfs", ]`.
---
@@ -357,7 +357,7 @@ ___
**`agent.python_binary`** (*string*)
* Set the Python version to use when creating the virtual environment, and when launching the experiment. For example, `/usr/bin/python3` or `/usr/local/bin/python3.6`.
* Set the Python version to use when creating the virtual environment, and when launching the task. For example, `/usr/bin/python3` or `/usr/local/bin/python3.6`.
---
@@ -376,7 +376,7 @@ ___
**`agent.venvs_dir`** (*string*)
* The target folder for virtual environments builds that are created when executing an experiment.
* The target folder for virtual environments builds that are created when executing a task.
---
@@ -628,7 +628,7 @@ ___
**`agent.package_manager.system_site_packages`** (*bool*)
* Indicates whether Python packages for virtual environments are inherited from the system when building a virtual environment
for an experiment.
for a task.
The values are:
@@ -716,7 +716,7 @@ Torch Nightly builds are ephemeral and are deleted from time to time.
**`agent.vcs_cache.path`** (*string*)
* The version control system cache clone folder when executing experiments.
* The version control system cache clone folder when executing tasks.
<br/>
@@ -1115,7 +1115,7 @@ URL to a CA bundle, or set this option to `false` to skip SSL certificate verifi
**`sdk.development.log_os_environments`** (*[string]*)
* Log specific environment variables. OS environments are listed in the UI, under an experiment's
* Log specific environment variables. OS environments are listed in the UI, under a task's
**CONFIGURATION > HYPERPARAMETERS > Environment** section.
Multiple selected variables are supported including the suffix `*`. For example: `"AWS_*"` will log any OS environment
variable starting with `"AWS_"`. Example: `log_os_environments: ["AWS_*", "CUDA_VERSION"]`
@@ -1126,7 +1126,7 @@ URL to a CA bundle, or set this option to `false` to skip SSL certificate verifi
**`sdk.development.store_uncommitted_code_diff`** (*bool*)
* For development mode, indicates whether to store the uncommitted `git diff` or `hg diff` in the experiment manifest.
* For development mode, indicates whether to store the uncommitted `git diff` or `hg diff` in the task manifest.
The values are:
@@ -1145,7 +1145,7 @@ URL to a CA bundle, or set this option to `false` to skip SSL certificate verifi
**`sdk.development.support_stopping`** (*bool*)
* For development mode, indicates whether to allow stopping an experiment if the experiment was aborted externally, its status was changed, or it was reset.
* For development mode, indicates whether to allow stopping a task if the task was aborted externally, its status was changed, or it was reset.
The values are:
@@ -1158,7 +1158,7 @@ URL to a CA bundle, or set this option to `false` to skip SSL certificate verifi
**`sdk.development.task_reuse_time_window_in_hours`** (*float*)
* For development mode, the number of hours after which an experiment with the same project name and experiment name is reused.
* For development mode, the number of hours after which a task with the same project name and task name is reused.
---

100
docs/faq.md
View File

@@ -17,26 +17,26 @@ title: FAQ
* [Models are not accessible from the UI after I migrated ClearML Server to a new address. How do I fix this?](#migrate_server_models)
* [Models are not accessible from the UI after I moved them (different bucket / server). How do I fix this?](#relocate_models)
**Experiments**
**Tasks**
* [I noticed I keep getting the message "warning: uncommitted code". What does it mean?](#uncommitted-code-warning)
* [I do not use argparse for hyperparameters. Do you have a solution?](#dont-want-argparser)
* [I noticed that all of my experiments appear as "Training". Are there other options?](#other-experiment-types)
* [Sometimes I see experiments as running when in fact they are not. What's going on?](#experiment-running-but-stopped)
* [My code throws an exception, but my experiment status is not "Failed". What happened?](#exception-not-failed)
* [CERTIFICATE_VERIFY_FAILED - When I run my experiment, I get an SSL Connection error. Do you have a solution?](#ssl-connection-error)
* [How do I modify experiment names once they have been created?](#modify_exp_names)
* [I noticed that all of my tasks appear as "Training". Are there other options?](#other-task-types)
* [Sometimes I see tasks as running when in fact they are not. What's going on?](#task-running-but-stopped)
* [My code throws an exception, but my task status is not "Failed". What happened?](#exception-not-failed)
* [CERTIFICATE_VERIFY_FAILED - When I run my task, I get an SSL Connection error. Do you have a solution?](#ssl-connection-error)
* [How do I modify task names once they have been created?](#modify_exp_names)
* [Using Conda and the "typing" package, I get the error "AttributeError: type object 'Callable' has no attribute '_abc_registry'". How do I fix this?](#typing)
* [My ClearML Server disk space usage is too high. What can I do about this?](#delete_exp)
* [Can I change the random seed my experiment uses?](#random_see)
* [In the Web UI, I can't access files that my experiment stored. Why not?](#access_files)
* [Can I change the random seed my task uses?](#random_seed)
* [In the Web UI, I can't access files that my task stored. Why not?](#access_files)
* [I get the message "ClearML Monitor: Could not detect iteration reporting, falling back to iterations as seconds-from-start". What does it mean?](#resource_monitoring)
* [Can I control what ClearML automatically logs?](#controlling_logging)
* [Can I run a ClearML Task while working offline?](#offline_mode)
**Graphs and Logs**
* [The first log lines are missing from the experiment console tab. Where did they go?](#first-log-lines-missing)
* [The first log lines are missing from the task console tab. Where did they go?](#first-log-lines-missing)
* [How do I create a graph comparing hyperparameters vs model accuracy?](#compare-graph-parameters)
* [I want to add more graphs, not just with TensorBoard. Is this supported?](#more-graph-types)
* [How can I report more than one scatter 2D series on the same plot?](#multiple-scatter2D)
@@ -65,11 +65,11 @@ title: FAQ
* [How do I explicitly specify the ClearML configuration file to be used?](#change-config-path)
* [How can I override ClearML credentials from the OS environment?](#credentials-os-env)
* [How can I track OS environment variables with experiments?](#track-env-vars)
* [How can I track OS environment variables with tasks?](#track-env-vars)
**ClearML Hosted Service**
* [I run my script, but my experiment is not in the ClearML Hosted Service Web UI. How do I fix this?](#hosted-service-no-config)
* [I run my script, but my task is not in the ClearML Hosted Service Web UI. How do I fix this?](#hosted-service-no-config)
**ClearML Server Deployment**
@@ -113,7 +113,7 @@ title: FAQ
#### How do I know a new version came out? <a className="tr_top_negative" id="new-version-auto-update"></a>
Starting with ClearML v0.9.3, ClearML issues a new version release notification, which appears in the log and is
output to the console, when a Python experiment script is run.
output to the console, when a Python task script is run.
For example, when a new ClearML Python Package version is available, the notification is:
@@ -165,8 +165,8 @@ add a [custom column](webapp/webapp_model_table.md#customizing-the-models-table)
that metric column. Available custom column options depend upon the models in the table and the metrics that have been
attached to them (see [Logging Metrics and Plots](clearml_sdk/model_sdk.md#logging-metrics-and-plots)).
ClearML associates models with the experiments that created them, so you can also add a [custom column](webapp/webapp_exp_table.md#customizing-the-task-table)
in an experiments table and sort by that metric column.
ClearML associates models with the tasks that created them, so you can also add a [custom column](webapp/webapp_exp_table.md#customizing-the-task-table)
in a task table and sort by that metric column.
<br/>
@@ -287,13 +287,13 @@ To fix this, the registered URL of each model needs to be replaced with its curr
mongo backend script.js
```
## Experiments
## Tasks
#### I noticed I keep getting the message "warning: uncommitted code". What does it mean? <a id="uncommitted-code-warning"></a>
This message is only a warning. ClearML not only detects your current repository and git commit, but also warns you
if you are using uncommitted code. ClearML does this because uncommitted code means this experiment will be difficult
to reproduce. You can see uncommitted changes in the ClearML Web UI, in the **EXECUTION** tab of the experiment info panel.
if you are using uncommitted code. ClearML does this because uncommitted code means this task will be difficult
to reproduce. You can see uncommitted changes in the ClearML Web UI, in the **EXECUTION** tab of the task info panel.
#### I do not use argparse for hyperparameters. Do you have a solution? <a id="dont-want-argparser"></a>
@@ -324,10 +324,10 @@ For more task configuration options, see [Hyperparameters](fundamentals/hyperpar
<br/>
#### I noticed that all of my experiments appear as "Training". Are there other options? <a id="other-experiment-types"></a>
#### I noticed that all of my tasks appear as "Training". Are there other options? <a id="other-experiment-types"></a>
Yes! ClearML supports [multiple experiment types](fundamentals/task.md#task-types). When creating experiments and
calling [`Task.init()`](references/sdk/task.md#taskinit), you can provide an experiment type. For example:
Yes! ClearML supports [multiple task types](fundamentals/task.md#task-types). When creating tasks and
calling [`Task.init()`](references/sdk/task.md#taskinit), you can provide a task type. For example:
```python
task = Task.init(project_name, task_name, Task.TaskTypes.testing)
@@ -336,13 +336,13 @@ task = Task.init(project_name, task_name, Task.TaskTypes.testing)
<br/>
#### Sometimes I see experiments as running when in fact they are not. What's going on? <a id="experiment-running-but-stopped"></a>
#### Sometimes I see tasks as running when in fact they are not. What's going on? <a id="experiment-running-but-stopped"></a>
ClearML monitors your Python process. When the process exits properly, ClearML closes the experiment. When the process crashes and terminates abnormally, it sometimes misses the stop signal. In this case, you can safely right-click the experiment in the WebApp and abort it.
ClearML monitors your Python process. When the process exits properly, ClearML closes the task. When the process crashes and terminates abnormally, it sometimes misses the stop signal. In this case, you can safely right-click the task in the WebApp and abort it.
<br/>
#### My code throws an exception, but my experiment status is not "Failed". What happened? <a id="exception-not-failed"></a>
#### My code throws an exception, but my task status is not "Failed". What happened? <a id="exception-not-failed"></a>
This issue was resolved in Trains v0.9.2. Upgrade to ClearML by executing the following command:
@@ -354,7 +354,7 @@ pip install -U clearml
<br/>
#### When I run my experiment, I get an SSL Connection error CERTIFICATE_VERIFY_FAILED. Do you have a solution?
#### When I run my task, I get an SSL Connection error CERTIFICATE_VERIFY_FAILED. Do you have a solution?
Your firewall may be preventing the connection. Try one of the following solutions:
@@ -382,11 +382,11 @@ Your firewall may be preventing the connection. Try one of the following solutio
<br/>
#### How do I modify experiment names once they have been created?
#### How do I modify task names once they have been created?
An experiment's name is a user-controlled property, which can be accessed via the `Task.name` property. This lets you use meaningful naming schemes to easily filter and compare experiments.
A task's name is a user-controlled property, which can be accessed via the `Task.name` property. This lets you use meaningful naming schemes to easily filter and compare tasks.
For example, to distinguish between different experiments, you can append the task ID to the task name:
For example, to distinguish between different tasks, you can append the task ID to the task name:
```python
task = Task.init(project_name='examples', task_name='train')
task.name += ' {}'.format(task.id)
@@ -407,7 +407,7 @@ for t in tasks:
if 'my_secret_parameter' in params:
t.name += ' my_secret_parameter={}'.format(params['my_secret_parameter'])
```
Use this experiment naming when creating automation pipelines with a naming convention.
Use this task naming when creating automation pipelines with a naming convention.
<a id="typing"></a>
@@ -427,10 +427,10 @@ To resolve the error, uninstall `typing` and rerun your script. If this does not
#### My ClearML Server disk space usage is too high. What can I do about this?
To clear up space, you can delete ClearML objects (e.g. experiments, models, datasets, etc.).
To clear up space, you can delete ClearML objects (e.g. tasks, models, datasets, etc.).
To delete an object via the UI:
1. Go to the relevant object table (e.g. [Experiments Table](webapp/webapp_exp_table.md), [Models Table](webapp/webapp_model_table.md), etc.)
1. Go to the relevant object table (e.g. [Task Table](webapp/webapp_exp_table.md), [Model Table](webapp/webapp_model_table.md), etc.)
1. Archive the object - Right-click the object in the table **>** click **Archive**
1. Click **Open Archive** on the top of the table
1. In the archive table, right-click the object **>** click **Delete**.
@@ -448,11 +448,11 @@ You cannot undo the deletion of a ClearML object.
:::
<a id="random_see"></a>
<a id="random_seed"></a>
<br/>
#### Can I change the random seed my experiment uses?
#### Can I change the random seed my task uses?
Yes! By default, ClearML initializes Tasks with an initial seed of `1337` to ensure reproducibility. To set a different
value for your task, use the [`Task.set_random_seed()`](references/sdk/task.md#taskset_random_seed) class method and
@@ -464,7 +464,7 @@ You can disable the deterministic behavior entirely by passing `Task.set_random_
<br/>
#### In the Web UI, I can't access files that my experiment stored. Why not?
#### In the Web UI, I can't access files that my task stored. Why not?
ClearML stores file locations. The machine running your browser must have access to the location where the machine
that ran the Task stored the file. This applies to debug samples and artifacts. If, for example, the machine running the browser does not have access, you may see `Unable to load image`, instead of the image.
@@ -586,11 +586,11 @@ See [Storing Task Data Offline](guides/set_offline.md).
## Graphs and Logs
#### The first log lines are missing from the experiment console tab. Where did they go? <a id="first-log-lines-missing"></a>
#### The first log lines are missing from the task console tab. Where did they go? <a id="first-log-lines-missing"></a>
Due to speed/optimization issues, the console displays only the last several hundred log lines.
You can always download the full log as a file using the ClearML Web UI. In the **ClearML Web UI >** experiment's **CONSOLE**
You can always download the full log as a file using the ClearML Web UI. In the **ClearML Web UI >** task's **CONSOLE**
tab, click `Download full log`.
![Download console log](img/faq_download_console_log.png)
@@ -599,20 +599,20 @@ tab, click `Download full log`.
#### How do I create a graph comparing hyperparameters vs. model accuracy? <a id="compare-graph-parameters"></a>
You can use the UI's [experiment comparison features](webapp/webapp_exp_comparing.md) to compare the logged hyperparameter
and accuracy values of several experiments. In the experiment comparison page, under the **HYPERPARAMETERS** tab
you can visualize experiments' hyperparameter values in relation to performance metrics in a scatter plot or parallel
You can use the UI's [task comparison features](webapp/webapp_exp_comparing.md) to compare the logged hyperparameter
and accuracy values of several tasks. In the task comparison page, under the **HYPERPARAMETERS** tab
you can visualize tasks' hyperparameter values in relation to performance metrics in a scatter plot or parallel
coordinates plot:
* [Scatter plot](webapp/webapp_exp_comparing.md#scatter-plot): View the correlation between a selected hyperparameter and
metric. For example, the image below shows a scatter plot that displays the values of a performance metric (`epoch_accuracy`)
and a hyperparameter (`epochs`) of a few experiments:
and a hyperparameter (`epochs`) of a few tasks:
![Scatter plot comparison](img/faq_compare_scatter.png)
* [Parallel coordinates plot](webapp/webapp_exp_comparing.md#parallel-coordinates-mode): View the impact of hyperparameters
on selected metric(s). For example, the image below shows
a parallel coordinates plot which displays the values of selected hyperparameters (`base_lr`, `batch_size`, and
`number_of_epochs`) and a performance metric (`accuracy`) of three experiments:
`number_of_epochs`) and a performance metric (`accuracy`) of three tasks:
![Parallel Coordinates](img/compare_parallel_coordinates.png)
@@ -658,8 +658,8 @@ logger.report_scatter2d(
#### Is there something ClearML can do about uncommitted code running? <a id="help-uncommitted-code"></a>
Yes! ClearML stores the git diff as part of the experiment's information. You can view the git diff in the **ClearML Web UI >**
experiment's **EXECUTION** tab.
Yes! ClearML stores the git diff as part of the task's information. You can view the git diff in the **ClearML Web UI >**
task's **EXECUTION** tab.
<br/>
@@ -675,7 +675,7 @@ task = Task.init(project_name, task_name, output_uri="/mnt/shared/folder")
```
ClearML will copy all stored snapshots into a subfolder under `/mnt/shared/folder`. The subfolder's name will contain
the experiment's ID. If the experiment's ID is `6ea4f0b56d994320a713aeaf13a86d9d`, the following folder will be used:
the task's ID. For example, if the task's ID is `6ea4f0b56d994320a713aeaf13a86d9d`, the following folder will be used:
```
/mnt/shared/folder/task.6ea4f0b56d994320a713aeaf13a86d9d/models/
```
@@ -849,9 +849,9 @@ export CLEARML_API_HOST="http://localhost:8008"
<br/>
#### How can I track OS environment variables with experiments? <a id="track-env-vars"></a>
#### How can I track OS environment variables with tasks? <a id="track-env-vars"></a>
You can set environment variables to track in an experiment by specifying them in the `sdk.development.log_os_environments`
You can set environment variables to track in a task by specifying them in the `sdk.development.log_os_environments`
field of the [`clearml.conf`](configs/clearml_conf.md#log_env_var) file:
```editorconfig
@@ -884,9 +884,9 @@ The `CLEARML_LOG_ENVIRONMENT` variable always overrides the `clearml.conf` file.
## ClearML Hosted Service
#### I run my script, but my experiment is not in the ClearML Hosted Service Web UI. How do I fix this? <a id="hosted-service-no-config"></a>
#### I run my script, but my task is not in the ClearML Hosted Service Web UI. How do I fix this? <a id="hosted-service-no-config"></a>
If you joined the ClearML Hosted Service and ran a script, but your experiment does not appear in Web UI, you may not have configured ClearML for the hosted service. Run the ClearML setup wizard. It will request your hosted service ClearML credentials and create the ClearML configuration you need.
If you joined the ClearML Hosted Service and ran a script, but your task does not appear in Web UI, you may not have configured ClearML for the hosted service. Run the ClearML setup wizard. It will request your hosted service ClearML credentials and create the ClearML configuration you need.
```
pip install clearml
@@ -961,7 +961,7 @@ For detailed instructions, see [Web Login Authentication](deploying_clearml/clea
#### Can I modify non-responsive task settings? <a id="watchdog"></a>
The non-responsive experiment watchdog monitors experiments that were not updated for a specified time interval, and
The non-responsive task watchdog monitors tasks that were not updated for a specified time interval, and
marks them as `aborted`. The watchdog is always active.
You can modify the following settings for the watchdog:
@@ -1137,9 +1137,9 @@ If your ClearML WebApp (UI) does not show anything, it may be an error authentic
The network definitions inside a virtual machine (or container) are different from those of the host. The virtual machine's
and the server machine's IP addresses are different, so you have to make sure that the machine that is executing the
experiment can access the server's machine.
task can access the server's machine.
Make sure to have an independent configuration file for the virtual machine where you are running your experiments.
Make sure to have an independent configuration file for the virtual machine where you are running your tasks.
Edit the `api` section of your `clearml.conf` file and insert IP addresses of the server machine that are accessible
from the VM. It should look something like this:

4
docs/fundamentals/agents_and_queues.md
View File

@@ -79,7 +79,7 @@ Agents can be deployed bare-metal, with multiple instances allocating
specific GPUs to the agents. They can also be deployed as dockers in a Kubernetes cluster.
The Agent supports the following running modes:
* **Virtual Environment Mode** - The agent creates a new virtual environment for the experiment, installs the required
* **Virtual Environment Mode** - The agent creates a new virtual environment for the task, installs the required
python packages based on the Task specification, clones the code repository, applies the uncommitted changes and
finally executes the code while monitoring it. This mode uses smart caching so packages and environments can be reused
over multiple tasks (see [Virtual Environment Reuse](../clearml_agent/clearml_agent_env_caching.md#virtual-environment-reuse)).
@@ -108,7 +108,7 @@ in parallel (each task will register itself as a sub-agent, visible in the [Orch
This mode is intended for running maintenance tasks. Some suitable tasks include:
- [Pipeline controller](../guides/pipeline/pipeline_controller.md) - Implementing the pipeline scheduling and logic
- [Hyperparameter Optimization](../guides/optimization/hyper-parameter-optimization/examples_hyperparam_opt.md) - Implementing an active selection of experiments
- [Hyperparameter Optimization](../guides/optimization/hyper-parameter-optimization/examples_hyperparam_opt.md) - Implementing an active selection of tasks
- [Control Service](../guides/services/aws_autoscaler.md) - AWS Autoscaler for example
- [External services](../guides/services/slack_alerts.md) - Such as Slack integration alert service

20
docs/fundamentals/hyperparameters.md
View File

@@ -5,14 +5,14 @@ title: Hyperparameters
Hyperparameters are a script's configuration options. Since hyperparameters can have substantial impact on
model performance, it is crucial to efficiently track and manage them.
ClearML supports tracking and managing hyperparameters in each experiment and provides a dedicated [hyperparameter
optimization module](hpo.md). With ClearML's logging and tracking capabilities, experiments can be reproduced, and their
ClearML supports tracking and managing hyperparameters in each task and provides a dedicated [hyperparameter
optimization module](hpo.md). With ClearML's logging and tracking capabilities, tasks can be reproduced, and their
hyperparameters and results can be saved and compared, which is key to understanding model behavior.
ClearML lets you easily try out different hyperparameter values without changing your original code. ClearML's [execution
agent](../clearml_agent.md) will override the original values with any new ones you specify through the web UI (see
[Configuration](../webapp/webapp_exp_tuning.md#configuration) in the Tuning Experiments page). You can also
programmatically set experiment parameters.
[Configuration](../webapp/webapp_exp_tuning.md#configuration) in the Tuning Tasks page). You can also
programmatically set task parameters.
## Tracking Hyperparameters
Hyperparameters can be added from anywhere in your code, and ClearML provides multiple ways to obtain them! ClearML logs
@@ -38,8 +38,8 @@ Automatic logging can be disabled. See [Control Automatic Logging](../clearml_sd
### Environment Variables
:::important Experiment Reproducibility
Relying on environment variables makes an experiment not fully reproducible, since ClearML Agent can't reproduce them at
:::important Task Reproducibility
Relying on environment variables makes a task not fully reproducible, since ClearML Agent can't reproduce them at
runtime.
:::
@@ -94,15 +94,15 @@ objects are saved as blobs i.e. ClearML is not aware of their internal structure
#### Setting and Updating Parameters
ClearML provides methods to set and update task parameters manually. Use [`Task.set_parameters()`](../references/sdk/task.md#set_parameters)
to define parameters manually. To update the parameters in an experiment, use [`Task.set_parameters_as_dict()`](../references/sdk/task.md#set_parameters_as_dict).
to define parameters manually. To update the parameters in a task, use [`Task.set_parameters_as_dict()`](../references/sdk/task.md#set_parameters_as_dict).
The `set_parameters_as_dict` method updates parameters while the `set_parameters` method overrides the parameters.
ClearML does not automatically track changes to explicitly set parameters.
### User Properties
User properties do not impact task execution and can be modified at any stage. They are convenient for setting
helpful values which are displayed in the [experiment table](../webapp/webapp_exp_table.md) (i.e. customize columns),
making it easier to search / filter experiments. Add user properties to an experiment with the
helpful values which are displayed in the [task table](../webapp/webapp_exp_table.md) (i.e. customize columns),
making it easier to search / filter tasks. Add user properties to a task with the
[`Task.set_user_properties`](../references/sdk/task.md#set_user_properties) method.
### Accessing Parameters
@@ -115,7 +115,7 @@ including their subsections (see [WebApp sections](#webapp-interface) below).
## WebApp Interface
Configurations can be viewed in web UI experiment pages, in the **CONFIGURATION** tab.
Configurations can be viewed in web UI task pages, in the **CONFIGURATION** tab.
The configuration panel is split into three sections according to type:
- **User Properties** - Modifiable section that can be edited post-execution

2
docs/fundamentals/logger.md
View File

@@ -2,7 +2,7 @@
title: Logger
---
The ClearML Logger class is used to report experiments' results such as metrics, graphs, and debug samples. It is provided
The ClearML Logger class is used to report task results such as metrics, graphs, and debug samples. It is provided
through the ClearML [Task](task.md) object.
A Logger object is used to do the following:

6
docs/fundamentals/projects.md
View File

@@ -13,14 +13,14 @@ computer, making organization easier.
Projects contain a textual description field for noting relevant information. The WebApp supports markdown rendering
of the description (see [overview](../webapp/webapp_project_overview.md)).
In addition, the project's default output URI can be specified. When new experiments from
In addition, the project's default output URI can be specified. When new tasks from
the project are executed, the model checkpoints (snapshots) and artifacts are stored in the default output location.
## WebApp
Users can create and modify projects, and see project details in the [WebApp](../webapp/webapp_home.md).
A project's description can be edited in its [overview](../webapp/webapp_project_overview.md) page. Each project's experiments,
models, and dataviews, can be viewed in the project's [experiments table](../webapp/webapp_exp_table.md),
A project's description can be edited in its [overview](../webapp/webapp_project_overview.md) page. Each project's tasks,
models, and dataviews, can be viewed in the project's [task table](../webapp/webapp_exp_table.md),
[models table](../webapp/webapp_model_table.md), and [dataviews table](../hyperdatasets/webapp/webapp_dataviews.md).
## Usage

30
docs/fundamentals/task.md
View File

@@ -31,7 +31,7 @@ You can create copies of a task ([clone](../webapp/webapp_exp_reproducing.md)) t
* Set hyperparameter and run-time configuration values
Modifying a task clone's configuration will have the executing ClearML agent override the original values:
* Modified package requirements will have the experiment script run with updated packages
* Modified package requirements will have the task script run with updated packages
* Modified recorded command line arguments will have the ClearML agent inject the new values in their stead
* Code-level configuration instrumented with [`Task.connect`](../references/sdk/task.md#connect) will be overridden by
modified hyperparameters
@@ -46,7 +46,7 @@ modified hyperparameters
The sections of ClearML Task are made up of the information that a task captures and stores, which consists of code
execution details and execution outputs. This information is used for tracking
and visualizing results, reproducing, tuning, and comparing experiments, and executing workflows.
and visualizing results, reproducing, tuning, and comparing tasks, and executing workflows.
The captured [code execution information](../webapp/webapp_exp_track_visual.md#execution) includes:
* Git information
@@ -61,11 +61,11 @@ The captured [execution output](../webapp/webapp_exp_track_visual.md#task-result
* [Debug samples](../webapp/webapp_exp_track_visual.md#debug-samples)
* [Models](models.md)
For a more in-depth description of each task section, see [Tracking Experiments and Visualizing Results](../webapp/webapp_exp_track_visual.md).
For a more in-depth description of each task section, see [Tracking Tasks and Visualizing Results](../webapp/webapp_exp_track_visual.md).
### Execution Configuration
ClearML logs a task's hyperparameters specified as command line arguments, environment or code level variables. This
allows experiments to be reproduced, and their hyperparameters and results can be saved and compared, which is key to
allows tasks to be reproduced, and their hyperparameters and results can be saved and compared, which is key to
understanding model behavior.
Hyperparameters can be added from anywhere in your code, and ClearML provides multiple ways to log them. If you specify
@@ -80,10 +80,10 @@ See [Hyperparameters](hyperparameters.md) for more information.
### Artifacts
ClearML allows easy storage of experiments' output products as artifacts that can later be accessed easily and used,
ClearML allows easy storage of tasks' output products as artifacts that can later be accessed easily and used,
through the [web UI](../webapp/webapp_overview.md) or programmatically.
ClearML provides methods to easily track files generated throughout your experiments' execution such as:
ClearML provides methods to easily track files generated throughout your tasks' execution such as:
- Numpy objects
- Pandas DataFrames
@@ -92,7 +92,7 @@ ClearML provides methods to easily track files generated throughout your experim
- Python objects
- and more!
Most importantly, ClearML also logs experiments' input and output models as well as interim model snapshots (see
Most importantly, ClearML also logs tasks' input and output models as well as interim model snapshots (see
[Models](models.md)).
#### Logging Artifacts
@@ -174,20 +174,20 @@ The above diagram demonstrates how a previously run task can be used as a baseli
## Task States
The state of a task represents its stage in the task lifecycle. It indicates whether the task is read-write (editable) or
read-only. For each state, a state transition indicates which actions can be performed on an experiment, and the new state
read-only. For each state, a state transition indicates which actions can be performed on a task, and the new state
after performing an action.
The following table describes the task states and state transitions.
| State | Description / Usage | State Transition |
|---|---|---|
| *Draft* | The experiment is editable. Only experiments in *Draft* mode are editable. The experiment is not running locally or remotely. | If the experiment is enqueued for a [worker](../fundamentals/agents_and_queues.md) to fetch and execute, the state becomes *Pending*. |
| *Pending* | The experiment was enqueued and is waiting in a queue for a worker to fetch and execute it. | If the experiment is dequeued, the state becomes *Draft*. |
| *Running* | The experiment is running locally or remotely. | If the experiment is manually or programmatically terminated, the state becomes *Aborted*. |
| *Completed* | The experiment ran and terminated successfully. | If the experiment is reset or cloned, the state of the cloned experiment or newly cloned experiment becomes *Draft*. Resetting deletes the logs and output of a previous run. Cloning creates an exact, editable copy. |
| *Failed* | The experiment ran and terminated with an error. | The same as *Completed*. |
| *Aborted* | The experiment ran, and was manually or programmatically terminated. The server's [non-responsive task monitor](../deploying_clearml/clearml_server_config.md#non-responsive-task-watchdog) aborts a task automatically after no activity has been detected for a specified time interval (configurable). | The same as *Completed*. |
| *Published* | The experiment is read-only. Publish an experiment to prevent changes to its inputs and outputs. | A *Published* experiment cannot be reset. If it is cloned, the state of the newly cloned experiment becomes *Draft*. |
| *Draft* | The task is editable. Only tasks in *Draft* mode are editable. The task is not running locally or remotely. | If the task is enqueued for a [worker](../fundamentals/agents_and_queues.md) to fetch and execute, the state becomes *Pending*. |
| *Pending* | The task was enqueued and is waiting in a queue for a worker to fetch and execute it. | If the task is dequeued, the state becomes *Draft*. |
| *Running* | The task is running locally or remotely. | If the task is manually or programmatically terminated, the state becomes *Aborted*. |
| *Completed* | The task ran and terminated successfully. | If the task is reset or cloned, the state of the cloned task or newly cloned task becomes *Draft*. Resetting deletes the logs and output of a previous run. Cloning creates an exact, editable copy. |
| *Failed* | The task ran and terminated with an error. | The same as *Completed*. |
| *Aborted* | The task ran, and was manually or programmatically terminated. The server's [non-responsive task monitor](../deploying_clearml/clearml_server_config.md#non-responsive-task-watchdog) aborts a task automatically after no activity has been detected for a specified time interval (configurable). | The same as *Completed*. |
| *Published* | The task is read-only. Publish a task to prevent changes to its inputs and outputs. | A *Published* task cannot be reset. If it is cloned, the state of the newly cloned task becomes *Draft*. |
## SDK Interface

2
docs/getting_started/main.md
View File

@@ -108,7 +108,7 @@ Want a more in depth introduction to ClearML? Choose where you want to get start
## What Can You Do with ClearML?
- [Track and upload](../fundamentals/task.md) metrics and models with only 2 lines of code
- [Reproduce](../webapp/webapp_exp_reproducing.md) experiments with 3 mouse clicks
- [Reproduce](../webapp/webapp_exp_reproducing.md) tasks with 3 mouse clicks
- [Create bots](../guides/services/slack_alerts.md) that send you Slack messages based on experiment behavior (for example,
alert you whenever your model improves in accuracy)
- Manage your [data](../clearml_data/clearml_data.md) - store, track, and version control

2
docs/getting_started/mlops/mlops_first_steps.md
View File

@@ -73,7 +73,7 @@ Cloning a task duplicates the task's configuration, but not its outputs.
The newly cloned experiment will appear and its info panel will slide open. The cloned experiment is in draft mode, so
it can be modified. You can edit the Git / code references, control the python packages to be installed, specify the
Docker container image to be used, or change the hyperparameters and configuration files. See [Modifying Experiments](../../webapp/webapp_exp_tuning.md#modifying-experiments) for more information about editing experiments in the UI.
Docker container image to be used, or change the hyperparameters and configuration files. See [Modifying Tasks](../../webapp/webapp_exp_tuning.md#modifying-experiments) for more information about editing experiments in the UI.
## Enqueue an Experiment
Once you have set up an experiment, it is now time to execute it.

14
docs/guides/advanced/execute_remotely.md
View File

@@ -28,7 +28,7 @@ moved to be executed by a stronger machine.
During the execution of the example script, the code does the following:
* Uses ClearML's automatic and explicit logging.
* Creates an experiment named `Remote_execution PyTorch MNIST train` in the `examples` project.
* Creates an task named `Remote_execution PyTorch MNIST train` in the `examples` project.
## Scalars
@@ -55,26 +55,26 @@ Logger.current_logger().report_scalar(
)
```
These scalars can be visualized in plots, which appear in the ClearML web UI, in the experiment's **SCALARS** tab.
These scalars can be visualized in plots, which appear in the ClearML web UI, in the task's **SCALARS** tab.
![Experiment Scalars](../../img/examples_pytorch_mnist_07.png)
![Task Scalars](../../img/examples_pytorch_mnist_07.png)
## Hyperparameters
ClearML automatically logs command line options defined with `argparse`. They appear in **CONFIGURATION** **>** **HYPERPARAMETERS** **>** **Args**.
![Experiment hyperparameters](../../img/examples_pytorch_mnist_01.png)
![Task hyperparameters](../../img/examples_pytorch_mnist_01.png)
## Console
Text printed to the console for training progress, as well as all other console output, appear in **CONSOLE**.
![Experiment console log](../../img/examples_pytorch_mnist_06.png)
![Task console log](../../img/examples_pytorch_mnist_06.png)
## Artifacts
Models created by the experiment appear in the experiment's **ARTIFACTS** tab. ClearML automatically logs and tracks models
Models created by the task appear in the task's **ARTIFACTS** tab. ClearML automatically logs and tracks models
and any snapshots created using PyTorch.
![Experiment artifacts](../../img/examples_remote_execution_artifacts.png)
![Task artifacts](../../img/examples_remote_execution_artifacts.png)

16
docs/guides/automation/manual_random_param_search_example.md
View File

@@ -3,32 +3,32 @@ title: Manual Random Parameter Search
---
The [manual_random_param_search_example.py](https://github.com/allegroai/clearml/blob/master/examples/automation/manual_random_param_search_example.py)
script demonstrates a random parameter search by automating the execution of an experiment multiple times, each time with
script demonstrates a random parameter search by automating the execution of a task multiple times, each time with
a different set of random hyperparameters.
This example accomplishes the automated random parameter search by doing the following:
1. Creating a template Task named `Keras HP optimization base`. To create it, run the [base_template_keras_simple.py](https://github.com/allegroai/clearml/blob/master/examples/optimization/hyper-parameter-optimization/base_template_keras_simple.py)
script. This experiment must be executed first, so it will be stored in the server, and then it can be accessed, cloned,
script. This task must be executed first, so it will be stored in the server, and then it can be accessed, cloned,
and modified by another Task.
1. Creating a parameter dictionary, which is connected to the Task by calling [`Task.connect()`](../../references/sdk/task.md#connect)
so that the parameters are logged by ClearML.
1. Adding the random search hyperparameters and parameters defining the search (e.g., the experiment name, and number of
times to run the experiment).
1. Creating a Task object referencing the template experiment, `Keras HP optimization base`. See [`Task.get_task`](../../references/sdk/task.md#taskget_task).
1. Adding the random search hyperparameters and parameters defining the search (e.g., the task name, and number of
times to run the task).
1. Creating a Task object referencing the template task, `Keras HP optimization base`. See [`Task.get_task`](../../references/sdk/task.md#taskget_task).
1. For each set of parameters:
1. Cloning the Task object. See [`Task.clone`](../../references/sdk/task.md#taskclone).
1. Getting the newly cloned Task's parameters. See [`Task.get_parameters`](../../references/sdk/task.md#get_parameters).
1. Setting the newly cloned Task's parameters to the search values in the parameter dictionary (Step 1). See [`Task.set_parameters`](../../references/sdk/task.md#set_parameters).
1. Enqueuing the newly cloned Task to execute. See [`Task.enqueue`](../../references/sdk/task.md#taskenqueue).
When the example script runs, it creates an experiment named `Random Hyper-Parameter Search Example` in
the `examples` project. This starts the parameter search, and creates the experiments:
When the example script runs, it creates a task named `Random Hyper-Parameter Search Example` in
the `examples` project. This starts the parameter search, and creates the tasks:
* `Keras HP optimization base 0`
* `Keras HP optimization base 1`
* `Keras HP optimization base 2`.
When these experiments are completed, their [results can be compared](../../webapp/webapp_exp_comparing.md).
When these tasks are completed, their [results can be compared](../../webapp/webapp_exp_comparing.md).
![Comparison parallel coordinates](../../img/examples_hpo_parallel_coordinates.png)

2
docs/guides/automation/task_piping.md
View File

@@ -22,4 +22,4 @@ This example accomplishes a task pipe by doing the following:
1. Setting the newly cloned Task's parameters to the search values in the parameter dictionary (Step 2). See [`Task.set_parameters`](../../references/sdk/task.md#set_parameters).
1. Enqueuing the newly cloned Task to execute. See [`Task.enqueue`](../../references/sdk/task.md#taskenqueue).
When the example script runs, it creates an instance of the template experiment, named `Auto generated cloned task` in the `examples` project. In the instance, the value of the customized parameter, `Example_Param` changed to `3`. You can see it in **CONFIGURATION** **>** **HYPERPARAMETERS**.
When the example script runs, it creates an instance of the template task, named `Auto generated cloned task` in the `examples` project. In the instance, the value of the customized parameter, `Example_Param` changed to `3`. You can see it in **CONFIGURATION** **>** **HYPERPARAMETERS**.

10
docs/guides/clearml-task/clearml_task_tutorial.md
View File

@@ -8,7 +8,7 @@ on a remote or local machine, from a remote repository and your local machine.
### Prerequisites
- [`clearml`](../../getting_started/ds/ds_first_steps.md) Python package installed and configured
- [`clearml-agent`](../../clearml_agent/clearml_agent_setup.md#installation) running on at least one machine (to execute the experiment), configured to listen to `default` queue
- [`clearml-agent`](../../clearml_agent/clearml_agent_setup.md#installation) running on at least one machine (to execute the task), configured to listen to `default` queue
### Executing Code from a Remote Repository
@@ -18,13 +18,13 @@ clearml-task --project keras_examples --name remote_test --repo https://github.c
This sets the following arguments:
* `--project keras_examples --name remote_test` - The project and experiment names
* `--project keras_examples --name remote_test` - The project and task names
* `--repo https://github.com/allegroai/events.git` - The repository's URL. By default, `clearml-task` uses the latest
commit from the master branch
* `--branch master` - The repository branch
* `--script /webinar-0620/keras_mnist.py` - The script to be executed
* `--args batch_size=64 epochs=1` - Arguments passed to the script. This uses the `argparse` object to get CLI parameters
* `--queue default` - Selected queue to send the experiment to
* `--queue default` - Selected queue to send the task to
:::note Adding Requirements
`clearml-task` automatically finds the requirements.txt file in remote repositories.
@@ -58,10 +58,10 @@ clearml-task --project keras --name local_test --script webinar-0620/keras_mnist
```
This sets the following arguments:
* `--project keras --name local_test` - The project and experiment names
* `--project keras --name local_test` - The project and task names
* `--script /webinar-0620/keras_mnist.py` - The local script to be executed
* `-requirements webinar-0620/requirements.txt` - The local python package requirements file
* `--args batch_size=64 epochs=1` - Arguments passed to the script. This uses the argparse object to capture CLI parameters
* `--queue default` - Selected queue to send the experiment to
* `--queue default` - Selected queue to send the task to
`clearml-task` creates a task according to the input parameters, and sends the task to the `default` queue for execution.

12
docs/guides/clearml_agent/executable_exp_containers.md
View File

@@ -1,9 +1,9 @@
---
title: Executable Experiment Containers
title: Executable Task Containers
---
This tutorial demonstrates using [`clearml-agent`](../../clearml_agent.md)'s [`build`](../../clearml_agent/clearml_agent_ref.md#build)
command to package an experiment into an executable container. In this example, you will build a Docker image that, when
command to package a task into an executable container. In this example, you will build a Docker image that, when
run, will automatically execute the [keras_tensorboard.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py)
script.
@@ -12,15 +12,15 @@ script.
* [`clearml`](../../getting_started/ds/ds_first_steps.md#install-clearml) installed and configured
* [clearml](https://github.com/allegroai/clearml) repo cloned (`git clone https://github.com/allegroai/clearml.git`)
## Creating the ClearML Experiment
1. Set up the experiment's execution environment:
## Creating the ClearML Task
1. Set up the task's execution environment:
```console
cd clearml/examples/frameworks/keras
pip install -r requirements.txt
```
1. Run the experiment:
1. Run the task:
```console
python keras_tensorboard.py
@@ -44,7 +44,7 @@ script.
If the container will not make use of a GPU, add the `--cpu-only` flag.
:::
This command will create a Docker container, set up with the execution environment for this experiment in the
This command will create a Docker container, set up with the execution environment for this task in the
specified `--target` folder. When the Docker container is launched, it will clone the task specified with `id` and
execute the clone (as designated by the `--entry-point` parameter).

20
docs/guides/clearml_agent/exp_environment_containers.md
View File

@@ -1,5 +1,5 @@
---
title: Experiment Environment Containers
title: Task Environment Containers
---
This tutorial demonstrates using [`clearml-agent`](../../clearml_agent.md)'s [`build`](../../clearml_agent/clearml_agent_ref.md#build)
@@ -14,15 +14,15 @@ be used when running optimization tasks.
* [`clearml`](../../getting_started/ds/ds_first_steps.md#install-clearml) installed and configured
* [clearml](https://github.com/allegroai/clearml) repo cloned (`git clone https://github.com/allegroai/clearml.git`)
## Creating the ClearML Experiment
1. Set up the experiment's execution environment:
## Creating the ClearML Task
1. Set up the task's execution environment:
```console
cd clearml/examples/frameworks/keras
pip install -r requirements.txt
```
1. Run the experiment:
1. Run the task:
```console
python keras_tensorboard.py
@@ -57,15 +57,15 @@ sha256:460453b93ct1989fd1c6637c236e544031c4d378581433fc0b961103ce206af1
```
## Using the New Docker Container
Make use of the container you've just built by having a ClearML agent make use of it for executing a new experiment:
Make use of the container you've just built by having a ClearML agent make use of it for executing a new task:
1. In the [ClearML Web UI](../../webapp/webapp_overview.md), go to the "examples" project, "Keras with TensorBoard
example" task (the one executed [above](#creating-the-clearml-experiment)).
1. [Clone](../../webapp/webapp_exp_reproducing.md) the experiment.
1. In the cloned experiment, go to the **EXECUTION** tab **>** **CONTAINER** section. Under **IMAGE**, insert the name
of the new Docker image, `new_docker`. See [Tuning Experiments](../../webapp/webapp_exp_tuning.md) for more task
example" task (the one executed [above](#creating-the-clearml-task)).
1. [Clone](../../webapp/webapp_exp_reproducing.md) the task.
1. In the cloned task, go to the **EXECUTION** tab **>** **CONTAINER** section. Under **IMAGE**, insert the name
of the new Docker image, `new_docker`. See [Tuning Tasks](../../webapp/webapp_exp_tuning.md) for more task
modification options.
1. Enqueue the cloned experiment to the `default` queue.
1. Enqueue the cloned task to the `default` queue.
1. Launch a `clearml-agent` in [Docker Mode](../../clearml_agent/clearml_agent_execution_env.md#docker-mode) and assign it to the `default` queue:
```console
clearml-agent daemon --docker --queue default

24
docs/guides/clearml_agent/reproduce_exp.md
View File

@@ -1,5 +1,5 @@
---
title: Recreating Experiment Environments
title: Recreating Task Environments
---
<div class="vid" >
@@ -14,21 +14,23 @@ title: Recreating Experiment Environments
<br/>
Sometimes, you may need to recreate your experiment environment on a different machine, but you haven't committed your
Sometimes, you may need to recreate your task environment on a different machine, but you haven't committed your
code.
ClearML logs everything needed to reproduce your experiment and its environment (uncommitted changes, used packages, and
more), making it easy to reproduce your experiment's execution environment using ClearML.
ClearML logs everything needed to reproduce your task and its environment (uncommitted changes, used packages, and
more), making it easy to reproduce your task's execution environment using ClearML.
You can reproduce the execution environment of any experiment you've run with ClearML on any workload:
You can reproduce the execution environment of any task you've run with ClearML on any workload:
1. Go to the task page of the task you want to reproduce in the [ClearML WebApp](../../webapp/webapp_overview.md)
1. Go to the experiment page of the task you want to reproduce in the [ClearML WebApp](../../webapp/webapp_overview.md),
:::tip
Use the UI's [filtering and sorting](../../webapp/webapp_exp_table.md#filtering-columns) to find the best performing experiments.
Use the UI's [filtering and sorting](../../webapp/webapp_exp_table.md#filtering-columns) to find the best performing tasks.
:::
1. Copy the desired experiment's ID
1. Use the ClearML Agent's [`build`](../../clearml_agent/clearml_agent_ref.md#build) command to rebuild the experiment's
execution environment. Input the experiment's ID and the target local folder, where the environment will be created:
1. Copy the task's ID
1. Use the ClearML Agent's [`build`](../../clearml_agent/clearml_agent_ref.md#build) command to rebuild the task's
execution environment. Input the task's ID and the target local folder, where the environment will be created:
```commandline
clearml-agent build --id <task_id> --target <target_folder>
@@ -38,4 +40,4 @@ You can reproduce the execution environment of any experiment you've run with Cl
2. Activate the virtual environment using the activation script. Once done, you'll find all of your environment's packages
already installed in the environment
And that's it! Your experiment's environment and your code has been reproduced!
And that's it! Your task's environment and your code has been reproduced!

12
docs/guides/distributed/distributed_pytorch_example.md
View File

@@ -17,7 +17,7 @@ dataset), and reports (uploads) the following to the main Task:
Each Task in a subprocess references the main Task by calling [`Task.current_task()`](../../references/sdk/task.md#taskcurrent_task), which always returns
the main Task.
When the script runs, it creates an experiment named `test torch distributed` in the `examples` project.
When the script runs, it creates a task named `test torch distributed` in the `examples` project.
## Artifacts
@@ -34,7 +34,7 @@ Task.current_task().upload_artifact(
All of these artifacts appear in the main Task under **ARTIFACTS** **>** **OTHER**.
![Experiment artifacts](../../img/examples_pytorch_distributed_example_09.png)
![Task artifacts](../../img/examples_pytorch_distributed_example_09.png)
## Scalars
@@ -54,7 +54,7 @@ Task.current_task().get_logger().report_scalar(
The single scalar plot for loss appears in **SCALARS**.
![Experiment scalars](../../img/examples_pytorch_distributed_example_08.png)
![Task scalars](../../img/examples_pytorch_distributed_example_08.png)
## Hyperparameters
@@ -69,12 +69,12 @@ Task.current_task().connect(param)
All the hyperparameters appear in **CONFIGURATION** **>** **HYPERPARAMETERS**.
![Experiment hyperparameters Args](../../img/examples_pytorch_distributed_example_01.png)
![Task hyperparameters Args](../../img/examples_pytorch_distributed_example_01.png)
![Experiment hyperparameters General ](../../img/examples_pytorch_distributed_example_01a.png)
![Task hyperparameters General ](../../img/examples_pytorch_distributed_example_01a.png)
## Console
Output to the console, including the text messages printed from the main Task object and each subprocess appear in **CONSOLE**.
![Experiment console log](../../img/examples_pytorch_distributed_example_06.png)
![Task console log](../../img/examples_pytorch_distributed_example_06.png)

2
docs/guides/distributed/subprocess_example.md
View File

@@ -10,7 +10,7 @@ which always returns the main Task.
* The Task in each subprocess reports the following to the main Task:
* Hyperparameters - Additional, different hyperparameters.
* Console - Text logged to the console as the Task in each subprocess executes.
* When the script runs, it creates an experiment named `Popen example` in the `examples` project.
* When the script runs, it creates a task named `Popen example` in the `examples` project.
## Hyperparameters

2
docs/guides/docker/extra_docker_shell_script.md
View File

@@ -9,7 +9,7 @@ but sometimes, when using a Docker container, a user may need to use additional,
## Tutorial
In this tutorial, you will learn how to use `extra_docker_shell_script` to reconfigure an Agent to execute
a shell script when a docker is started, but before an experiment is run.
a shell script when a docker is started, but before a task is run.
## Prerequisites
* `clearml-agent` downloaded and configured - work on a machine which has access to the configuration file of the Agent

4
docs/guides/frameworks/autokeras/autokeras_imdb_example.md
View File

@@ -9,7 +9,7 @@ The example does the following:
the autokeras [TextClassifier](https://autokeras.com/text_classifier/) class, and searches for the best model.
* Uses two TensorBoard callbacks, one for training and one for testing.
* ClearML automatically logs everything the code sends to TensorBoard.
* Creates an experiment named `autokeras imdb example with scalars` in the `autokeras` project.
* Creates a task named `autokeras imdb example with scalars` in the `autokeras` project.
## Scalars
@@ -32,7 +32,7 @@ Text printed to the console for training progress, as well as all other console
## Artifacts
Models created by the experiment appear in the experiment's **ARTIFACTS** tab.
Models created by the task appear in the task's **ARTIFACTS** tab.
![image](../../../img/examples_keras_18.png)

14
docs/guides/frameworks/catboost/catboost.md
View File

@@ -8,30 +8,30 @@ example demonstrates the integration of ClearML into code that uses [CatBoost](h
The example script does the following:
* Trains a simple deep neural network on the CatBoost built-in [msrank](https://catboost.ai/en/docs/concepts/python-reference_datasets_msrank)
dataset.
* Creates an experiment named `CatBoost simple example` in the `examples` project
* Creates a task named `CatBoost simple example` in the `examples` project
* ClearML automatically logs argparse command line options, and models and metrics logged by CatBoost
## Scalars
ClearML automatically captures scalars logged by CatBoost. These scalars can be visualized in plots, which appear in the
[ClearML web UI](../../../webapp/webapp_overview.md), in the experiment's page **> SCALARS**.
[ClearML web UI](../../../webapp/webapp_overview.md), in the task's **SCALARS** tab.
![Experiment scalars](../../../img/examples_catboost_scalars.png)
![Task scalars](../../../img/examples_catboost_scalars.png)
## Hyperparameters
ClearML automatically logs command line options defined with argparse. They appear in **CONFIGURATIONS > HYPERPARAMETERS > Args**.
![Experiment hyperparameters](../../../img/examples_catboost_configurations.png)
![Task hyperparameters](../../../img/examples_catboost_configurations.png)
## Console
Text printed to the console for training progress, as well as all other console output, appear in **CONSOLE**.
![Experiment console](../../../img/examples_catboost_console.png)
![Task console](../../../img/examples_catboost_console.png)
## Artifacts
Models created by the experiment appear in the experiment's **ARTIFACTS** tab. ClearML automatically logs and tracks
Models created by the task appear in the task's **ARTIFACTS** tab. ClearML automatically logs and tracks
models created using CatBoost.
![Experiment artifacts](../../../img/examples_catboost_artifacts.png)
![Task artifacts](../../../img/examples_catboost_artifacts.png)
Clicking on the model name takes you to the [model's page](../../../webapp/webapp_model_viewing.md), where you can view
the model's details and access the model.

2
docs/guides/frameworks/fastai/fastai_with_tensorboard.md
View File

@@ -12,7 +12,7 @@ The ClearML repository also includes [examples using FastAI v2](https://github.c
The example code does the following:
1. Trains a simple deep neural network on the fastai built-in MNIST dataset (see the [fast.ai](https://fastai1.fast.ai) documentation).
1. Uses the fastai `LearnerTensorboardWriter` callback, and ClearML automatically logs TensorBoard through the callback.
1. During script execution, creates an experiment named `fastai with tensorboard callback` in the `examples` project.
1. During script execution, creates a task named `fastai with tensorboard callback` in the `examples` project.
## Scalars

6
docs/guides/frameworks/huggingface/transformers.md
View File

@@ -34,7 +34,7 @@ training_args = TrainingArguments(
ClearML captures all of the `TrainingArguments` passed to the Trainer.
View these parameters in the experiment's **CONFIGURATION** tab **> Hyperparameters** section.
View these parameters in the task's **CONFIGURATION** tab **> Hyperparameters** section.
![Transformers params](../../../img/examples_transformers_params.png)
@@ -45,12 +45,12 @@ In order for ClearML to log the models created during training in this example,
variable is set to `True`.
ClearML automatically captures the model snapshots created by the Trainer, and saves them as artifacts. View the snapshots in the
experiment's **ARTIFACTS** tab.
task's **ARTIFACTS** tab.
![Transformers models](../../../img/examples_transformers_artifacts.png)
### Scalars
ClearML automatically captures the Trainer's scalars, which can be viewed in the experiment's **Scalars** tab.
ClearML automatically captures the Trainer's scalars, which can be viewed in the task's **Scalars** tab.
![Transformers scalars](../../../img/integrations_transformers_scalars.png)

6
docs/guides/frameworks/keras/jupyter.md
View File

@@ -12,7 +12,7 @@ The example does the following:
1. Specifies accuracy as the metric, and uses two callbacks: a TensorBoard callback and a model checkpoint callback.
1. During script execution, creates an experiment named `notebook example` in the `examples` project.
1. During script execution, creates a task named `notebook example` in the `examples` project.
## Scalars
@@ -69,10 +69,10 @@ Text printed to the console for training appears in **CONSOLE**.
## Artifacts
Models created by the experiment appear in the experiment's **ARTIFACTS** tab. ClearML automatically logs and tracks models
Models created by the task appear in the task's **ARTIFACTS** tab. ClearML automatically logs and tracks models
created using Keras.
The experiment info panel shows model tracking, including the model name and design in **ARTIFACTS** **>** **Output Model**.
The task info panel shows model tracking, including the model name and design in **ARTIFACTS** **>** **Output Model**.
![image](../../../img/examples_keras_jupyter_23.png)

4
docs/guides/frameworks/keras/keras_tensorboard.md
View File

@@ -16,7 +16,7 @@ The example script does the following:
dataset.
1. Builds a sequential model using a categorical cross entropy loss objective function.
1. Specifies accuracy as the metric, and uses two callbacks: a TensorBoard callback and a model checkpoint callback.
1. During script execution, creates an experiment named `Keras with TensorBoard example` in the
1. During script execution, creates a task named `Keras with TensorBoard example` in the
`examples` project (in script) or the `Colab notebooks` project (in Jupyter Notebook).
@@ -53,7 +53,7 @@ Text printed to the console for training progress, as well as all other console
## Configuration Objects
In the experiment code, a configuration dictionary is connected to the Task by calling [`Task.connect()`](../../../references/sdk/task.md#connect).
A configuration dictionary is connected to the Task by calling [`Task.connect()`](../../../references/sdk/task.md#connect).
```python
task.connect_configuration(

6
docs/guides/frameworks/lightgbm/lightgbm_example.md
View File

@@ -9,11 +9,11 @@ The example script does the following:
* Creates a dataset for LightGBM to train a model
* Specifies configuration which are automatically captured by ClearML
* Saves model which ClearML automatically captures
* Creates an experiment named `LightGBM` in the `examples` project.
* Creates a task named `LightGBM` in the `examples` project.
## Scalars
The scalars logged in the experiment can be visualized in a plot, which appears in the ClearML web UI, in the experiment's **SCALARS** tab.
The scalars logged in the task can be visualized in a plot, which appears in the ClearML web UI, in the task's **SCALARS** tab.
![LightGBM scalars](../../../img/examples_lightgbm_scalars.png)
@@ -25,7 +25,7 @@ ClearML automatically logs the configurations applied to LightGBM. They appear i
## Artifacts
Models created by the experiment appear in the experiment's **ARTIFACTS** tab. ClearML automatically logs and tracks
Models created by the task appear in the task's **ARTIFACTS** tab. ClearML automatically logs and tracks
models and any snapshots created using LightGBM.
![LightGBM model](../../../img/examples_lightgbm_model.png)

2
docs/guides/frameworks/matplotlib/matplotlib_example.md
View File

@@ -13,7 +13,7 @@ The example in [Jupyter Notebook](https://github.com/allegroai/clearml/blob/mast
includes a clickable icon to open the notebook in Google Colab.
:::
When the example runs, it creates an experiment named `Matplotlib example`
When the example runs, it creates a task named `Matplotlib example`
in the `examples` project (in script) or the `Colab notebooks` project (in Jupyter Notebook).

10
docs/guides/frameworks/megengine/megengine_mnist.md
View File

@@ -10,11 +10,11 @@ The example script does the following:
* Trains a simple deep neural network on MegEngine's built-in [MNIST](https://www.megengine.org.cn/doc/master/en/reference/api/megengine.data.dataset.MNIST.html)
dataset.
* Creates a TensorBoardX `SummaryWriter` object to log scalars during training.
* Creates a ClearML experiment named `megengine mnist train` in the `examples` project.
* Creates a ClearML task named `megengine mnist train` in the `examples` project.
## Hyperparameters
ClearML automatically logs command line options defined with `argparse`. They appear in the experiment's **CONFIGURATION**
ClearML automatically logs command line options defined with `argparse`. They appear in the task's **CONFIGURATION**
tab under **HYPERPARAMETERS** **>** **Args**.
![Configuration tab](../../../img/examples_megengine_mnist_config.png)
@@ -25,7 +25,7 @@ The example script's `train` function calls TensorBoardX's `SummaryWriter.add_sc
ClearML automatically captures the data that is added to the `SummaryWriter` object.
These scalars can be visualized in plots, which appear in the ClearML [WebApp](../../../webapp/webapp_home.md), in the
experiment's **SCALARS** tab.
task's **SCALARS** tab.
![Scalars tab](../../../img/examples_megengine_mnist_scalars.png)
@@ -34,7 +34,7 @@ experiment's **SCALARS** tab.
ClearML automatically captures the model logged using the `megengine.save` method, and saves it as an artifact.
View saved snapshots in the experiment's **ARTIFACTS** tab.
View saved snapshots in the task's **ARTIFACTS** tab.
![Artifacts tab](../../../img/examples_megengine_models_1.png)
@@ -49,7 +49,7 @@ The model info panel contains the model details, including:
## Console
All console output during the script's execution appears in the experiment's **CONSOLE** page.
All console output during the script's execution appears in the task's **CONSOLE** page.
![Console tab](../../../img/examples_megengine_console.png)

2
docs/guides/frameworks/pytorch/notebooks/audio/audio_classification_UrbanSound8K.md
View File

@@ -2,7 +2,7 @@
title: Audio Classification - Jupyter Notebooks
---
The [audio_classification_UrbanSound8K.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/audio/audio_classifier_UrbanSound8K.ipynb) example script demonstrates integrating ClearML into a Jupyter Notebook which uses PyTorch, TensorBoard, and TorchVision to train a neural network on the UrbanSound8K dataset for audio classification. The example calls TensorBoard methods in training and testing to report scalars, audio debug samples, and spectrogram visualizations. The spectrogram visualizations are plotted by calling Matplotlib methods. The example also demonstrates connecting parameters to a Task and logging them. When the script runs, it creates an experiment named `audio classification UrbanSound8K` in the `Audio Example` project.
The [audio_classification_UrbanSound8K.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/audio/audio_classifier_UrbanSound8K.ipynb) example script demonstrates integrating ClearML into a Jupyter Notebook which uses PyTorch, TensorBoard, and TorchVision to train a neural network on the UrbanSound8K dataset for audio classification. The example calls TensorBoard methods in training and testing to report scalars, audio debug samples, and spectrogram visualizations. The spectrogram visualizations are plotted by calling Matplotlib methods. The example also demonstrates connecting parameters to a Task and logging them. When the script runs, it creates a task named `audio classification UrbanSound8K` in the `Audio Example` project.
## Scalars

2
docs/guides/frameworks/pytorch/notebooks/audio/audio_preprocessing_example.md
View File

@@ -3,7 +3,7 @@ title: Audio Preprocessing - Jupyter Notebook
---
The example [audio_preprocessing_example.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/audio/audio_preprocessing_example.ipynb)
demonstrates integrating ClearML into a Jupyter Notebook which uses PyTorch and preprocesses audio samples. ClearML automatically logs spectrogram visualizations reported by calling Matplotlib methods, and audio samples reported by calling TensorBoard methods. The example also demonstrates connecting parameters to a Task and logging them. When the script runs, it creates an experiment named `data pre-processing` in the `Audio Example` project.
demonstrates integrating ClearML into a Jupyter Notebook which uses PyTorch and preprocesses audio samples. ClearML automatically logs spectrogram visualizations reported by calling Matplotlib methods, and audio samples reported by calling TensorBoard methods. The example also demonstrates connecting parameters to a Task and logging them. When the script runs, it creates a task named `data pre-processing` in the `Audio Example` project.
## Plots

38
docs/guides/frameworks/pytorch/notebooks/image/hyperparameter_search.md
View File

@@ -9,13 +9,13 @@ class to perform automated hyperparameter optimization (HPO).
The code creates a HyperParameterOptimizer object, which is a search controller. The search controller uses the
[Optuna](../../../../../references/sdk/hpo_optuna_optuna_optimizeroptuna.md) search strategy optimizer.
The example maximizes total accuracy by finding an optimal number of epochs, batch size, base learning rate, and dropout. ClearML
automatically logs the optimization's top performing experiments.
automatically logs the optimization's top performing tasks.
The experiment whose hyperparameters are optimized is named `image_classification_CIFAR10`. It is created by running another
The task whose hyperparameters are optimized is named `image_classification_CIFAR10`. It is created by running another
ClearML example, [image_classification_CIFAR10.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/image/image_classification_CIFAR10.ipynb),
which must run before `hyperparameter_search.ipynb`.
The optimizer Task, `Hyperparameter Optimization`, and the experiments appear individually in the [ClearML Web UI](../../../../../webapp/webapp_overview.md).
The optimizer Task, `Hyperparameter Optimization`, and the tasks appear individually in the [ClearML Web UI](../../../../../webapp/webapp_overview.md).
## Optimizer Task
@@ -26,7 +26,7 @@ optimizer task's **CONFIGURATION > HYPERPARAMETERS > General** section.
```python
optimizer = HyperParameterOptimizer(
base_task_id=TEMPLATE_TASK_ID, # This is the experiment we want to optimize
base_task_id=TEMPLATE_TASK_ID, # This is the task we want to optimize
# here we define the hyperparameters to optimize
hyper_parameters=[
UniformIntegerParameterRange('number_of_epochs', min_value=2, max_value=12, step_size=2),
@@ -43,33 +43,33 @@ optimizer = HyperParameterOptimizer(
optimizer_class=OptimizerOptuna,
# Configuring optimization parameters
execution_queue='dan_queue', # queue to schedule the experiments for execution
max_number_of_concurrent_tasks=2, # number of concurrent experiments
execution_queue='dan_queue', # queue to schedule the tasks for execution
max_number_of_concurrent_tasks=2, # number of concurrent tasks
optimization_time_limit=60, # set the time limit for the optimization process
compute_time_limit=120, # set the compute time limit (sum of execution time on all machines)
total_max_jobs=20, # set the maximum number of experiments for the optimization.
total_max_jobs=20, # set the maximum number of tasks for the optimization.
# Converted to total number of iteration for OptimizerBOHB
min_iteration_per_job=15000, # minimum number of iterations per experiment, till early stopping
max_iteration_per_job=150000, # maximum number of iterations per experiment
min_iteration_per_job=15000, # minimum number of iterations per task, till early stopping
max_iteration_per_job=150000, # maximum number of iterations per task
)
```
![Experiment hyperparameters](../../../../../img/examples_hyperparameter_search_01.png)
![Task hyperparameters](../../../../../img/examples_hyperparameter_search_01.png)
### Console
All console output appears in the optimizer task's **CONSOLE**.
![Experiment console](../../../../../img/examples_hyperparameter_search_03.png)
![Task console](../../../../../img/examples_hyperparameter_search_03.png)
### Scalars
Scalar metrics for total accuracy and remaining budget by iteration, and a plot of total accuracy by iteration appear in the
experiment's **SCALARS** tab. Remaining budget indicates the percentage of total iterations for all jobs left before that total is reached.
task's **SCALARS** tab. Remaining budget indicates the percentage of total iterations for all jobs left before that total is reached.
ClearML automatically reports the scalars generated by `HyperParameterOptimizer`.
![Experiment Scalars](../../../../../img/examples_hyperparameter_search_04.png)
![Task Scalars](../../../../../img/examples_hyperparameter_search_04.png)
### Plots
@@ -77,17 +77,17 @@ The optimization task automatically records and monitors the different trial tas
provides a summary of the optimization results in tabular and parallel coordinate formats. View these plots in the task's
**PLOTS**.
![Experiment scatter plot](../../../../../img/examples_hyperparameter_search_05.png)
![Task scatter plot](../../../../../img/examples_hyperparameter_search_05.png)
![Experiment parallel coordinates](../../../../../img/examples_hyperparameter_search_02a.png)
![Task parallel coordinates](../../../../../img/examples_hyperparameter_search_02a.png)
![Experiment summary plot](../../../../../img/examples_hyperparameter_search_02b.png)
![Task summary plot](../../../../../img/examples_hyperparameter_search_02b.png)
## Experiments Comparison
## Task Comparison
ClearML automatically logs each job, meaning each experiment that executes with a set of hyperparameters, separately. Each appears as an individual experiment in the ClearML Web UI, where the Task name is `image_classification_CIFAR10` and the hyperparameters appended.
ClearML automatically logs each job, meaning each task that executes with a set of hyperparameters, separately. Each appears as an individual task in the ClearML Web UI, where the Task name is `image_classification_CIFAR10` and the hyperparameters appended.
For example: `image_classification_CIFAR10: base_lr=0.0075 batch_size=12 dropout=0.05 number_of_epochs=6`.
Compare the experiments' configuration, results, plots, debug samples, and more, using the WebApp [comparison features](../../../../../webapp/webapp_exp_comparing.md).
Compare the tasks' configuration, results, plots, debug samples, and more, using the WebApp [comparison features](../../../../../webapp/webapp_exp_comparing.md).
![Comparison scalars](../../../../../img/examples_hyperparameter_search_09.png)

2
docs/guides/frameworks/pytorch/notebooks/image/image_classification_CIFAR10.md
View File

@@ -7,7 +7,7 @@ demonstrates integrating ClearML into a Jupyter Notebook, which uses PyTorch, Te
neural network on the CIFAR10 dataset for image classification. ClearML automatically logs the example script's
calls to TensorBoard methods in training and testing which report scalars and image debug samples, as well as the model
and console log. The example also demonstrates connecting parameters to a Task and logging them. When the script runs,
it creates an experiment named `image_classification_CIFAR10` in the `Image Example` project.
it creates a task named `image_classification_CIFAR10` in the `Image Example` project.
Another example optimizes the hyperparameters for this image classification example (see the [Hyperparameter Optimization - Jupyter Notebook](hyperparameter_search.md) documentation page). This image classification example must run before the hyperparameter optimization example.

2
docs/guides/frameworks/pytorch/notebooks/table/download_and_preprocessing.md
View File

@@ -2,7 +2,7 @@
title: Tabular Data Downloading and Preprocessing - Jupyter Notebook
---
The [download_and_preprocessing.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/download_and_preprocessing.ipynb) example demonstrates ClearML storing preprocessed tabular data as artifacts, and explicitly reporting the tabular data in the **ClearML Web UI**. When the script runs, it creates an experiment named `tabular preprocessing` in the `Table Example` project.
The [download_and_preprocessing.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/download_and_preprocessing.ipynb) example demonstrates ClearML storing preprocessed tabular data as artifacts, and explicitly reporting the tabular data in the **ClearML Web UI**. When the script runs, it creates a task named `tabular preprocessing` in the `Table Example` project.
This tabular data is prepared for another script, [train_tabular_predictor.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/train_tabular_predictor.ipynb), which trains a network with it.

6
docs/guides/frameworks/pytorch/notebooks/text/text_classification_AG_NEWS.md
View File

@@ -6,7 +6,7 @@ The example [text_classification_AG_NEWS.ipynb](https://github.com/allegroai/cle
demonstrates using Jupyter Notebook for ClearML, and the integration of ClearML into code which trains a network
to classify text in the `torchtext` [AG_NEWS](https://pytorch.org/text/stable/datasets.html#ag-news) dataset, and then applies the model to predict the classification of sample text.
ClearML automatically logs the scalars and text samples reported with TensorBoard methods. The example code explicitly logs parameters to the Task. When the script runs, it creates an experiment named `text classifier` in the `Text Example` project.
ClearML automatically logs the scalars and text samples reported with TensorBoard methods. The example code explicitly logs parameters to the Task. When the script runs, it creates a task named `text classifier` in the `Text Example` project.
## Scalars
@@ -16,7 +16,7 @@ Accuracy, learning rate, and training loss appear in **SCALARS**, along with the
## Debug Samples
ClearML automatically logs the text samples reported to TensorBoard. They are displayed in the experiment's **DEBUG SAMPLES**.
ClearML automatically logs the text samples reported to TensorBoard. They are displayed in the task's **DEBUG SAMPLES**.
![Debug samples](../../../../../img/text_classification_AG_NEWS_04.png)
@@ -32,7 +32,7 @@ configuration_dict = {
configuration_dict = task.connect(configuration_dict)
```
The parameters are displayed in the experiment's **CONFIGURATION** **>** **HYPERPARAMETERS** **>** **General** section.
The parameters are displayed in the task's **CONFIGURATION** **>** **HYPERPARAMETERS** **>** **General** section.
![Hyperparameters](../../../../../img/text_classification_AG_NEWS_01.png)

6
docs/guides/frameworks/pytorch/pytorch_abseil.md
View File

@@ -8,7 +8,7 @@ example demonstrates the integration of ClearML into code that uses PyTorch and
The example script does the following:
* Trains a simple deep neural network on the PyTorch built-in [MNIST](https://pytorch.org/vision/stable/datasets.html#mnist)
dataset
* Creates an experiment named `pytorch mnist train with abseil` in the `examples` project
* Creates a task named `pytorch mnist train with abseil` in the `examples` project
* ClearML automatically logs the absl.flags, and the models (and their snapshots) created by PyTorch
* Additional metrics are logged by calling [`Logger.report_scalar()`](../../../references/sdk/logger.md#report_scalar)
@@ -40,7 +40,7 @@ Logger.current_logger().report_scalar(
```
These scalars can be visualized in plots, which appear in the [ClearML web UI](../../../webapp/webapp_overview.md), in
the experiment's **SCALARS** tab.
the task's **SCALARS** tab.
![image](../../../img/examples_pytorch_mnist_07.png)
@@ -59,7 +59,7 @@ Text printed to the console for training progress, as well as all other console
## Artifacts
Models created by the experiment appear in the experiment's **ARTIFACTS** tab. ClearML automatically logs and tracks
Models created by the task appear in the task's **ARTIFACTS** tab. ClearML automatically logs and tracks
models and any snapshots created using PyTorch.
![image](../../../img/examples_pytorch_abseil_models.png)

2
docs/guides/frameworks/pytorch/pytorch_distributed_example.md
View File

@@ -19,7 +19,7 @@ The script does the following:
Each Task in a subprocess references the main Task by calling [`Task.current_task()`](../../../references/sdk/task.md#taskcurrent_task),
which always returns the main Task.
1. When the script runs, it creates an experiment named `test torch distributed` in the `examples` project in the **ClearML Web UI**.
1. When the script runs, it creates a task named `test torch distributed` in the `examples` project in the **ClearML Web UI**.
### Artifacts

2
docs/guides/frameworks/pytorch/pytorch_matplotlib.md
View File

@@ -6,7 +6,7 @@ The [pytorch_matplotlib.py](https://github.com/allegroai/clearml/blob/master/exa
example demonstrates the integration of ClearML into code that uses PyTorch and Matplotlib.
The example does the following:
* Creates an experiment named `pytorch with matplotlib example`, in the `examples` project.
* Creates a task named `pytorch with matplotlib example`, in the `examples` project.
* The script calls Matplotlib methods to show images, each with a different title.
* ClearML automatically logs the images as debug samples.

6
docs/guides/frameworks/pytorch/pytorch_mnist.md
View File

@@ -8,7 +8,7 @@ demonstrates the integration of ClearML into code that uses PyTorch.
The example script does the following:
* Trains a simple deep neural network on the PyTorch built-in [MNIST](https://pytorch.org/vision/stable/datasets.html#mnist)
dataset.
* Creates an experiment named `pytorch mnist train` in the `examples` project.
* Creates a task named `pytorch mnist train` in the `examples` project.
* ClearML automatically logs `argparse` command line options, and models (and their snapshots) created by PyTorch.
* Additional metrics are logged by calling [`Logger.report_scalar()`](../../../references/sdk/logger.md#report_scalar).
@@ -34,7 +34,7 @@ Logger.current_logger().report_scalar(
```
These scalars can be visualized in plots, which appear in the ClearML [web UI](../../../webapp/webapp_overview.md),
in the experiment's page **>** **SCALARS**.
in the task's **SCALARS** tab.
![image](../../../img/examples_pytorch_mnist_07.png)
@@ -52,7 +52,7 @@ Text printed to the console for training progress, as well as all other console
## Artifacts
Models created by the experiment appear in the experiment's **ARTIFACTS** tab. ClearML automatically logs and tracks models
Models created by the task appear in the task's **ARTIFACTS** tab. ClearML automatically logs and tracks models
and any snapshots created using PyTorch.
![image](../../../img/examples_pytorch_mnist_02.png)

6
docs/guides/frameworks/pytorch/pytorch_tensorboard.md
View File

@@ -8,14 +8,14 @@ example demonstrates the integration of ClearML into code that uses PyTorch and
The example does the following:
* Trains a simple deep neural network on the PyTorch built-in [MNIST](https://pytorch.org/vision/stable/datasets.html#mnist)
dataset.
* Creates an experiment named `pytorch with tensorboard` in the `examples` project.
* Creates a task named `pytorch with tensorboard` in the `examples` project.
* ClearML automatically captures scalars and text logged using the TensorBoard `SummaryWriter` object, and
the model created by PyTorch.
## Scalars
In the example script, the `train` and `test` functions call the TensorBoard `SummaryWriter.add_scalar` method to log loss.
These scalars, along with the resource utilization plots, which are titled **:monitor: machine**, appear in the experiment's
These scalars, along with the resource utilization plots, which are titled **:monitor: machine**, appear in the task's
page in the [ClearML web UI](../../../webapp/webapp_overview.md) under **SCALARS**.
![image](../../../img/examples_pytorch_tensorboard_07.png)
@@ -40,7 +40,7 @@ Text printed to the console for training progress, as well as all other console
## Artifacts
Models created by the experiment appear in the experiment's **ARTIFACTS** tab. ClearML automatically logs and tracks
Models created by the task appear in the task's **ARTIFACTS** tab. ClearML automatically logs and tracks
models and any snapshots created using PyTorch.
![image](../../../img/examples_pytorch_tensorboard_02.png)

6
docs/guides/frameworks/pytorch/pytorch_tensorboardx.md
View File

@@ -8,14 +8,14 @@ example demonstrates the integration of ClearML into code that uses PyTorch and
The example does the following:
* Trains a simple deep neural network on the PyTorch built-in [MNIST](https://pytorch.org/vision/stable/datasets.html#mnist)
dataset.
* Creates an experiment named `pytorch with tensorboardX` in the `examples` project.
* Creates a task named `pytorch with tensorboardX` in the `examples` project.
* ClearML automatically captures scalars and text logged using the TensorBoardX `SummaryWriter` object, and
the model created by PyTorch.
## Scalars
The loss and accuracy metric scalar plots, along with the resource utilization plots, which are titled **:monitor: machine**,
appear in the experiment's page in the [web UI](../../../webapp/webapp_overview.md), under **SCALARS**.
appear in the task's page in the [web UI](../../../webapp/webapp_overview.md), under **SCALARS**.
![image](../../../img/examples_pytorch_tensorboardx_03.png)
@@ -35,7 +35,7 @@ Text printed to the console for training progress, as well as all other console
## Artifacts
Models created by the experiment appear in the experiment's **ARTIFACTS** tab. ClearML automatically logs and tracks
Models created by the task appear in the task's **ARTIFACTS** tab. ClearML automatically logs and tracks
models and any snapshots created using PyTorch.
![image](../../../img/examples_pytorch_tensorboardx_04.png)

4
docs/guides/frameworks/pytorch/tensorboard_toy_pytorch.md
View File

@@ -4,12 +4,12 @@ title: PyTorch TensorBoard Toy
The [tensorboard_toy_pytorch.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/tensorboard_toy_pytorch.py)
example demonstrates the integration of ClearML into code, which creates a TensorBoard `SummaryWriter` object to log
debug sample images. When the script runs, it creates an experiment named `pytorch tensorboard toy example`, which is
debug sample images. When the script runs, it creates a task named `pytorch tensorboard toy example`, which is
associated with the `examples` project.
## Debug Samples
The debug sample images appear according to metric, in the experiment's **DEBUG SAMPLES** tab.
The debug sample images appear according to metric, in the task's **DEBUG SAMPLES** tab.
![image](../../../img/examples_tensorboard_toy_pytorch_02.png)

14
docs/guides/frameworks/pytorch_ignite/integration_pytorch_ignite.md
View File

@@ -9,7 +9,7 @@ The example script does the following:
* Trains a neural network on the CIFAR10 dataset for image classification.
* Creates a [ClearML Task](../../../fundamentals/task.md) named `image classification CIFAR10` in
the `examples` project.
* Calls the [`Task.connect`](../../../references/sdk/task.md#connect) method to track experiment configuration.
* Calls the [`Task.connect`](../../../references/sdk/task.md#connect) method to track task configuration.
* Uses `ignite`'s `TensorboardLogger` and attaches handlers to it. See [`TensorboardLogger`](https://github.com/pytorch/ignite/blob/master/ignite/contrib/handlers/tensorboard_logger.py).
ClearML's automatic logging captures information and outputs logged with `TensorboardLogger`.
@@ -22,7 +22,7 @@ Parameters are explicitly reported to ClearML using the [`Task.connect`](../../.
params = {'number_of_epochs': 20, 'batch_size': 64, 'dropout': 0.25, 'base_lr': 0.001, 'momentum': 0.9, 'loss_report': 100}
params = task.connect(params) # enabling configuration override by clearml
```
The hyperparameter configurations can be viewed in the WebApp in the experiment's **CONFIGURATION** tab.
The hyperparameter configurations can be viewed in the WebApp in the task's **CONFIGURATION** tab.
![image](../../../img/examples_integration_pytorch_ignite_config.png)
@@ -35,22 +35,22 @@ into a script which uses `TensorboardLogger`, all information logged through the
ClearML automatically captures scalars logged through `TensorboardLogger`.
View the scalars in the experiment's page in the **ClearML Web UI**, in **SCALARS**.
View the scalars in the task's page in the **ClearML Web UI**, in **SCALARS**.
![image](../../../img/examples_cifar_scalars.png)
![Task scalars](../../../img/examples_cifar_scalars.png)
## Model Snapshots
ClearML automatically captures the model logged with Torch, and saves it as an artifact.
View saved snapshots in the experiment's **ARTIFACTS** tab.
View saved snapshots in the task's **ARTIFACTS** tab.
![image](../../../img/examples_cifar_artifacts.png)
![Task models](../../../img/examples_cifar_artifacts.png)
To view the model, in the **ARTIFACTS** tab, click the model name (or download it).
![image](../../../img/examples_cifar_model.png)
![Model details](../../../img/examples_cifar_model.png)
## Debug Samples

12
docs/guides/frameworks/pytorch_ignite/pytorch_ignite_mnist.md
View File

@@ -45,9 +45,9 @@ Integrate ClearML with the following steps:
### Parameters
The following are the `ClearMLLogger` parameters:
* `project_name` - The name of the project in which the experiment will be created.
* `project_name` - The name of the project in which the task will be created.
* `task_name` – The name of task.
* `task_type` – The type of experiment (see [task types](../../../fundamentals/task.md#task-types)).
* `task_type` – The type of task (see [task types](../../../fundamentals/task.md#task-types)).
* `report_freq` – The histogram processing frequency (handles histogram values every X calls to the handler). Affects
`GradsHistHandler` and `WeightsHistHandler`. Default value is 100.
* `histogram_update_freq_multiplier` – The histogram report frequency (report first X histograms and once every X
@@ -152,16 +152,16 @@ validation_evaluator.add_event_handler(Events.EPOCH_COMPLETED, handler)
```
## Visualizing Experiment Results
## Visualizing Task Results
When the code runs, the experiment results can be viewed in the [ClearML Web UI](../../../webapp/webapp_overview.md).
When the code runs, the task results can be viewed in the [ClearML Web UI](../../../webapp/webapp_overview.md).
### Scalars
View the scalars, including training and validation metrics, in the experiment's page in the ClearML Web UI, under
View the scalars, including training and validation metrics, in the task's page in the ClearML Web UI, under
**SCALARS**.
![image](../../../img/ignite_training.png)
![Task scalars](../../../img/ignite_training.png)
### Model Snapshots

6
docs/guides/frameworks/pytorch_lightning/pytorch_lightning_example.md
View File

@@ -8,11 +8,11 @@ script demonstrates the integration of ClearML into code that uses [PyTorch Ligh
The example script does the following:
* Trains a simple deep neural network on the PyTorch built-in MNIST dataset
* Defines Argparse command line options, which are automatically captured by ClearML
* Creates an experiment named `pytorch lightning mnist example` in the `examples` project.
* Creates a task named `pytorch lightning mnist example` in the `examples` project.
## Scalars
The test loss and validation loss plots appear in the experiment's page in the ClearML web UI under **SCALARS**.
The test loss and validation loss plots appear in the task's page in the ClearML web UI under **SCALARS**.
Resource utilization plots, which are titled **:monitor: machine**, also appear in the **SCALARS** tab. All of these
plots are automatically captured by ClearML.
@@ -28,7 +28,7 @@ ClearML automatically logs command line options defined with argparse and Tensor
## Artifacts
Models created by the experiment appear in the experiment's **ARTIFACTS** tab.
Models created by the task appear in the task's **ARTIFACTS** tab.
![PyTorch Lightning model](../../../img/examples_pytorch_lightning_model.png)

6
docs/guides/frameworks/scikit-learn/sklearn_joblib_example.md
View File

@@ -4,19 +4,19 @@ title: scikit-learn with Joblib
The [sklearn_joblib_example.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/scikit-learn/sklearn_joblib_example.py)
demonstrates the integration of ClearML into code that uses `scikit-learn` and `joblib` to store a model and model snapshots,
and `matplotlib` to create a scatter diagram. When the script runs, it creates an experiment named
and `matplotlib` to create a scatter diagram. When the script runs, it creates a task named
`scikit-learn joblib example` in the `examples` project.
## Plots
ClearML automatically logs the scatter plot, which appears in the [experiment's page](../../../webapp/webapp_exp_track_visual.md)
ClearML automatically logs the scatter plot, which appears in the [task's page](../../../webapp/webapp_exp_track_visual.md)
in the ClearML web UI, under **PLOTS**.
![image](../../../img/examples_sklearn_joblib_example_06.png)
## Artifacts
Models created by the experiment appear in the experiment's **ARTIFACTS** tab.
Models created by the task appear in the task's **ARTIFACTS** tab.
![image](../../../img/examples_sklearn_joblib_example_01.png)

2
docs/guides/frameworks/scikit-learn/sklearn_matplotlib_example.md
View File

@@ -9,7 +9,7 @@ The example does the following:
* Uses `scikit-learn` to determine cross-validated training and test scores.
* Uses `matplotlib` to plot the learning curves.
* ClearML automatically logs the scatter diagrams for the learning curves.
* Creates an experiment named `scikit-learn matplotlib example` in the `examples` project.
* Creates a task named `scikit-learn matplotlib example` in the `examples` project.
## Plots

6
docs/guides/frameworks/tensorboardx/tensorboardx.md
View File

@@ -7,13 +7,13 @@ example demonstrates the integration of ClearML into code that uses PyTorch and
The script does the following:
* Trains a simple deep neural network on the PyTorch built-in [MNIST](https://pytorch.org/vision/stable/datasets.html#mnist) dataset
* Creates an experiment named `pytorch with tensorboardX` in the `examples` project
* Creates a task named `pytorch with tensorboardX` in the `examples` project
* ClearML automatically captures scalars and text logged using the TensorBoardX `SummaryWriter` object, and
the model created by PyTorch
## Scalars
The loss and accuracy metric scalar plots appear in the experiment's page in the **ClearML web UI**, under
The loss and accuracy metric scalar plots appear in the task's page in the **ClearML web UI**, under
**SCALARS**. The also includes resource utilization plots, which are titled **:monitor: machine**.
![image](../../../img/examples_pytorch_tensorboardx_03.png)
@@ -33,7 +33,7 @@ Text printed to the console for training progress, as well as all other console
## Artifacts
Models created by the experiment appear in the experiment's **ARTIFACTS** tab. ClearML automatically logs and tracks
Models created by the task appear in the task's **ARTIFACTS** tab. ClearML automatically logs and tracks
models and any snapshots created using PyTorch.
![image](../../../img/examples_pytorch_tensorboardx_04.png)

4
docs/guides/frameworks/tensorboardx/video_tensorboardx.md
View File

@@ -6,13 +6,13 @@ The [moveiepy_tensorboardx.py](https://github.com/allegroai/clearml/blob/master/
example demonstrates the integration of ClearML into code, which creates a TensorBoardX `SummaryWriter` object to log
video data.
When the script runs, it creates an experiment named `pytorch with video tensorboardX` in
When the script runs, it creates a task named `pytorch with video tensorboardX` in
the `examples` project.
## Debug Samples
ClearML automatically captures the video data that is added to the `SummaryWriter` object, using the `add_video` method.
The video appears in the experiment's **DEBUG SAMPLES** tab.
The video appears in the task's **DEBUG SAMPLES** tab.
![Debug Samples](../../../img/examples_tensorboardx_debug.png)

4
docs/guides/frameworks/tensorflow/integration_keras_tuner.md
View File

@@ -41,14 +41,14 @@ When the script runs, it logs:
## Scalars
ClearML logs the scalars from training each network. They appear in the experiment's page in the **ClearML web UI**, under
ClearML logs the scalars from training each network. They appear in the task's page in the **ClearML web UI**, under
**SCALARS**.
![image](../../../img/integration_keras_tuner_06.png)
## Summary of Hyperparameter Optimization
ClearML automatically logs the parameters of each experiment run in the hyperparameter search. They appear in tabular
ClearML automatically logs the parameters of each task run in the hyperparameter search. They appear in tabular
form in **PLOTS**.
![image](../../../img/integration_keras_tuner_07.png)

4
docs/guides/frameworks/tensorflow/tensorboard_pr_curve.md
View File

@@ -6,7 +6,7 @@ The [tensorboard_pr_curve.py](https://github.com/allegroai/clearml/blob/master/e
example demonstrates the integration of ClearML into code that uses TensorFlow and TensorBoard.
The example script does the following:
* Creates an experiment named `tensorboard pr_curve` in the `examples` project.
* Creates a task named `tensorboard pr_curve` in the `examples` project.
* Creates three classes, R, G, and B, and generates colors within the RGB space from normal distributions. The true
label of each random color is associated with the normal distribution that generated it.
* Computes the probability that each color belongs to the class, using three other normal distributions.
@@ -16,7 +16,7 @@ The example script does the following:
## Plots
In the **ClearML Web UI**, the PR Curve summaries appear in the experiment's page under **PLOTS**.
In the **ClearML Web UI**, the PR Curve summaries appear in the task's page under **PLOTS**.
* Blue PR curves

4
docs/guides/frameworks/tensorflow/tensorboard_toy.md
View File

@@ -6,12 +6,12 @@ The [tensorboard_toy.py](https://github.com/allegroai/clearml/blob/master/exampl
example demonstrates ClearML's automatic logging of TensorBoard scalars, histograms, images, and text, as well as
all other console output and TensorFlow Definitions.
When the script runs, it creates an experiment named `tensorboard toy example` in the `examples`
When the script runs, it creates a task named `tensorboard toy example` in the `examples`
project.
## Scalars
The `tf.summary.scalar` output appears in the ClearML web UI, in the experiment's
The `tf.summary.scalar` output appears in the ClearML web UI, in the task's
**SCALARS**. Resource utilization plots, which are titled **:monitor: machine**, also appear in the **SCALARS** tab.
![image](../../../img/examples_tensorboard_toy_03.png)

16
docs/guides/frameworks/tensorflow/tensorflow_mnist.md
View File

@@ -6,37 +6,37 @@ The [tensorflow_mnist.py](https://github.com/allegroai/clearml/blob/master/examp
example demonstrates the integration of ClearML into code that uses TensorFlow and Keras to train a neural network on
the Keras built-in [MNIST](https://www.tensorflow.org/api_docs/python/tf/keras/datasets/mnist) handwritten digits dataset.
When the script runs, it creates an experiment named `Tensorflow v2 mnist with summaries` in the `examples` project.
When the script runs, it creates a task named `Tensorflow v2 mnist with summaries` in the `examples` project.
## Scalars
The loss and accuracy metric scalar plots appear in the experiment's page in the **ClearML web UI** under
The loss and accuracy metric scalar plots appear in the task's page in the **ClearML web UI** under
**SCALARS**. Resource utilization plots, which are titled **:monitor: machine**, also appear in the **SCALARS** tab.
![image](../../../img/examples_tensorflow_mnist_06.png)
![Task scalars](../../../img/examples_tensorflow_mnist_06.png)
## Hyperparameters
ClearML automatically logs TensorFlow Definitions. They appear in **CONFIGURATION** **>** **HYPERPARAMETERS**
**>** **TF_DEFINE**.
![image](../../../img/examples_tensorflow_mnist_01.png)
![Task hyperparameters](../../../img/examples_tensorflow_mnist_01.png)
## Console
All console output appears in **CONSOLE**.
![image](../../../img/examples_tensorflow_mnist_05.png)
![Task console](../../../img/examples_tensorflow_mnist_05.png)
## Artifacts
Models created by the experiment appear in the experiment's **ARTIFACTS** tab. ClearML automatically logs and tracks
Models created by the task appear in the task's **ARTIFACTS** tab. ClearML automatically logs and tracks
models and any snapshots created using TensorFlow.
![image](../../../img/examples_tensorflow_mnist_03.png)
![Task models](../../../img/examples_tensorflow_mnist_03.png)
Clicking on a model's name takes you to the [model's page](../../../webapp/webapp_model_viewing.md), where you can
view the model's details and access the model.
![image](../../../img/examples_tensorflow_mnist_10.png)
![Model details](../../../img/examples_tensorflow_mnist_10.png)

8
docs/guides/frameworks/xgboost/xgboost_metrics.md
View File

@@ -6,12 +6,12 @@ The [xgboost_metrics.py](https://github.com/allegroai/clearml/blob/master/exampl
example demonstrates the integration of ClearML into code that uses XGBoost to train a network on the scikit-learn [iris](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_iris.html#sklearn.datasets.load_iris)
classification dataset. ClearML automatically captures models and scalars logged with XGBoost.
When the script runs, it creates a ClearML experiment named `xgboost metric auto reporting` in
When the script runs, it creates a ClearML task named `xgboost metric auto reporting` in
the `examples` project.
## Scalars
ClearML automatically captures scalars logged with XGBoost, which can be visualized in plots in the
ClearML WebApp, in the experiment's **SCALARS** tab.
ClearML WebApp, in the task's **SCALARS** tab.
![Scalars](../../../img/examples_xgboost_metric_scalars.png)
@@ -19,7 +19,7 @@ ClearML WebApp, in the experiment's **SCALARS** tab.
ClearML automatically captures the model logged using the `xgboost.save` method, and saves it as an artifact.
View saved snapshots in the experiment's **ARTIFACTS** tab.
View saved snapshots in the task's **ARTIFACTS** tab.
![Artifacts tab](../../../img/examples_xgboost_metric_artifacts.png)
@@ -29,6 +29,6 @@ To view the model details, click the model name in the **ARTIFACTS** page, which
## Console
All console output during the script's execution appears in the experiment's **CONSOLE** page.
All console output during the script's execution appears in the task's **CONSOLE** page.
![Console output](../../../img/examples_xgboost_metric_console.png)

6
docs/guides/frameworks/xgboost/xgboost_sample.md
View File

@@ -11,11 +11,11 @@ classification dataset using XGBoost
* Scores accuracy using scikit-learn
* ClearML automatically logs the input model registered by XGBoost, and the output model (and its checkpoints),
feature importance plot, and tree plot created with XGBoost.
* Creates an experiment named `XGBoost simple example` in the `examples` project.
* Creates a task named `XGBoost simple example` in the `examples` project.
## Plots
The feature importance plot and tree plot appear in the experiment's page in the **ClearML web UI**, under
The feature importance plot and tree plot appear in the task's page in the **ClearML web UI**, under
**PLOTS**.
![Feature importance plot](../../../img/examples_xgboost_sample_06.png)
@@ -31,7 +31,7 @@ All other console output appear in **CONSOLE**.
## Artifacts
Models created by the experiment appear in the experiment's **ARTIFACTS** tab. ClearML automatically logs and tracks
Models created by the task appear in the task's **ARTIFACTS** tab. ClearML automatically logs and tracks
models and any snapshots created using XGBoost.
![image](../../../img/examples_xgboost_sample_10.png)

4
docs/guides/ide/google_colab.md
View File

@@ -54,7 +54,7 @@ and running, users can send Tasks to be executed on Google Colab's hardware.
For additional options for running `clearml-agent`, see the [clearml-agent reference](../../clearml_agent/clearml_agent_ref.md).
After executing cell 4, the worker appears in the [**Orchestration**](../../webapp/webapp_workers_queues.md)
page of your server. Clone experiments and enqueue them to your hearts content! The `clearml-agent` will fetch
experiments and execute them using the Google Colab hardware.
page of your server. Clone tasks and enqueue them to your hearts content! The `clearml-agent` will fetch
tasks and execute them using the Google Colab hardware.

26
docs/guides/optimization/hyper-parameter-optimization/examples_hyperparam_opt.md
View File

@@ -71,9 +71,9 @@ def job_complete_callback(
Initialize the Task, which will be stored in ClearML Server when the code runs. After the code runs at least once, it
can be [reproduced](../../../webapp/webapp_exp_reproducing.md) and [tuned](../../../webapp/webapp_exp_tuning.md).
Set the Task type to `optimizer`, and create a new experiment (and Task object) each time the optimizer runs (`reuse_last_task_id=False`).
Set the Task type to `optimizer`, and create a new task each time the optimizer runs (`reuse_last_task_id=False`).
When the code runs, it creates an experiment named **Automatic Hyper-Parameter Optimization** in
When the code runs, it creates a task named **Automatic Hyper-Parameter Optimization** in
the **Hyper-Parameter Optimization** project, which can be seen in the **ClearML Web UI**.
```python
@@ -91,21 +91,21 @@ task = Task.init(
Create an arguments dictionary that contains the ID of the Task to optimize, and a Boolean indicating whether the
optimizer will run as a service, see [Running as a service](#running-as-a-service).
In this example, an experiment named **Keras HP optimization base** is being optimized. The experiment must have run at
In this example, a task named **Keras HP optimization base** is being optimized. The task must have run at
least once so that it is stored in ClearML Server, and, therefore, can be cloned.
Since the arguments dictionary is connected to the Task, after the code runs once, the `template_task_id` can be changed
to optimize a different experiment.
to optimize a different task.
```python
# experiment template to optimize in the hyperparameter optimization
# task template to optimize in the hyperparameter optimization
args = {
'template_task_id': None,
'run_as_service': False,
}
args = task.connect(args)
# Get the template task experiment that we want to optimize
# Get the template task that we want to optimize
if not args['template_task_id']:
args['template_task_id'] = Task.get_task(
project_name='examples', task_name='Keras HP optimization base').id
@@ -120,7 +120,7 @@ object, setting the following optimization parameters:
```python
an_optimizer = HyperParameterOptimizer(
# This is the experiment we want to optimize
# This is the task we want to optimize
base_task_id=args['template_task_id'],
```
@@ -174,10 +174,10 @@ object, setting the following optimization parameters:
* Remaining parameters, including the time limit per Task (minutes), period for checking the optimization (minutes),
maximum number of jobs to launch, minimum and maximum number of iterations for each Task:
```python
# Optional: Limit the execution time of a single experiment, in minutes.
# Optional: Limit the execution time of a single task, in minutes.
# (this is optional, and if using OptimizerBOHB, it is ignored)
time_limit_per_job=10.,
# Check the experiments every 6 seconds is way too often, we should probably set it to 5 min,
# Check the tasks every 6 seconds is way too often, we should probably set it to 5 min,
# assuming a single experiment is usually hours...
pool_period_min=0.1,
# set the maximum number of jobs to launch for the optimization, default (None) unlimited
@@ -185,9 +185,9 @@ object, setting the following optimization parameters:
# basically the cumulative number of iterations will not exceed total_max_jobs * max_iteration_per_job
total_max_jobs=10,
# This is only applicable for OptimizerBOHB and ignore by the rest
# set the minimum number of iterations for an experiment, before early stopping
# set the minimum number of iterations for a task before early stopping
min_iteration_per_job=10,
# Set the maximum number of iterations for an experiment to execute
# Set the maximum number of iterations for a task to execute
# (This is optional, unless using OptimizerBOHB where this is a must)
max_iteration_per_job=30,
@@ -215,7 +215,7 @@ it, providing the callback method to report the best performance:
```python
# report every 12 seconds, this is way too often, but we are testing here J
an_optimizer.set_report_period(0.2)
# start the optimization process, callback function to be called every time an experiment is completed
# start the optimization process, callback function to be called every time a task is completed
# this function returns immediately
an_optimizer.start(job_complete_callback=job_complete_callback)
# set the time limit for the optimization process (2 hours)
@@ -233,7 +233,7 @@ Now that it is running:
an_optimizer.set_time_limit(in_minutes=90.0)
# wait until process is done (notice we are controlling the optimization process in the background)
an_optimizer.wait()
# optimization is completed, print the top performing experiments id
# optimization is completed, print the top performing tasks id
top_exp = an_optimizer.get_top_experiments(top_k=3)
print([t.id for t in top_exp])
# make sure background optimization stopped

2
docs/guides/pipeline/pipeline_controller.md
View File

@@ -210,7 +210,7 @@ does the following:
## WebApp
When the experiment is executed, the console output displays the task ID, and links to the pipeline controller task page and
When the task is executed, the console output displays the task ID, and links to the pipeline controller task page and
pipeline page.
```

2
docs/guides/pipeline/pipeline_decorator.md
View File

@@ -76,7 +76,7 @@ To run the pipeline, call the pipeline controller function.
## WebApp
When the experiment is executed, the console output displays the task ID, and links to the pipeline controller task page and pipeline page.
When the task is executed, the console output displays the task ID, and links to the pipeline controller task page and pipeline page.
```
ClearML Task: created new task id=bc93610688f242ecbbe70f413ff2cf5f

2
docs/guides/pipeline/pipeline_functions.md
View File

@@ -105,7 +105,7 @@ the function will be automatically logged as required packages for the pipeline
The pipeline will be launched remotely, through the `services` queue, unless otherwise specified.
## WebApp
When the experiment is executed, the console output displays the task ID, and links to the pipeline controller task page and pipeline page.
When the pipeline is executed, the console output displays the task ID, and links to the pipeline controller task page and pipeline page.
```
ClearML Task: created new task id=bc93610688f242ecbbe70f413ff2cf5f

4
docs/guides/reporting/3d_plots_reporting.md
View File

@@ -5,9 +5,9 @@ title: 3D Plots Reporting
The [3d_plots_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/3d_plots_reporting.py)
example demonstrates reporting a series as a surface plot and as a 3D scatter plot.
When the script runs, it creates an experiment named `3D plot reporting` in the `examples` project.
When the script runs, it creates a task named `3D plot reporting` in the `examples` project.
ClearML reports these plots in the experiment's **PLOTS** tab.
ClearML reports these plots in the task's **PLOTS** tab.
## Surface Plot

12
docs/guides/reporting/artifacts.md
View File

@@ -3,7 +3,7 @@ title: Artifacts Reporting
---
The [artifacts.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/artifacts.py) example demonstrates
uploading objects (other than models) to storage as experiment artifacts.
uploading objects (other than models) to storage as task artifacts.
These artifacts include:
* Pandas DataFrames
@@ -19,14 +19,14 @@ S3 buckets, Google Cloud Storage, and Azure Storage ([debug sample storage](../.
is different). Configure ClearML in any of the following ways:
* In the configuration file, set [`default_output_uri`](../../configs/clearml_conf.md#config_default_output_uri).
* In code, when [initializing a Task](../../references/sdk/task.md#taskinit), use the `output_uri` parameter.
* In the **ClearML Web UI**, when [modifying an experiment](../../webapp/webapp_exp_tuning.md#output-destination).
* In code, when [initializing a task](../../references/sdk/task.md#taskinit), use the `output_uri` parameter.
* In the **ClearML Web UI**, when [modifying a task](../../webapp/webapp_exp_tuning.md#output-destination).
When the script runs, it creates an experiment named `artifacts example` in the `examples` project.
When the script runs, it creates a task named `artifacts example` in the `examples` project.
ClearML reports artifacts in the **ClearML Web UI** **>** experiment details **>** **ARTIFACTS** tab.
ClearML reports artifacts in the **ClearML Web UI** **>** task details **>** **ARTIFACTS** tab.
![Experiment artifacts](../../img/examples_reporting_03.png)
![Task artifacts](../../img/examples_reporting_03.png)
## Dynamically Tracked Artifacts

10
docs/guides/reporting/explicit_reporting.md
View File

@@ -29,7 +29,7 @@ cp pytorch_mnist.py pytorch_mnist_tutorial.py
## Step 1: Setting an Output Destination for Model Checkpoints
Specify a default output location, which is where model checkpoints (snapshots) and artifacts will be stored when the
experiment runs. Some possible destinations include:
task runs. Some possible destinations include:
* Local destination
* Shared folder
* Cloud storage:
@@ -234,7 +234,7 @@ task.register_artifact(
### Reference the Registered Artifact
Once an artifact is registered, it can be referenced and utilized in the Python experiment script.
Once an artifact is registered, it can be referenced and utilized in the Python task script.
In the tutorial script, add [`Task.current_task()`](../../references/sdk/task.md#taskcurrent_task) and
[`Task.get_registered_artifacts()`](../../references/sdk/task.md#get_registered_artifacts)
@@ -279,16 +279,16 @@ task.upload_artifact(
## Additional Information
After extending the Python experiment script, run it and view the results in the **ClearML Web UI**.
After extending the Python task script, run it and view the results in the **ClearML Web UI**.
```bash
python pytorch_mnist_tutorial.py
```
**To view the experiment results, do the following:**
**To view the task results:**
1. In the **ClearML Web UI**, on the Projects page, click the examples project.
1. In the experiments table, click the **Extending automagical ClearML example** experiment.
1. In the task table, click the **Extending automagical ClearML example** task.
1. In the **ARTIFACTS** tab, **DATA AUDIT** section, click **Test_Loss_Correct**. The registered Pandas DataFrame appears,
including the file path, size, hash, metadata, and a preview.
1. In the **OTHER** section, click **Loss**. The uploaded numpy array appears, including its related information.

5
docs/guides/reporting/html_reporting.md
View File

@@ -5,10 +5,9 @@ title: HTML Reporting
The [html_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/html_reporting.py) example
demonstrates reporting local HTML files and HTML by URL using [`Logger.report_media()`](../../references/sdk/logger.md#report_media).
ClearML reports these HTML debug samples in the **ClearML Web UI** **>** experiment details **>**
**DEBUG SAMPLES** tab.
ClearML reports these HTML debug samples in the **ClearML Web UI** **>** task's **DEBUG SAMPLES** tab.
When the script runs, it creates an experiment named `html samples reporting` in the `examples` project.
When the script runs, it creates a task named `html samples reporting` in the `examples` project.
![image](../../img/examples_reporting_05.png)

6
docs/guides/reporting/hyper_parameters.md
View File

@@ -7,11 +7,11 @@ script demonstrates:
* ClearML's automatic logging of `argparse` command line options and TensorFlow Definitions
* Logging user-defined hyperparameters with a parameter dictionary and connecting the dictionary to a Task.
Hyperparameters appear in the **web UI** in the experiment's page, under **CONFIGURATION** **>** **HYPERPARAMETERS**.
Each type is in its own subsection. Parameters from older experiments are grouped together with the ``argparse`` command
Hyperparameters appear in the **web UI** in the task's page, under **CONFIGURATION** **>** **HYPERPARAMETERS**.
Each type is in its own subsection. Parameters from older tasks are grouped together with the ``argparse`` command
line options (in the **Args** subsection).
When the script runs, it creates an experiment named `hyper-parameters example` in the `examples` project.
When the script runs, it creates a task named `hyper-parameters example` in the `examples` project.
## Argparse Command Line Options

4
docs/guides/reporting/image_reporting.md
View File

@@ -15,7 +15,7 @@ or ClearML can be configured for image storage, see [`Logger.set_default_upload_
(storage for [artifacts](../../clearml_sdk/task_sdk.md#setting-upload-destination) is different). Set credentials for
storage in the ClearML configuration file.
When the script runs, it creates an experiment named `image reporting` in the `examples` project.
When the script runs, it creates a task named `image reporting` in the `examples` project.
Report images using several formats by calling [`Logger.report_image()`](../../references/sdk/logger.md#report_image):
@@ -47,7 +47,7 @@ Logger.current_logger().report_image(
)
```
ClearML reports these images as debug samples in the **ClearML Web UI**, under the experiment's
ClearML reports these images as debug samples in the **ClearML Web UI**, under the task's
**DEBUG SAMPLES** tab.
![Debug samples](../../img/examples_reporting_07.png)

10
docs/guides/reporting/manual_matplotlib_reporting.md
View File

@@ -8,15 +8,15 @@ example demonstrates using ClearML to log plots and images generated by Matplotl
## Plots
The Matplotlib and Seaborn plots reported using [`Logger.report_matplotlib_figure()`](../../references/sdk/logger.md#report_matplotlib_figure)
appear in the experiment's **PLOTS**.
appear in the task's **PLOTS** tab.
![Experiment Matplotlib plots](../../img/manual_matplotlib_reporting_01.png)
![Task Matplotlib plots](../../img/manual_matplotlib_reporting_01.png)
![Experiment Seaborn plot](../../img/manual_matplotlib_reporting_02.png)
![Task Seaborn plot](../../img/manual_matplotlib_reporting_02.png)
## Debug Samples
Matplotlib figures can be logged as images by using [`Logger.report_matplotlib_figure()`](../../references/sdk/logger.md#report_matplotlib_figure),
and passing `report_image=True`. The images are displayed in the experiment's **DEBUG SAMPLES**.
and passing `report_image=True`. The images are displayed in the task's **DEBUG SAMPLES**.
![Experiment debug sample](../../img/manual_matplotlib_reporting_03.png)
![Task debug sample](../../img/manual_matplotlib_reporting_03.png)

4
docs/guides/reporting/media_reporting.md
View File

@@ -14,10 +14,10 @@ storage using [`Logger.set_default_upload_destination()`](../../references/sdk/l
(note that [artifact storage](../../clearml_sdk/task_sdk.md#setting-upload-destination) is handled differently).
Set the storage credentials in the [clearml.conf file](../../configs/clearml_conf.md#sdk-section).
ClearML reports media in the **ClearML Web UI** **>** experiment details **>** **DEBUG SAMPLES**
ClearML reports media in the **ClearML Web UI** **>** task details **>** **DEBUG SAMPLES**
tab.
When the script runs, it creates an experiment named `audio and video reporting` in the `examples`
When the script runs, it creates a task named `audio and video reporting` in the `examples`
project.
## Reporting (Uploading) Media from a Source by URL

4
docs/guides/reporting/pandas_reporting.md
View File

@@ -4,10 +4,10 @@ title: Tables Reporting (Pandas and CSV Files)
The [pandas_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/pandas_reporting.py) example demonstrates reporting tabular data from Pandas DataFrames and CSV files as tables.
ClearML reports these tables, and displays them in the **ClearML Web UI** **>** experiment details **>** **PLOTS**
ClearML reports these tables, and displays them in the **ClearML Web UI** **>** task's **PLOTS**
tab.
When the script runs, it creates an experiment named `table reporting` in the `examples` project.
When the script runs, it creates a task named `table reporting` in the `examples` project.
## Reporting Pandas DataFrames as Tables

6
docs/guides/reporting/plotly_reporting.md
View File

@@ -30,9 +30,9 @@ task.get_logger().report_plotly(
)
```
When the script runs, it creates an experiment named `plotly reporting` in the examples project.
When the script runs, it creates a task named `plotly reporting` in the examples project.
ClearML reports Plotly figures, and displays them in the **ClearML Web UI** **>** experiment details **>** **PLOTS**
ClearML reports Plotly figures, and displays them in the **ClearML Web UI** **>** task's **PLOTS**
tab.
![Web UI experiment plots](../../img/examples_reporting_13.png)
![Web UI task plots](../../img/examples_reporting_13.png)

5
docs/guides/reporting/scalar_reporting.md
View File

@@ -3,10 +3,9 @@ title: Scalars Reporting
---
The [scalar_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/scalar_reporting.py) script
demonstrates explicit scalar reporting. ClearML reports scalars in the **ClearML Web UI** **>** experiment details
**>** **SCALARS** tab.
demonstrates explicit scalar reporting. ClearML reports scalars in the **ClearML Web UI** **>** task's **SCALARS** tab.
When the script runs, it creates an experiment named `scalar reporting` in the `examples` project.
When the script runs, it creates a task named `scalar reporting` in the `examples` project.
To reports scalars, call [`Logger.report_scalar()`](../../references/sdk/logger.md#report_scalar).
To report more than one series on the same plot, use the same `title` argument. For different plots, use different

4
docs/guides/reporting/scatter_hist_confusion_mat_reporting.md
View File

@@ -8,9 +8,9 @@ example demonstrates reporting series data in the following 2D formats:
* [Confusion matrices](#confusion-matrices)
* [Scatter plots](#2d-scatter-plots)
ClearML reports these tables in the **ClearML Web UI**, experiment details **>** **PLOTS** tab.
ClearML reports these tables in the **ClearML Web UI** **>** task's **PLOTS** tab.
When the script runs, it creates an experiment named `2D plots reporting` in the `examples` project.
When the script runs, it creates a task named `2D plots reporting` in the `examples` project.
## Histograms

4
docs/guides/reporting/text_reporting.md
View File

@@ -5,9 +5,9 @@ title: Text Reporting
The [text_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/text_reporting.py) script
demonstrates reporting explicit text by calling [`Logger.report_text()`](../../references/sdk/logger.md#report_text).
ClearML reports the text in the **ClearML Web UI**, in the experiment's **CONSOLE** tab.
ClearML reports the text in the **ClearML Web UI**, in the task's **CONSOLE** tab.
When the script runs, it creates an experiment named `text reporting` in the `examples` project.
When the script runs, it creates a task named `text reporting` in the `examples` project.
```python
# report text

2
docs/guides/reporting/using_artifacts.md
View File

@@ -21,7 +21,7 @@ task1.upload_artifact(name='data file', artifact_object='data_samples/sample.jso
The task is then closed, using [`Task.close()`](../../references/sdk/task.md#close), so another task can be
initialized in the same script.
Artifact details (location and size) can be viewed in ClearML's **web UI > experiment details > ARTIFACTS tab > OTHER section**.
Artifact details (location and size) can be viewed in ClearML's **web UI > task details > ARTIFACTS tab > OTHER section**.
![Artifacts in WebApp](../../img/examples_using_artifacts_1.png)

6
docs/guides/services/aws_autoscaler.md
View File

@@ -166,8 +166,8 @@ The values configured through the wizard are stored in the task's hyperparameter
[`Task.connect()`](../../references/sdk/task.md#connect) and [`Task.set_configuration_object()`](../../references/sdk/task.md#set_configuration_object)
methods respectively. They can be viewed in the WebApp, in the task's **CONFIGURATION** page under **HYPERPARAMETERS** and **CONFIGURATION OBJECTS > General**.
ClearML automatically logs command line arguments defined with argparse. View them in the experiments **CONFIGURATION**
page under **HYPERPARAMETERS > General**.
ClearML automatically logs command line arguments defined with argparse. View them in the task's **CONFIGURATION**
tab under **HYPERPARAMETERS > General**.
![Autoscaler configuration](../../img/examples_aws_autoscaler_config.png)
@@ -177,6 +177,6 @@ in [services mode](../../clearml_agent/clearml_agent_services_mode.md) for such
### Console
All other console output appears in the experiment's **CONSOLE**.
All other console output appears in the task's **CONSOLE**.
![Autoscaler console](../../img/examples_aws_autoscaler_console.png)

8
docs/guides/services/cleanup_service.md
View File

@@ -6,7 +6,7 @@ The [cleanup service](https://github.com/allegroai/clearml/blob/master/examples/
demonstrates how to use the `clearml.backend_api.session.client.APIClient` class to implement a service that deletes old
archived tasks and their associated files: model checkpoints, other artifacts, and debug samples.
Modify the cleanup service's parameters to specify which archived experiments to delete and when to delete them.
Modify the cleanup service's parameters to specify which archived tasks to delete and when to delete them.
### Running the Cleanup Service
@@ -52,14 +52,14 @@ an `APIClient` object that establishes a session with the ClearML Server, and ac
* [`Task.delete`](../../references/sdk/task.md#delete) - Delete a Task.
## Configuration
The experiment's hyperparameters are explicitly logged to ClearML using the [`Task.connect()`](../../references/sdk/task.md#connect)
method. View them in the WebApp, in the experiment's **CONFIGURATION** page under **HYPERPARAMETERS > General**.
The task's hyperparameters are explicitly logged to ClearML using the [`Task.connect()`](../../references/sdk/task.md#connect)
method. View them in the WebApp, in the task's **CONFIGURATION** tab under **HYPERPARAMETERS > General**.
The task can be reused. Clone the task, edit its parameters, and enqueue the task to run in ClearML Agent [services mode](../../clearml_agent/clearml_agent_services_mode.md).
![Cleanup service configuration](../../img/example_cleanup_configuration.png)
## Console
All console output appears in the experiment's **CONSOLE**.
All console output appears in the task's **CONSOLE** tab.
![Cleanup service console](../../img/examples_cleanup_console.png)

14
docs/guides/services/slack_alerts.md
View File

@@ -60,17 +60,17 @@ The script supports the following additional command line options:
* `service_queue` - The queue to use when running remotely as a service. The default value is `services` (make sure
your workspace has such a queue and to assign a ClearML Agent to this queue).
* `message_prefix` - A message prefix for Slack alerts. For example, to alert all channel members use: `"Hey <!here>"`.
* `min_num_iterations` - Minimal iteration threshold below which experiments are ignored. Use this option to eliminate
* `min_num_iterations` - Minimal iteration threshold below which tasks are ignored. Use this option to eliminate
debug sessions that fail quickly. The default value is 0.
* `project` - The name of the project to monitor. By default, all projects are monitored.
* `include_manual_experiments` - Whether to include experiments that are running locally:
* `True` - Monitor all experiments (both local and remote, executed by ClearML Agent).
* `False` (default) - Monitor only remote experiments.
* `include_manual_experiments` - Whether to include tasks that are running locally:
* `True` - Monitor all tasks (both local and remote, executed by ClearML Agent).
* `False` (default) - Monitor only remote tasks.
* `include_completed_experiments` - If `False` (default), send alerts only for failed tasks. If `True`, send an alert
for completed and failed tasks.
* `include_archived` - If `False` (default), only tasks that are not archived will be reported. This option can be
useful if a task is archived between polling.
* `refresh_rate` - How often to monitor the experiments in seconds. The default value is 10.0.
* `refresh_rate` - How often to monitor the tasks in seconds. The default value is 10.0.
* `include_users` - Only report tasks that were initiated by these users (usernames and user IDs are accepted).
Mutually exclusive to `exclude_users`.
* `exclude_users` - Only report tasks that were NOT initiated by these users (usernames and user IDs are accepted).
@@ -79,7 +79,7 @@ The script supports the following additional command line options:
## Configuration
ClearML automatically logs command line options defined with argparse. They appear in the experiment's **CONFIGURATION**
ClearML automatically logs command line options defined with argparse. They appear in the task's **CONFIGURATION**
page under **HYPERPARAMETERS > Args**.
![Monitoring configuration](../../img/examples_slack_config.png)
@@ -89,7 +89,7 @@ execution (you'll typically want to use a ClearML Agent running in [services mod
for such service tasks).
## Console
All console output appears in the experiment's **CONSOLE** page.
All console output appears in the task's **CONSOLE** page.
## Additional Information about slack_alerts.py

2
docs/guides/set_offline.md
View File

@@ -98,5 +98,5 @@ ClearML: Importing offline session from /home/user/.clearml/cache/offline/b78684
ClearML results page: https://app.clear.ml/projects/4043a1657f374e9298649c6ba72ad233/experiments/bb8b0f6fa0f94536a0d27fb55f02d3a5/output/log
```
The session details can be viewed in the ClearML WebApp, in the "my_task" experiment of the "examples"
The session details can be viewed in the ClearML WebApp, in the "my_task" task of the "examples"
project, as specified when initializing the Task.

50
docs/guides/ui/building_leader_board.md
View File

@@ -2,12 +2,12 @@
title: Tracking Leaderboards
---
In this tutorial, you will set up a tracking leaderboard. A tracking leaderboard allows easy monitoring of experiments
using a customized [experiments table](../../webapp/webapp_exp_table.md) with auto refresh for continual updates.
In this tutorial, you will set up a tracking leaderboard. A tracking leaderboard allows easy monitoring of tasks
using a customized [task table](../../webapp/webapp_exp_table.md) with auto refresh for continual updates.
The tracking leaderboard can be customized to include columns with information about:
* Experiment status
* Elapsed time since the experiment started and last updated
* Task status
* Elapsed time since the task started and last updated
* Most recent iteration
* Metrics
* Hyperparameters
@@ -17,24 +17,24 @@ Bookmark the leaderboard in the browser to reuse and share the leaderboard URL w
For this tutorial, use one of the following as a project:
* Recommended: One of your projects with more than one experiment.
* Recommended: One of your projects with more than one task.
* Clone the [clearml](https://github.com/allegroai/clearml) repository and execute some of the example scripts.
## Step 1: Select a Project
The leaderboard will track experiments in one or all projects.
The leaderboard will track tasks in one or all projects.
Begin by opening the [ClearML Web UI](../../webapp/webapp_overview.md) and selecting a project, by doing one of the following:
* On the Dashboard, click a project card or **VIEW ALL**.
* On the Projects page, click a project card or the **All projects** card.
## Step 2: Filter the Experiments
## Step 2: Filter the Tasks
The experiments table allows filtering experiments by experiment name, type, and status.
The task table allows filtering tasks by name, type, and status.
**Apply any combination of the following filters:**
* Name - Use the search bar to find experiments containing any search term.
* Name - Use the search bar to find tasks containing any search term.
* Type - Click the **TYPE** column's filter and then on the list select any of the following:
* **Training**
* **Testing**
@@ -43,13 +43,13 @@ The experiments table allows filtering experiments by experiment name, type, and
* And others, depending on the types of tasks in the project
* Status - Click the **STATUS** column's filter and then on the list select any of the following:
* **Draft** - The experiment is not running.
* **Pending** - The experiment is in a queue waiting to be run by a worker.
* **Running** - The experiment is currently running by a worker.
* **Completed** - The experiment ran and terminated successfully.
* **Failed** - The experiment ran and terminated with an error.
* **Aborted** - The experiment ran and was manually or programmatically terminated.
* **Published** - The experiment is not running, it is preserved as read-only.
* **Draft** - The task is not running.
* **Pending** - The task is in a queue waiting to be run by a worker.
* **Running** - The task is currently running by a worker.
* **Completed** - The task ran and terminated successfully.
* **Failed** - The task ran and terminated with an error.
* **Aborted** - The task ran and was manually or programmatically terminated.
* **Published** - The task is not running, it is preserved as read-only.
## Step 3: Hide the Default Columns
@@ -60,14 +60,14 @@ Customize the columns on the tracking leaderboard by hiding any of the default c
1. Click settings.
1. Select / clear any combination of the default columns from the following:
* **TYPE** - The type of experiment: training, testing, or other, depending on the types of experiments in the project.
* **NAME** - The name of the experiment.
* **STATUS** - The status of the experiment.
* **TYPE** - The type of task: training, testing, or other, depending on the types of tasks in the project.
* **NAME** - The name of the task.
* **STATUS** - The status of the task.
* **PROJECT** - The project name.
* **USER** - The user creating or running the experiment.
* **STARTED** - The elapsed time since the experiment started running.
* **UPDATED** - The elapsed time since the experiment update.
* **ITERATION** - The last iteration of the experiment. For experiments with a status of Running, this is the most recent iteration. For Completed, Aborted, and Failed experiments, this is the final iteration.
* **USER** - The user creating or running the task.
* **STARTED** - The elapsed time since the task started running.
* **UPDATED** - The elapsed time since the task update.
* **ITERATION** - The last iteration of the task. For tasks with a status of `Running`, this is the most recent iteration. For `Completed`, `Aborted`, and `Failed` tasks, this is the final iteration.
## Step 4: Show Metrics or Hyperparameters
@@ -83,7 +83,7 @@ recent), minimum, and/or maximum values.
## Step 5: Enable Auto Refresh
Auto refresh allows monitoring the progress of experiments in real time. It is enabled by default.
Auto refresh allows monitoring the progress of tasks in real time. It is enabled by default.
**To enable / disable auto refresh:**
@@ -92,7 +92,7 @@ Auto refresh allows monitoring the progress of experiments in real time. It is e
## Step 6: Save the Tracking Leaderboard
The URL for **ClearML Web UI** now contains parameters and values for the customized leaderboard. Bookmark it to be able
to return to the leaderboard and monitor the experiments.
to return to the leaderboard and monitor the tasks.
:::important
Share the leaderboard with another user by copying and sharing the URL.

46
docs/guides/ui/tuning_exp.md
View File

@@ -1,8 +1,8 @@
---
title: Tuning Experiments
title: Tuning Tasks
---
In this tutorial, learn how to tune an experiment. The experiment that will be tuned is created by the [pytorch_mnist.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py)
In this tutorial, learn how to tune a task. The task that will be tuned is created by the [pytorch_mnist.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py)
example script.
## Prerequisites
@@ -12,25 +12,25 @@ example script.
for the TensorFlow examples.
* Have **ClearML Agent** [installed and configured](../../clearml_agent/clearml_agent_setup.md#installation).
## Step 1: Run the Experiment
## Step 1: Run the Task
In the `examples/frameworks/pytorch` directory, run the experiment script:
In the `examples/frameworks/pytorch` directory, run the task script:
```commandline
python pytorch_mnist.py
```
## Step 2: Clone the Experiment
## Step 2: Clone the Task
Clone the experiment to create an editable copy for tuning.
Clone the task to create an editable copy for tuning.
1. In the ClearML WebApp (UI), on the Projects page, click the `examples` project card.
1. In the experiments table, right-click the experiment `pytorch mnist train`.
1. In the task table, right-click the task `pytorch mnist train`.
1. In the context menu, click **Clone** **>** **CLONE**. The newly cloned experiment appears and its info panel slides open.
1. In the context menu, click **Clone** **>** **CLONE**. The newly cloned task appears and its info panel slides open.
## Step 3: Tune the Cloned Experiment
## Step 3: Tune the Cloned Task
To demonstrate tuning, change two hyperparameter values.
@@ -44,7 +44,7 @@ To demonstrate tuning, change two hyperparameter values.
## Step 4: Run a Worker Daemon Listening to a Queue
To execute the cloned experiment, use a [ClearML Agent](../../fundamentals/agents_and_queues.md).
To execute the cloned task, use a [ClearML Agent](../../fundamentals/agents_and_queues.md).
Run the agent on the local development machine:
1. Open a terminal session.
@@ -78,44 +78,44 @@ Run the agent on the local development machine:
Running CLEARML-AGENT daemon in background mode, writing stdout/stderr to /home/<username>/.clearml_agent_daemon_outym6lqxrz.txt
```
## Step 5: Enqueue the Tuned Experiment
## Step 5: Enqueue the Tuned Task
Enqueue the tuned experiment.
Enqueue the tuned task.
1. In the ClearML WebApp > experiments table, right-click the experiment `Clone Of pytorch mnist train`.
1. In the ClearML WebApp > task table, right-click the task `Clone Of pytorch mnist train`.
1. In the context menu, click **Enqueue**.
1. Select **Default** queue.
1. Click **ENQUEUE**. The experiment's status becomes Pending. When the worker fetches the experiment from the queue,
the status becomes Running. The progress of the experiment can be viewed in the info panel. When the status becomes
1. Click **ENQUEUE**. The task's status becomes Pending. When the worker fetches the task from the queue,
the status becomes Running. The progress of the task can be viewed in the info panel. When the status becomes
Completed, continue to the next step.
## Step 6: Compare the Experiments
## Step 6: Compare the Tasks
To compare the original and tuned experiments:
To compare the original and tuned tasks:
1. In the ClearML WebApp (UI), on the Projects page, click the `examples` project.
1. In the experiments table, select the checkboxes for the two experiments: `pytorch mnist train` and `Clone Of pytorch mnist train`.
1. On the menu bar at the bottom of the experiments table, click **COMPARE**. The experiment comparison window appears.
1. In the task table, select the checkboxes for the two tasks: `pytorch mnist train` and `Clone Of pytorch mnist train`.
1. On the menu bar at the bottom of the task table, click **COMPARE**. The task comparison window appears.
All differences appear with a different background color to highlight them.
The experiment comparison window is organized in the following tabs:
The task comparison window is organized in the following tabs:
* **DETAILS** - The **ARTIFACTS** section, including input and output models with their network designs, and other artifacts;
the **EXECUTION** section execution, including source code control, installed Python packages and versions,
uncommitted changes, and the Docker image name which, in this case, is empty.
* **HYPERPARAMETERS** - The hyperparameters and their values.
* **SCALARS** - Scalar metrics with the option to view them as charts or values.
* **PLOTS** - Plots of any data with the option to view them as charts or values.
* **DEBUG SAMPLES** - Media including images, audio, and video uploaded by your experiment shown as thumbnails.
* **DEBUG SAMPLES** - Media including images, audio, and video uploaded by your task shown as thumbnails.
1. Examine the differences.
1. Compare the hyperparameters. In the **HYPERPARAMETERS** tab, expand **ARGS**. The hyperparameters `batch_size`
and `lr` are shown with a different background color. The values are different.
1. Compare the metrics. In the **SCALARS** tab, to the right of **Add Experiment**, select the plot or value comparison:
1. Compare the metrics. In the **SCALARS** tab, to the right of **Add Task**, select the plot or value comparison:
* **Graph** - The scalar metrics plots show `pytorch mnist train` and `Clone of pytorch mnist train`.
* **Last Values** - Expand a metric and variant.
## Next Steps
* For more information about editing experiments, see [modifying experiments](../../webapp/webapp_exp_tuning.md#modifying-experiments).
* For more information about editing tasks, see [modifying tasks](../../webapp/webapp_exp_tuning.md#modifying-tasks).

4
docs/hyper_datasets.md
View File

@@ -31,9 +31,9 @@ information.
The basic premise of Hyper-Datasets is that a user-formed query is a full representation of the dataset used by the ML/DL
process. Hyper-Datasets decouple metadata from raw data files, allowing you to manipulate metadata through sophisticated
queries and parameters that can be tracked through the experiment manager. You can clone experiments using different
queries and parameters that can be tracked through the task manager. You can clone tasks using different
data manipulations--or [**DataViews**](hyperdatasets/dataviews.md)--without changing any of the hard coded values, making these manipulations part of
the experiment.
the task.
ClearML **Enterprise**'s Hyper-Datasets supports rapid prototyping, creating new opportunities such as:
* Hyperparameter optimization of the data itself

4
docs/hyperdatasets/code_examples.md
View File

@@ -19,9 +19,9 @@ After executing either of these scripts, you can view your DatasetVersion conten
The [dataview_example_framegroup.py](https://github.com/allegroai/clearml/blob/master/examples/hyperdatasets/data-ingestion/dataview_example_framegroup.py)
and [dataview_example_singleframe.py](https://github.com/allegroai/clearml/blob/master/examples/hyperdatasets/data-ingestion/dataview_example_singleframe.py)
examples demonstrate how to use a [DataView](dataviews.md) to retrieve your data as SingleFrames and FrameGroups as
part of a running experiment. This is done by creating a DataView query and then retrieving the corresponding frames.
part of a running task. This is done by creating a DataView query and then retrieving the corresponding frames.
DataView details are displayed in the UI in an experiment's **DATAVIEWS** tab.
DataView details are displayed in the UI in a task's **DATAVIEWS** tab.
### Data Ingestion

6
docs/hyperdatasets/dataset.md
View File

@@ -6,7 +6,7 @@ ClearML Enterprise's **Datasets** and **Dataset versions** provide the internal
and functionality for the following purposes:
* Connecting source data to the ClearML Enterprise platform
* Using ClearML Enterprise's Git-like [Dataset versioning](#dataset-versioning)
* Integrating the powerful features of [Dataviews](dataviews.md) with an experiment
* Integrating the powerful features of [Dataviews](dataviews.md) with a task
* [Annotating](webapp/webapp_datasets_frames.md#annotations) images and videos
Datasets consist of versions with SingleFrames and/or FrameGroups. Each Dataset can contain multiple versions, which
@@ -128,7 +128,7 @@ Use the [`Dataset.delete`](../references/hyperdataset/hyperdataset.md#datasetdel
### Tagging Datasets
Tags can be added to datasets, allowing to easily identify and group experiments.
Tags can be added to datasets, allowing to easily identify and group tasks.
Add tags to a dataset:
```python
@@ -159,7 +159,7 @@ Dataset versions can have either *Draft* or *Published* state.
A *Draft* version is editable, so frames can be added to and deleted and/or modified.
A *Published* version is read-only, which ensures reproducible experiments and preserves the Dataset version contents.
A *Published* version is read-only, which ensures reproducible tasks and preserves the Dataset version contents.
Child versions can only be created from *Published* versions, as they inherit their predecessor version contents.
## Dataset Version Structure

14
docs/hyperdatasets/dataviews.md
View File

@@ -14,19 +14,19 @@ Dataviews support:
* Class label enumeration
* Controls for the frame iteration, such as sequential or random iteration, limited or infinite iteration, and reproducibility.
Dataviews are lazy and optimize processing. When an experiment script runs in a local environment, Dataview pointers
are initialized. If the experiment is cloned or extended, and that newly cloned or extended experiment is tuned and run,
Dataviews are lazy and optimize processing. When a task script runs in a local environment, Dataview pointers
are initialized. If the task is cloned or extended, and that newly cloned or extended task is tuned and run,
only changed pointers are initialized. The pointers that did not change are reused.
## Dataview State
Dataviews can be in either *Draft* or *Published* state.
A *Draft* Dataview is editable. A *Published* Dataview is read-only, which ensures reproducible experiments and
A *Draft* Dataview is editable. A *Published* Dataview is read-only, which ensures reproducible tasks and
preserves the Dataview's settings.
## Filtering
A Dataview filters experiment input data, using one or more frame filters. A frame filter defines the criteria for the
A Dataview filters task input data, using one or more frame filters. A frame filter defines the criteria for the
selection of SingleFrames iterated by a Dataview.
A frame filter contains the following criteria:
@@ -92,11 +92,11 @@ may repeat. The settings include the following:
the maximum, then the actual number of SingleFrames are iterated. If the order is sequential, then no SingleFrames
repeat. If the order is random, then some SingleFrames may repeat.
* Infinite Iterations - Iterate SingleFrames until the experiment is manually terminated. If the order is sequential,
then all SingleFrames are iterated (unless the experiment is manually terminated before all iterate) and SingleFrames
* Infinite Iterations - Iterate SingleFrames until the task is manually terminated. If the order is sequential,
then all SingleFrames are iterated (unless the task is manually terminated before all iterate) and SingleFrames
repeat. If the order is random, then all SingleFrames may not be iterated, and some SingleFrames may repeat.
* Random Seed - If the experiment is rerun and the seed remains unchanged, the SingleFrames iteration is the same.
* Random Seed - If the task is rerun and the seed remains unchanged, the SingleFrames iteration is the same.
* Clip Length - For video data sources, in the number of sequential SingleFrames from a clip to iterate.

2
docs/hyperdatasets/frames.md
View File

@@ -11,4 +11,4 @@ Two types of frames are supported:
**SingleFrames** and **FrameGroups** contain data sources, metadata, and other data. A Frame can be added to [Datasets](dataset.md)
and then modified or removed. [Versions](dataset.md#dataset-versioning) of the Datasets can be created, which enables
documenting changes and reproducing data for experiments.
documenting changes and reproducing data for tasks.

4
docs/hyperdatasets/overview.md
View File

@@ -32,7 +32,7 @@ These components interact in a way that enables revising data and tracking and a
Frames are the basic units of data in ClearML Enterprise. SingleFrames and FrameGroups make up a Dataset version.
Dataset versions can be created, modified, and removed. The different versions are recorded and available,
so experiments, and their data are reproducible and traceable.
so tasks, and their data are reproducible and traceable.
Lastly, Dataviews manage views of the dataset with queries, so the input data to an experiment can be defined from a
Lastly, Dataviews manage views of the dataset with queries, so a task's input data can be defined from a
subset of a Dataset or combinations of Datasets.

2
docs/hyperdatasets/webapp/webapp_annotator.md
View File

@@ -53,7 +53,7 @@ Sort the annotation tasks by either using **RECENT** or **NAME** option.
1. In **ITERATION**, in the **ORDER** list, choose either:
* **Sequential** - Frames are sorted by the frame top-level `context_id` (primary sort key) and `timestamp` (secondary sort key) metadata key values, and returned by the iterator in the sorted order.
* **Random** - Frames are randomly returned using the value of the `random_seed` argument. The random seed is maintained with the experiments. Therefore, the random order is reproducible if the experiment is rerun.
* **Random** - Frames are randomly returned using the value of the `random_seed` argument. The random seed is maintained with the tasks. Therefore, the random order is reproducible if the task is rerun.
1. In **REPETITION**, choose either **Use Each Frame Once** or **Limit Frames**. If you select **Limit Frames**, then in **Use Max. Frames**, type the number of frames to annotate.
1. If iterating randomly, in **RANDOM SEED** type your seed or leave blank, and the ClearML Enterprise platform generates a seed for you.

Some files were not shown because too many files have changed in this diff Show More