mirror of
https://github.com/clearml/clearml-docs
synced 2025-06-26 18:17:44 +00:00
Small edits (#1031)
This commit is contained in:
@@ -27,7 +27,7 @@ The goal of this phase is to get a code, dataset, and environment set up, so you
|
||||
- [ClearML SDK](../../clearml_sdk/clearml_sdk.md) should be integrated into your code (check out [Getting Started](ds_first_steps.md)).
|
||||
This helps visualizing the results and tracking progress.
|
||||
- [ClearML Agent](../../clearml_agent.md) helps moving your work to other machines without the hassle of rebuilding the environment every time,
|
||||
while also creating an easy queue interface that easily lets you drop your experiments to be executed one by one
|
||||
while also creating an easy queue interface that easily lets you drop your tasks to be executed one by one
|
||||
(great for ensuring that the GPUs are churning during the weekend).
|
||||
- [ClearML Session](../../apps/clearml_session.md) helps with developing on remote machines, in the same way that you'd develop on your local laptop!
|
||||
|
||||
@@ -38,7 +38,7 @@ yields the best performing model for your task!
|
||||
|
||||
- The real training (usually) should **not** be executed on your development machine.
|
||||
- Training sessions should be launched and monitored from a web UI.
|
||||
- You should continue coding while experiments are being executed without interrupting them.
|
||||
- You should continue coding while tasks are being executed without interrupting them.
|
||||
- Stop optimizing your code because your machine struggles, and run it on a beefier machine (cloud / on-prem).
|
||||
|
||||
Visualization and comparison dashboards keep your sanity at bay! At this stage you usually have a docker container with all the binaries
|
||||
@@ -58,23 +58,23 @@ that you need.
|
||||
Track everything--from obscure parameters to weird metrics, it's impossible to know what will end up
|
||||
improving your results later on!
|
||||
|
||||
- Make sure experiments are reproducible! ClearML logs code, parameters, and environment in a single, easily searchable place.
|
||||
- Make sure tasks are reproducible! ClearML logs code, parameters, and environment in a single, easily searchable place.
|
||||
- Development is not linear. Configuration / Parameters should not be stored in your git, as
|
||||
they are temporary and constantly changing. They still need to be logged because who knows, one day...
|
||||
- Uncommitted changes to your code should be stored for later forensics in case that magic number actually saved the day. Not every line change should be committed.
|
||||
- Mark potentially good experiments, make them the new baseline for comparison.
|
||||
- Mark potentially good tasks, make them the new baseline for comparison.
|
||||
|
||||
## Visibility Matters
|
||||
|
||||
While you can track experiments with one tool, and pipeline them with another, having
|
||||
While you can track tasks with one tool, and pipeline them with another, having
|
||||
everything under the same roof has its benefits!
|
||||
|
||||
Being able to track experiment progress and compare experiments, and, based on that, send experiments to execution on remote
|
||||
Being able to track task progress and compare tasks, and, based on that, send tasks to execution on remote
|
||||
machines (that also build the environment themselves) has tremendous benefits in terms of visibility and ease of integration.
|
||||
|
||||
Being able to have visibility in your pipeline, while using experiments already defined in the platform,
|
||||
Being able to have visibility in your pipeline, while using tasks already defined in the platform,
|
||||
enables users to have a clearer picture of the pipeline's status
|
||||
and makes it easier to start using pipelines earlier in the process by simplifying chaining tasks.
|
||||
|
||||
Managing datasets with the same tools and APIs that manage the experiments also lowers the barrier of entry into
|
||||
experiment and data provenance.
|
||||
Managing datasets with the same tools and APIs that manage the tasks also lowers the barrier of entry into
|
||||
task and data provenance.
|
||||
|
||||
@@ -99,7 +99,7 @@ Now you can use ClearML in your notebook!
|
||||
|
||||
In ClearML, experiments are organized as [Tasks](../../fundamentals/task.md).
|
||||
|
||||
ClearML automatically logs your experiment and code, including outputs and parameters from popular ML frameworks,
|
||||
ClearML automatically logs your task and code, including outputs and parameters from popular ML frameworks,
|
||||
once you integrate the ClearML [SDK](../../clearml_sdk/clearml_sdk.md) with your code. To control what ClearML automatically logs, see this [FAQ](../../faq.md#controlling_logging).
|
||||
|
||||
At the beginning of your code, import the `clearml` package:
|
||||
@@ -115,7 +115,7 @@ To ensure full automatic logging, it is recommended to import the `clearml` pack
|
||||
Then initialize the Task object in your `main()` function, or the beginning of the script.
|
||||
|
||||
```python
|
||||
task = Task.init(project_name='great project', task_name='best experiment')
|
||||
task = Task.init(project_name='great project', task_name='best task')
|
||||
```
|
||||
|
||||
If the project does not already exist, a new one is created automatically.
|
||||
|
||||
@@ -151,14 +151,14 @@ Once everything is neatly logged and displayed, use the [comparison tool](../../
|
||||
|
||||
## Track Experiments
|
||||
|
||||
The experiments table is a powerful tool for creating dashboards and views of your own projects, your team's projects, or the entire development.
|
||||
The task table is a powerful tool for creating dashboards and views of your own projects, your team's projects, or the entire development.
|
||||
|
||||

|
||||

|
||||

|
||||

|
||||
|
||||
|
||||
### Creating Leaderboards
|
||||
Customize the [experiments table](../../webapp/webapp_exp_table.md) to fit your own needs, adding desired views of parameters, metrics, and tags.
|
||||
Customize the [task table](../../webapp/webapp_exp_table.md) to fit your own needs, adding desired views of parameters, metrics, and tags.
|
||||
You can filter and sort based on parameters and metrics, so creating custom views is simple and flexible.
|
||||
|
||||
Create a dashboard for a project, presenting the latest Models and their accuracy scores, for immediate insights.
|
||||
|
||||
@@ -12,7 +12,7 @@ If you are afraid of clutter, use the archive option, and set up your own [clean
|
||||
- Track the code base. There is no reason not to add metrics to any process in your workflow, even if it is not directly ML. Visibility is key to iterative improvement of your code / workflow.
|
||||
- Create per-project [leaderboards](../../guides/ui/building_leader_board.md) based on custom columns
|
||||
(hyperparameters and performance accuracy), and bookmark them (full URL will always reproduce the same view and table).
|
||||
- Share experiments with your colleagues and team-leaders.
|
||||
- Share tasks with your colleagues and team-leaders.
|
||||
Invite more people to see how your project is progressing, and suggest they add metric reporting for their own.
|
||||
These metrics can later be part of your own in-house monitoring solution, don't let good data go to waste :)
|
||||
|
||||
@@ -26,10 +26,10 @@ Once you have a Task in ClearML, you can clone and edit its definitions in the U
|
||||
## Advanced Automation
|
||||
- Create daily / weekly cron jobs for retraining best performing models on.
|
||||
- Create data monitoring & scheduling and launch inference jobs to test performance on any new coming dataset.
|
||||
- Once there are two or more experiments that run after another, group them together into a [pipeline](../../pipelines/pipelines.md).
|
||||
- Once there are two or more tasks that run after another, group them together into a [pipeline](../../pipelines/pipelines.md).
|
||||
|
||||
## Manage Your Data
|
||||
Use [ClearML Data](../../clearml_data/clearml_data.md) to version your data, then link it to running experiments for easy reproduction.
|
||||
Use [ClearML Data](../../clearml_data/clearml_data.md) to version your data, then link it to running tasks for easy reproduction.
|
||||
Make datasets machine agnostic (i.e. store original dataset in a shared storage location, e.g. shared-folder / S3 / Gs / Azure).
|
||||
ClearML Data supports efficient Dataset storage and caching, differentiable and compressed.
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@ are fully utilized at all times become daunting tasks.
|
||||
|
||||
This can create overhead that derails you from your core work!
|
||||
|
||||
ClearML Agent was designed to deal with such issues and more! It is a tool responsible for executing experiments on remote machines: on-premises or in the cloud! ClearML Agent provides the means to reproduce and track experiments in your
|
||||
ClearML Agent was designed to deal with such issues and more! It is a tool responsible for executing tasks on remote machines: on-premises or in the cloud! ClearML Agent provides the means to reproduce and track tasks in your
|
||||
machine of choice through the ClearML WebApp with no need for additional code.
|
||||
|
||||
The agent will set up the environment for a specific Task's execution (inside a Docker, or bare-metal), install the
|
||||
@@ -54,40 +54,40 @@ required python packages, and execute and monitor the process.
|
||||
|
||||
:::tip Agent Deployment Modes
|
||||
ClearML Agents can be deployed in:
|
||||
* [Virtual environment mode](../../clearml_agent/clearml_agent_execution_env.md): Agent creates a new venv to execute an experiment.
|
||||
* [Docker mode](../../clearml_agent/clearml_agent_execution_env.md#docker-mode): Agent executes an experiment inside a
|
||||
* [Virtual environment mode](../../clearml_agent/clearml_agent_execution_env.md): Agent creates a new venv to execute a task.
|
||||
* [Docker mode](../../clearml_agent/clearml_agent_execution_env.md#docker-mode): Agent executes a task inside a
|
||||
Docker container.
|
||||
|
||||
For more information, see [Running Modes](../../fundamentals/agents_and_queues.md#running-modes).
|
||||
:::
|
||||
|
||||
## Clone an Experiment
|
||||
Experiments can be reproduced (cloned) for validation or as a baseline for further experimentation.
|
||||
## Clone a Task
|
||||
Tasks can be reproduced (cloned) for validation or as a baseline for further experimentation.
|
||||
Cloning a task duplicates the task's configuration, but not its outputs.
|
||||
|
||||
**To clone an experiment in the ClearML WebApp:**
|
||||
1. Click on any project card to open its [experiments table](../../webapp/webapp_exp_table.md).
|
||||
1. Right-click one of the experiments on the table.
|
||||
1. Click **Clone** in the context menu, which will open a **CLONE EXPERIMENT** window.
|
||||
**To clone a task in the ClearML WebApp:**
|
||||
1. Click on any project card to open its [task table](../../webapp/webapp_exp_table.md).
|
||||
1. Right-click one of the tasks on the table.
|
||||
1. Click **Clone** in the context menu, which will open a **CLONE TASK** window.
|
||||
1. Click **CLONE** in the window.
|
||||
|
||||
The newly cloned experiment will appear and its info panel will slide open. The cloned experiment is in draft mode, so
|
||||
The newly cloned task will appear and its info panel will slide open. The cloned task is in draft mode, so
|
||||
it can be modified. You can edit the Git / code references, control the python packages to be installed, specify the
|
||||
Docker container image to be used, or change the hyperparameters and configuration files. See [Modifying Tasks](../../webapp/webapp_exp_tuning.md#modifying-experiments) for more information about editing experiments in the UI.
|
||||
Docker container image to be used, or change the hyperparameters and configuration files. See [Modifying Tasks](../../webapp/webapp_exp_tuning.md#modifying-tasks) for more information about editing tasks in the UI.
|
||||
|
||||
## Enqueue an Experiment
|
||||
Once you have set up an experiment, it is now time to execute it.
|
||||
## Enqueue a Task
|
||||
Once you have set up a task, it is now time to execute it.
|
||||
|
||||
**To execute an experiment through the ClearML WebApp:**
|
||||
1. Right-click your draft experiment (the context menu is also available through the <img src="/docs/latest/icons/ico-bars-menu.svg" alt="Menu" className="icon size-md space-sm" />
|
||||
button on the top right of the experiment's info panel)
|
||||
1. Click **ENQUEUE,** which will open the **ENQUEUE EXPERIMENT** window
|
||||
**To execute a task through the ClearML WebApp:**
|
||||
1. Right-click your draft task (the context menu is also available through the <img src="/docs/latest/icons/ico-bars-menu.svg" alt="Menu" className="icon size-md space-sm" />
|
||||
button on the top right of the task's info panel)
|
||||
1. Click **ENQUEUE,** which will open the **ENQUEUE TASK** window
|
||||
1. In the window, select `default` in the queue menu
|
||||
1. Click **ENQUEUE**
|
||||
|
||||
This action pushes the experiment into the `default` queue. The experiment's status becomes *Pending* until an agent
|
||||
assigned to the queue fetches it, at which time the experiment's status becomes *Running*. The agent executes the
|
||||
experiment, and the experiment can be [tracked and its results visualized](../../webapp/webapp_exp_track_visual.md).
|
||||
This action pushes the task into the `default` queue. The task's status becomes *Pending* until an agent
|
||||
assigned to the queue fetches it, at which time the task's status becomes *Running*. The agent executes the
|
||||
task, and the task can be [tracked and its results visualized](../../webapp/webapp_exp_track_visual.md).
|
||||
|
||||
|
||||
## Programmatic Interface
|
||||
@@ -95,7 +95,7 @@ experiment, and the experiment can be [tracked and its results visualized](../..
|
||||
The cloning, modifying, and enqueuing actions described above can also be performed programmatically.
|
||||
|
||||
### First Steps
|
||||
#### Access Previously Executed Experiments
|
||||
#### Access Previously Executed Tasks
|
||||
All Tasks in the system can be accessed through their unique Task ID, or based on their properties using the [`Task.get_task`](../../references/sdk/task.md#taskget_task)
|
||||
method. For example:
|
||||
```python
|
||||
@@ -106,15 +106,15 @@ executed_task = Task.get_task(task_id='aabbcc')
|
||||
|
||||
Once a specific Task object has been obtained, it can be cloned, modified, and more. See [Advanced Usage](#advanced-usage).
|
||||
|
||||
#### Clone an Experiment
|
||||
#### Clone a Task
|
||||
|
||||
To duplicate an experiment, use the [`Task.clone`](../../references/sdk/task.md#taskclone) method, and input either a
|
||||
To duplicate a task, use the [`Task.clone`](../../references/sdk/task.md#taskclone) method, and input either a
|
||||
Task object or the Task's ID as the `source_task` argument.
|
||||
```python
|
||||
cloned_task = Task.clone(source_task=executed_task)
|
||||
```
|
||||
|
||||
#### Enqueue an Experiment
|
||||
#### Enqueue a Task
|
||||
To enqueue the task, use the [`Task.enqueue`](../../references/sdk/task.md#taskenqueue) method, and input the Task object
|
||||
with the `task` argument, and the queue to push the task into with `queue_name`.
|
||||
|
||||
@@ -129,7 +129,7 @@ Before execution, use a variety of programmatic methods to manipulate a task obj
|
||||
[Hyperparameters](../../fundamentals/hyperparameters.md) are an integral part of Machine Learning code as they let you
|
||||
control the code without directly modifying it. Hyperparameters can be added from anywhere in your code, and ClearML supports multiple ways to obtain them!
|
||||
|
||||
Users can programmatically change cloned experiments' parameters.
|
||||
Users can programmatically change cloned tasks' parameters.
|
||||
|
||||
For example:
|
||||
```python
|
||||
@@ -200,7 +200,7 @@ min_max_values = executed_task.get_last_scalar_metrics()
|
||||
full_scalars = executed_task.get_reported_scalars()
|
||||
```
|
||||
|
||||
#### Query Experiments
|
||||
#### Query Tasks
|
||||
You can also search and query Tasks in the system. Use the [`Task.get_tasks`](../../references/sdk/task.md#taskget_tasks)
|
||||
class method to retrieve Task objects and filter based on the specific values of the Task - status, parameters, metrics and more!
|
||||
|
||||
@@ -219,7 +219,7 @@ Data is probably one of the biggest factors that determines the success of a pro
|
||||
the model's configuration, code, and results (such as accuracy) is key to deducing meaningful insights into model behavior.
|
||||
|
||||
[ClearML Data](../../clearml_data/clearml_data.md) lets you version your data, so it's never lost, fetch it from every
|
||||
machine with minimal code changes, and associate data to experiment results.
|
||||
machine with minimal code changes, and associate data to task results.
|
||||
|
||||
Logging data can be done via command line, or programmatically. If any preprocessing code is involved, ClearML logs it
|
||||
as well! Once data is logged, it can be used by other experiments.
|
||||
as well! Once data is logged, it can be used by other tasks.
|
||||
|
||||
Reference in New Issue
Block a user