Add ClearML Agent flow clarification (#490)

This commit is contained in:
pollfly 2023-03-12 10:30:13 +02:00 committed by GitHub
parent 7aef970ab6
commit aa8947cd0e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 29 additions and 8 deletions

View File

@ -27,11 +27,18 @@ The diagram above demonstrates a typical flow where an agent executes a task:
While the agent is running, it continuously reports system metrics to the ClearML Server (These can be monitored in the **Workers and Queues** page).
Continue using **ClearML Agent** once it is running on a target machine. Reproduce experiments and execute
Continue using ClearML Agent once it is running on a target machine. Reproduce experiments and execute
automated workflows in one (or both) of the following ways:
* Programmatically
* By using the **ClearML Web UI** (without directly working with code), by enqueuing experiments
to the queue that a **ClearML Agent** is listening to.
* Programmatically (using [`Task.enqueue`](references/sdk/task.md#taskenqueue) or [`Task.execute_remotely`](references/sdk/task.md#execute_remotely))
* Through the ClearML Web UI (without working directly with code), by cloning experiments and enqueuing them to the
queue that a ClearML Agent is servicing.
The agent facilitates [overriding task execution detail](webapp/webapp_exp_tuning.md) values through the UI without
code modification. Modifying a task clones configuration will have the ClearML agent executing it override the
original values:
* Modified package requirements will have the experiment script run with updated packages
* Modified recorded command line arguments will have the ClearML agent inject the new values in their stead
* Code-level configuration instrumented with [`Task.connect`](references/sdk/task.md#connect) will be overridden by modified hyperparameters
For more information, see [ClearML Agent Reference](clearml_agent/clearml_agent_ref.md),
and [configuration options](configs/clearml_conf.md#agent-section).

View File

@ -17,13 +17,27 @@ model checkpoints, artifacts, and charts to cloud storage (see [Storage](../inte
you can work with tasks in Offline Mode, in which all information is saved in a local folder (see
[Storing Task Data Offline](../guides/set_offline.md)).
In the UI and code, tasks are grouped into [projects](projects.md), which are logical entities similar to folders. Users can decide
Tasks are grouped into a [project](projects.md) hierarchical structure, similar to file folders. Users can decide
how to group tasks, though different models or objectives are usually grouped into different projects.
Tasks that are in the system can be accessed and utilized with code. To [access a task](../clearml_sdk/task_sdk.md#accessing-tasks), it can be identified either by a
project name & task name combination or by a unique ID.
Tasks can be accessed and utilized with code. [Access a task](../clearml_sdk/task_sdk.md#accessing-tasks) by
specifying project name & task name combination or by a unique ID.
It's possible to create copies of a task ([clone](../webapp/webapp_exp_reproducing.md)) then execute them with
[ClearML Agent](../clearml_agent.md). When an agent executes a task, it uses the specified configuration to:
* Install required Python packages
* Apply code patches
* Set hyperparameter and run-time configuration values
Modifying a task clone's configuration will have the executing ClearML agent override the original values:
* Modified package requirements will have the experiment script run with updated packages.
* Modified recorded command line arguments will have the ClearML agent inject the new values in their stead
* Code-level configuration instrumented with [`Task.connect`](../references/sdk/task.md#connect) will be overridden by
modified hyperparameters.
It's possible to copy ([clone](../webapp/webapp_exp_reproducing.md)) a task multiple times and to modify it for re-execution.
![Task](../img/fundamentals_task.png)