ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-03-03 18:53:37 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Add ClearML Agent flow clarification (#490)

Browse Source
This commit is contained in:
pollfly 2023-03-12 10:30:13 +02:00 committed by GitHub
parent 7aef970ab6
commit aa8947cd0e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 29 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
docs/clearml_agent.md
View File

@ -27,11 +27,18 @@ The diagram above demonstrates a typical flow where an agent executes a task:
While the agent is running, it continuously reports system metrics to the ClearML Server (These can be monitored in the **Workers and Queues** page). While the agent is running, it continuously reports system metrics to the ClearML Server (These can be monitored in the **Workers and Queues** page).
Continue using **ClearML Agent** once it is running on a target machine. Reproduce experiments and execute Continue using ClearML Agent once it is running on a target machine. Reproduce experiments and execute
automated workflows in one (or both) of the following ways: automated workflows in one (or both) of the following ways:
* Programmatically * Programmatically (using [`Task.enqueue`](references/sdk/task.md#taskenqueue) or [`Task.execute_remotely`](references/sdk/task.md#execute_remotely))
* By using the **ClearML Web UI** (without directly working with code), by enqueuing experiments * Through the ClearML Web UI (without working directly with code), by cloning experiments and enqueuing them to the
to the queue that a **ClearML Agent** is listening to. queue that a ClearML Agent is servicing.
The agent facilitates [overriding task execution detail](webapp/webapp_exp_tuning.md) values through the UI without
code modification. Modifying a task clone’s configuration will have the ClearML agent executing it override the
original values:
* Modified package requirements will have the experiment script run with updated packages
* Modified recorded command line arguments will have the ClearML agent inject the new values in their stead
* Code-level configuration instrumented with [`Task.connect`](references/sdk/task.md#connect) will be overridden by modified hyperparameters
For more information, see [ClearML Agent Reference](clearml_agent/clearml_agent_ref.md), For more information, see [ClearML Agent Reference](clearml_agent/clearml_agent_ref.md),
and [configuration options](configs/clearml_conf.md#agent-section). and [configuration options](configs/clearml_conf.md#agent-section).

22
docs/fundamentals/task.md
View File

@ -17,13 +17,27 @@ model checkpoints, artifacts, and charts to cloud storage (see [Storage](../inte
you can work with tasks in Offline Mode, in which all information is saved in a local folder (see you can work with tasks in Offline Mode, in which all information is saved in a local folder (see
[Storing Task Data Offline](../guides/set_offline.md)). [Storing Task Data Offline](../guides/set_offline.md)).
In the UI and code, tasks are grouped into [projects](projects.md), which are logical entities similar to folders. Users can decide Tasks are grouped into a [project](projects.md) hierarchical structure, similar to file folders. Users can decide
how to group tasks, though different models or objectives are usually grouped into different projects. how to group tasks, though different models or objectives are usually grouped into different projects.
Tasks that are in the system can be accessed and utilized with code. To [access a task](../clearml_sdk/task_sdk.md#accessing-tasks), it can be identified either by a Tasks can be accessed and utilized with code. [Access a task](../clearml_sdk/task_sdk.md#accessing-tasks) by
project name & task name combination or by a unique ID. specifying project name & task name combination or by a unique ID.
It's possible to create copies of a task ([clone](../webapp/webapp_exp_reproducing.md)) then execute them with
[ClearML Agent](../clearml_agent.md). When an agent executes a task, it uses the specified configuration to:
* Install required Python packages
* Apply code patches
* Set hyperparameter and run-time configuration values
Modifying a task clone's configuration will have the executing ClearML agent override the original values:
* Modified package requirements will have the experiment script run with updated packages.
* Modified recorded command line arguments will have the ClearML agent inject the new values in their stead
* Code-level configuration instrumented with [`Task.connect`](../references/sdk/task.md#connect) will be overridden by
modified hyperparameters.
It's possible to copy ([clone](../webapp/webapp_exp_reproducing.md)) a task multiple times and to modify it for re-execution.
![Task](../img/fundamentals_task.png) ![Task](../img/fundamentals_task.png)