ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-03-03 18:53:37 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Add ClearML Agent info (#662)

Browse Source
This commit is contained in:
pollfly 2023-09-04 14:37:36 +03:00 committed by GitHub
parent 751e6592dc
commit cd12d80e19
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 26 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

28
docs/clearml_agent.md
View File

@ -2,6 +2,17 @@
title: ClearML Agent title: ClearML Agent
--- ---
<div class="vid" >
<iframe style={{position: 'absolute', top: '0', left: '0', bottom: '0', right: '0', width: '100%', height: '100%'}}
src="https://www.youtube.com/embed/MX3BrXnaULs"
title="YouTube video player"
frameborder="0"
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; fullscreen"
allowfullscreen>
</iframe>
</div>
**ClearML Agent** is a virtual environment and execution manager for DL / ML solutions on GPU machines. It integrates with the **ClearML Python Package** and ClearML Server to provide a full AI cluster solution. <br/> **ClearML Agent** is a virtual environment and execution manager for DL / ML solutions on GPU machines. It integrates with the **ClearML Python Package** and ClearML Server to provide a full AI cluster solution. <br/>
Its main focus is around: Its main focus is around:
- Reproducing experiments, including their complete environments. - Reproducing experiments, including their complete environments.
@ -25,7 +36,8 @@ The preceding diagram demonstrates a typical flow where an agent executes a task
1. Set up the python environment and required packages. 1. Set up the python environment and required packages.
1. The task's script/code is executed. 1. The task's script/code is executed.
While the agent is running, it continuously reports system metrics to the ClearML Server (These can be monitored in the **Orchestration** page). While the agent is running, it continuously reports system metrics to the ClearML Server (these can be monitored in the
[**Orchestration**](webapp/webapp_workers_queues.md) page).
Continue using ClearML Agent once it is running on a target machine. Reproduce experiments and execute Continue using ClearML Agent once it is running on a target machine. Reproduce experiments and execute
automated workflows in one (or both) of the following ways: automated workflows in one (or both) of the following ways:
@ -364,6 +376,18 @@ When executing the ClearML Agent in Docker mode, it will:
ClearML Agent uses the provided default Docker container, which can be overridden from the UI. ClearML Agent uses the provided default Docker container, which can be overridden from the UI.
:::tip Setting Docker Container via UI
You can set the docker container via the UI:
1. Clone the experiment
2. Set the Docker in the cloned task's **Execution** tab **> Container** section
![Container section](../img/webapp_exp_container.png)
3. Enqueue the cloned task
The task will be executed in the container specified in the UI.
:::
All ClearML Agent flags (such as `--gpus` and `--foreground`) are applicable to Docker mode as well. All ClearML Agent flags (such as `--gpus` and `--foreground`) are applicable to Docker mode as well.
To execute ClearML Agent in Docker mode, run: To execute ClearML Agent in Docker mode, run:
@ -389,7 +413,7 @@ CLEARML_AGENT_K8S_HOST_MOUNT=/mnt/host/data:/root/.clearml
ClearML Agent caches virtual environments so when running experiments multiple times, there's no need to spend time reinstalling ClearML Agent caches virtual environments so when running experiments multiple times, there's no need to spend time reinstalling
pre-installed packages. To make use of the cached virtual environments, enable the virtual environment reuse mechanism. pre-installed packages. To make use of the cached virtual environments, enable the virtual environment reuse mechanism.
#### Virtual Environment Reuse ### Virtual Environment Reuse
The virtual environment reuse feature may reduce experiment startup time dramatically. The virtual environment reuse feature may reduce experiment startup time dramatically.