ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-06-26 18:17:44 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Clarify ClearML Agent modes (#685)

Browse Source
This commit is contained in:
pollfly
2023-10-03 10:44:37 +03:00
committed by GitHub
parent ffb1d51072
commit 61806b5178
2 changed files with 35 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files

41
docs/clearml_agent.md
View File

@@ -340,40 +340,37 @@ Run a `clearml-agent` daemon in foreground mode, sending all output to the conso
clearml-agent daemon --queue default --foreground
```
## Execution Environments
ClearML Agent supports executing tasks in multiple environments.
ClearML Agent has two primary execution modes: [Virtual Environment Mode](#virtual-environment-mode) and [Docker Mode](#docker-mode).
### PIP Mode
By default, ClearML Agent works in PIP Mode, in which it uses [pip](https://en.wikipedia.org/wiki/Pip_(package_manager))
as the package manager. When ClearML runs, it will create a virtual environment
(or [reuse an existing one](clearml_agent.md#virtual-environment-reuse)).
Task dependencies (Python packages) will be installed in the virtual environment.
### Virtual Environment Mode
### Conda Mode
This mode is similar to the PIP mode but uses [Conda](https://docs.conda.io/en/latest/) as the package
manager. To enable Conda mode, edit the `clearml.conf` file, and modify the `type: pip` to `type: conda` in the “package_manager” section.
If extra conda channels are needed, look for “conda_channels” under “package_manager”, and add the missing channel.
In Virtual Environment Mode, the agent creates a virtual environment for the experiment, installs the required Python
packages based on the task specification, clones the code repository, applies the uncommitted changes and finally
executes the code while monitoring it. This mode uses smart caching so packages and environments can be reused over
multiple tasks (see [Virtual Environment Reuse](#virtual-environment-reuse)).
### Poetry Mode
This mode is similar to the PIP mode but uses [Poetry](https://python-poetry.org/) as the package manager.
To enable Poetry mode, edit the `clearml.conf` file, and modify the `type: pip` to `type: poetry` in the “package_manager”
section.
ClearML Agent supports working with one of the following package managers:
* [`pip`](https://en.wikipedia.org/wiki/Pip_(package_manager)) (default)
* [`conda`](https://docs.conda.io/en/latest/)
* [`poetry`](https://python-poetry.org/)
To change the package manager used by the agent, edit the [`package_manager.type`](configs/clearml_conf.md#agentpackagemanager)
field in the of the `clearml.conf`. If extra channels are needed for `conda`, add the missing channels in the
`package_manager.conda_channels` field in the `clearml.conf`.
:::note Using Poetry with Pyenv
Some versions of poetry (using `install-poetry.py`) do not respect `pyenv global`.
If you are using pyenv to control the environment where you use ClearML Agent, you can:
* Use poetry v1.2 and above (which [fixes this issue](https://github.com/python-poetry/poetry/issues/5077))
* Use poetry v1.2 and above (which fixes [this issue](https://github.com/python-poetry/poetry/issues/5077))
* Install poetry with the deprecated `get-poetry.py` installer
:::
### Docker Mode
:::note
Docker Mode is only supported in linux.<br/>
Docker Mode requires docker service v19.03 or higher installed.
:::note notes
* Docker Mode is only supported in linux.
* Docker Mode requires docker service v19.03 or higher installed.
:::
When executing the ClearML Agent in Docker mode, it will:
@@ -401,7 +398,7 @@ clearml-agent daemon --queue <execution_queue_to_pull_from> --docker [optional d
```
To use the current `clearml-agent` version in the Docker container, instead of the latest `clearml-agent` version that is
automatically installed, run:
automatically installed, pass the `--force-current-version` flag:
```bash
clearml-agent daemon --queue default --docker --force-current-version
```