mirror of
https://github.com/clearml/clearml-docs
synced 2025-02-07 13:21:46 +00:00
Clarify ClearML Agent modes (#685)
This commit is contained in:
parent
ffb1d51072
commit
61806b5178
@ -340,40 +340,37 @@ Run a `clearml-agent` daemon in foreground mode, sending all output to the conso
|
||||
clearml-agent daemon --queue default --foreground
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Execution Environments
|
||||
|
||||
ClearML Agent supports executing tasks in multiple environments.
|
||||
ClearML Agent has two primary execution modes: [Virtual Environment Mode](#virtual-environment-mode) and [Docker Mode](#docker-mode).
|
||||
|
||||
### PIP Mode
|
||||
By default, ClearML Agent works in PIP Mode, in which it uses [pip](https://en.wikipedia.org/wiki/Pip_(package_manager))
|
||||
as the package manager. When ClearML runs, it will create a virtual environment
|
||||
(or [reuse an existing one](clearml_agent.md#virtual-environment-reuse)).
|
||||
Task dependencies (Python packages) will be installed in the virtual environment.
|
||||
### Virtual Environment Mode
|
||||
|
||||
### Conda Mode
|
||||
This mode is similar to the PIP mode but uses [Conda](https://docs.conda.io/en/latest/) as the package
|
||||
manager. To enable Conda mode, edit the `clearml.conf` file, and modify the `type: pip` to `type: conda` in the “package_manager” section.
|
||||
If extra conda channels are needed, look for “conda_channels” under “package_manager”, and add the missing channel.
|
||||
In Virtual Environment Mode, the agent creates a virtual environment for the experiment, installs the required Python
|
||||
packages based on the task specification, clones the code repository, applies the uncommitted changes and finally
|
||||
executes the code while monitoring it. This mode uses smart caching so packages and environments can be reused over
|
||||
multiple tasks (see [Virtual Environment Reuse](#virtual-environment-reuse)).
|
||||
|
||||
### Poetry Mode
|
||||
This mode is similar to the PIP mode but uses [Poetry](https://python-poetry.org/) as the package manager.
|
||||
To enable Poetry mode, edit the `clearml.conf` file, and modify the `type: pip` to `type: poetry` in the “package_manager”
|
||||
section.
|
||||
ClearML Agent supports working with one of the following package managers:
|
||||
* [`pip`](https://en.wikipedia.org/wiki/Pip_(package_manager)) (default)
|
||||
* [`conda`](https://docs.conda.io/en/latest/)
|
||||
* [`poetry`](https://python-poetry.org/)
|
||||
|
||||
To change the package manager used by the agent, edit the [`package_manager.type`](configs/clearml_conf.md#agentpackagemanager)
|
||||
field in the of the `clearml.conf`. If extra channels are needed for `conda`, add the missing channels in the
|
||||
`package_manager.conda_channels` field in the `clearml.conf`.
|
||||
|
||||
:::note Using Poetry with Pyenv
|
||||
Some versions of poetry (using `install-poetry.py`) do not respect `pyenv global`.
|
||||
If you are using pyenv to control the environment where you use ClearML Agent, you can:
|
||||
* Use poetry v1.2 and above (which [fixes this issue](https://github.com/python-poetry/poetry/issues/5077))
|
||||
* Use poetry v1.2 and above (which fixes [this issue](https://github.com/python-poetry/poetry/issues/5077))
|
||||
* Install poetry with the deprecated `get-poetry.py` installer
|
||||
|
||||
:::
|
||||
|
||||
### Docker Mode
|
||||
:::note
|
||||
Docker Mode is only supported in linux.<br/>
|
||||
Docker Mode requires docker service v19.03 or higher installed.
|
||||
:::note notes
|
||||
* Docker Mode is only supported in linux.
|
||||
* Docker Mode requires docker service v19.03 or higher installed.
|
||||
:::
|
||||
|
||||
When executing the ClearML Agent in Docker mode, it will:
|
||||
@ -401,7 +398,7 @@ clearml-agent daemon --queue <execution_queue_to_pull_from> --docker [optional d
|
||||
```
|
||||
|
||||
To use the current `clearml-agent` version in the Docker container, instead of the latest `clearml-agent` version that is
|
||||
automatically installed, run:
|
||||
automatically installed, pass the `--force-current-version` flag:
|
||||
```bash
|
||||
clearml-agent daemon --queue default --docker --force-current-version
|
||||
```
|
||||
|
@ -77,21 +77,23 @@ reuse machines without the need for any dedicated containers or images.
|
||||
Agents can be deployed bare-metal, with multiple instances allocating
|
||||
specific GPUs to the agents. They can also be deployed as dockers in a Kubernetes cluster.
|
||||
|
||||
The Agent has three running modes:
|
||||
- Docker Mode: The agent spins a docker image based on the Task’s definition then inside the docker the agent will clone
|
||||
the specified repository/code, apply the original execution’s uncommitted changes, install the required python packages
|
||||
and start executing the code while monitoring it.
|
||||
- Virtual Environment Mode: The agent creates a new virtual environment for the experiment, installs the required python
|
||||
packages based on the Task specification, clones the code repository, applies the uncommitted changes and finally
|
||||
executes the code while monitoring it.
|
||||
- Conda Environment Mode: Similar to the Virtual Environment mode, only instead of using pip, it uses conda install and
|
||||
pip combination. Notice this mode is quite brittle due to the Conda package version support table.
|
||||
The Agent supports the following running modes:
|
||||
* **Virtual Environment Mode** - The agent creates a new virtual environment for the experiment, installs the required
|
||||
python packages based on the Task specification, clones the code repository, applies the uncommitted changes and
|
||||
finally executes the code while monitoring it. This mode uses smart caching so packages and environments can be reused
|
||||
over multiple tasks (see [Virtual Environment Reuse](../clearml_agent.md#virtual-environment-reuse)).
|
||||
|
||||
:::tip Agents and virtual environments
|
||||
An agent that runs in Virtual Environment Mode or Conda Environment Mode needs to create virtual environments, and
|
||||
it can't do that when running from a virtual environment. You can run agents in these modes with
|
||||
system Python.
|
||||
:::
|
||||
ClearML Agent supports using the following package managers: `pip` (default), `conda`, `poetry`.
|
||||
|
||||
:::tip Agents and virtual environments
|
||||
An agent that runs in Virtual Environment Mode needs to create virtual environments, and
|
||||
it can't do that when running from a virtual environment. You can run agents in these modes with
|
||||
system Python.
|
||||
:::
|
||||
|
||||
* **Docker Mode** - The agent spins a Docker image based on the Task's definitions. Inside the Docker the agent clones
|
||||
the specified repository/code, applies the original execution's uncommitted changes, sets up the Python environment and
|
||||
required packages, and starts executing the code while monitoring it.
|
||||
|
||||
## Services Mode
|
||||
|
||||
|
Loading…
Reference in New Issue
Block a user