Add Docker mode notes (#890)

This commit is contained in:
pollfly 2024-08-04 18:50:09 +03:00 committed by GitHub
parent 9371e861f4
commit 5353bd3cf6
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 40 additions and 8 deletions

View File

@ -140,7 +140,7 @@ the same argument is passed in both. If set to `False`, a task's docker argument
Compatible with Docker versions 0.6.5 and above Compatible with Docker versions 0.6.5 and above
::: :::
* Set a name format for Docker containers created by an agent * Set a name format for Docker containers created by an agent running in [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode)
* The following variables can be used: * The following variables can be used:
* `task_id` * `task_id`
@ -206,7 +206,7 @@ For example:
**`agent.docker_use_activated_venv`** (*bool*) **`agent.docker_use_activated_venv`** (*bool*)
* In Docker mode, if the container's entrypoint automatically activates a virtual environment, the activated virtual * In [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode), if the container's entrypoint automatically activates a virtual environment, the activated virtual
environment is used and everything is installed in it. Set to `false` to disable, and always create a new venv inheriting environment is used and everything is installed in it. Set to `false` to disable, and always create a new venv inheriting
from `system_site_packages` from `system_site_packages`
@ -232,13 +232,14 @@ from `system_site_packages`
**`agent.extra_docker_arguments`** (*[string]*) **`agent.extra_docker_arguments`** (*[string]*)
* Optional arguments to pass to the Docker image. These are local for this agent, and will not be updated in the experiment's `docker_cmd` section. For example, `["--ipc=host", ]`. * Optional arguments to pass to the Docker image when ClearML Agent is running in [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode). These are local for this agent, and will not be updated in the experiment's `docker_cmd` section. For example, `["--ipc=host", ]`.
--- ---
**`agent.extra_docker_shell_script`** (*[string]*) **`agent.extra_docker_shell_script`** (*[string]*)
* An optional shell script to run in the Docker, when the Docker starts, before the experiment starts. For example, `["apt-get install -y bindfs", ]` * When ClearML Agent is running in [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode), this
optional shell script executes inside the Docker on startup, before the experiment starts. For example, `["apt-get install -y bindfs", ]`.
--- ---
@ -397,19 +398,19 @@ ___
**`agent.default_docker`** (*dict*) **`agent.default_docker`** (*dict*)
* Dictionary containing the default options for workers in Docker mode. * Dictionary containing the default options for workers in [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode).
--- ---
**`agent.default_docker.arguments`** (*string*) **`agent.default_docker.arguments`** (*string*)
* If running a worker in Docker mode, this option specifies the options to pass to the Docker container. * If running a worker in [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode), this option specifies the options to pass to the Docker container.
--- ---
**`agent.default_docker.image`** (*string*) **`agent.default_docker.image`** (*string*)
* If running a worker in Docker mode, this option specifies the default Docker image to use. * If running a worker in [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode), this option specifies the default Docker image to use.
--- ---
@ -419,7 +420,8 @@ ___
This feature is available under the ClearML Enterprise plan This feature is available under the ClearML Enterprise plan
::: :::
* Lookup table of rules for default container. The first matched rule will be picked, according to rule order. * Lookup table of rules for default container if running a worker in [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode).
The first matched rule will be picked, according to rule order.
* Each dictionary in the list lays out rules, and the container to be used if the rules are matched. The * Each dictionary in the list lays out rules, and the container to be used if the rules are matched. The
rules can be script requirements, Git details, and/or Python binary, and/or the task's project. rules can be script requirements, Git details, and/or Python binary, and/or the task's project.

View File

@ -72,6 +72,17 @@ To create an experiment through the UI interface:
* Image - Docker image to use for running the experiment * Image - Docker image to use for running the experiment
* Arguments - Add Docker arguments as a single string * Arguments - Add Docker arguments as a single string
* Startup Script - Add a bash script to be executed inside the Docker before setting up the experiment's environment * Startup Script - Add a bash script to be executed inside the Docker before setting up the experiment's environment
:::important
For a task to run in the specified container, the ClearML Agent executing the task must be running in
Docker mode:
```bash
clearml-agent daemon --queue <execution_queue_to_pull_from> --docker
```
For more information, see [Docker Mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode).
:::
* **Run** * **Run**
* Queue - [ClearML Queue](../fundamentals/agents_and_queues.md#what-is-a-queue) where the experiment should be * Queue - [ClearML Queue](../fundamentals/agents_and_queues.md#what-is-a-queue) where the experiment should be
enqueued for execution enqueued for execution

View File

@ -92,6 +92,16 @@ The Container section list the following information:
* Arguments - add Docker arguments * Arguments - add Docker arguments
* Setup shell script - a bash script to be executed inside the Docker before setting up the experiment's environment * Setup shell script - a bash script to be executed inside the Docker before setting up the experiment's environment
:::important
To [rerun](webapp_exp_tuning.md) an experiment through the UI in the listed container, the ClearML Agent executing the experiment must be running in
Docker mode:
```bash
clearml-agent daemon --queue <execution_queue_to_pull_from> --docker [optional default docker image to use]
```
For more information, see [Docker Mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode).
:::
![Container section](../img/webapp_exp_container.png) ![Container section](../img/webapp_exp_container.png)

View File

@ -77,7 +77,16 @@ Select a pre-configured Docker that **ClearML Agent** will use to remotely execu
* In **EXECUTION** **>** **AGENT CONFIGURATION** **>** **BASE DOCKER IMAGE** **>** hover **>** **EDIT** **>** * In **EXECUTION** **>** **AGENT CONFIGURATION** **>** **BASE DOCKER IMAGE** **>** hover **>** **EDIT** **>**
Enter the base Docker image. Enter the base Docker image.
:::important
For a ClearML Agent to execute the task in a container, the agent must be running in
Docker mode:
```bash
clearml-agent daemon --queue <execution_queue_to_pull_from> --docker [optional default docker image to use]
```
For more information, see [Docker Mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode).
:::
#### Output Destination #### Output Destination