From 5353bd3cf690e272fa85ff6d9cf8af40740ed179 Mon Sep 17 00:00:00 2001 From: pollfly <75068813+pollfly@users.noreply.github.com> Date: Sun, 4 Aug 2024 18:50:09 +0300 Subject: [PATCH] Add Docker mode notes (#890) --- docs/configs/clearml_conf.md | 18 ++++++++++-------- docs/webapp/webapp_exp_table.md | 11 +++++++++++ docs/webapp/webapp_exp_track_visual.md | 10 ++++++++++ docs/webapp/webapp_exp_tuning.md | 9 +++++++++ 4 files changed, 40 insertions(+), 8 deletions(-) diff --git a/docs/configs/clearml_conf.md b/docs/configs/clearml_conf.md index 2be7edfd..15e76932 100644 --- a/docs/configs/clearml_conf.md +++ b/docs/configs/clearml_conf.md @@ -140,7 +140,7 @@ the same argument is passed in both. If set to `False`, a task's docker argument Compatible with Docker versions 0.6.5 and above ::: -* Set a name format for Docker containers created by an agent +* Set a name format for Docker containers created by an agent running in [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode) * The following variables can be used: * `task_id` @@ -206,7 +206,7 @@ For example: **`agent.docker_use_activated_venv`** (*bool*) -* In Docker mode, if the container's entrypoint automatically activates a virtual environment, the activated virtual +* In [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode), if the container's entrypoint automatically activates a virtual environment, the activated virtual environment is used and everything is installed in it. Set to `false` to disable, and always create a new venv inheriting from `system_site_packages` @@ -232,13 +232,14 @@ from `system_site_packages` **`agent.extra_docker_arguments`** (*[string]*) -* Optional arguments to pass to the Docker image. These are local for this agent, and will not be updated in the experiment's `docker_cmd` section. For example, `["--ipc=host", ]`. +* Optional arguments to pass to the Docker image when ClearML Agent is running in [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode). These are local for this agent, and will not be updated in the experiment's `docker_cmd` section. For example, `["--ipc=host", ]`. --- **`agent.extra_docker_shell_script`** (*[string]*) -* An optional shell script to run in the Docker, when the Docker starts, before the experiment starts. For example, `["apt-get install -y bindfs", ]` +* When ClearML Agent is running in [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode), this +optional shell script executes inside the Docker on startup, before the experiment starts. For example, `["apt-get install -y bindfs", ]`. --- @@ -397,19 +398,19 @@ ___ **`agent.default_docker`** (*dict*) -* Dictionary containing the default options for workers in Docker mode. +* Dictionary containing the default options for workers in [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode). --- **`agent.default_docker.arguments`** (*string*) -* If running a worker in Docker mode, this option specifies the options to pass to the Docker container. +* If running a worker in [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode), this option specifies the options to pass to the Docker container. --- **`agent.default_docker.image`** (*string*) -* If running a worker in Docker mode, this option specifies the default Docker image to use. +* If running a worker in [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode), this option specifies the default Docker image to use. --- @@ -419,7 +420,8 @@ ___ This feature is available under the ClearML Enterprise plan ::: -* Lookup table of rules for default container. The first matched rule will be picked, according to rule order. +* Lookup table of rules for default container if running a worker in [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode). +The first matched rule will be picked, according to rule order. * Each dictionary in the list lays out rules, and the container to be used if the rules are matched. The rules can be script requirements, Git details, and/or Python binary, and/or the task's project. diff --git a/docs/webapp/webapp_exp_table.md b/docs/webapp/webapp_exp_table.md index e5c27b71..b728e14e 100644 --- a/docs/webapp/webapp_exp_table.md +++ b/docs/webapp/webapp_exp_table.md @@ -72,6 +72,17 @@ To create an experiment through the UI interface: * Image - Docker image to use for running the experiment * Arguments - Add Docker arguments as a single string * Startup Script - Add a bash script to be executed inside the Docker before setting up the experiment's environment + + :::important + For a task to run in the specified container, the ClearML Agent executing the task must be running in + Docker mode: + + ```bash + clearml-agent daemon --queue --docker + ``` + For more information, see [Docker Mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode). + ::: + * **Run** * Queue - [ClearML Queue](../fundamentals/agents_and_queues.md#what-is-a-queue) where the experiment should be enqueued for execution diff --git a/docs/webapp/webapp_exp_track_visual.md b/docs/webapp/webapp_exp_track_visual.md index d1b503b7..d8a2ffc3 100644 --- a/docs/webapp/webapp_exp_track_visual.md +++ b/docs/webapp/webapp_exp_track_visual.md @@ -92,6 +92,16 @@ The Container section list the following information: * Arguments - add Docker arguments * Setup shell script - a bash script to be executed inside the Docker before setting up the experiment's environment +:::important +To [rerun](webapp_exp_tuning.md) an experiment through the UI in the listed container, the ClearML Agent executing the experiment must be running in +Docker mode: + +```bash +clearml-agent daemon --queue --docker [optional default docker image to use] +``` + +For more information, see [Docker Mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode). +::: ![Container section](../img/webapp_exp_container.png) diff --git a/docs/webapp/webapp_exp_tuning.md b/docs/webapp/webapp_exp_tuning.md index 21b0df6d..6c4f055d 100644 --- a/docs/webapp/webapp_exp_tuning.md +++ b/docs/webapp/webapp_exp_tuning.md @@ -77,7 +77,16 @@ Select a pre-configured Docker that **ClearML Agent** will use to remotely execu * In **EXECUTION** **>** **AGENT CONFIGURATION** **>** **BASE DOCKER IMAGE** **>** hover **>** **EDIT** **>** Enter the base Docker image. +:::important +For a ClearML Agent to execute the task in a container, the agent must be running in +Docker mode: +```bash +clearml-agent daemon --queue --docker [optional default docker image to use] +``` + +For more information, see [Docker Mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode). +::: #### Output Destination