From 50ead21ad9e049d8b338d8e1281160664489f94f Mon Sep 17 00:00:00 2001 From: pollfly <75068813+pollfly@users.noreply.github.com> Date: Wed, 28 Aug 2024 17:03:43 +0300 Subject: [PATCH] Fix code examples (#912) --- docs/clearml_agent/clearml_agent_dynamic_gpus.md | 10 +++++++--- docs/clearml_agent/clearml_agent_fractional_gpus.md | 4 ++-- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/docs/clearml_agent/clearml_agent_dynamic_gpus.md b/docs/clearml_agent/clearml_agent_dynamic_gpus.md index 36f8fbad..4206fff2 100644 --- a/docs/clearml_agent/clearml_agent_dynamic_gpus.md +++ b/docs/clearml_agent/clearml_agent_dynamic_gpus.md @@ -14,9 +14,13 @@ To configure the number of GPUs for a queue, use the `--gpus` flag to specify th flag to specify the queue name and number of GPUs: ```console -clearml-agent daemon --dynamic-gpus --gpus 0-2 --queue dual_gpus=2 single_gpu=1 +clearml-agent daemon --dynamic-gpus --gpus 0-2 --queue dual_gpus=2 single_gpu=1 --docker ``` +:::note Docker mode +Make sure to include the `--docker` flag, as dynamic GPU allocation is only supported in [Docker Mode](clearml_agent_execution_env.md#docker-mode). +::: + ## Example Let's say a server has three queues: @@ -28,7 +32,7 @@ An agent can be spun on multiple GPUs (for example: 8 GPUs, `--gpus 0-7`), and t queues that are configured to run with a certain amount of resources: ```console -clearml-agent daemon --dynamic-gpus --gpus 0-7 --queue quad_gpu=4 dual_gpu=2 +clearml-agent daemon --dynamic-gpus --gpus 0-7 --queue quad_gpu=4 dual_gpu=2 --docker ``` The agent can now spin multiple Tasks from the different queues based on the number of GPUs configured to the queue. @@ -38,7 +42,7 @@ queue, look for available GPUs again and spin on GPUs 4-5. Another option for allocating GPUs: ```console -clearml-agent daemon --dynamic-gpus --gpus 0-7 --queue dual=2 opportunistic=1-4 +clearml-agent daemon --dynamic-gpus --gpus 0-7 --queue dual=2 opportunistic=1-4 --docker ``` Notice that a minimum and maximum value of GPUs is specified for the `opportunistic` queue. This means the agent diff --git a/docs/clearml_agent/clearml_agent_fractional_gpus.md b/docs/clearml_agent/clearml_agent_fractional_gpus.md index 1fd8c54b..42e8870d 100644 --- a/docs/clearml_agent/clearml_agent_fractional_gpus.md +++ b/docs/clearml_agent/clearml_agent_fractional_gpus.md @@ -41,7 +41,7 @@ both MIG-enabled and non-MIG devices. flag to specify the queue name(s) and number (or fraction) of GPUs to allocate to them. ``` - clearml-agent daemon --dynamic-gpus --gpus 0, 1 --queue half_gpu=0.5 + clearml-agent daemon --dynamic-gpus --gpus 0, 1 --queue half_gpu=0.5 --docker ``` The agent can utilize 2 GPUs (GPUs 0 and 1). Every task enqueued to the `half_gpu` queue will be run by the agent and @@ -57,7 +57,7 @@ You can set up multiple queues, each allocated a different number of GPUs per ta are listed is their order of priority, so the agent will service tasks from the first listed queue before servicing subsequent queues: ``` -clearml-agent daemon --dynamic-gpus --gpus 0-2 --queue dual_gpus=2 quarter_gpu=0.25 half_gpu=0.5 single_gpu=1 +clearml-agent daemon --dynamic-gpus --gpus 0-2 --queue dual_gpus=2 quarter_gpu=0.25 half_gpu=0.5 single_gpu=1 --docker ``` This agent will utilize 3 GPUs (GPUs 0, 1, and 2). The agent can spin multiple jobs from the different queues based on