clearml-docs/docs/clearml_agent/clearml_agent_ref.md

title
ClearML Agent CLI

The following page provides a reference to clearml-agent's CLI commands:

• build - Create a worker environment without executing an experiment.
• config - List your ClearML Agent configuration data.
• daemon - Run a worker daemon listening to a queue for Tasks (experiments) to execute.
• execute - Execute an experiment, locally without a queue.
• list - List the current workers.

build

Create a worker environment without executing the experiment.

clearml-agent build [-h] --id TASK_ID [--target TARGET]
                    [--install-globally]
                    [--docker [DOCKER [DOCKER ...]]] [--force-docker] 
                    [--python-version PYTHON_VERSION]
                    [--entry-point {reuse_task,clone_task}] [-O]
                    [--git-user GIT_USER] [--git-pass GIT_PASS]
                    [--log-level {DEBUG,INFO,WARN,WARNING,ERROR,CRITICAL}]
                    [--gpus GPUS] [--cpu-only]

Parameters

Name	Description	Optional
--id	Build a worker environment for this Task ID.
--cpu-only	Disable GPU access for the Docker container.
--docker	Run agent in Docker mode. Specify Docker container that a worker will execute at launch. To specify the image name and optional arguments, use one of the following: --docker <image_name> <args> on the command line --docker on the command line, and specify the default image name and arguments in the configuration file. Environment variable settings for Docker containers: CLEARML_DOCKER_SKIP_GPUS_FLAG - Ignore the --gpus flag inside the Docker container. This also allows you to execute ClearML Agent using Docker versions earlier than 19.03. NVIDIA_VISIBLE_DEVICES - Limit GPU visibility for the Docker container. CLEARML_AGENT_GIT_USER and CLEARML_AGENT_GIT_PASS - Pass these credentials to the Docker container at execution. To limit GPU visibility for Docker, set the NVIDIA_VISIBLE_DEVICES environment variable.
--entry-point	Used in conjunction with --docker, indicates how to run the Task specified by --task-id on Docker startup. The --entry-point options are: reuse - Overwrite the existing Task data. clone_task - Clone the Task, and execute the cloned Task.
--force-docker	Force using the agent-specified docker image (either explicitly in the --docker argument or using the agent's default docker image). If provided, the agent will not use any docker container information stored in the task itself (default False)
--git-pass	Git password for repository access.
--git-user	Git username for repository access.
--gpus	Specify the active GPUs for the Docker containers to use. These are the same GPUs set in the NVIDIA_VISIBLE_DEVICES environment variable. For example: --gpus 0 --gpu 0,1,2 --gpus all
-h, --help	Get help for this command.
--install-globally	Install the required Python packages before creating the virtual environment. Use agent.package_manager.system_site_packages to control the installation of the system packages. When --docker is used, --install-globally is always true.
--log-level	SDK log level. The values are: DEBUG INFO WARN WARNING ERROR CRITICAL
--python-version	Virtual environment Python version to use.
-O	Compile optimized pyc code (see python documentation). Repeat for more optimization.
--target	The target folder for the virtual environment and source code that will be used at launch.

config

List your ClearML Agent configuration.

clearml-agent config [-h]

daemon

Run a worker, optionally in a Docker container, listening to a queue.

clearml-agent daemon [-h] [--foreground] [--queue QUEUES [QUEUES ...]] [--order-fairness] 
                     [--standalone-mode] [--services-mode [SERVICES_MODE]] 
                     [--child-report-tags CHILD_REPORT_TAGS [CHILD_REPORT_TAGS ...]]
                     [--create-queue] [--detached] [--stop] [--dynamic-gpus] 
                     [--uptime [UPTIME [UPTIME ...]]] [--downtime [DOWNTIME [DOWNTIME ...]]] 
                     [--status] [--use-owner-token] [-O] 
                     [--git-user GIT_USER] [--git-pass GIT_PASS] 
                     [--log-level {DEBUG,INFO,WARN,WARNING,ERROR,CRITICAL}] 
                     [--gpus GPUS] [--cpu-only] 
                     [--docker [DOCKER [DOCKER ...]]] [--force-current-version]

Parameters

Name	Description	Optional
--child-report-tags	List of tags to send with the status reports from the worker that executes a task.
--cpu-only	If running in Docker mode (see the --docker option), disable GPU access for the Docker container or virtual environment.
--create-queue	If the queue name provided with --queue does not exist in the server, create it on-the-fly and use it.
--detached	Run agent in the background. The clearml-agent command returns immediately.
--docker	Run in Docker mode. Execute the Task inside a Docker container. To specify the image name and optional arguments, either: Use --docker <image_name> <args> on the command line, or Use --docker on the command line, and specify the default image name and arguments in the configuration file. Environment variable settings for Docker containers: CLEARML_DOCKER_SKIP_GPUS_FLAG - Ignore the --gpus flag inside the Docker container. This also allows you to execute ClearML Agent using Docker versions earlier than 19.03. NVIDIA_VISIBLE_DEVICES - Limit GPU visibility for the Docker container. CLEARML_AGENT_GIT_USER and CLEARML_AGENT_GIT_PASS - Pass these credentials to the Docker container at execution.
--downtime	Specify downtime for clearml-agent in <hours> <days> format. For example, use 09-13 TUE to set Tuesday's downtime to 09-13. NOTE: This feature is available under the ClearML Enterprise plan Make sure to have only one of uptime / downtime configuration and not both.
--dynamic-gpus	Allow to dynamically allocate GPUs based on queue properties, configure with --queue <queue_name>=<num_gpus>. For example: --dynamic-gpus --queue dual_gpus=2 single_gpu=1 NOTE: This feature is available under the ClearML Enterprise plan
--force-current-version	To use your current version of ClearML Agent when running in Docker mode (the --docker argument is specified), instead of the latest ClearML Agent version which is automatically installed, specify force-current-version. For example, if your current ClearML Agent version is 0.13.1, but the latest version of ClearML Agent is 0.13.3, use --force-current-version and your Task will execute in the Docker container with ClearML Agent version 0.13.1.
--foreground	Pipe full log to stdout/stderr. Do not use this option if running in background.
--git-pass	Git password for repository access.
--git-user	Git username for repository access
--gpus	If running in Docker mode (see the --docker option), specify the active GPUs for the Docker containers to use. These are the same GPUs set in the NVIDIA_VISIBLE_DEVICES environment variable. For example: --gpus 0 --gpu 0,1,2 --gpus all
-h, --help	Get help for this command.
--log-level	SDK log level. The values are: DEBUG INFO WARN WARNING ERROR CRITICAL
-O	Compile optimized pyc code (see python documentation). Repeat for more optimization.
--order-fairness	Pull from each queue in a round-robin order, instead of priority order.
--queue	Specify the queues which the worker is listening to. The values can be any combination of: One or more queue IDs One or more queue names default indicating the default queue
--services-mode	Launch multiple long-term docker services. Spin multiple, simultaneous Tasks, each in its own Docker container, on the same machine. Each Task will be registered as a new node in the system, providing tracking and transparency capabilities. Start up and shutdown of each Docker is verified. Use in CPU mode (--cpu-only) only. To limit the number of simultaneous tasks run in services mode, pass the maximum number immediately after the --services-mode option (e.g. --services-mode 5)
--standalone-mode	Do not use any network connects. This assumes all requirements are pre-installed.
--status	Print the worker's schedule (uptime properties, server's runtime properties and listening queues)
--stop	Terminate a running ClearML Agent, if other arguments are the same. If no additional arguments are provided, agents are terminated in lexicographical order.
--uptime	Specify uptime for clearml-agent in <hours> <days> format. For example, use 17-20 TUE to set Tuesday's uptime to 17-20. NOTES This feature is available under the ClearML Enterprise plan Make sure to have only one of uptime / downtime configuration and not both.
--use-owner-token	Generate and use the task owner's token for the execution of the task.

execute

Build and execute an experiment without a queue.

clearml-agent execute [-h] --id TASK_ID [--log-file LOG_FILE] [--disable-monitoring] 
                      [--full-monitoring] [--require-queue]
                      [--standalone-mode] [--docker [DOCKER [DOCKER ...]]] [--clone] 
                      [-O] [--git-user GIT_USER] [--git-pass GIT_PASS] 
                      [--log-level {DEBUG,INFO,WARN,WARNING,ERROR,CRITICAL}] 
                      [--gpus GPUS] [--cpu-only]

Parameters

Name	Description	Optional
--id	The ID of the Task to build
--clone	Clone the Task specified by --id, and then execute that cloned Task.
--cpu-only	Disable GPU access for the daemon, only use CPU in either docker or virtual environment.
--docker	Run in Docker mode. Execute the Task inside a Docker container. To specify the image name and optional arguments, use one of the following: --docker <image_name> <args> on the command line --docker on the command line, and specify the default image name and arguments in the configuration file. Environment variable settings for Dockers containers: CLEARML_DOCKER_SKIP_GPUS_FLAG - Ignore the --gpus flag inside the Docker container. This also allows you to execute ClearML Agent using Docker versions earlier than 19.03. NVIDIA_VISIBLE_DEVICES - Limit GPU visibility for the Docker container. CLEARML_AGENT_GIT_USER and CLEARML_AGENT_GIT_PASS - Pass these credentials to the Docker container at execution.
--disable-monitoring	Disable logging and monitoring, except for stdout.
--full-monitoring	Create a full log, including the environment setup log, Task log, and monitoring, as well as stdout.
--git-pass	Git password for repository access.
--git-user	Git username for repository access.
--gpus	Specify active GPUs for the daemon to use (docker / virtual environment), Equivalent to setting NVIDIA_VISIBLE_DEVICES. For example: --gpus 0 --gpu 0,1,2 --gpus all
-h, --help	Get help for this command.
--log-file	The log file for Task execution output (stdout / stderr) to a text file.
--log-level	SDK log level. The values are: DEBUG INFO WARN WARNING ERROR CRITICAL
-O	Compile optimized pyc code (see python documentation). Repeat for more optimization.
--require-queue	If the specified task is not queued (in any queue), the execution will fail. (Used for 3rd party scheduler integration, e.g. K8s, SLURM, etc.)
--standalone-mode	Do not use any network connects, assume everything is pre-installed

list

List information about all active workers.

clearml-agent list [-h]