diff --git a/docs/clearml_agent.md b/docs/clearml_agent.md index 2fc57017..8011d895 100644 --- a/docs/clearml_agent.md +++ b/docs/clearml_agent.md @@ -1,5 +1,5 @@ --- -title: ClearML Agent Overview +title: ClearML Agent --- **ClearML Agent** is a virtual environment and execution manager for DL / ML solutions on GPU machines. It integrates with the **ClearML Python Package** and ClearML Server to provide a full AI cluster solution.
@@ -649,3 +649,9 @@ Or, force workers on for a queue using the REST API: ```bash curl --user : --header "Content-Type: application/json" --data '{"queue":"","tags":["force_workers:on"]}' http://:8008/queues.update ``` + +## References + +* See [ClearML Agent CLI](clearml_agent/clearml_agent_ref.md) for a reference for `clearml-agent`'s CLI commands. +* See [ClearML Agent Environment Variables](clearml_agent/clearml_agent_env_var.md) for a list of environment variables +to configure ClearML Agent \ No newline at end of file diff --git a/docs/clearml_agent/clearml_agent_build.md b/docs/clearml_agent/clearml_agent_build.md deleted file mode 100644 index 870df483..00000000 --- a/docs/clearml_agent/clearml_agent_build.md +++ /dev/null @@ -1,141 +0,0 @@ ---- -title: build ---- - -This reference page provides detailed information about ClearML Agent's `build` subcommand, which you can use to create -a worker environment without executing the experiment. - -## Syntax - -```bash -clearml-agent build [-h] --id TASK_ID [--target TARGET] - [--install-globally] - [--docker [DOCKER [DOCKER ...]]] [--force-docker] - [--python-version PYTHON_VERSION] - [--entry-point {reuse_task,clone_task}] [-O] - [--git-user GIT_USER] [--git-pass GIT_PASS] - [--log-level {DEBUG,INFO,WARN,WARNING,ERROR,CRITICAL}] - [--gpus GPUS] [--cpu-only] -``` - -## Arguments - ---- - -**`id`** -(*mandatory*) - -* Build a worker environment for this Task ID. - ---- - -**`cpu-only`** - -* Disable GPU access for the Docker container. - ---- - -**`docker`** - -* Docker mode. A Docker container that a worker will execute at launch. - - To specify the image name and optional arguments, either: - - * Use `--docker ` on the command line, or - * Use `--docker` on the command line, and specify the image name and arguments in the configuration file. - - Environment variable settings for Dockers: - - * `CLEARML_DOCKER_SKIP_GPUS_FLAG` - Ignore the `gpus` flag inside the Docker container. This also allows you to execute **ClearML Agent** using Docker versions earlier than 19.03. - * `NVIDIA_VISIBLE_DEVICES` - Limit GPU visibility for the Docker container. - * `CLEARML_AGENT_GIT_USER` and `CLEARML_AGENT_GIT_PASS` - Pass these credentials to the Docker container at execution. - - To limit GPU visibility for Docker, set the `NVIDIA_VISIBLE_DEVICES` environment variable. - ---- - -**`entry-point`** - -* Used in conjunction with `--docker`, indicates how to run the Task specified by `task-id` on Docker startup. - - The `entry-point` options are: - - * `reuse` - Overwrite the existing Task data. - * `clone_task` - Clone the Task, and execute the cloned Task. - ---- - -**`force-docker`** - -* Force using the agent-specified docker image (either explicitly in the `--docker` argument or using the agent's - default docker image). If provided, the agent will not use any docker container information stored in the task itself - (default `False`) - - ---- - -**`git-pass`** - -* Git password for repository access. - ---- - -**`git-user`** - -* Git username for repository access. - ---- - -**`gpus`** - -* Specify the active GPUs for the Docker containers to use. These are the same GPUs set in the `NVIDIA_VISIBLE_DEVICES` environment variable. - - For example: - - * `--gpus 0` - * `--gpu 0,1,2` - * `--gpus all` - ---- - -**`h`, `help`** - -* Get help for this command. - ---- - -**`install-globally`** - -* Install the required Python packages before creating the virtual environment. Use `agent.package_manager.system_site_packages` to control the installation of the system packages. When `--docker` is used, `install-globally` is always true. - ---- - -**`log-level`** - -* SDK log level. The values are: - - * `DEBUG` - * `INFO` - * `WARN` - * `WARNING` - * `ERROR` - * `CRITICAL` - ---- - -**`python-version`** - -* Virtual environment Python version to use. - ---- - -**`O`** - -* Compile optimized pyc code (see python documentation). Repeat for more optimization. - ---- - -**`target`** - -* The target folder for the virtual environment and source code that will be used at launch. - diff --git a/docs/clearml_agent/clearml_agent_config.md b/docs/clearml_agent/clearml_agent_config.md deleted file mode 100644 index 937bbe5d..00000000 --- a/docs/clearml_agent/clearml_agent_config.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: config ---- - -This reference page provides detailed information about ClearML Agent's `config` subcommand, which you can use to -list your ClearML Agent configuration. - -## Syntax - -``` -clearml-agent config [-h] -``` \ No newline at end of file diff --git a/docs/clearml_agent/clearml_agent_daemon.md b/docs/clearml_agent/clearml_agent_daemon.md deleted file mode 100644 index a0943c4c..00000000 --- a/docs/clearml_agent/clearml_agent_daemon.md +++ /dev/null @@ -1,223 +0,0 @@ ---- -title: daemon ---- - -This reference page provides detailed information about ClearML Agent's `daemon` subcommand, which you can use to -run a worker, optionally in a Docker container, listening to a queue. - -## Syntax - -```bash -clearml-agent daemon [-h] [--foreground] [--queue QUEUES [QUEUES ...]] [--order-fairness] - [--standalone-mode] [--services-mode [SERVICES_MODE]] - [--child-report-tags CHILD_REPORT_TAGS [CHILD_REPORT_TAGS ...]] - [--create-queue] [--detached] [--stop] [--dynamic-gpus] - [--uptime [UPTIME [UPTIME ...]]] [--downtime [DOWNTIME [DOWNTIME ...]]] - [--status] [--use-owner-token] [-O] - [--git-user GIT_USER] [--git-pass GIT_PASS] - [--log-level {DEBUG,INFO,WARN,WARNING,ERROR,CRITICAL}] - [--gpus GPUS] [--cpu-only] - [--docker [DOCKER [DOCKER ...]]] [--force-current-version] -``` - - -## Arguments - ---- - -**`child-report-tags`** - -List of tags to send with the status reports from the worker that executes a task. - ---- - -**`cpu-only`** - -* If running in Docker mode (see the `docker` option), disable GPU access for the Docker container or virtual environment. - ---- - -**`create-queue`** - -* If the queue name provided does not exist in the server, create and use it. - ---- - -**`detached`** - -* Run agent in the background. The `clearml-agent` command returns immediately. - ---- - -**`docker`** - -* Run in Docker mode. Execute the Task inside a Docker container. - - To specify the image name and optional arguments, either: - - * use `--docker ` on the command line, or - * use `--docker` on the command line, and specify the image name and arguments in the configuration file. - - Environment variable settings for Dockers: - - * `CLEARML_DOCKER_SKIP_GPUS_FLAG` - Ignore the `gpus` flag inside the Docker container. This also allows you to execute **ClearML Agent** using Docker versions earlier than 19.03. - * `NVIDIA_VISIBLE_DEVICES` - Limit GPU visibility for the Docker container. - * `ClearML_AGENT_GIT_USER` and `ClearML_AGENT_GIT_PASS` - Pass these credentials to the Docker container at execution. - ---- - -**`downtime`** - -* Specify downtime for clearml-agent in ` ` format. - - For example, use `09-13 TUE` to set Tuesday's downtime to 09-13. - -:::info -* This feature is available under the ClearML Enterprise plan -* Make sure to have only one of uptime / downtime configuration and not both. -::: - ---- - -**`dynamic-gpus`** - -* Allow to dynamically allocate gpus based on queue properties, configure with `--queue =`. - - For example: `--dynamic-gpus --queue dual_gpus=2 single_gpu=1` - -:::info Enterprise Feature -This feature is available under the ClearML Enterprise plan -::: - - - ---- - -**`force-current-version`** - -* To use your current version of **ClearML Agent** when running in Docker mode (the `docker` argument is specified), - instead of the latest **ClearML Agent** version which is automatically installed, specify `force-current-version`. - - For example, if your current **ClearML Agent** version is `0.13.1`, but the latest version of **ClearML Agent** is - `0.13.3`, use `--force-current-version` and your Task will execute in the Docker container with **ClearML Agent** version `0.13.1`. - ---- - -**`foreground`** - -* Pipe full log to stdout/stderr. Do not use this option if running in background. - ---- - -**`git-pass`** - -* Git password for repository access. - ---- - -**`git-user`** - -* Git username for repository access. - ---- - -**`gpus`** - -* If running in Docker mode (see the `docker` option), specify the active GPUs for the Docker containers to use. - - These are the same GPUs set in the `NVIDIA_VISIBLE_DEVICES` environment variable. - - For example: - - * `--gpus 0` - * `--gpu 0,1,2` - * `--gpus all` - ---- - -**`h`, `help`** - -* Get help for this command. - ---- -**`log-level`** - -* SDK log level. The values are: - - * `DEBUG` - * `INFO` - * `WARN` - * `WARNING` - * `ERROR` - * `CRITICAL` - ---- - -**`O`** - -* Compile optimized pyc code (see python documentation). Repeat for more optimization. - ---- - -**`order-fairness`** - -* Pull from each queue in a round-robin order, instead of priority order. - ---- - -**`queue`** - -* Specify the queues which the worker is listening to. The values can be any combination of: - - * One or more queue IDs. - * One or more queue names. - * `default` indicating the default queue. - ---- - -**`services-mode`** - -* Launch multiple long-term docker services. Spin multiple, simultaneous Tasks, each in its own Docker container, on the same machine. Each Task will be registered - as a new node in the system, providing tracking and transparency capabilities. Start up and shutdown of each Docker is - verified. Use in CPU mode (`--cpu-only`), only. - -* To limit the number of simultaneous tasks run in services mode, pass the maximum number immediately after the -`--services-mode` option (e.g. `--services-mode 5`) - - ---- - -**`standalone-mode`** - -* Do not use any network connects. This assumes all requirements are pre-installed. - ---- - -**`status`** - -* Print the worker's schedule (uptime properties, server's runtime properties and listening queues) - - ---- - -**`stop`** - -* Terminate a running ClearML Agent, if other arguments are the same. If no additional arguments are provided, agents are - terminated in lexicographical order. - ---- - -**`uptime`** -* Specify uptime for clearml-agent in ` ` format. for example, use `17-20 TUE` to set Tuesday's - uptime to 17-20 - -:::info -* This feature is available under the ClearML Enterprise plan -* Make sure to have only one of uptime / downtime configuration and not both. -::: - ---- - -**`use-owner-token`** - -Generate and use the task owner's token for the execution of the task. diff --git a/docs/clearml_agent/clearml_agent_env_var.md b/docs/clearml_agent/clearml_agent_env_var.md index 6662cc30..45bf73df 100644 --- a/docs/clearml_agent/clearml_agent_env_var.md +++ b/docs/clearml_agent/clearml_agent_env_var.md @@ -1,11 +1,12 @@ --- -title: Environment Variables +title: ClearML Agent Environment Variables --- This page lists the available environment variables for configuring ClearML Agent. :::info -ClearML's environment variables override the clearml.conf file, SDK, and [configuration vault](../webapp/webapp_profile.md#configuration-vault), +ClearML's environment variables override the [clearml.conf file](../configs/clearml_conf.md), SDK, and +[configuration vault](../webapp/webapp_profile.md#configuration-vault), but can be overridden by command-line arguments. ::: diff --git a/docs/clearml_agent/clearml_agent_execute.md b/docs/clearml_agent/clearml_agent_execute.md deleted file mode 100644 index 22c6205f..00000000 --- a/docs/clearml_agent/clearml_agent_execute.md +++ /dev/null @@ -1,130 +0,0 @@ ---- -title: execute ---- - -This reference page provides detailed information about ClearML Agent's `execute` subcommand, which you can use to -build and execute an experiment without a queue. - -## Syntax - -```bash -clearml-agent execute [-h] --id TASK_ID [--log-file LOG_FILE] [--disable-monitoring] - [--full-monitoring] [--require-queue] - [--standalone-mode] [--docker [DOCKER [DOCKER ...]]] [--clone] - [-O] [--git-user GIT_USER] [--git-pass GIT_PASS] - [--log-level {DEBUG,INFO,WARN,WARNING,ERROR,CRITICAL}] - [--gpus GPUS] [--cpu-only] -``` - - -## Arguments - ---- - -**`id`** (*mandatory*) - -* The ID of the Task to build. - ---- - -**`clone`** - -* Clone the Task specified by `id`, and then execute that cloned Task. - ---- - -**`cpu-only`** -* Disable GPU access for the daemon, only use CPU in either docker or virtual environment. - - ---- - -**`docker`** - -* Run in Docker mode. Execute the Task inside a Docker container. - - To specify the image name and optional arguments, either: - - * use `--docker ` on the command line, or - * use `--docker` on the command line, and specify the default image name and arguments in the configuration file. - - Environment variable settings for Dockers containers: - - * `ClearML_DOCKER_SKIP_GPUS_FLAG` - Ignore the `gpus` flag inside the Docker container. This also allows you to execute **ClearML Agent** using Docker versions earlier than 19.03. - * `NVIDIA_VISIBLE_DEVICES` - Limit GPU visibility for the Docker container. - * `ClearML_AGENT_GIT_USER` and `ClearML_AGENT_GIT_PASS` - Pass these credentials to the Docker container at execution. - ---- - -**`disable-monitoring`** - -* Disable logging and monitoring, except for stdout. - ---- - -**`full-monitoring`** - -* Create a full log, including the environment setup log, Task log, and monitoring, as well as stdout. - ---- - -**`git-pass`** - -* Git password for repository access. - ---- - -**`git-user`** - -* Git username for repository access. - ---- - -**`gpus`** -* Specify active GPUs for the daemon to use (docker / virtual environment), Equivalent to setting - `NVIDIA_VISIBLE_DEVICES`. Examples: `--gpus 0` or `--gpu 0,1,2` or `--gpus all` - - ---- - -**`h`, `help`** - -* Get help for this command. - ---- - -**`log-file`** - -* The log file for Task execution output (stdout / stderr) to a text file. - ---- - -**`log-level`** - -* SDK log level. The values are: - - * `DEBUG` - * `INFO` - * `WARN` - * `WARNING` - * `ERROR` - * `CRITICAL` - ---- - -**`O`** - -* Compile optimized pyc code (see python documentation). Repeat for more optimization. - ---- - -**`require-queue`** - -* If the specified task is not queued (in any Queue), the execution will fail. (Used for 3rd party scheduler - integration, e.g. K8s, SLURM, etc.) - ---- - -**`standalone-mode`** - -* Do not use any network connects, assume everything is pre-installed diff --git a/docs/clearml_agent/clearml_agent_list.md b/docs/clearml_agent/clearml_agent_list.md deleted file mode 100644 index 495a9a07..00000000 --- a/docs/clearml_agent/clearml_agent_list.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -title: list ---- - -This reference page provides detailed information about ClearML Agent's `list` subcommand, which you can use to list -information about all active workers. - -## Syntax - - clearml-agent list [-h] \ No newline at end of file diff --git a/docs/clearml_agent/clearml_agent_ref.md b/docs/clearml_agent/clearml_agent_ref.md index 2c663f22..61d4b919 100644 --- a/docs/clearml_agent/clearml_agent_ref.md +++ b/docs/clearml_agent/clearml_agent_ref.md @@ -1,13 +1,141 @@ --- -title: ClearML Agent Reference +title: ClearML Agent CLI --- -The following reference pages provide detailed information about using ClearML Agent: +The following page provides a reference to `clearml-agent`'s CLI commands: +* [build](#build) - Create a worker environment without executing an experiment. +* [config](#config) - List your ClearML Agent configuration data. +* [daemon](#daemon) - Run a worker daemon listening to a queue for Tasks (experiments) to execute. +* [execute](#execute) - Execute an experiment, locally without a queue. +* [list](#list) - List the current workers. + + +## build + +Create a worker environment without executing the experiment. + +```bash +clearml-agent build [-h] --id TASK_ID [--target TARGET] + [--install-globally] + [--docker [DOCKER [DOCKER ...]]] [--force-docker] + [--python-version PYTHON_VERSION] + [--entry-point {reuse_task,clone_task}] [-O] + [--git-user GIT_USER] [--git-pass GIT_PASS] + [--log-level {DEBUG,INFO,WARN,WARNING,ERROR,CRITICAL}] + [--gpus GPUS] [--cpu-only] +``` + +### Parameters + +|Name | Description| Optional | +|---|----|---| +|`--id`| Build a worker environment for this Task ID.|No| +|`--cpu-only`| Disable GPU access for the Docker container.|Yes| +|`--docker`| Run agent in Docker mode. Specify Docker container that a worker will execute at launch. To specify the image name and optional arguments, use one of the following:
  • `--docker ` on the command line
  • `--docker` on the command line, and specify the default image name and arguments in the configuration file.
Environment variable settings for Docker containers:
  • `CLEARML_DOCKER_SKIP_GPUS_FLAG` - Ignore the `--gpus` flag inside the Docker container. This also allows you to execute ClearML Agent using Docker versions earlier than 19.03.
  • `NVIDIA_VISIBLE_DEVICES` - Limit GPU visibility for the Docker container.
  • `CLEARML_AGENT_GIT_USER` and `CLEARML_AGENT_GIT_PASS` - Pass these credentials to the Docker container at execution.
To limit GPU visibility for Docker, set the `NVIDIA_VISIBLE_DEVICES` environment variable.| Yes| +|`--entry-point`| Used in conjunction with `--docker`, indicates how to run the Task specified by `--task-id` on Docker startup. The `--entry-point` options are:
  • `reuse` - Overwrite the existing Task data.
  • `clone_task` - Clone the Task, and execute the cloned Task.
|Yes| +|`--force-docker`| Force using the agent-specified docker image (either explicitly in the `--docker` argument or using the agent's default docker image). If provided, the agent will not use any docker container information stored in the task itself (default `False`)|Yes| +|`--git-pass`| Git password for repository access.|Yes| +|`--git-user`| Git username for repository access.|Yes| +|`--gpus`| Specify the active GPUs for the Docker containers to use. These are the same GPUs set in the `NVIDIA_VISIBLE_DEVICES` environment variable. For example:
  • `--gpus 0`
  • `--gpu 0,1,2`
  • `--gpus all`
|Yes| +|`-h`, `--help`| Get help for this command.|Yes| +|`--install-globally`| Install the required Python packages before creating the virtual environment. Use `agent.package_manager.system_site_packages` to control the installation of the system packages. When `--docker` is used, `--install-globally` is always true.|Yes| +|`--log-level`| SDK log level. The values are:
  • `DEBUG`
  • `INFO`
  • `WARN`
  • `WARNING`
  • `ERROR`
  • `CRITICAL`
|Yes| +|`--python-version`| Virtual environment Python version to use.|Yes| +|`-O`| Compile optimized pyc code (see python documentation). Repeat for more optimization.|Yes| +|`--target`| The target folder for the virtual environment and source code that will be used at launch.|Yes| + +## config +List your ClearML Agent configuration. + +``` +clearml-agent config [-h] +``` + +## daemon + +Run a worker, optionally in a Docker container, listening to a queue. + +```bash +clearml-agent daemon [-h] [--foreground] [--queue QUEUES [QUEUES ...]] [--order-fairness] + [--standalone-mode] [--services-mode [SERVICES_MODE]] + [--child-report-tags CHILD_REPORT_TAGS [CHILD_REPORT_TAGS ...]] + [--create-queue] [--detached] [--stop] [--dynamic-gpus] + [--uptime [UPTIME [UPTIME ...]]] [--downtime [DOWNTIME [DOWNTIME ...]]] + [--status] [--use-owner-token] [-O] + [--git-user GIT_USER] [--git-pass GIT_PASS] + [--log-level {DEBUG,INFO,WARN,WARNING,ERROR,CRITICAL}] + [--gpus GPUS] [--cpu-only] + [--docker [DOCKER [DOCKER ...]]] [--force-current-version] +``` + + +### Parameters + +|Name | Description| Optional | +|---|----|---| +|`--child-report-tags`| List of tags to send with the status reports from the worker that executes a task.|Yes| +|`--cpu-only`| If running in Docker mode (see the `--docker` option), disable GPU access for the Docker container or virtual environment.|Yes| +|`--create-queue`| If the queue name provided with `--queue` does not exist in the server, create it on-the-fly and use it.|Yes| +|`--detached`| Run agent in the background. The `clearml-agent` command returns immediately.|Yes| +|`--docker`| Run in Docker mode. Execute the Task inside a Docker container. To specify the image name and optional arguments, either:
  • Use `--docker ` on the command line, or
  • Use `--docker` on the command line, and specify the default image name and arguments in the configuration file.
Environment variable settings for Docker containers:
  • `CLEARML_DOCKER_SKIP_GPUS_FLAG` - Ignore the `--gpus` flag inside the Docker container. This also allows you to execute ClearML Agent using Docker versions earlier than 19.03.
  • `NVIDIA_VISIBLE_DEVICES` - Limit GPU visibility for the Docker container.
  • `CLEARML_AGENT_GIT_USER` and `CLEARML_AGENT_GIT_PASS` - Pass these credentials to the Docker container at execution.
|Yes| +|`--downtime`| Specify downtime for clearml-agent in ` ` format. For example, use `09-13 TUE` to set Tuesday's downtime to 09-13.

NOTE:
  • This feature is available under the ClearML Enterprise plan
  • Make sure to have only one of uptime / downtime configuration and not both.
|Yes| +|`--dynamic-gpus`| Allow to dynamically allocate GPUs based on queue properties, configure with `--queue =`. For example: `--dynamic-gpus --queue dual_gpus=2 single_gpu=1`

NOTE: This feature is available under the ClearML Enterprise plan|Yes| +|`--force-current-version`| To use your current version of ClearML Agent when running in Docker mode (the `--docker` argument is specified), instead of the latest ClearML Agent version which is automatically installed, specify `force-current-version`.

For example, if your current ClearML Agent version is `0.13.1`, but the latest version of ClearML Agent is `0.13.3`, use `--force-current-version` and your Task will execute in the Docker container with ClearML Agent version `0.13.1`.|Yes| +|`--foreground`| Pipe full log to stdout/stderr. Do not use this option if running in background.|Yes| +|`--git-pass`| Git password for repository access.|Yes| +|`--git-user`| Git username for repository access|Yes| +|`--gpus`| If running in Docker mode (see the `--docker` option), specify the active GPUs for the Docker containers to use. These are the same GPUs set in the `NVIDIA_VISIBLE_DEVICES` environment variable. For example:
  • `--gpus 0`
  • `--gpu 0,1,2`
  • `--gpus all`
|Yes| +|`-h`, `--help`| Get help for this command.|Yes| +|`--log-level`| SDK log level. The values are:
  • `DEBUG`
  • `INFO`
  • `WARN`
  • `WARNING`
  • `ERROR`
  • `CRITICAL`
|Yes| +|`-O`| Compile optimized pyc code (see python documentation). Repeat for more optimization.|Yes| +|`--order-fairness`| Pull from each queue in a round-robin order, instead of priority order.|Yes| +|`--queue`| Specify the queues which the worker is listening to. The values can be any combination of:
  • One or more queue IDs
  • One or more queue names
  • `default` indicating the default queue
|Yes| +|`--services-mode`| Launch multiple long-term docker services. Spin multiple, simultaneous Tasks, each in its own Docker container, on the same machine. Each Task will be registered as a new node in the system, providing tracking and transparency capabilities. Start up and shutdown of each Docker is verified. Use in CPU mode (`--cpu-only`) only.
To limit the number of simultaneous tasks run in services mode, pass the maximum number immediately after the `--services-mode` option (e.g. `--services-mode 5`)|Yes| +|`--standalone-mode`| Do not use any network connects. This assumes all requirements are pre-installed.|Yes| +|`--status`| Print the worker's schedule (uptime properties, server's runtime properties and listening queues)|Yes| +|`--stop`| Terminate a running ClearML Agent, if other arguments are the same. If no additional arguments are provided, agents are terminated in lexicographical order.|Yes| +|`--uptime`| Specify uptime for clearml-agent in ` ` format. For example, use `17-20 TUE` to set Tuesday's uptime to 17-20.

NOTES
  • This feature is available under the ClearML Enterprise plan
  • Make sure to have only one of uptime / downtime configuration and not both.
|Yes| +|`--use-owner-token`| Generate and use the task owner's token for the execution of the task.|Yes| + +## execute + +Build and execute an experiment without a queue. + +```bash +clearml-agent execute [-h] --id TASK_ID [--log-file LOG_FILE] [--disable-monitoring] + [--full-monitoring] [--require-queue] + [--standalone-mode] [--docker [DOCKER [DOCKER ...]]] [--clone] + [-O] [--git-user GIT_USER] [--git-pass GIT_PASS] + [--log-level {DEBUG,INFO,WARN,WARNING,ERROR,CRITICAL}] + [--gpus GPUS] [--cpu-only] +``` + + +### Parameters + +|Name | Description| Optional | +|---|----|---| +|`--id`| The ID of the Task to build|No| +|`--clone`| Clone the Task specified by `--id`, and then execute that cloned Task.|Yes| +|`--cpu-only`| Disable GPU access for the daemon, only use CPU in either docker or virtual environment.|Yes| +|`--docker`| Run in Docker mode. Execute the Task inside a Docker container. To specify the image name and optional arguments, use one of the following:
  • `--docker ` on the command line
  • `--docker` on the command line, and specify the default image name and arguments in the configuration file.
Environment variable settings for Dockers containers:
  • `CLEARML_DOCKER_SKIP_GPUS_FLAG` - Ignore the `--gpus` flag inside the Docker container. This also allows you to execute ClearML Agent using Docker versions earlier than 19.03.
  • `NVIDIA_VISIBLE_DEVICES` - Limit GPU visibility for the Docker container.
  • `CLEARML_AGENT_GIT_USER` and `CLEARML_AGENT_GIT_PASS` - Pass these credentials to the Docker container at execution.
| Yes| +|`--disable-monitoring`| Disable logging and monitoring, except for stdout.|Yes| +|`--full-monitoring`| Create a full log, including the environment setup log, Task log, and monitoring, as well as stdout.|Yes| +|`--git-pass`| Git password for repository access.|Yes| +|`--git-user`| Git username for repository access.|Yes| +|`--gpus`| Specify active GPUs for the daemon to use (docker / virtual environment), Equivalent to setting `NVIDIA_VISIBLE_DEVICES`. For example:
  • `--gpus 0`
  • `--gpu 0,1,2`
  • `--gpus all`
|Yes| +|`-h`, `--help`| Get help for this command.|Yes| +|`--log-file`| The log file for Task execution output (stdout / stderr) to a text file.|Yes| +|`--log-level`| SDK log level. The values are:
  • `DEBUG`
  • `INFO`
  • `WARN`
  • `WARNING`
  • `ERROR`
  • `CRITICAL`
|Yes| +|`-O`| Compile optimized pyc code (see python documentation). Repeat for more optimization.|Yes| +|`--require-queue`| If the specified task is not queued (in any queue), the execution will fail. (Used for 3rd party scheduler integration, e.g. K8s, SLURM, etc.)|Yes| +|`--standalone-mode`| Do not use any network connects, assume everything is pre-installed|Yes| + +## list + +List information about all active workers. + +```bash +clearml-agent list [-h] +``` -* `clearml-agent` CLI commands: - * [build](clearml_agent_build.md) - Create a worker environment, without executing an experiment. - * [config](clearml_agent_config.md) - List your ClearML Agent configuration. - * [daemon](clearml_agent_daemon.md) - Run a worker daemon listening to a queue for Task (experiments) to execute. - * [execute](clearml_agent_execute.md) - Execute an experiment, locally without a queue. - * [list](clearml_agent_list.md) - List the current workers. -* [Environment variables](clearml_agent_env_var.md) diff --git a/docs/guides/clearml_agent/executable_exp_containers.md b/docs/guides/clearml_agent/executable_exp_containers.md index 0d0ac67f..42d8eaee 100644 --- a/docs/guides/clearml_agent/executable_exp_containers.md +++ b/docs/guides/clearml_agent/executable_exp_containers.md @@ -2,7 +2,7 @@ title: Executable Experiment Containers --- -This tutorial demonstrates using [`clearml-agent`](../../clearml_agent.md)’s [`build`](../../clearml_agent/clearml_agent_build.md) +This tutorial demonstrates using [`clearml-agent`](../../clearml_agent.md)’s [`build`](../../clearml_agent/clearml_agent__ref#build) command to package an experiment into an executable container. In this example, you will build a Docker image that, when run, will automatically execute the [keras_tensorboard.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py) script. diff --git a/docs/guides/clearml_agent/exp_environment_containers.md b/docs/guides/clearml_agent/exp_environment_containers.md index 943b6b55..c50dc5f5 100644 --- a/docs/guides/clearml_agent/exp_environment_containers.md +++ b/docs/guides/clearml_agent/exp_environment_containers.md @@ -2,7 +2,7 @@ title: Experiment Environment Containers --- -This tutorial demonstrates using [`clearml-agent`](../../clearml_agent.md)’s [`build`](../../clearml_agent/clearml_agent_build.md) +This tutorial demonstrates using [`clearml-agent`](../../clearml_agent.md)’s [`build`](../../clearml_agent/clearml_agent_ref.md#build) command to build a Docker container replicating the execution environment of an existing task. ClearML Agents can make use of such containers to execute tasks without having to set up their environment every time. diff --git a/docusaurus.config.js b/docusaurus.config.js index e1696b03..07c77198 100644 --- a/docusaurus.config.js +++ b/docusaurus.config.js @@ -79,13 +79,18 @@ module.exports = { to: '/docs/references/sdk/task', }, { - label: 'Hyper-Datasets', - to: '/docs/references/hyperdataset', + label: 'ClearML Agent', + to: '/docs/clearml_agent/clearml_agent_ref', }, { label: 'Server API', to: '/docs/references/api', }, + { + label: 'Hyper-Datasets', + to: '/docs/references/hyperdataset', + }, + { label: 'Release Notes', to: '/docs/release_notes/ver_1_9', diff --git a/sidebars.js b/sidebars.js index e4fad094..b73486fe 100644 --- a/sidebars.js +++ b/sidebars.js @@ -36,19 +36,7 @@ module.exports = { {'ClearML Fundamentals': ['fundamentals/projects', 'fundamentals/task', 'fundamentals/hyperparameters', 'fundamentals/artifacts', 'fundamentals/logger', 'fundamentals/agents_and_queues', 'fundamentals/hpo']}, {'ClearML SDK': ['clearml_sdk/clearml_sdk', 'clearml_sdk/task_sdk', 'clearml_sdk/model_sdk', 'clearml_sdk/apiclient_sdk']}, - {'ClearML Agent': - [ - 'clearml_agent', - {'Reference': - [ - 'clearml_agent/clearml_agent_ref', 'clearml_agent/clearml_agent_build', - 'clearml_agent/clearml_agent_config', 'clearml_agent/clearml_agent_daemon', - 'clearml_agent/clearml_agent_execute', 'clearml_agent/clearml_agent_list', - 'clearml_agent/clearml_agent_env_var' - ] - } - ] - }, + 'clearml_agent', {'ClearML Pipelines':['pipelines/pipelines', {"Building Pipelines": ['pipelines/pipelines_sdk_tasks', 'pipelines/pipelines_sdk_function_decorators'] @@ -208,6 +196,9 @@ module.exports = { 'references/sdk/hpo_parameters_parameterset', ]}, ], + clearmlAgentSidebar: [ + 'clearml_agent/clearml_agent_ref', 'clearml_agent/clearml_agent_env_var' + ], hyperdatasetsSidebar: [ 'hyperdatasets/overview', {'Frames': [