Refactor ClearML Agent pages (#312)

docs/clearml_agent.md

@@ -1,5 +1,5 @@
 ---
-title: ClearML Agent
+title: ClearML Agent Overview
 ---
 
 **ClearML Agent** is a virtual environment and execution manager for DL / ML solutions on GPU machines. It integrates with the **ClearML Python Package** and ClearML Server to provide a full AI cluster solution. <br/>
@@ -33,7 +33,7 @@ automated workflows in one (or both) of the following ways:
 * By using the **ClearML Web UI** (without directly working with code), by enqueuing experiments 
 to the queue that a **ClearML Agent** is listening to.
 
-For more information, see [ClearML Agent Reference](references/clearml_agent_ref.md), 
+For more information, see [ClearML Agent Reference](clearml_agent/clearml_agent_ref.md), 
 and [configuration options](configs/clearml_conf.md#agent-section).
 
 

docs/clearml_agent/clearml_agent_build.md

@@ -0,0 +1,141 @@
+---
+title: build
+---
+
+This reference page provides detailed information about ClearML Agent's `build` subcommand, which you can use to create 
+a worker environment without executing the experiment.
+
+## Syntax
+
+```bash
+clearml-agent build [-h] --id TASK_ID [--target TARGET]
+                    [--install-globally]
+                    [--docker [DOCKER [DOCKER ...]]] [--force-docker] 
+                    [--python-version PYTHON_VERSION]
+                    [--entry-point {reuse_task,clone_task}] [-O]
+                    [--git-user GIT_USER] [--git-pass GIT_PASS]
+                    [--log-level {DEBUG,INFO,WARN,WARNING,ERROR,CRITICAL}]
+                    [--gpus GPUS] [--cpu-only]
+```
+                          
+## Arguments                          
+
+---
+
+**`id`** 
+(*mandatory*)
+
+* Build a worker environment for this Task ID.
+
+---
+
+**`cpu-only`**
+        
+* Disable GPU access for the Docker container.
+
+---
+
+**`docker`**
+
+* Docker mode. A Docker container that a worker will execute at launch. 
+
+    To specify the image name and optional arguments, either:
+    
+    * Use `--docker <image_name> <args>` on the command line, or
+    * Use `--docker` on the command line, and specify the image name and arguments in the configuration file.
+    
+    Environment variable settings for Dockers:
+    
+    * `CLEARML_DOCKER_SKIP_GPUS_FLAG` - Ignore the `gpus` flag inside the Docker container. This also allows you to execute **ClearML Agent** using Docker versions earlier than 19.03.
+    * `NVIDIA_VISIBLE_DEVICES` - Limit GPU visibility for the Docker container. 
+    * `CLEARML_AGENT_GIT_USER` and `CLEARML_AGENT_GIT_PASS` - Pass these credentials to the Docker container at execution.        
+
+    To limit GPU visibility for Docker, set the `NVIDIA_VISIBLE_DEVICES` environment variable.       
+
+---
+
+**`entry-point`** 
+        
+* Used in conjunction with `--docker`, indicates how to run the Task specified by `task-id` on Docker startup.
+     
+    The `entry-point` options are:
+        
+    * `reuse` - Overwrite the existing Task data.
+    * `clone_task` - Clone the Task, and execute the cloned Task.
+        
+---
+
+**`force-docker`**
+
+* Force using the agent-specified docker image (either explicitly in the `--docker` argument or using the agent's 
+  default docker image). If provided, the agent will not use any docker container information stored in the task itself 
+  (default `False`)
+
+
+---
+
+**`git-pass`** 
+        
+* Git password for repository access.
+        
+---
+
+**`git-user`**
+        
+* Git username for repository access.
+        
+---
+
+**`gpus`**
+        
+* Specify the active GPUs for the Docker containers to use. These are the same GPUs set in the `NVIDIA_VISIBLE_DEVICES` environment variable.
+    
+    For example:
+    
+    * `--gpus 0`
+    * `--gpu 0,1,2`
+    * `--gpus all`
+
+---
+
+**`h`, `help`**
+        
+* Get help for this command.
+
+---
+
+**`install-globally`**
+        
+* Install the required Python packages before creating the virtual environment. Use `agent.package_manager.system_site_packages` to control the installation of the system packages. When `--docker` is used, `install-globally` is always true.
+
+---
+
+**`log-level`**
+        
+* SDK log level. The values are:
+       
+    * `DEBUG`
+    * `INFO`
+    * `WARN`
+    * `WARNING`
+    * `ERROR`
+    * `CRITICAL`
+
+---
+
+**`python-version`**
+        
+* Virtual environment Python version to use.
+        
+---
+
+**`O`**
+        
+* Compile optimized pyc code (see python documentation). Repeat for more optimization.
+
+---
+
+**`target`**
+        
+* The target folder for the virtual environment and source code that will be used at launch.
+        

docs/clearml_agent/clearml_agent_config.md

@@ -0,0 +1,12 @@
+---
+title: config
+---
+
+This reference page provides detailed information about ClearML Agent's `config` subcommand, which you can use to
+list your ClearML Agent configuration.
+
+## Syntax
+
+```
+clearml-agent config [-h]
+```

docs/clearml_agent/clearml_agent_daemon.md

@@ -0,0 +1,223 @@
+---
+title: daemon
+---
+
+This reference page provides detailed information about ClearML Agent's `daemon` subcommand, which you can use to
+run a worker, optionally in a Docker container, listening to a queue.
+
+## Syntax
+
+```bash
+clearml-agent daemon [-h] [--foreground] [--queue QUEUES [QUEUES ...]] [--order-fairness] 
+                     [--standalone-mode] [--services-mode [SERVICES_MODE]] 
+                     [--child-report-tags CHILD_REPORT_TAGS [CHILD_REPORT_TAGS ...]]
+                     [--create-queue] [--detached] [--stop] [--dynamic-gpus] 
+                     [--uptime [UPTIME [UPTIME ...]]] [--downtime [DOWNTIME [DOWNTIME ...]]] 
+                     [--status] [--use-owner-token] [-O] 
+                     [--git-user GIT_USER] [--git-pass GIT_PASS] 
+                     [--log-level {DEBUG,INFO,WARN,WARNING,ERROR,CRITICAL}] 
+                     [--gpus GPUS] [--cpu-only] 
+                     [--docker [DOCKER [DOCKER ...]]] [--force-current-version]
+```
+
+                           
+## Arguments
+
+---
+
+**`child-report-tags`**
+
+List of tags to send with the status reports from the worker that executes a task.
+
+---
+
+**`cpu-only`**
+        
+* If running in Docker mode (see the `docker` option), disable GPU access for the Docker container or virtual environment.
+        
+---
+
+**`create-queue`**
+        
+* If the queue name provided does not exist in the server, create and use it.
+
+---
+
+**`detached`**
+        
+* Run agent in the background. The `clearml-agent` command returns immediately.  
+        
+---
+
+**`docker`**
+        
+* Run in Docker mode. Execute the Task inside a Docker container.
+
+    To specify the image name and optional arguments, either:
+    
+    * use `--docker <image_name> <args>` on the command line, or
+    * use `--docker` on the command line, and specify the image name and arguments in the configuration file.
+    
+    Environment variable settings for Dockers:  
+    
+    * `CLEARML_DOCKER_SKIP_GPUS_FLAG` - Ignore the `gpus` flag inside the Docker container. This also allows you to execute **ClearML Agent** using Docker versions earlier than 19.03.
+    * `NVIDIA_VISIBLE_DEVICES` - Limit GPU visibility for the Docker container. 
+    * `ClearML_AGENT_GIT_USER` and `ClearML_AGENT_GIT_PASS` - Pass these credentials to the Docker container at execution.        
+
+---
+
+**`downtime`** 
+
+* Specify downtime for clearml-agent in `<hours> <days>` format.
+
+  For example, use `09-13 TUE` to set Tuesday's downtime to 09-13. 
+  
+:::info
+* This feature is available under the ClearML Enterprise plan 
+* Make sure to have only one of uptime / downtime configuration and not both.
+::: 
+  
+---
+
+**`dynamic-gpus`**
+
+*  Allow to dynamically allocate gpus based on queue properties, configure with `--queue <queue_name>=<num_gpus>`.
+
+  For example: `--dynamic-gpus --queue dual_gpus=2 single_gpu=1`
+
+:::info Enterprise Feature
+This feature is available under the ClearML Enterprise plan
+:::
+
+
+   
+---
+
+**`force-current-version`**
+        
+* To use your current version of **ClearML Agent** when running in Docker mode (the `docker` argument is specified),
+ instead of the latest **ClearML Agent** version which is automatically installed, specify `force-current-version`.
+
+    For example, if your current **ClearML Agent** version is `0.13.1`, but the latest version of **ClearML Agent** is 
+  `0.13.3`, use `--force-current-version` and your Task will execute in the Docker container with **ClearML Agent** version `0.13.1`.
+        
+---
+
+**`foreground`**
+        
+* Pipe full log to stdout/stderr. Do not use this option if running in background.
+
+---
+
+**`git-pass`**
+        
+* Git password for repository access.
+
+---
+
+**`git-user`**
+        
+* Git username for repository access.
+        
+---
+
+**`gpus`**
+        
+* If running in Docker mode (see the `docker` option), specify the active GPUs for the Docker containers to use. 
+        
+    These are the same GPUs set in the `NVIDIA_VISIBLE_DEVICES` environment variable.
+    
+    For example:
+    
+    * `--gpus 0`
+    * `--gpu 0,1,2`
+    * `--gpus all`
+
+---
+
+**`h`, `help`**
+        
+* Get help for this command.
+
+---
+**`log-level`**
+        
+* SDK log level. The values are:
+    
+    * `DEBUG`
+    * `INFO`
+    * `WARN`
+    * `WARNING`
+    * `ERROR`
+    * `CRITICAL`
+
+---
+
+**`O`**
+        
+* Compile optimized pyc code (see python documentation). Repeat for more optimization.
+        
+---
+
+**`order-fairness`**
+        
+* Pull from each queue in a round-robin order, instead of priority order.
+        
+---
+
+**`queue`**
+        
+* Specify the queues which the worker is listening to. The values can be any combination of:
+
+    * One or more queue IDs.
+    * One or more queue names.
+    * `default` indicating the default queue.
+
+---
+
+**`services-mode`** <a class="tr_top_negative" name="services_mode"></a>
+        
+* Launch multiple long-term docker services. Spin multiple, simultaneous Tasks, each in its own Docker container, on the same machine. Each Task will be registered 
+  as a new node in the system, providing tracking and transparency capabilities. Start up and shutdown of each Docker is 
+  verified. Use in CPU mode (`--cpu-only`), only. 
+
+* To limit the number of simultaneous tasks run in services mode, pass the maximum number immediately after the 
+`--services-mode` option (e.g. `--services-mode 5`)
+   
+        
+---
+
+**`standalone-mode`**
+        
+* Do not use any network connects. This assumes all requirements are pre-installed.            
+  
+---
+
+**`status`**
+
+* Print the worker's schedule (uptime properties, server's runtime properties and listening queues)
+ 
+
+---
+
+**`stop`**
+        
+* Terminate a running ClearML Agent, if other arguments are the same. If no additional arguments are provided, agents are 
+  terminated in lexicographical order.  
+
+---
+
+**`uptime`** 
+* Specify uptime for clearml-agent in `<hours> <days>` format. for example, use `17-20 TUE` to set Tuesday's
+  uptime to 17-20
+  
+:::info
+* This feature is available under the ClearML Enterprise plan
+* Make sure to have only one of uptime / downtime configuration and not both.
+::: 
+  
+---
+
+**`use-owner-token`**
+
+Generate and use the task owner's token for the execution of the task.

docs/clearml_agent/clearml_agent_env_var.md

@@ -0,0 +1,31 @@
+---
+title: Environment Variables
+---
+
+This page lists the available environment variables for configuring ClearML Agent. 
+
+:::info
+ClearML's environment variables override the clearml.conf file, SDK, and [configuration vault](../webapp/webapp_profile.md#configuration-vault), 
+but can be overridden by command-line arguments. 
+:::
+
+|Name|Description|
+|---|---|
+|**CLEARML_DOCKER_IMAGE** | Default ClearML Agent docker image|
+|**CLEARML_WORKER_NAME** | Sets the Worker's name|
+|**CLEARML_WORKER_ID** | Sets the Worker ID|
+|**CLEARML_CUDA_VERSION** | Sets the CUDA version to be used|
+|**CLEARML_CUDNN_VERSION** | Sets the CUDNN version to be used|
+|**CLEARML_CPU_ONLY** | Force CPU only mode|
+|**CLEARML_DOCKER_SKIP_GPUS_FLAG**| Skips the GPUs flag (support for docker V18|
+|**CLEARML_AGENT_DISABLE_SSH_MOUNT**| Disables the auto `.ssh` mount into the docker|
+|**CLEARML_AGENT_GIT_USER** | Sets the Git user for ClearML Agent|
+|**CLEARML_AGENT_GIT_PASS** | Sets the Git password for ClearML Agent|
+|**CLEARML_AGENT_GIT_HOST** | Sets Git host (only sending login to this host)|
+|**CLEARML_AGENT_EXEC_USER**| User for Agent executing tasks (root by default)|
+|**CLEARML_AGENT_EXTRA_DOCKER_ARGS**| Overrides extra docker args configuration |
+|**CLEARML_AGENT_EXTRA_PYTHON_PATH**| Sets extra python path|
+|**CLEARML_AGENT_INITIAL_CONNECT_RETRY_OVERRIDE**| Overrides initial server connection behavior (true by default), allows explicit number to specify number of connect retries) | 
+|**CLEARML_AGENT_K8S_HOST_MOUNT / CLEARML_AGENT_DOCKER_HOST_MOUNT**| Specifies Agent's mount point for Docker / K8s|
+|**CLEARML_AGENT_SKIP_PIP_VENV_INSTALL**| Skips Python virtual env installation on execute and provides a custom venv binary |
+|**CLEARML_AGENT_VENV_CACHE_PATH**|Overrides venv cache folder configuration|

docs/clearml_agent/clearml_agent_execute.md

@@ -0,0 +1,130 @@
+---
+title: execute
+---
+
+This reference page provides detailed information about ClearML Agent's `execute` subcommand, which you can use to 
+build and execute an experiment without a queue.
+
+## Syntax
+
+```bash
+clearml-agent execute [-h] --id TASK_ID [--log-file LOG_FILE] [--disable-monitoring] 
+                      [--full-monitoring] [--require-queue]
+                      [--standalone-mode] [--docker [DOCKER [DOCKER ...]]] [--clone] 
+                      [-O] [--git-user GIT_USER] [--git-pass GIT_PASS] 
+                      [--log-level {DEBUG,INFO,WARN,WARNING,ERROR,CRITICAL}] 
+                      [--gpus GPUS] [--cpu-only]
+```
+
+                            
+## Arguments                            
+
+---
+
+**`id`** (*mandatory*)
+        
+* The ID of the Task to build.
+        
+---
+
+**`clone`**
+        
+* Clone the Task specified by `id`, and then execute that cloned Task.
+        
+---
+
+**`cpu-only`**
+* Disable GPU access for the daemon, only use CPU in either docker or virtual environment.
+
+
+---
+
+**`docker`**
+        
+* Run in Docker mode. Execute the Task inside a Docker container.
+
+    To specify the image name and optional arguments, either:
+
+    * use `--docker <image_name> <args>` on the command line, or
+    * use `--docker` on the command line, and specify the default image name and arguments in the configuration file.
+
+    Environment variable settings for Dockers containers:
+    
+    * `ClearML_DOCKER_SKIP_GPUS_FLAG` - Ignore the `gpus` flag inside the Docker container. This also allows you to execute **ClearML Agent** using Docker versions earlier than 19.03.
+    * `NVIDIA_VISIBLE_DEVICES` - Limit GPU visibility for the Docker container. 
+    * `ClearML_AGENT_GIT_USER` and `ClearML_AGENT_GIT_PASS` - Pass these credentials to the Docker container at execution.        
+
+---
+
+**`disable-monitoring`**
+        
+* Disable logging and monitoring, except for stdout.
+        
+---
+
+**`full-monitoring`**
+        
+* Create a full log, including the environment setup log, Task log, and monitoring, as well as stdout.
+        
+---
+
+**`git-pass`** 
+        
+* Git password for repository access.
+        
+---
+
+**`git-user`**
+        
+* Git username for repository access.
+        
+---
+
+**`gpus`** 
+* Specify active GPUs for the daemon to use (docker / virtual environment), Equivalent to setting
+  `NVIDIA_VISIBLE_DEVICES`. Examples: `--gpus 0` or `--gpu 0,1,2` or `--gpus all`
+
+
+---
+
+**`h`, `help`**
+        
+* Get help for this command.
+        
+---
+
+**`log-file`**
+        
+* The log file for Task execution output (stdout / stderr) to a text file.
+        
+---
+
+**`log-level`**
+        
+* SDK log level. The values are:
+    
+    * `DEBUG`
+    * `INFO`
+    * `WARN`
+    * `WARNING`
+    * `ERROR`
+    * `CRITICAL`
+
+---
+
+**`O`**
+        
+* Compile optimized pyc code (see python documentation). Repeat for more optimization.
+
+---
+
+**`require-queue`** 
+
+* If the specified task is not queued (in any Queue), the execution will fail. (Used for 3rd party scheduler
+  integration, e.g. K8s, SLURM, etc.)
+ 
+---
+
+**`standalone-mode`** 
+
+* Do not use any network connects, assume everything is pre-installed

docs/clearml_agent/clearml_agent_list.md

@@ -0,0 +1,10 @@
+---
+title: list
+---
+
+This reference page provides detailed information about ClearML Agent's `list` subcommand, which you can use to list 
+information about all active workers.
+
+## Syntax
+
+    clearml-agent list [-h]

docs/clearml_agent/clearml_agent_ref.md

@@ -0,0 +1,13 @@
+---
+title: ClearML Agent Reference
+---
+
+The following reference pages provide detailed information about using ClearML Agent: 
+
+* `clearml-agent` CLI commands:
+    * [build](clearml_agent_build.md) - Create a worker environment, without executing an experiment.
+    * [config](clearml_agent_config.md) - List your ClearML Agent configuration.
+    * [daemon](clearml_agent_daemon.md) - Run a worker daemon listening to a queue for Task (experiments) to execute.
+    * [execute](clearml_agent_execute.md) - Execute an experiment, locally without a queue.
+    * [list](clearml_agent_list.md) - List the current workers.
+* [Environment variables](clearml_agent_env_var.md)

docs/configs/configuring_clearml.md

@@ -17,7 +17,7 @@ from the vault are applied on top of the configuration specified in `clearml.con
 
 The different ClearML configuration methods take precedence as summarized in the following list (higher ordered methods 
 override the lower ones):
-1. Command-line arguments (e.g. [clearml-task](../apps/clearml_task.md#command-line-options), [clearml-agent](../references/clearml_agent_ref.md), 
+1. Command-line arguments (e.g. [clearml-task](../apps/clearml_task.md#command-line-options), [clearml-agent](../clearml_agent/clearml_agent_ref.md), 
    [clearml-session](../apps/clearml_session.md#command-line-options), [clearml-data](../clearml_data/clearml_data_cli.md) 
    arguments) 
 1. Environment variables

docs/configs/env_vars.md

@@ -2,6 +2,9 @@
 title: Environment Variables
 ---
 
+This page lists the available environment variables for configuring ClearML. See [here](../clearml_agent/clearml_agent_env_var.md)
+for environment variables to configure ClearML Agent. 
+
 :::info
 ClearML's environment variables override the clearml.conf file, SDK, and [configuration vault](../webapp/webapp_profile.md#configuration-vault), 
 but can be overridden by command-line arguments. 
@@ -42,24 +45,3 @@ Overrides Repository Auto-logging
 |**CLEARML_OFFLINE_MODE** | Sets Offline mode|
 |**CLEARML_NO_DEFAULT_SERVER** | Disables sending information to demo server when no HOST server is set|
 
-## ClearML Agent Variables
-|Name|Description|
-|---|---|
-|**CLEARML_DOCKER_IMAGE** | Default ClearML Agent docker image|
-|**CLEARML_WORKER_NAME** | Sets the Worker's name|
-|**CLEARML_WORKER_ID** | Sets the Worker ID|
-|**CLEARML_CUDA_VERSION** | Sets the CUDA version to be used|
-|**CLEARML_CUDNN_VERSION** | Sets the CUDNN version to be used|
-|**CLEARML_CPU_ONLY** | Force CPU only mode|
-|**CLEARML_DOCKER_SKIP_GPUS_FLAG**| Skips the GPUs flag (support for docker V18|
-|**CLEARML_AGENT_DISABLE_SSH_MOUNT**| Disables the auto `.ssh` mount into the docker|
-|**CLEARML_AGENT_GIT_USER** | Sets the Git user for ClearML Agent|
-|**CLEARML_AGENT_GIT_PASS** | Sets the Git password for ClearML Agent|
-|**CLEARML_AGENT_GIT_HOST** | Sets Git host (only sending login to this host)|
-|**CLEARML_AGENT_EXEC_USER**| User for Agent executing tasks (root by default)|
-|**CLEARML_AGENT_EXTRA_DOCKER_ARGS**| Overrides extra docker args configuration |
-|**CLEARML_AGENT_EXTRA_PYTHON_PATH**| Sets extra python path|
-|**CLEARML_AGENT_INITIAL_CONNECT_RETRY_OVERRIDE**| Overrides initial server connection behavior (true by default), allows explicit number to specify number of connect retries) | 
-|**CLEARML_AGENT_K8S_HOST_MOUNT / CLEARML_AGENT_DOCKER_HOST_MOUNT**| Specifies Agent's mount point for Docker / K8s|
-|**CLEARML_AGENT_SKIP_PIP_VENV_INSTALL**| Skips Python virtual env installation on execute and provides a custom venv binary |
-|**CLEARML_AGENT_VENV_CACHE_PATH**|Overrides venv cache folder configuration|

docs/deploying_clearml/clearml_config_for_clearml_server.md

@@ -33,7 +33,7 @@ by setting [configuration options](../configs/clearml_conf.md).
         CLEARML_CONFIG_FILE = MyOtherClearML.conf
 
    For more information about running experiments inside Docker containers, see [ClearML Agent Execution](../clearml_agent.md#execution)
-   and [ClearML Agent Reference](../references/clearml_agent_ref.md).
+   and [ClearML Agent Reference](../clearml_agent/clearml_agent_ref.md).
     
    </div>
    </details>

docs/guides/ide/google_colab.md

@@ -53,7 +53,7 @@ and running, users can send Tasks to be executed on the Google Colab's HW.
    !clearml-agent daemon --queue default
    ```
    
-   For additional options for running `clearml-agent`, see the [clearml-agent reference](../../references/clearml_agent_ref.md).  
+   For additional options for running `clearml-agent`, see the [clearml-agent reference](../../clearml_agent/clearml_agent_ref.md).  
    
    After cell 4 is executed, the worker should now appear in the [**Workers & Queues**](../../webapp/webapp_workers_queues.md) 
    page of your server. Clone experiments and enqueue them to your hearts content! The `clearml-agent` will fetch 

docs/references/clearml_agent_ref.md

@@ -1,516 +0,0 @@
----
-title: ClearML Agent Reference
----
-This reference page provides detailed information about **ClearML Agent** commands, including:
-
-* [build](#build) - Create a worker environment, without executing an experiment.
-* [config](#config) - List your **ClearML Agent** configuration.
-* [daemon](#daemon) - Run a worker daemon listening to a queue for Task (experiments) to execute.
-* [execute](#execute) - Execute an experiment, locally without a queue.
-* [list](#list) - List the current workers.
-
-## build
-
-Use the `build` subcommand to create a worker environment without executing the experiment.
-
-### Syntax 
-
-```bash
-clearml-agent build [-h] --id TASK_ID [--target TARGET]
-                    [--install-globally]
-                    [--docker [DOCKER [DOCKER ...]]] [--force-docker] 
-                    [--python-version PYTHON_VERSION]
-                    [--entry-point {reuse_task,clone_task}] [-O]
-                    [--git-user GIT_USER] [--git-pass GIT_PASS]
-                    [--log-level {DEBUG,INFO,WARN,WARNING,ERROR,CRITICAL}]
-                    [--gpus GPUS] [--cpu-only]
-```
-                          
-### Arguments                          
-
----
-
-**`id`** 
-(*mandatory*)
-
-* Build a worker environment for this Task ID.
-
----
-
-**`cpu-only`**
-        
-* Disable GPU access for the Docker container.
-
----
-
-**`docker`**
-
-* Docker mode. A Docker container that a worker will execute at launch. 
-
-    To specify the image name and optional arguments, either:
-    
-    * Use `--docker <image_name> <args>` on the command line, or
-    * Use `--docker` on the command line, and specify the image name and arguments in the configuration file.
-    
-    Environment variable settings for Dockers:
-    
-    * `CLEARML_DOCKER_SKIP_GPUS_FLAG` - Ignore the `gpus` flag inside the Docker container. This also allows you to execute **ClearML Agent** using Docker versions earlier than 19.03.
-    * `NVIDIA_VISIBLE_DEVICES` - Limit GPU visibility for the Docker container. 
-    * `CLEARML_AGENT_GIT_USER` and `CLEARML_AGENT_GIT_PASS` - Pass these credentials to the Docker container at execution.        
-
-    To limit GPU visibility for Docker, set the `NVIDIA_VISIBLE_DEVICES` environment variable.       
-
----
-
-**`entry-point`** 
-        
-* Used in conjunction with `--docker`, indicates how to run the Task specified by `task-id` on Docker startup.
-     
-    The `entry-point` options are:
-        
-    * `reuse` - Overwrite the existing Task data.
-    * `clone_task` - Clone the Task, and execute the cloned Task.
-        
----
-
-**`force-docker`**
-
-* Force using the agent-specified docker image (either explicitly in the `--docker` argument or using the agent's 
-  default docker image). If provided, the agent will not use any docker container information stored in the task itself 
-  (default `False`)
-
-
----
-
-**`git-pass`** 
-        
-* Git password for repository access.
-        
----
-
-**`git-user`**
-        
-* Git username for repository access.
-        
----
-
-**`gpus`**
-        
-* Specify the active GPUs for the Docker containers to use. These are the same GPUs set in the `NVIDIA_VISIBLE_DEVICES` environment variable.
-    
-    For example:
-    
-    * `--gpus 0`
-    * `--gpu 0,1,2`
-    * `--gpus all`
-
----
-
-**`h`, `help`**
-        
-* Get help for this command.
-
----
-
-**`install-globally`**
-        
-* Install the required Python packages before creating the virtual environment. Use `agent.package_manager.system_site_packages` to control the installation of the system packages. When `--docker` is used, `install-globally` is always true.
-
----
-
-**`log-level`**
-        
-* SDK log level. The values are:
-       
-    * `DEBUG`
-    * `INFO`
-    * `WARN`
-    * `WARNING`
-    * `ERROR`
-    * `CRITICAL`
-
----
-
-**`python-version`**
-        
-* Virtual environment Python version to use.
-        
----
-
-**`O`**
-        
-* Compile optimized pyc code (see python documentation). Repeat for more optimization.
-
----
-
-**`target`**
-        
-* The target folder for the virtual environment and source code that will be used at launch.
-        
-
-## config
-
-Use the `config` subcommand to list your **ClearML Agent** configuration.
-
-### Syntax
-
-    clearml-agent config [-h]
-
-
-## daemon
-
-Use the `daemon` subcommand to run a worker, optionally in a Docker container, listening to a queue.
-
-### Syntax
-
-```bash
-clearml-agent daemon [-h] [--foreground] [--queue QUEUES [QUEUES ...]] [--order-fairness] 
-                     [--standalone-mode] [--services-mode [SERVICES_MODE]] 
-                     [--child-report-tags CHILD_REPORT_TAGS [CHILD_REPORT_TAGS ...]]
-                     [--create-queue] [--detached] [--stop] [--dynamic-gpus] 
-                     [--uptime [UPTIME [UPTIME ...]]] [--downtime [DOWNTIME [DOWNTIME ...]]] 
-                     [--status] [--use-owner-token] [-O] 
-                     [--git-user GIT_USER] [--git-pass GIT_PASS] 
-                     [--log-level {DEBUG,INFO,WARN,WARNING,ERROR,CRITICAL}] 
-                     [--gpus GPUS] [--cpu-only] 
-                     [--docker [DOCKER [DOCKER ...]]] [--force-current-version]
-```
-
-                           
-### Arguments
-
----
-
-**`child-report-tags`**
-
-List of tags to send with the status reports from the worker that executes a task.
-
----
-
-**`cpu-only`**
-        
-* If running in Docker mode (see the `docker` option), disable GPU access for the Docker container or virtual environment.
-        
----
-
-**`create-queue`**
-        
-* If the queue name provided does not exist in the server, create and use it.
-
----
-
-**`detached`**
-        
-* Run agent in the background. The `clearml-agent` command returns immediately.  
-        
----
-
-**`docker`**
-        
-* Run in Docker mode. Execute the Task inside a Docker container.
-
-    To specify the image name and optional arguments, either:
-    
-    * use `--docker <image_name> <args>` on the command line, or
-    * use `--docker` on the command line, and specify the image name and arguments in the configuration file.
-    
-    Environment variable settings for Dockers:  
-    
-    * `CLEARML_DOCKER_SKIP_GPUS_FLAG` - Ignore the `gpus` flag inside the Docker container. This also allows you to execute **ClearML Agent** using Docker versions earlier than 19.03.
-    * `NVIDIA_VISIBLE_DEVICES` - Limit GPU visibility for the Docker container. 
-    * `ClearML_AGENT_GIT_USER` and `ClearML_AGENT_GIT_PASS` - Pass these credentials to the Docker container at execution.        
-
----
-
-**`downtime`** 
-
-* Specify downtime for clearml-agent in `<hours> <days>` format.
-
-  For example, use `09-13 TUE` to set Tuesday's downtime to 09-13. 
-  
-:::info
-* This feature is available under the ClearML Enterprise plan 
-* Make sure to have only one of uptime / downtime configuration and not both.
-::: 
-  
----
-
-**`dynamic-gpus`**
-
-*  Allow to dynamically allocate gpus based on queue properties, configure with `--queue <queue_name>=<num_gpus>`.
-
-  For example: `--dynamic-gpus --queue dual_gpus=2 single_gpu=1`
-
-:::info Enterprise Feature
-This feature is available under the ClearML Enterprise plan
-:::
-
-
-   
----
-
-**`force-current-version`**
-        
-* To use your current version of **ClearML Agent** when running in Docker mode (the `docker` argument is specified),
- instead of the latest **ClearML Agent** version which is automatically installed, specify `force-current-version`.
-
-    For example, if your current **ClearML Agent** version is `0.13.1`, but the latest version of **ClearML Agent** is 
-  `0.13.3`, use `--force-current-version` and your Task will execute in the Docker container with **ClearML Agent** version `0.13.1`.
-        
----
-
-**`foreground`**
-        
-* Pipe full log to stdout/stderr. Do not use this option if running in background.
-
----
-
-**`git-pass`**
-        
-* Git password for repository access.
-
----
-
-**`git-user`**
-        
-* Git username for repository access.
-        
----
-
-**`gpus`**
-        
-* If running in Docker mode (see the `docker` option), specify the active GPUs for the Docker containers to use. 
-        
-    These are the same GPUs set in the `NVIDIA_VISIBLE_DEVICES` environment variable.
-    
-    For example:
-    
-    * `--gpus 0`
-    * `--gpu 0,1,2`
-    * `--gpus all`
-
----
-
-**`h`, `help`**
-        
-* Get help for this command.
-
----
-**`log-level`**
-        
-* SDK log level. The values are:
-    
-    * `DEBUG`
-    * `INFO`
-    * `WARN`
-    * `WARNING`
-    * `ERROR`
-    * `CRITICAL`
-
----
-
-**`O`**
-        
-* Compile optimized pyc code (see python documentation). Repeat for more optimization.
-        
----
-
-**`order-fairness`**
-        
-* Pull from each queue in a round-robin order, instead of priority order.
-        
----
-
-**`queue`**
-        
-* Specify the queues which the worker is listening to. The values can be any combination of:
-
-    * One or more queue IDs.
-    * One or more queue names.
-    * `default` indicating the default queue.
-
----
-
-**`services-mode`** <a class="tr_top_negative" name="services_mode"></a>
-        
-* Launch multiple long-term docker services. Spin multiple, simultaneous Tasks, each in its own Docker container, on the same machine. Each Task will be registered 
-  as a new node in the system, providing tracking and transparency capabilities. Start up and shutdown of each Docker is 
-  verified. Use in CPU mode (`--cpu-only`), only. 
-
-* To limit the number of simultaneous tasks run in services mode, pass the maximum number immediately after the 
-`--services-mode` option (e.g. `--services-mode 5`)
-   
-        
----
-
-**`standalone-mode`**
-        
-* Do not use any network connects. This assumes all requirements are pre-installed.            
-  
----
-
-**`status`**
-
-* Print the worker's schedule (uptime properties, server's runtime properties and listening queues)
- 
-
----
-
-**`stop`**
-        
-* Terminate a running ClearML Agent, if other arguments are the same. If no additional arguments are provided, agents are 
-  terminated in lexicographical order.  
-
----
-
-**`uptime`** 
-* Specify uptime for clearml-agent in `<hours> <days>` format. for example, use `17-20 TUE` to set Tuesday's
-  uptime to 17-20
-  
-:::info
-* This feature is available under the ClearML Enterprise plan
-* Make sure to have only one of uptime / downtime configuration and not both.
-::: 
-  
----
-
-**`use-owner-token`**
-
-Generate and use the task owner's token for the execution of the task.
-  
-## execute
-
-Use the `execute` subcommand to build and execute an experiment without a queue.
-
-### Syntax
-
-```bash
-clearml-agent execute [-h] --id TASK_ID [--log-file LOG_FILE] [--disable-monitoring] 
-                      [--full-monitoring] [--require-queue]
-                      [--standalone-mode] [--docker [DOCKER [DOCKER ...]]] [--clone] 
-                      [-O] [--git-user GIT_USER] [--git-pass GIT_PASS] 
-                      [--log-level {DEBUG,INFO,WARN,WARNING,ERROR,CRITICAL}] 
-                      [--gpus GPUS] [--cpu-only]
-```
-
-                            
-### Arguments                            
-
----
-
-**`id`** (*mandatory*)
-        
-* The ID of the Task to build.
-        
----
-
-**`clone`**
-        
-* Clone the Task specified by `id`, and then execute that cloned Task.
-        
----
-
-**`cpu-only`**
-* Disable GPU access for the daemon, only use CPU in either docker or virtual environment.
-
-
----
-
-**`docker`**
-        
-* Run in Docker mode. Execute the Task inside a Docker container.
-
-    To specify the image name and optional arguments, either:
-
-    * use `--docker <image_name> <args>` on the command line, or
-    * use `--docker` on the command line, and specify the default image name and arguments in the configuration file.
-
-    Environment variable settings for Dockers containers:
-    
-    * `ClearML_DOCKER_SKIP_GPUS_FLAG` - Ignore the `gpus` flag inside the Docker container. This also allows you to execute **ClearML Agent** using Docker versions earlier than 19.03.
-    * `NVIDIA_VISIBLE_DEVICES` - Limit GPU visibility for the Docker container. 
-    * `ClearML_AGENT_GIT_USER` and `ClearML_AGENT_GIT_PASS` - Pass these credentials to the Docker container at execution.        
-
----
-
-**`disable-monitoring`**
-        
-* Disable logging and monitoring, except for stdout.
-        
----
-
-**`full-monitoring`**
-        
-* Create a full log, including the environment setup log, Task log, and monitoring, as well as stdout.
-        
----
-
-**`git-pass`** 
-        
-* Git password for repository access.
-        
----
-
-**`git-user`**
-        
-* Git username for repository access.
-        
----
-
-**`gpus`** 
-* Specify active GPUs for the daemon to use (docker / virtual environment), Equivalent to setting
-  `NVIDIA_VISIBLE_DEVICES`. Examples: `--gpus 0` or `--gpu 0,1,2` or `--gpus all`
-
-
----
-
-**`h`, `help`**
-        
-* Get help for this command.
-        
----
-
-**`log-file`**
-        
-* The log file for Task execution output (stdout / stderr) to a text file.
-        
----
-
-**`log-level`**
-        
-* SDK log level. The values are:
-    
-    * `DEBUG`
-    * `INFO`
-    * `WARN`
-    * `WARNING`
-    * `ERROR`
-    * `CRITICAL`
-
----
-
-**`O`**
-        
-* Compile optimized pyc code (see python documentation). Repeat for more optimization.
-
----
-
-**`require-queue`** 
-
-* If the specified task is not queued (in any Queue), the execution will fail. (Used for 3rd party scheduler
-  integration, e.g. K8s, SLURM, etc.)
- 
----
-
-**`standalone-mode`** 
-
-* Do not use any network connects, assume everything is pre-installed
-
-        
-## list
-
-Use the `list` subcommand to list information about all workers
-
-### Syntax
-
-    clearml-agent list [-h]

docusaurus.config.js

@@ -88,10 +88,6 @@ module.exports = {
               label: 'Server API',
               to: '/docs/references/api/index',
             },
-            {
-              label: 'Agent Reference',
-              to: '/docs/references/clearml_agent_ref',
-            },
           ],
         },
         {

sidebars.js

@@ -16,7 +16,19 @@ module.exports = {
         {'ClearML Fundamentals': ['fundamentals/projects', 'fundamentals/task', 'fundamentals/hyperparameters', 'fundamentals/artifacts', 'fundamentals/logger', 'fundamentals/agents_and_queues',
             'fundamentals/hpo']},
         {'ClearML SDK': ['clearml_sdk/clearml_sdk', 'clearml_sdk/task_sdk', 'clearml_sdk/model_sdk', 'clearml_sdk/apiclient_sdk']},
-        'clearml_agent',
+        {'ClearML Agent':
+                [
+                    'clearml_agent',
+                    {'Reference':
+                            [
+                                'clearml_agent/clearml_agent_ref', 'clearml_agent/clearml_agent_build',
+                                'clearml_agent/clearml_agent_config', 'clearml_agent/clearml_agent_daemon',
+                                'clearml_agent/clearml_agent_execute', 'clearml_agent/clearml_agent_list',
+                                'clearml_agent/clearml_agent_env_var'
+                            ]
+                    }
+                ]
+        },
         {'ClearML Pipelines':['pipelines/pipelines',
                 {"Building Pipelines":
                         ['pipelines/pipelines_sdk_tasks', 'pipelines/pipelines_sdk_function_decorators']