From a7954101b4deed727105a8f023cd010c09829cff Mon Sep 17 00:00:00 2001 From: pollfly <75068813+pollfly@users.noreply.github.com> Date: Wed, 13 Sep 2023 10:56:47 +0300 Subject: [PATCH] Add ClearML Agent config options (#669) --- docs/clearml_agent/clearml_agent_env_var.md | 4 +++ docs/configs/clearml_conf.md | 27 +++++++++++++++++++++ 2 files changed, 31 insertions(+) diff --git a/docs/clearml_agent/clearml_agent_env_var.md b/docs/clearml_agent/clearml_agent_env_var.md index 45bf73df..b557ccf1 100644 --- a/docs/clearml_agent/clearml_agent_env_var.md +++ b/docs/clearml_agent/clearml_agent_env_var.md @@ -27,10 +27,14 @@ but can be overridden by command-line arguments. |**CLEARML_AGENT_GIT_HOST** | Sets Git host (only sending login to this host) | |**CLEARML_AGENT_EXEC_USER** | User for Agent executing tasks (root by default) | |**CLEARML_AGENT_EXTRA_DOCKER_ARGS** | Overrides extra docker args configuration | +|**CLEARML_AGENT_EXTRA_DOCKER_LABELS** | List of labels to add to docker container. See [Docker documentation](https://docs.docker.com/config/labels-custom-metadata/). | +|**CLEARML_EXTRA_PIP_INSTALL_FLAGS**| List of additional flags to use when the agent install packages. For example: `["--use-deprecated=legacy-resolver", ]`| |**CLEARML_AGENT_EXTRA_PYTHON_PATH** | Sets extra python path | |**CLEARML_AGENT_INITIAL_CONNECT_RETRY_OVERRIDE** | Overrides initial server connection behavior (true by default), allows explicit number to specify number of connect retries) | +|**CLEARML_AGENT_NO_UPDATE** | Boolean. Set to `true` to skip agent update in the k8s pod container before the agent executes the task | |**CLEARML_AGENT_K8S_HOST_MOUNT / CLEARML_AGENT_DOCKER_HOST_MOUNT** | Specifies Agent's mount point for Docker / K8s | |**CLEARML_K8S_GLUE_START_AGENT_SCRIPT_PATH** | Provide an alternate path to place the agent startup script generated inside a k8s task pod (instead of the default `~/~/__start_agent__.sh`) | +|**CLEARML_AGENT_PACKAGE_PYTORCH_RESOLVE**|Sets the PyTorch resolving mode. The options are: | |**CLEARML_AGENT_DEBUG_INFO** | Provide additional debug information for a specific context (currently only the `docker` value is supported) | |**CLEARML_AGENT_CHILD_AGENTS_COUNT_CMD** | Provide an alternate bash command to list child agents while working in services mode | |**CLEARML_AGENT_SKIP_PIP_VENV_INSTALL** | Skips Python virtual env installation on execute and provides a custom venv binary | diff --git a/docs/configs/clearml_conf.md b/docs/configs/clearml_conf.md index 9c00ad82..f16dd9c2 100644 --- a/docs/configs/clearml_conf.md +++ b/docs/configs/clearml_conf.md @@ -110,6 +110,16 @@ in either case). --- +**`agent.docker_allow_host_environ`** (*bool*) + +* Set to `true` to allow passing host environments into docker container with Task's docker container arguments. For example: `"-e HOST_NAME=$HOST_NAME"`. + +:::caution +Use with care! This might introduce security risks by allowing access to keys/secret on the host machine. +::: + +--- + **`agent.docker_apt_cache`** (*string*) * The apt (Linux package tool) cache folder for mapping Ubuntu package caching into Docker. @@ -460,6 +470,12 @@ ___ * A list of URLs for additional artifact repositories when installing Python packages. +--- + +**`agent.package_manager.extra_pip_install_flags`** (*[string]*) + +* A list of additional flags to use when the agent install packages. For example: `["--use-deprecated=legacy-resolver", ]` + --- **`agent.package_manager.force_upgrade`** (*bool*) @@ -518,6 +534,17 @@ ___ --- +**`agent.package_manager.pytorch_resolve`** (*str*) + +* Set the PyTorch resolving mode. The options are: + * `pip` (default) - Sets extra index based on cuda and lets pip resolve + * `none` - No resolving. Install PyTorch like any other package + * `direct` - Resolve a direct link to the PyTorch wheel by parsing the pytorch.org pip repository and matching the + automatically detected cuda version with the required PyTorch wheel. If the exact cuda version is not found for the + required PyTorch wheel, it will try a lower cuda version until a match is found + +--- + **`agent.package_manager.system_site_packages`** (*bool*)