ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-05-16 02:16:37 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Fix CLEARML_LOG_ENVIRONMENT info (#887)

Browse Source
This commit is contained in:
pollfly 2024-08-04 18:43:02 +03:00 committed by GitHub
parent e08cb72fac
commit b377e940aa
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
6 changed files with 44 additions and 13 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/configs/clearml_conf.md
View File

@ -1044,7 +1044,7 @@ and limitations on bucket naming.
Multiple selected variables are supported including the suffix "\*". For example: "AWS\_\*" will log any OS environment Multiple selected variables are supported including the suffix "\*". For example: "AWS\_\*" will log any OS environment
variable starting with "AWS\_". Example: `log_os_environments: ["AWS_*", "CUDA_VERSION"]` variable starting with "AWS\_". Example: `log_os_environments: ["AWS_*", "CUDA_VERSION"]`
* This value can be overwritten with OS environment variable `CLEARML_LOG_ENVIRONMENT="[AWS_*, CUDA_VERSION]"`. * This value can be overwritten with OS environment variable `CLEARML_LOG_ENVIRONMENT=AWS_*,CUDA_VERSION`.
--- ---

2
docs/configs/env_vars.md
View File

@ -17,7 +17,7 @@ but can be overridden by command-line arguments.
### General ### General
|Name| Description | |Name| Description |
|---|--------------------------------------------------------------------------------| |---|--------------------------------------------------------------------------------|
|**CLEARML_LOG_ENVIRONMENT** | List of Environment variable names. These environment variables will be logged in the ClearML task's configuration hyperparameters `Environment` section. When executed by a ClearML agent, these values will be set in the task's execution environment. | |**CLEARML_LOG_ENVIRONMENT** | List of Environment variable names. These environment variables will be logged in the ClearML task's configuration hyperparameters `Environment` section. When executed by a ClearML agent, these values will be set in the task's execution environment. The list should be specified in the following format: `CLEARML_LOG_ENVIRONMENT=VAR_1,VAR_2`. |
|**CLEARML_TASK_NO_REUSE** | Boolean. <br/> When set to `1`, a new task is created for every execution (see Task [reuse](../clearml_sdk/task_sdk.md#task-reuse)). | |**CLEARML_TASK_NO_REUSE** | Boolean. <br/> When set to `1`, a new task is created for every execution (see Task [reuse](../clearml_sdk/task_sdk.md#task-reuse)). |
|**CLEARML_CACHE_DIR** | Set the path for the ClearML cache directory, where ClearML stores all downloaded content. | |**CLEARML_CACHE_DIR** | Set the path for the ClearML cache directory, where ClearML stores all downloaded content. |
|**CLEARML_DOCKER_IMAGE** | Sets the default docker image to use when running an agent in [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode). | |**CLEARML_DOCKER_IMAGE** | Sets the default docker image to use when running an agent in [Docker mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode). |

17
docs/faq.md
View File

@ -854,18 +854,25 @@ export CLEARML_API_HOST="http://localhost:8008"
#### How can I track OS environment variables with experiments? <a id="track-env-vars"></a> #### How can I track OS environment variables with experiments? <a id="track-env-vars"></a>
Set the OS environment variable `CLEARML_LOG_ENVIRONMENT` with the variables you need track, either: You can set environment variables to track in an experiment by specifying them in the `sdk.development.log_os_environments`
field of the [`clearml.conf`](configs/clearml_conf.md) file:
```editorconfig
log_os_environments: ["AWS_*", "CUDA_VERSION"]
```
You can also use the `CLEARML_LOG_ENVIRONMENT` variable to track environment variables:
* All environment variables: * All environment variables:
``` ```
export CLEARML_LOG_ENVIRONMENT="*" export CLEARML_LOG_ENVIRONMENT=*
``` ```
* Specific environment variables, for example, log `PWD` and `PYTHONPATH`: * Specific environment variables, for example, log `PWD` and `PYTHONPATH`:
``` ```
export CLEARML_LOG_ENVIRONMENT="PWD,PYTHONPATH" export CLEARML_LOG_ENVIRONMENT=PWD,PYTHONPATH
``` ```
* No environment variables: * No environment variables:
@ -874,6 +881,10 @@ Set the OS environment variable `CLEARML_LOG_ENVIRONMENT` with the variables you
export CLEARML_LOG_ENVIRONMENT= export CLEARML_LOG_ENVIRONMENT=
``` ```
:::note Overriding clearml.conf
The `CLEARML_LOG_ENVIRONMENT` variable always overrides the `clearml.conf` file.
:::
## ClearML Hosted Service ## ClearML Hosted Service
#### I run my script, but my experiment is not in the ClearML Hosted Service Web UI. How do I fix this? <a id="hosted-service-no-config"></a> #### I run my script, but my experiment is not in the ClearML Hosted Service Web UI. How do I fix this? <a id="hosted-service-no-config"></a>

24
docs/fundamentals/hyperparameters.md
View File

@ -43,20 +43,34 @@ Relying on environment variables makes an experiment not fully reproducible, sin
runtime. runtime.
::: :::
Environment variables can be logged by modifying the [clearml.conf](../configs/clearml_conf.md) file. Modify the `log_os_environments` Environment variables can be logged to a task by specifying the variables in the `sdk.development.log_os_environments`
parameter specifying parameters to log. field of the [`clearml.conf`](../configs/clearml_conf.md) file:
```editorconfig ```editorconfig
log_os_environments: ["AWS_*", "CUDA_VERSION"] log_os_environments: ["AWS_*", "CUDA_VERSION"]
``` ```
You can also specify environment variables using the `CLEARML_LOG_ENVIRONMENT` variable. You can also specify environment variables using the `CLEARML_LOG_ENVIRONMENT` environment variable:
* All environment variables:
```
export CLEARML_LOG_ENVIRONMENT=*
```
* Specific environment variables. For example, log `PWD` and `PYTHONPATH`:
```
export CLEARML_LOG_ENVIRONMENT=PWD,PYTHONPATH
```
* No environment variables
```
export CLEARML_LOG_ENVIRONMENT=
```
:::note Overriding clearml.conf :::note Overriding clearml.conf
The `CLEARML_LOG_ENVIRONMENT` always overrides the `clearml.conf` file. The `CLEARML_LOG_ENVIRONMENT` variable always overrides the `clearml.conf` file.
::: :::
When a script that has integrated ClearML is executed, the environment variables listed in `clearml.conf` or specified by When a script that has integrated ClearML is executed, the environment variables listed in the `clearml.conf` or specified by
the `CLEARML_LOG_ENVIRONMENT` variable are logged by ClearML. the `CLEARML_LOG_ENVIRONMENT` variable are logged by ClearML.
### Explicit Logging ### Explicit Logging

7
docs/webapp/webapp_exp_track_visual.md
View File

@ -122,7 +122,12 @@ parameter's line, and the type, description, and default value appear, if they w
#### Environment Variables #### Environment Variables
If the `CLEARML_LOG_ENVIRONMENT` variable was set, the **Environment** group will show environment variables (see [this FAQ](../faq.md#track-env-vars)). If environment variables were listed in the `CLEARML_LOG_ENVIRONMENT` environment variable or the `sdk.development.log_os_environments`
field of the `clearml.conf` file, the **Environment** group displays the listed environment variables (see [this FAQ](../faq.md#track-env-vars)).
:::note
The `CLEARML_LOG_ENVIRONMENT` variable always overrides the `clearml.conf` file.
:::
![Environment variables configuration group](../img/webapp_tracking_23.png) ![Environment variables configuration group](../img/webapp_tracking_23.png)

3
docs/webapp/webapp_exp_tuning.md
View File

@ -117,7 +117,8 @@ Add, change, or delete hyperparameters, which are organized in the **ClearML Web
* **General** - Parameter dictionaries (from code, connected to the Task by calling [`Task.connect()`](../references/sdk/task.md#connect)). * **General** - Parameter dictionaries (from code, connected to the Task by calling [`Task.connect()`](../references/sdk/task.md#connect)).
* Environment variables - Tracked if the `CLEARML_LOG_ENVIRONMENT` environment variable was set (see this [FAQ](../faq.md#track-env-vars)). * Environment variables - Tracked if variables were listed in the `CLEARML_LOG_ENVIRONMENT` environment variable
or the `sdk.development.log_os_environments` field of the `clearml.conf` file (see this [FAQ](../faq.md#track-env-vars)).
* Custom named parameter groups (see the `name` parameter in [`Task.connect`](../references/sdk/task.md#connect)). * Custom named parameter groups (see the `name` parameter in [`Task.connect`](../references/sdk/task.md#connect)).