Update configuration file reference (#542)

This commit is contained in:
pollfly 2023-04-27 17:24:57 +03:00 committed by GitHub
parent ba1236f9be
commit d5290e57d7
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 89 additions and 3 deletions

View File

@ -125,7 +125,7 @@ auto_connect_frameworks={'tensorboard': {'report_hparams': False}}
### Task Reuse ### Task Reuse
Every `Task.init` call will create a new task for the current execution. Every `Task.init` call will create a new task for the current execution.
In order to mitigate the clutter that a multitude of debugging tasks might create, a task will be reused if: In order to mitigate the clutter that a multitude of debugging tasks might create, a task will be reused if:
* The last time it was executed (on this machine) was under 72 hours ago (configurable, see * The last time it was executed (on this machine) was under 24 hours ago (configurable, see
[`sdk.development.task_reuse_time_window_in_hours`](../configs/clearml_conf.md#task_reuse) in [`sdk.development.task_reuse_time_window_in_hours`](../configs/clearml_conf.md#task_reuse) in
the ClearML configuration reference) the ClearML configuration reference)
* The previous task execution did not have any artifacts / models * The previous task execution did not have any artifacts / models

View File

@ -10,6 +10,9 @@ This reference page is organized by configuration file section:
file will not have an `agent` section. file will not have an `agent` section.
* [api](#api-section) - Contains ClearML and ClearML Agent configuration options for ClearML Server. * [api](#api-section) - Contains ClearML and ClearML Agent configuration options for ClearML Server.
* [sdk](#sdk-section) - Contains ClearML and ClearML Agent configuration options for ClearML Python Package and ClearML Server. * [sdk](#sdk-section) - Contains ClearML and ClearML Agent configuration options for ClearML Python Package and ClearML Server.
* [environment](#environment-section) - Define environment variables to apply to the OS environment
* [files](#files-section) - Define auto-generated files to apply into local file system
An example configuration file is located [here](https://github.com/allegroai/clearml-agent/blob/master/docs/clearml.conf), An example configuration file is located [here](https://github.com/allegroai/clearml-agent/blob/master/docs/clearml.conf),
in the ClearML Agent GitHub repository. in the ClearML Agent GitHub repository.
@ -183,6 +186,14 @@ For example:
--- ---
**`agent.docker_use_activated_venv`** (*bool*)
* In Docker mode, if the container's entrypoint automatically activates a virtual environment, the activated virtual
environment is used and everything is installed in it. Set to `false` to disable, and always create a new venv inheriting
from `system_site_packages`
---
**`agent.enable_git_ask_pass`** (*bool*) **`agent.enable_git_ask_pass`** (*bool*)
:::note :::note
@ -213,6 +224,13 @@ For example:
--- ---
**`agent.force_git_root_python_path`** (*bool*)
* Force the root folder of the git repository (instead of the working directory) into the `PYHTONPATH` environment variable.
`false` by default, so only the working directory will be added to `PYHTONPATH`
---
**`agent.force_git_ssh_protocol`** (*bool*) **`agent.force_git_ssh_protocol`** (*bool*)
* Force Git protocol to use SSH regardless of the Git URL. This assumes the Git user/pass are blank. * Force Git protocol to use SSH regardless of the Git URL. This assumes the Git user/pass are blank.
@ -468,6 +486,11 @@ ___
--- ---
**`agent.package_manager.poetry_install_extra_args`** (*list*)
* List extra command-line arguments to pass when using `poetry`
---
**`agent.package_manager.post_optional_packages`** (*string*) **`agent.package_manager.post_optional_packages`** (*string*)
* A list of optional packages that will be installed after the required packages. If the installation of an optional post * A list of optional packages that will be installed after the required packages. If the installation of an optional post
@ -1287,6 +1310,69 @@ every 5MB
* Specify how often in MB the `StorageManager` reports its upload progress to the console. By default, it reports every * Specify how often in MB the `StorageManager` reports its upload progress to the console. By default, it reports every
5MB 5MB
### environment section
**`environment`** (*dict*)
Dictionary of environment variables and values which are applied to the OS environment as `key=value` for each key-value
pair.
Enable by setting `agent.apply_environment` OR `sdk.apply_environment` to `true`.
Example:
```
environment {
key_a: value_a
key_b: value_b
}
```
### files section
**`files`** (*dict*)
The `files` section allows to define files which will be auto-generated at designated paths with predefined content and
target format.
Enable by setting `agent.apply_files` OR `sdk.apply_files` to `true`.
Define each file's contents in a dictionary. Files content options include:
* `contents` - Target file's content, typically a string (or any base type int/float/list/dict etc.)
* `format` - Custom format for the contents. Currently supports `base64` to automatically decode a
base64-encoded contents string, otherwise ignored
* `path` - Target file's path, may include `~` and inplace env vars
* `target_format` - Format used to encode contents before writing into the target file. Supported values are `json`, `yaml`,
`yml`, and `bytes` (in which case the file will be written in binary mode). Default is text mode.
* `overwrite` - Overwrite the target file in case it exists. Default is `true`.
Example:
```
files {
myfile1 {
contents: "The quick brown fox jumped over the lazy dog"
path: "/tmp/fox.txt"
}
myjsonfile {
contents: {
some {
nested {
value: [1, 2, 3, 4]
}
}
}
path: "/tmp/test.json"
target_format: json
}
}
# Apply top-level files section from configuration into local file system
sdk {
apply_files: true
}
```
## Configuration Vault ## Configuration Vault
:::note Enterprise Feature :::note Enterprise Feature