mirror of
https://github.com/clearml/clearml-docs
synced 2025-03-03 18:53:37 +00:00
Update clearml.conf options (#837)
This commit is contained in:
pollfly
GitHub
parent
fe8b8ab888
commit
3cc6988663
@ -413,7 +413,7 @@ ___
|
||||
|
||||
---
|
||||
|
||||
**`agent.match_rules`** (*[dict]*)
|
||||
**`agent.default_docker.match_rules`** (*[dict]*)
|
||||
|
||||
:::important Enterprise Feature
|
||||
This feature is available under the ClearML Enterprise plan
|
||||
@ -478,6 +478,12 @@ match_rules: [
|
||||
|
||||
* Uses conda environment for execution (like a docker)
|
||||
|
||||
---
|
||||
|
||||
**`agent.package_manager.use_conda_base_env`** (*bool*)
|
||||
|
||||
* When set to `True`, installation will be performed into the base Conda environment. Use in [Docker mode](../clearml_agent.md#docker-mode).
|
||||
|
||||
___
|
||||
|
||||
**`agent.package_manager.extra_index_url`** (*[string]*)
|
||||
@ -685,6 +691,10 @@ Torch Nightly builds are ephemeral and are deleted from time to time.
|
||||
|
||||
#### agent.venv_update
|
||||
|
||||
:::note
|
||||
This option is deprecated. Use `venvs_cache` and set `venvs_cache.path` instead.
|
||||
:::
|
||||
|
||||
**`agent.venv_update`** (*dict*)
|
||||
|
||||
* Dictionary containing virtual environment update options.
|
||||
@ -833,6 +843,12 @@ metrics, network, AWS S3 buckets and credentials, Google Cloud Storage, Azure St
|
||||
|
||||
---
|
||||
|
||||
**`sdk.aws.s3.profile`** (*string*)
|
||||
|
||||
* For AWS S3, the default profile name for any bucket that is not specified in the `sdk.aws.s3.credentials` section.
|
||||
|
||||
---
|
||||
|
||||
**`sdk.aws.s3.region`** (*string*)
|
||||
|
||||
* For AWS S3, the default region name for any bucket that is not specified in the `sdk.aws.s3.credentials` section.
|
||||
@ -955,6 +971,25 @@ and limitations on bucket naming.
|
||||
|
||||
<br/>
|
||||
|
||||
#### sdk.dataset
|
||||
|
||||
**`sdk.dataset.preview`** (*[dict]*)
|
||||
|
||||
* Set limits for the objects that are logged as dataset previews:
|
||||
* `**sdk.dataset.preview.media`** (*dict*) - Set limits for media files that are logged as dataset previews. Available
|
||||
options:
|
||||
* **`sdk.dataset.preview.media.max_file_size`** (*int*) - Maximum file size in bytes of a preview object (e.g. image,
|
||||
video, html, etc.). Files exceeding this size will not be reported as previews.
|
||||
* **`sdk.dataset.preview.media.image_count`** (*int*) - The maximum number of image files reported as previews
|
||||
* **`sdk.dataset.preview.media.video_count`** (*int*) - The maximum number of video files reported as previews
|
||||
* **`sdk.dataset.preview.media.audio_count`** (*int*) - The maximum number of image files reported as previews
|
||||
* **`sdk.dataset.preview.media.html_count`** (*int*) - The maximum number of html files reported as previews
|
||||
* **`sdk.dataset.preview.media.json_count`** (*int*) - The maximum number of json files reported as previews
|
||||
* `**sdk.dataset.preview.tabular`** (*dict*) - Set limits for tabular files that are logged as dataset previews. Available
|
||||
options:
|
||||
* **`sdk.dataset.preview.tabular.row_count`** (*int*) - The maximum number of rows for each tabular file reported as a preview. By default, it will report only the first 10 rows from a file
|
||||
* **`sdk.dataset.preview.tabular.table_count`** (*int*) - The maximum number of tables reported as preview in a dataset
|
||||
|
||||
|
||||
#### sdk.development
|
||||
|
||||
@ -1013,7 +1048,7 @@ and limitations on bucket naming.
|
||||
|
||||
---
|
||||
|
||||
**`sdk.development.store_uncommitted_code_diff_on_train`** (*bool*)
|
||||
**`sdk.development.store_uncommitted_code_diff`** (*bool*)
|
||||
|
||||
* For development mode, indicates whether to store the uncommitted `git diff` or `hg diff` in the experiment manifest.
|
||||
|
||||
@ -1081,6 +1116,14 @@ and limitations on bucket naming.
|
||||
|
||||
---
|
||||
|
||||
**`sdk.development.worker. max_wait_for_first_iteration_to_start_sec`** (*integer*)
|
||||
|
||||
* Maximum time (in seconds) for allowing the resource monitoring to switch back to reporting iterations as the x-axis
|
||||
after initially starting to report "seconds from start." If the specified time limit is exceeded, the resource monitoring
|
||||
will continue reporting using "seconds from start" as the x-axis.
|
||||
|
||||
---
|
||||
|
||||
**`sdk.development.worker.ping_period_sec`** (*integer*)
|
||||
|
||||
* For development mode workers, the interval in seconds for a worker to ping the server testing connectivity.
|
||||
@ -1107,6 +1150,19 @@ and limitations on bucket naming.
|
||||
|
||||
* For development mode workers, the interval in seconds for a development mode ClearML worker to report.
|
||||
|
||||
---
|
||||
|
||||
**`sdk.development.worker.report_start_sec`** (*integer*)
|
||||
|
||||
* The number of seconds after which the development mode worker starts resource reporting.
|
||||
|
||||
---
|
||||
|
||||
**`sdk.development.worker.wait_for_first_iteration_to_start_sec`** (*integer*)
|
||||
|
||||
* Controls how long (in seconds) to wait for iteration reporting to be used as x-axis for resource monitoring. If iteration
|
||||
reporting is unavailable once time is exceeded, "seconds from start" will be used for the x-axis.
|
||||
|
||||
<br/>
|
||||
|
||||
#### sdk.google.storage
|
||||
@ -1228,7 +1284,7 @@ will not exceed the value of `matplotlib_untitled_history_size`
|
||||
**`sdk.metrics.tensorboard_single_series_per_graph`** (*bool*)
|
||||
|
||||
:::note
|
||||
This configuration is deprecated. This plot behavior is now controlled via the UI
|
||||
This option is deprecated. This plot behavior is now controlled via the UI
|
||||
:::
|
||||
|
||||
* Indicates whether plots appear using TensorBoard behavior where each series is plotted in its own graph (plot-per-graph).
|
||||
@ -1333,7 +1389,7 @@ The ClearML Enterprise plan also supports the following configuration options un
|
||||
* `size.max_used_bytes` (*str*) - Maximum size of the local cache directory. If set to `-1`, the directory can use
|
||||
the available disk space. Specified in storage units (for example: `1GB`, `2TB`, `500MB`).
|
||||
* `size.min_free_bytes` (*str*) - Minimum amount of free disk space that should be left. If `size.max_used_bytes` is
|
||||
set to `-1`, this configuration will limit the cache directory maximum size to `free disk space - size.min_free_bytes`.
|
||||
set to `-1`, this configuration option will limit the cache directory maximum size to `free disk space - size.min_free_bytes`.
|
||||
Specified in storage units (for example: `1GB`, `2TB`, `500MB`).
|
||||
* `zero_file_size_check` (*bool*)- If set to `True`, each cache hit will also check the cached file size, making sure
|
||||
it is not zero (default `False`)
|
||||
@ -1372,6 +1428,36 @@ every 5MB
|
||||
* Specify how often in MB the `StorageManager` reports its upload progress to the console. By default, it reports every
|
||||
5MB
|
||||
|
||||
##### sdk.storage.path_substitution
|
||||
|
||||
**`sdk.storage.path_substitution`** (*[dict]*)
|
||||
|
||||
* List of dictionaries, where each dictionary contains a registered link and a link to replace it. This is useful in
|
||||
cases where data was registered under a path that was later renamed, or when data was pushed to a bucket but later
|
||||
mounted to a local drive.
|
||||
* Each dictionary contains a `registered_prefix` and a `local_prefix`. The `registered_prefix` is the prefix to find and
|
||||
replace. The `local_prefix` is the prefix where
|
||||
the data is actually saved under, which will replace the `registered_prefix`.
|
||||
|
||||
For example:
|
||||
|
||||
```
|
||||
sdk {
|
||||
storage {
|
||||
path_substitution = [
|
||||
{
|
||||
registered_prefix = "s3://bucket/research"
|
||||
local_prefix = "file:///mnt/shared/bucket/research"
|
||||
},
|
||||
{
|
||||
registered_prefix = "file:///mnt/shared/folder/"
|
||||
local_prefix = "file:///home/user/shared/folder"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### environment section
|
||||
|
||||
**`environment`** (*dict*)
|
||||
|
||||
Loading…
Reference in New Issue
Block a user