ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-02-07 13:21:46 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Add cache configuration information (#607)

Browse Source
This commit is contained in:
pollfly 2023-06-29 17:11:27 +03:00 committed by GitHub
parent e65453b1f4
commit 8051d0df13
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 21 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
docs/configs/clearml_conf.md
View File

@ -1286,16 +1286,29 @@ will not exceed the value of `matplotlib_untitled_history_size`
**`sdk.storage.cache`** (*dict*)
* Dictionary of storage cache options.
* Dictionary of storage cache options. The keys include:
* `default_base_dir` (*str*) - The default base directory for caching. The default is the `<system_temp_folder>/clearml_cache`.
* `default_cache_manager_size` (*int*) - Maximum number of files in the cache (default 100 files).
:::important Enterprise features
The ClearML Enterprise plan also supports the following configuration options under `sdk.storage.cache`:
* `size.max_used_bytes` (*str*) - Maximum size of the local cache directory. If set to `-1`, the directory can use
the available disk space. Specified in storage units (e.g. `1GB`, `2TB`, `500MB`).
* `size.min_free_bytes` (*str*) - Minimum amount of free disk space that should be left. If `size.max_used_bytes` is
set to `-1`, this configuration will limit the cache directory maximum size to `free disk space - size.min_free_bytes`.
Specified in storage units (e.g. `1GB`, `2TB`, `500MB`).
* `zero_file_size_check` (*bool*)- If set to `True`, each cache hit will also check the cached file size, making sure
it is not zero (default `False`)
* `secondary` (*dict*) - Set up a secondary cache (acts as an L2 cache). When a request is made, the primary cache is
queried first. If the data is not in the primary cache, the secondary cache is queried. In case of a cache
miss, the data will be pulled to the primary cache, and then copied to the secondary cache. The
`sdk.storage.cache.secondary` dictionary supports the same option as the primary cache: `default_base_dir` (required), `size.max_used_bytes`,
`size.min_free_bytes`, etc. If an option is unspecified, it defaults to the primary cache's value.
:::
---
**`sdk.storage.cache.default_base_dir`** (*string*)
* The default base directory for caching. The default is the system temp folder for caching.
<br/>
##### sdk.storage.direct_access
**`sdk.storage.direct_access`** (*dict*)

2
docs/integrations/storage.md
View File

@ -230,7 +230,7 @@ Configure cache location by modifying the [clearml.conf](../configs/clearml_conf
```
storage {
cache {
# Defaults to system temp folder / cache
# Defaults to <system_temp_folder>/clearml_cache
default_base_dir: "~/.clearml/cache"
}