ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-06-26 18:17:44 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Add example links (#57)

Browse Source
This commit is contained in:
pollfly
2021-09-01 09:41:27 +03:00
committed by GitHub
parent 397a7d5a41
commit c52697fd1b
9 changed files with 86 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
docs/integrations/storage.md
View File

@@ -99,31 +99,11 @@ It's also possible to specify credentials for a specific bucket.
## Storage Manager
ClearML Offers a package to manage downloading, uploading and caching of content directly from code.
ClearML offers the [StorageManager](../references/sdk/storage.md) class to manage downloading, uploading, and caching of
content directly from code.
### Uploading files
To upload a file using storage manager, just run the following line specifying the path to a local file or folder, and the
remote destination.
```python
from clearml import StorageManager
See [Storage Examples](../guides/storage/examples_storagehelper.md).
StorageManager.upload_file(local_file='path_to_file',remote_url='s3://my_bucket')
```
### Downloading files
To download files into cache, run the following line, specifying the remote destination's URL.
```python
StorageManager.get_local_copy(remote_url='s3://my_bucket/path_to_file')
```
:::note
Zip and tar.gz files will be automatically extracted to cache. This can be controlled with the`extract_archive` flag.
:::
### Controling cache file limit
It's possible to control the maximum cache size by limiting the number of files it stores.
This is done by calling the ```StorageManager.set_cache_file_limit()``` method.
## Caching
ClearML also manages a cache of all downloaded content so nothing is duplicated, and code won't need to download the same