mirror of
https://github.com/clearml/clearml-docs
synced 2025-04-08 23:25:32 +00:00
Fix storage examples (#811)
This commit is contained in:
parent
ec1f4d069f
commit
57be45d2a8
@ -45,6 +45,7 @@ You can specify additional [ExtraArgs](https://boto3.amazonaws.com/v1/documentat
|
||||
to pass to boto3 when uploading files. You can set this on a per-bucket basis.
|
||||
|
||||
```
|
||||
sdk {
|
||||
aws {
|
||||
s3 {
|
||||
# S3 credentials, used for read/write access by various SDK elements
|
||||
@ -73,12 +74,14 @@ aws {
|
||||
max_multipart_concurrency: 16
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
AWS's S3 access parameters can be specified by referencing the standard environment variables if already defined.
|
||||
|
||||
For example:
|
||||
```
|
||||
sdk {
|
||||
aws {
|
||||
s3 {
|
||||
# default, used for any bucket not specified below
|
||||
@ -87,10 +90,12 @@ aws {
|
||||
region: ${AWS_DEFAULT_REGION}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
ClearML also supports [MinIO](https://github.com/minio/minio) by adding this configuration:
|
||||
```
|
||||
sdk {
|
||||
aws {
|
||||
s3 {
|
||||
# default, used for any bucket not specified below
|
||||
@ -110,6 +115,7 @@ aws {
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
:::info non-AWS Endpoints
|
||||
@ -121,6 +127,7 @@ To enable TLS, pass `secure: true`.
|
||||
To configure Azure blob storage specify the account name and key.
|
||||
|
||||
```
|
||||
sdk {
|
||||
azure.storage {
|
||||
containers: [
|
||||
{
|
||||
@ -130,12 +137,14 @@ azure.storage {
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Azure's storage access parameters can be specified by referencing the standard environment variables if already defined.
|
||||
|
||||
For example:
|
||||
```
|
||||
sdk {
|
||||
azure.storage {
|
||||
containers: [
|
||||
{
|
||||
@ -145,6 +154,7 @@ azure.storage {
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Configuring Google Storage
|
||||
@ -154,6 +164,7 @@ It's also possible to specify credentials for a specific bucket in the `google.s
|
||||
configuration provided in the `google.storage` section is applied to any bucket without a bucket-specific configuration.
|
||||
|
||||
```
|
||||
sdk {
|
||||
google.storage {
|
||||
# Default project and credentials file
|
||||
# Will be used when no bucket configuration is found
|
||||
@ -170,11 +181,13 @@ google.storage {
|
||||
},
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
GCP's storage access parameters can be specified by referencing the standard environment variables if already defined.
|
||||
|
||||
```
|
||||
sdk {
|
||||
google.storage {
|
||||
credentials = [
|
||||
{
|
||||
@ -185,6 +198,7 @@ google.storage {
|
||||
},
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
:::tip Direct Decoding
|
||||
@ -233,6 +247,7 @@ piece twice!
|
||||
Configure cache location by modifying the [clearml.conf](../configs/clearml_conf.md) file:
|
||||
|
||||
```
|
||||
sdk {
|
||||
storage {
|
||||
cache {
|
||||
# Defaults to <system_temp_folder>/clearml_cache
|
||||
@ -246,6 +261,7 @@ storage {
|
||||
{ url: "file://*" } # file-urls are always directly referenced
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Direct Access
|
||||
|
Loading…
Reference in New Issue
Block a user