Merge branch 'main' of https://github.com/allegroai/clearml-docs

docs/clearml_data/clearml_data_cli.md

@@ -107,7 +107,7 @@ Upload the local dataset changes to the server. By default, it's uploaded to the
 medium by entering an upload destination. For example: 
 * A shared folder: `/mnt/shared/folder`
 * S3: `s3://bucket/folder`
-* Non-AWS S3-like services (such as MinIO): `s3://host_addr:port/bucket` 
+* Non-AWS S3-like services (such as MinIO): `s3://host_addr:port/bucket`. **Note that port specification is required**. 
 * Google Cloud Storage: `gs://bucket-name/folder`
 * Azure Storage: `azure://<account name>.blob.core.windows.net/path/to/file`
 

docs/clearml_data/clearml_data_sdk.md

@@ -65,7 +65,7 @@ Use the `output_uri` parameter to specify a network storage target to upload the
 (such as previews) to. For example:
 * A shared folder: `/mnt/share/folder`
 * S3: `s3://bucket/folder`
-* Non-AWS S3-like services (such as MinIO): `s3://host_addr:port/bucket` 
+* Non-AWS S3-like services (such as MinIO): `s3://host_addr:port/bucket`. **Note that port specification is required**. 
 * Google Cloud Storage: `gs://bucket-name/folder`
 * Azure Storage: `azure://<account name>.blob.core.windows.net/path/to/file`
 
@@ -264,7 +264,7 @@ To upload the dataset files to network storage, use [`Dataset.upload()`](../refe
 Use the `output_url` parameter to specify storage target, such as S3 / GS / Azure. For example:
 * A shared folder: `/mnt/share/folder`
 * S3: `s3://bucket/folder`
-* Non-AWS S3-like services (such as MinIO): `s3://host_addr:port/bucket` 
+* Non-AWS S3-like services (such as MinIO): `s3://host_addr:port/bucket`. **Note that port specification is required**. 
 * Google Cloud Storage: `gs://bucket-name/folder`
 * Azure Storage: `azure://<account name>.blob.core.windows.net/path/to/file`
 

docs/clearml_sdk/task_sdk.md

@@ -800,7 +800,7 @@ task = Task.init(
 Specify the model storage URI location using the relevant format: 
 * A shared folder: `/mnt/share/folder`
 * S3: `s3://bucket/folder`
-* Non-AWS S3-like services (such as MinIO): `s3://host_addr:port/bucket` 
+* Non-AWS S3-like services (such as MinIO): `s3://host_addr:port/bucket`. **Note that port specification is required**. 
 * Google Cloud Storage: `gs://bucket-name/folder`
 * Azure Storage: `azure://<account name>.blob.core.windows.net/path/to/file`
 :::

docs/clearml_serving/clearml_serving_cli.md

@@ -202,7 +202,7 @@ clearml-serving model upload [-h] --name NAME [--tags TAGS [TAGS ...]] --project
 |`--publish`| Publish the newly created model (change model state to "published" (i.e. locked and ready to deploy)|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
 |`--path`|Specify a model file/folder to be uploaded and registered| <img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
 |`--url`| Specify an already uploaded model url (e.g. `s3://bucket/model.bin`, `gs://bucket/model.bin`)|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
-|`--destination`|Specify the target destination for the model to be uploaded. For example: `s3://bucket/folder/`, `s3://host_addr:port/bucket` (for non-AWS S3-like services like MinIO), `gs://bucket-name/folder`, `azure://<account name>.blob.core.windows.net/path/to/file`|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
+|`--destination`|Specify the target destination for the model to be uploaded. For example: <ul><li>S3: `s3://bucket/folder`</li><li>Non-AWS S3-like services (such as MinIO): `s3://host_addr:port/bucket`. **Note that port specification is required**.</li><li>Google Cloud Storage: `gs://bucket-name/folder`</li><li>Azure Storage: `azure://<account name>.blob.core.windows.net/path/to/file`</li></ul>|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
 
 </div>
 

docs/clearml_serving/clearml_serving_tutorial.md

@@ -107,8 +107,15 @@ or with the `clearml-serving` CLI.
    ```
 
 :::info Model Storage
-You can also provide a different storage destination for the model, such as S3/GS/Azure, by passing
-`--destination="s3://bucket/folder"`, `s3://host_addr:port/bucket` (for non-AWS S3-like services like MinIO), `gs://bucket/folder`, `azure://<account name>.blob.core.windows.net/path/to/file`. There is no need to provide a unique 
+You can also provide a different storage destination for the model, such as S3/GS/Azure, by using
+`--destination`. For example:
+
+* S3: `s3://bucket/folder`
+* Non-AWS S3-like services (such as MinIO): `s3://host_addr:port/bucket`. **Note that port specification is required**. 
+* Google Cloud Storage: `gs://bucket-name/folder`
+* Azure Storage: `azure://<account name>.blob.core.windows.net/path/to/file`
+
+There is no need to provide a unique 
 path to the destination argument, the location of the model will be a unique path based on the serving service ID and the 
 model name.
 :::

docs/faq.md

@@ -682,7 +682,7 @@ the experiment's ID. If the experiment's ID is `6ea4f0b56d994320a713aeaf13a86d9d
 
 ClearML supports other storage types for `output_uri`:
 * S3: `s3://bucket/folder`
-* Non-AWS S3-like services (such as MinIO): `s3://host_addr:port/bucket` 
+* Non-AWS S3-like services (such as MinIO): `s3://host_addr:port/bucket`. **Note that port specification is required**. 
 * Google Cloud Storage: `gs://bucket-name/folder`
 * Azure Storage: `azure://<account name>.blob.core.windows.net/path/to/file`
 

docs/hyperdatasets/single_frames.md

@@ -222,7 +222,7 @@ frame = SingleFrame(
 
 :::tip Previewing Frames in non-AWS S3-like services
 For the ClearML UI to be able to show frames stored in non-AWS S3-like services (e.g. MinIO), make sure the `preview_uri` link
-uses the `s3://` prefix and explicitly specifies the port number in the URL (e.g. `s3://my_address.com:80/bucket/my_image.png`).
+uses the `s3://` prefix and **explicitly specifies the port number** in the URL (e.g. `s3://my_address.com:80/bucket/my_image.png`).
 
 Additionally, make sure to provide cloud storage access in the WebApp [**Settings > Configuration > Web App Cloud Access**](../webapp/settings/webapp_settings_profile.md#browser-cloud-storage-access). 
 Input `<host_address>:<port_number>` in the **Host** field.

docs/integrations/storage.md

@@ -124,6 +124,18 @@ sdk {
 To force usage of a non-AWS endpoint, port declaration is *always* needed (e.g. `host: "my-minio-host:9000"`), 
 even for standard ports like `433` for HTTPS (e.g. `host: "my-minio-host:433"`).
 
+:::important
+Port specification is mandatory whenever you specify non-AWS S3 endpoint access. Use the following URI 
+format: `s3://<hostname>:<port>/<bucket-name>/path`.
+
+This applies when:
+* Setting output URIs for tasks (via SDK or UI)
+* Registering Hyper-Dataset frames
+* All fields where endpoint access is specified. 
+:::
+
+
+##### TLS
 To enable TLS, pass `secure: true`. For example: 
 ```
 sdk {

docs/webapp/applications/apps_aws_autoscaler.md

@@ -320,6 +320,16 @@ to an IAM user, and create credentials keys for that user to configure in the au
                    "ssm:GetParameter"
                ],
                "Resource": "arn:aws:ssm:*::parameter/aws/service/marketplace/*"
+           },
+           {
+               "Sid": "AllowUsingDeeplearningAMIAliases",
+               "Effect": "Allow",
+               "Action": [
+                   "ssm:GetParametersByPath",
+                   "ssm:GetParameters",
+                   "ssm:GetParameter"
+               ],
+               "Resource": "arn:aws:ssm:*::parameter/aws/service/deeplearning/*"
            }
        ]
    }

docs/webapp/webapp_exp_tuning.md

@@ -97,7 +97,7 @@ and formats for specifying locations include:
 
 * A shared folder: `/mnt/share/folder`
 * S3: `s3://bucket/folder`
-* Non-AWS S3-like services (e.g. MinIO): `s3://host_addr:port/bucket` 
+* Non-AWS S3-like services (e.g. MinIO): `s3://host_addr:port/bucket`. **Note that port specification is required**. 
 * Google Cloud Storage: `gs://bucket-name/folder`
 * Azure Storage: `azure://<account name>.blob.core.windows.net/path/to/file`