ClearML/clearml
1
0
Fork 0
mirror of https://github.com/clearml/clearml synced 2025-02-07 05:18:50 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Update errata, clarify a few points

Browse Source
This commit is contained in:
alex-burlacu-clear-ml 2023-07-19 20:23:03 +03:00 committed by GitHub
parent 618a625b74
commit ac1ae7f447
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 10 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
docs/errata_breaking_change_gcs_sdk_1_11_x.md
View File

@ -2,14 +2,18 @@
## Rationale
Due to an issue with ClearML SDK versions 1.11.x, URLs of objects uploaded to the Google Cloud Storage were stored in the ClearML backend as a quoted string. This causes issues accessing these objects directly from the ClearML SDK. This is relevant for URLs of models, datasets, artifacts, and media files/debug samples. In case you have such objects uploaded with the affected ClearML SDK versions and wish to be able to access them programmatically using the ClearML SDK (note that access from the ClearML UI is still possible), you should perform one of the actions in the section below.
Due to an issue with ClearML SDK versions 1.11.x, URLs of objects uploaded to the Google Cloud Storage were stored in the ClearML backend as a quoted string. This causes issues accessing these objects directly from the ClearML SDK. This is relevant for URLs of models, datasets, artifacts, and media files/debug samples. In case you have such objects uploaded with the affected ClearML SDK versions and wish to be able to access them programmatically using the ClearML SDK using the version 1.12 and above (note that access from the ClearML UI is still possible), you should perform one of the actions in the section below.
## Recommended Steps
The code snippets below should serve as an example rather than an actual conversion script. Depending on what object you're trying to fix, you should pick the respective lines of code from step 1 and 2.
The code snippets below should serve as an example rather than an actual conversion script.
The general flow is that you will first need to download these files by a custom access method, then upload them with the fixed SDK version. Depending on what object you're trying to fix, you should pick the respective lines of code from step 1 and 2.
1. You need to be able to download objects (models, datasets, media, artifacts) registered by affected versions. See the code snippet below and adjust it according to your use case to be able to get a local copy of the object
```
```python
from clearml import Task, ImportModel
from urllib.parse import unquote # <- you will need this
@ -38,7 +42,7 @@ local_path = StorageManager.get_local_copy(url)
```
2. Once the object is downloaded locally, you can re-register it with the new version. See the snipped below and adjust according to your use case
```
```python
from clearml import Task, Dataset, OutputModel
import os
@ -63,10 +67,10 @@ for sample in samples:
## Alternative methods
These methods are more advanced (read "more likely to mess up"). If you're unsure whether to use them or not, better don't. Both methods described below will alter the existing objects. Note that you still need to run the code from step 1 to have access to all required metadata.
These methods are more advanced (read "more likely to mess up"). If you're unsure whether to use them or not, better don't. Both methods described below will alter (i.e. modify **in-place**) the existing objects. Note that you still need to run the code from step 1 to have access to all required metadata.
**Method 1**: You can try to alter the existing unpublished experiments/models using the lower-level `APIClient`
```
```python
from clearml.backend_api.session.client import APIClient