ClearML/clearml
1
0
Fork 0
mirror of https://github.com/clearml/clearml synced 2025-06-25 17:46:28 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
1,747 Commits 4 Branches 180 Tags 56 MiB
301d68d2d3
Commit Graph

69 Commits

Author SHA1 Message Date
allegroai
36481a1337 Change thread Event/Lock to a process fork safe threading objects 2022-07-28 18:40:46 +03:00
allegroai
f9895a8e3b Fix StorageManager list() is not working for Windows full local path 2022-07-28 18:36:52 +03:00
allegroai
beb1bd447b Fix upload to S3 guess mimetypes 2022-07-21 17:18:53 +03:00
allegroai
a1edcea481 Fix Python 3.6 support guess mime type support 2022-07-17 20:00:49 +03:00
Giacomo Ferroni
397dcfacda
Release lock after dataset cache is downloaded (#708) 2022-07-13 17:33:55 +03:00
allegroai
e2208fc4c5 Fix last merge 2022-07-13 17:32:45 +03:00
Zied ANDOLSI
3f9cf400fa
Specify an endpoint_url when creating S3 resource service (#679) 
* Specify an endppoint_url when creating S3 resource service

* code review

Co-authored-by: Zied ANDOLSI <zandolsi@prophesee.ai>
 2022-07-13 17:24:53 +03:00
allegroai
a486881a56 Fix format string sometimes causing delayed evaluation issues 2022-07-04 11:53:40 +03:00
allegroai
b84963fea0 Add support for renaming very long filenames to avoid file system errors when downloading files 2022-06-28 21:22:51 +03:00
allegroai
11fc238886 Fix StorageManager.upload_folder() returns None for both successful and unsuccessful uploads 2022-06-28 21:14:50 +03:00
allegroai
21712a709d Improve error message when checking permissions 2022-06-28 21:08:40 +03:00
allegroai
68cf2745ff Add support for dataset version, fix trigger for datasets 2022-06-28 21:07:46 +03:00
allegroai
70de28ce7b Show better error when failing to download an artifact 2022-06-26 18:17:59 +03:00
allegroai
6c9e02f1b2 Fix wrong mimetype used for any file or folder uploaded to S3 using StorageManager 
Add support for boto3's aws_session_token
 2022-06-06 14:13:45 +03:00
allegroai
85c27fda82 Fix permission check fails when using local storage folder that does not exist 2022-05-30 19:50:12 +03:00
allegroai
3c396b1f7e Fix pipelines can't handle None return value 2022-05-12 23:46:32 +03:00
allegroai
24da3e3e08 Add clearml-data support for links (#585) 2022-04-27 17:00:09 +03:00
allegroai
42fa0dde65 Add support for artifacts with different formats (#634) 2022-04-27 16:57:20 +03:00
allegroai
a1709d5d41 Add support for AWS Session Token in AWS Storage configuration 2022-04-15 19:20:05 +03:00
allegroai
32a71d0025 Support custom configuration in StorageHelper.get() 2022-04-13 14:36:09 +03:00
allegroai
6330c879f8 Fix StorageManager.get_local_copy() returning None for a valid path in Windows 2022-04-13 14:34:01 +03:00
allegroai
619d59ea2e Fix sub-project separators are incorrectly quoted in generated URLs (#584) 2022-03-21 22:51:27 +02:00
allegroai
237c7241b3 Fix Azure storage upload (#598) 2022-03-15 20:01:41 +02:00
allegroai
05791ba6f8 Fix handling of legacy fileserver 2022-03-11 23:53:07 +02:00
allegroai
ffdbef8391 Fix invalid method called on error 2022-03-06 18:50:57 +02:00
allegroai
ecee4eaf64 Fix HTTP download fails constructing URL (#593) 2022-03-01 20:30:40 +02:00
allegroai
83a71d0fb0 Fix filserver upload does not support path in fileserver URL 2022-02-26 13:40:26 +02:00
allegroai
a5b16db95c Add missing skip_zero_size_check argument in StorageManager.download_foler() 
PEP8
 2022-02-26 13:38:42 +02:00
eugen-ajechiloae-clearml
5be1c8b2b0
Support newer azure storage python version (#548) 
Add support for Azure Blob Storage >=12.0.0. Backwards compatibility remains for >=2.0.1,<=2.1
 2022-02-12 17:32:43 +02:00
allegroai
5f1345b148 Add skip_zero_size_check to StorageManager.download_folder() 2022-01-31 11:07:15 +02:00
allegroai
ad735f93e0 Fix StorageManager.download_folder() fails on Windows path 2022-01-18 20:34:45 +02:00
allegroai
fa95d041e2 Add support for not raising error on downloaded files with 0 size 2022-01-18 20:33:26 +02:00
allegroai
977d68169b Fix cleanup fail silently does not remove all entries 2022-01-15 23:03:58 +02:00
allegroai
b3176a223b Change HTTP driver timeout and retry codes (connection timeout will now trigger a retry) 2022-01-13 11:57:44 +02:00
allegroai
ddffffa51c Control S3/GS _stream_download_pool_connections from clearml.conf "stream_connections", default 128 2021-12-30 12:22:11 +02:00
allegroai
d9aee85821 Fix forked StorageHelper should use its own ThreadExecuter 2021-12-22 13:27:57 +02:00
allegroai
1a0a7b6738 Fix local StorageHelper.delete() raising exception on non existing file instead of returning false 2021-12-22 13:08:32 +02:00
allegroai
69846d200a Add HTTP download timeout control (change default connection timeout to 30sec) 2021-12-22 12:53:37 +02:00
allegroai
6f6c5e9f5e Fix Storage Helper rename partial file throwing errors on multiple access 2021-12-17 11:56:24 +02:00
allegroai
11892a2145 Add support for Google Cloud Storage pool_connections and pool_maxsize overrides 2021-11-30 21:13:48 +02:00
allegroai
73cfbcb947 Fix download progress bar based on sdk.storage.log.report_download_chunk_size_mb configuration 2021-11-29 12:46:40 +02:00
allegroai
0bc43a31f4 PEP8 2021-10-19 10:39:55 +03:00
allegroai
844c01e15b Add clearml-Data (Datasets) multi-chunk support 2021-09-25 23:07:49 +03:00
allegroai
8ebf9be604 Add support for deferred configuration 2021-09-09 22:00:40 +03:00
allegroai
6e15349b76 Fix unsafe Google Storage delete object 2021-09-02 15:48:09 +03:00
allegroai
aff39ae061 Fix documentation and prints 2021-08-25 16:46:44 +03:00
allegroai
5295708e18 Add "sdk.storage.log.report_upload_chunk_size_mb" and "sdk.storage.log.report_download_chunk_size_mb", control upload/download log reporting (issue #424) 2021-08-20 00:34:20 +03:00
allegroai
62946c04f1 Fix wrong path for downloading the content 2021-08-12 13:19:56 +03:00
allegroai
82009a5935 Fix check permissions fail in HTTPDriver (issue #394) 2021-07-23 16:00:23 +03:00
allegroai
0765d18868 Add return_full_path argument to StorageManager.list() 2021-06-16 18:27:02 +03:00