ClearML/clearml
1
0
Fork 0
mirror of https://github.com/clearml/clearml synced 2025-06-26 18:16:07 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
1,619 Commits 4 Branches 181 Tags 56 MiB
0c0739379b
Commit Graph

591 Commits

Author SHA1 Message Date
allegroai
f93c0dddf0 Fix querying class attribute 2021-04-28 18:45:00 +03:00
allegroai
bdd8d224ac Fix get_active_config_file() crashing 2021-04-28 18:44:13 +03:00
allegroai
b1dbc9b7a2 Fix Task.set_comment/name() passing None should be equivalent to '' 2021-04-28 13:13:00 +03:00
allegroai
fe76680eda Fix Task.runtime API v2.13 2021-04-28 13:12:30 +03:00
allegroai
20d91ee16c Add PipelineController into main namespace 2021-04-28 13:11:36 +03:00
allegroai
19c88f93d0 PEP8 2021-04-28 13:11:16 +03:00
allegroai
84025a75b2 Add Task.runtime to API v2.13 2021-04-28 13:10:02 +03:00
allegroai
9d108d855f Add Pipeline controller caching, improve pipeline plot reporting 2021-04-25 10:43:39 +03:00
allegroai
a9f52a468c Rename TF patching functions for better readability 2021-04-25 10:43:11 +03:00
allegroai
dc6c8cfddc Add option to enable/disable tensorboard (default: enable) 2021-04-25 10:42:50 +03:00
allegroai
f9e555b464 Add pipeline step caching 2021-04-25 10:41:51 +03:00
allegroai
696034ac75 Add Task.get_configuration_objects() add Task.get_task() filtering support 2021-04-25 10:41:29 +03:00
allegroai
07fca61572 Fix PyCharm debugger Keyboard interrupt set Task as failed (instead of aborted) 2021-04-25 10:39:10 +03:00
allegroai
b64c43ba54 Change Logger plot iteration argument default to 0 (or None) 
Breaking changes! arguments order changed in Logger.report_line_plot(), Logger.report_plotly() and Logger.report_matplotlib_figure()
 2021-04-25 10:38:38 +03:00
allegroai
1637c1a270 Improve Task.artifacts for safer Task multi node use cases 2021-04-25 10:37:14 +03:00
allegroai
d3cdb0ac42 Fix ClearmlJob docstrings 2021-04-25 10:36:46 +03:00
allegroai
272ed44c12 Fix Task.set_credetnials() should override config file 2021-04-25 10:35:08 +03:00
allegroai
22d795f68f Fix Matplotlib automagic export legend not showing series names (issue #337) 2021-04-20 18:09:22 +03:00
allegroai
4832de5a65 Wait for subprocess when terminating 2021-04-20 18:09:22 +03:00
allegroai
1c6685e273 Change default metric report flushing to every 5 seconds 2021-04-20 18:09:22 +03:00
allegroai
8417e4f7a0 Add jupyter notebook preview update time stamp 2021-04-20 18:09:22 +03:00
allegroai
8c309afef4 Fix Google API credentials without storage section raises exception on import (issue #331) 2021-04-20 18:09:22 +03:00
allegroai
3e61e3b39b Add hash utility functions 2021-04-15 00:08:16 +03:00
allegroai
33009cc1c7 Fix flushing on CR timeout (default 10sec) 2021-04-15 00:07:48 +03:00
allegroai
6ed4826c65 Fix hang when leaving process 2021-04-15 00:07:23 +03:00
allegroai
91d63df488 Fix shutdown should be called after python code detected KeyboardInterrupt exception 2021-04-15 00:07:02 +03:00
allegroai
59d531d1c8 Remove unnecessary legacy model design update 2021-04-15 00:06:27 +03:00
allegroai
60b6da5d8d Fix typo 2021-04-13 22:54:37 +03:00
allegroai
4e13d434a1 Version bump 2021-04-12 22:58:55 +03:00
allegroai
28abc9e69a Fix Task.set_base_docker() 
Add docker bash script to Task.create()
 2021-04-12 22:58:22 +03:00
allegroai
50d620cc9a Add force argument to Task.mark_failed() 2021-04-12 22:56:33 +03:00
allegroai
c4e9a7d387 Fix Model.task 2021-04-12 22:54:52 +03:00
allegroai
f8c28bd98d Fix schema typo 2021-04-12 22:54:23 +03:00
allegroai
86148a928b Support ClearML server API v2.13 2021-04-10 22:32:49 +03:00
allegroai
139c4ffe86 Remove redundant code 2021-04-10 22:26:48 +03:00
allegroai
0013c5851e Improve pipeline controller, add callbacks 2021-04-10 22:26:05 +03:00
allegroai
cf28551d21 Support storing dict with "." in the keys using Task.connect_configuration 2021-04-10 22:22:03 +03:00
allegroai
bf9ec770c6 Add ClearmlJob support for Task parameters override (issue #214) 2021-04-10 22:21:17 +03:00
allegroai
6415722798 Fix dataset create single root plot 2021-04-10 22:20:11 +03:00
allegroai
adf199b43b Add Model query interface 2021-04-10 22:18:58 +03:00
allegroai
5ee6425b4f Fix remove deprecation warning with matplotlib 3.1 2021-04-07 18:38:58 +03:00
allegroai
c07c45d2b3 Add OS env var CLEARML_DEFAULT_OUTPUT_URI to override default output_uri for automatic models upload (issue #328) 2021-04-07 18:38:32 +03:00
allegroai
e8752c54ff Fix docstring 2021-03-31 23:51:48 +03:00
allegroai
31d3b6dbc5 Support stopping instead of resetting in Task.execute_remotely() in case server supports enqueueing stopped tasks 2021-03-31 23:51:25 +03:00
allegroai
1986ec43fd Fix reporting after task closing 2021-03-31 23:47:27 +03:00
allegroai
9b962bae4b Fix \r and \n in debug samples title/series (issue #323) 2021-03-26 12:30:27 +03:00
allegroai
fa77d0f380 Add Task.force_requirements_env_freeze() forcing pip freeze instead of package analysis (issue #304) 2021-03-25 01:03:38 +02:00
allegroai
2a299c51db Fix Task requirements should conform to strings 2021-03-25 01:02:25 +02:00
allegroai
c76fe55c03 Add PipelineController node skip visualization and post execution callback 2021-03-25 01:00:56 +02:00
allegroai
9ff52a8699 Fix Task.execute_remotely() with sub process 2021-03-18 09:50:36 +02:00
allegroai
56825f9e7a Fix Keras h5 model storage 2021-03-18 09:49:53 +02:00
allegroai
20f10f8fbb Allow to continue running from previous execution iteration 2021-03-18 09:48:47 +02:00
allegroai
3f56411bcc Version bump to v0.17.5 2021-03-16 10:20:56 +02:00
allegroai
c3bb260b92 Remove auto comment on new projects 2021-03-15 17:39:10 +02:00
allegroai
d37aa23fbf Fix spawn logger/reporting 2021-03-11 09:42:35 +02:00
allegroai
737ca91d2a Fix TF 2.4 keras load/save model 2021-03-11 09:41:14 +02:00
allegroai
dacf097ebb Fix missing required argument --files in clearml-data remove 2021-03-11 09:40:56 +02:00
allegroai
532010c93d Add tags to Dataset listing 2021-03-11 09:40:31 +02:00
allegroai
b33ac21764 Fix clearml-data sync requires --name 2021-03-11 09:39:56 +02:00
allegroai
880e2b7507 Fix Dataset should set system tag on current_task 2021-03-11 09:38:46 +02:00
H4dr1en
f067caca52
Add force_download argument to Artifact.get() and Artifact.get_local_copy() (#319) 2021-03-09 17:42:45 +02:00
allegroai
41cad87b77 Fix artifact preview limit 64kb 2021-03-06 01:28:33 +02:00
allegroai
0f65d6345d version bump 2021-03-06 00:48:04 +02:00
allegroai
e1d92e2419 Improve background reporting speed 2021-03-06 00:47:58 +02:00
allegroai
4106aa9c49 Fix set_base_docker in remote execution 2021-03-06 00:47:06 +02:00
allegroai
1374543b93 Fix python 2.7 support 2021-03-06 00:46:37 +02:00
allegroai
2682754aaf Fix TF bind keyboard interrupt 2021-03-06 00:46:22 +02:00
allegroai
912f6f5ba2 Fix matplotlib with Agg backend (or in remote execution) 2021-03-03 15:05:22 +02:00
allegroai
ecc539ffb6 Add Dataset.get_logger() 
Fix using Dataset on current_task set Task type
 2021-02-28 19:49:38 +02:00
allegroai
f91645fdaf Add Dataset.get() dataset_tags argument 2021-02-28 19:49:06 +02:00
allegroai
70a37b9d54 Fix PyYAML scientific notation parsing (issue #313) 2021-02-27 23:53:56 +02:00
allegroai
97a39c9f1a Fix utility nested_from_flat_dictionary 2021-02-27 23:53:27 +02:00
allegroai
88cd7071f2 Change pipeline aborted Task in diff shade of blue 2021-02-27 23:52:33 +02:00
allegroai
a2557c253c Disable progress upload callback at the end of http driver upload until we have full upload progress 2021-02-27 23:52:14 +02:00
allegroai
7e6158dd9b Add Task.add_requirements specify version <>=~ etc. 2021-02-27 23:51:40 +02:00
allegroai
e24a421457 Fix Hydra 1.1 support (argparse description) 2021-02-27 23:51:06 +02:00
allegroai
ae6caf4404 PEP8 2021-02-23 10:38:02 +02:00
allegroai
60b2f5bd74 Dataset remove files should not mark "dirty" flag as there is nothing to upload 2021-02-23 10:37:23 +02:00
allegroai
5641cb5a7c Fix JupyterLab notebook detection 2021-02-23 10:36:55 +02:00
allegroai
cf3c6ba815 PEP8 2021-02-21 15:00:15 +02:00
allegroai
132e76cfb9 Fix multiple Task.init()/close() calls in the same process 2021-02-21 14:59:50 +02:00
allegroai
ee9dfb8b35 Fix TensorBoard multiple Task.init()/close() calls in the same process (issue #312) 2021-02-21 14:58:40 +02:00
allegroai
c6d22c2d0a Add links to Tasks in optimization summary table (not just Task IDs) 2021-02-21 14:57:55 +02:00
allegroai
ff7e55756c Fix server updated with the argparse in remote before Task.init() is called (respect skipped args) 
Fix nonstandard argparse with default value that is not of defined type
 2021-02-21 14:56:30 +02:00
allegroai
41230ac2c7 Add progress report logging for StorageHelper.upload_from_stream, StorageHelper.upload and StorageHelper.upload_from_stream 2021-02-21 14:55:04 +02:00
allegroai
7dbf5471cc Allow Dataset to be created on a running Task 2021-02-21 14:51:24 +02:00
allegroai
c58e8a4c6a Fix make Pipeline Summary Table link to Task step logs 2021-02-14 13:52:09 +02:00
allegroai
aad9f5c812 Fix Task.create with repo and script that exists locally 2021-02-14 13:51:27 +02:00
allegroai
e5747d587d Add PipelineController disable clone base task option 2021-02-14 13:50:58 +02:00
allegroai
0c017a7331 Add "clearml-data publish" to allow publishing a dataset task 2021-02-11 14:37:41 +02:00
allegroai
1600101f94 Allow publishing a completed task 2021-02-11 14:35:39 +02:00
allegroai
787079669f Fix exception at exit in python 3.8+ on macOS/Windows 2021-02-11 14:28:05 +02:00
allegroai
7a25f8afd9 Fix hydra multi-run support (issue #306) 2021-02-11 14:27:28 +02:00
allegroai
f64d1a0993 Fix clearml-task patching local git diff 2021-02-11 14:27:00 +02:00
allegroai
1b6b1e12ee Fix clearml-task not passing --docker 2021-02-11 14:26:20 +02:00
allegroai
8babddd654 Fix pr_curve in TensorBoard 2+ 2021-02-11 14:25:43 +02:00
allegroai
e9f8fc949d Fix argparser logging always captures defaults (Windows only, ignored cmd) 2021-02-08 13:25:31 +02:00
allegroai
5a76ffd460 Fix Task.connect(object) should always return the same object instance 2021-02-08 13:23:16 +02:00
glemarivero
fbbae0b8bc
Fix Dataset.remove_files() does not match when files are in the root of the dataset (#303) 
Co-authored-by: Gabriel Lema <glema@ulta.com>
 2021-02-07 17:12:01 +02:00
allegroai
28b85028fe Fix close task after logger is closed 2021-02-04 10:41:51 +02:00
allegroai
71c45fcdd3 Update Auto Scaler default values and configuration format 2021-02-01 23:44:01 +02:00
allegroai
12d51dae92 Fix crash in case get_logger() cannot get the file name 2021-02-01 23:42:41 +02:00
allegroai
0d89497cc9 Clean up TF binding graph execution operator 2021-02-01 23:41:53 +02:00
allegroai
6778b982a7 Fix TF +2.3 mixed eager mode execution summary metrics not reported 2021-02-01 23:41:18 +02:00
allegroai
3890477056 Fix creating subprocess from a daemon process 2021-02-01 23:40:48 +02:00
allegroai
1c1fecf7d0 Improve docstring 2021-02-01 23:39:44 +02:00
allegroai
2f5080a34c Fix python package detection sklearn -> scikit-learn (issue #296) 2021-02-01 23:39:07 +02:00
allegroai
4931429054 Fix argparse with [None] in default parameter (issue #297) 2021-02-01 23:38:21 +02:00
allegroai
e3109c96d8 Add upload HTML preview of jupyter notebook as an artifact 2021-02-01 23:37:44 +02:00
tankeco
ebc0733357
Fix strip remote diff (#295) 
Co-authored-by: wjn <wjn@megvii.com>
 2021-01-25 13:20:21 +02:00
allegroai
0f401545b8 Replace humanfriendly with utility functions 2021-01-24 09:22:56 +02:00
allegroai
1c84b31056 Fix Dataset generate removed / modified / added of summary table 2021-01-24 09:21:28 +02:00
allegroai
698977d05e Fix reporter from non main tasks should always use threads (not subprocesses) 2021-01-24 09:20:42 +02:00
allegroai
2676e14d4d Fix reporter should skip report if report thread service is closed 2021-01-24 09:20:09 +02:00
allegroai
03f9bf0c2d Add clearml-data CLI sync can now create, sync and close a dataset in a single command 2021-01-24 09:19:33 +02:00
allegroai
088eae0ded Fix closing a dataset with only files removed 
Add system_tag `dataset` to a Dataset Project
 2021-01-24 09:18:49 +02:00
allegroai
381ae21cef Fix issue #292 pytorch-lighting multi node store 2021-01-24 09:18:18 +02:00
allegroai
828af76ffe Support all reporting using subprocesses instead of threads. Configure using sdk.development.report_use_subprocess 2021-01-24 09:17:52 +02:00
allegroai
4b4fa3eff0 Add Jupyter auto-magic store jupyter notebook as an artifact on the Task. Use sdk.development.store_jupyter_notebook_artifact=True by default 2021-01-24 09:15:24 +02:00
allegroai
0962e53f0b Fix argparse nargs passed in command line --nargs 1 2 should be stored as '[1, 2]' not as "['1', '2']" 2021-01-24 09:14:27 +02:00
allegroai
6b8f21b30e Add support for datetime in scatter plots + matplotlib support 2021-01-24 09:13:32 +02:00
allegroai
dd03ca0d1d Fix issue #291 2021-01-24 09:12:12 +02:00
allegroai
1988cf8adf Fix populate Task called from jupyter notebook. Added packages=True to auto populate based on locally installed packages 2021-01-24 09:11:09 +02:00
allegroai
fef2ddb43a Fix broken packages on package importlib detection failed the entire requirements detection 2021-01-24 09:06:17 +02:00
allegroai
29a5df407f Fix remote_execution from jupyter notebook 2021-01-24 09:05:18 +02:00
allegroai
715a0d7edc Fix pr_curve should not be inverted 2021-01-18 20:23:37 +02:00
allegroai
59d26e675b Improve plotly value type conforming 2021-01-18 11:36:10 +02:00
allegroai
8c4d3aca5b Add StorageManager.upload_folder() and StorageManager.download_folder() 2021-01-18 11:35:23 +02:00
allegroai
d06504c32c Fix plotly plot with numpy containing NaN/datetime 2021-01-18 11:34:31 +02:00
allegroai
8c0c85ae9c Fix clearml-task error when script cannot be found 2021-01-18 11:33:04 +02:00
allegroai
c8350a0438 Remove import imp (deprecated) 2021-01-18 11:32:41 +02:00
allegroai
661fac95a2 Flake8 2021-01-18 11:32:12 +02:00
allegroai
c8a690569f Fix trying to upload model file as a folder, automatically package the folder 2021-01-18 11:31:37 +02:00
allegroai
eb5d5da59e Fix None values in URI lists 2021-01-18 11:30:55 +02:00
allegroai
b0000df575 Fix unsafe call to set_active() 2021-01-18 11:30:17 +02:00
allegroai
34c41cfc8c Version bump to v0.17.4 2021-01-13 14:45:28 +02:00
allegroai
b123fb0cdd Fix python package detection should not list file:// links 2021-01-13 14:01:18 +02:00
allegroai
a4126bab01 Fix PyJWT v2.0 token parsing 2021-01-13 13:58:19 +02:00
allegroai
20fa70b33a Change HPO parallel coordinates color scale, yellow is low 2021-01-12 20:23:26 +02:00
allegroai
d4317f8b87 Fix argparse with action="append" 2021-01-12 20:23:01 +02:00
allegroai
794d97ca7c Change HyperParameterOptimizer spawn_task_project to spawn_project 
Add HyperParameterOptimizer parallel coordinates (issues #279)
 2021-01-12 20:22:05 +02:00
allegroai
54132e2505 Add Task.init() argument tags 2021-01-12 18:57:10 +02:00
allegroai
a35d172e5b Fix argparse nargs support broken 2021-01-12 18:56:45 +02:00
allegroai
e862a6de60 Version bump to v0.17.3 2021-01-11 18:48:58 +02:00
allegroai
1f87d25903 Change clearml-data "id=" to "id " for easier copy/paste 2021-01-11 18:30:14 +02:00
allegroai
f1f595b886 Fix Dataset.get dataset version without changeset 2021-01-11 18:29:34 +02:00
allegroai
f679c80535 Add HyperParameterOptimizer arguments: spawn_task_project, save_top_k_tasks_only 2021-01-10 13:07:57 +02:00
allegroai
49ba89587f Add internal Task._get_task_status() 2021-01-10 13:07:24 +02:00
allegroai
bb648bdb02 Fix dataset genealogy, graph and restoring data 2021-01-10 13:06:50 +02:00
allegroai
4beb21eb2b Add clearml-data close now auto uploads 2021-01-10 13:06:30 +02:00
allegroai
cc40b04a3c Add Task.set_credentials() option to store into credentials file 2021-01-10 13:06:10 +02:00
allegroai
ce5fd31070 Fix dataset upload aborted on server watchdog 2021-01-10 13:03:40 +02:00
allegroai
1b3574a7ca Remove pandas from HyperParameter optimizer requirements 2021-01-10 13:02:28 +02:00
allegroai
189a2b54de Fix PyJWT support v2 2021-01-10 13:02:08 +02:00
allegroai
d39f7bc195 Add Task.set_archived() / Task.get_archived() 2021-01-10 13:01:04 +02:00
allegroai
c9026ad4a1 Fix auto python package detection installed directly from URLs 2021-01-10 13:00:39 +02:00
allegroai
e905743fbf Added HyperParameterOptimizer.get_optimizer_top_experiments() for querying post execution optimization pipeline 2021-01-10 13:00:23 +02:00
allegroai
4d5e77f67c Fix Optuna optimizer failing on tasks with None value in scalar query 2021-01-10 12:59:50 +02:00
allegroai
0d0cc642e1 Fix optimizer callback best experiment 2021-01-10 12:59:25 +02:00
allegroai
a80802ba6e Fix model upload 2021-01-10 12:59:03 +02:00
allegroai
fe86fbcac3 Fix mpl exporter. Added support for legend. 2021-01-10 12:58:44 +02:00
allegroai
14706c03e1 Fix dataset add files 
Fix docs typo
 2021-01-10 12:58:13 +02:00
allegroai
12e575a155 Make clearml-data CLI stateful, remember last dataset id as default dataset 2021-01-10 12:57:18 +02:00
allegroai
dfb39249c4 Fix Dataset add single file 
Fix get project name from parent dataset if not specified
 2021-01-10 12:56:10 +02:00
allegroai
918d685ed1 Add Task.init() argument output_uri=True will use the default files_server as output uri 2021-01-10 12:55:31 +02:00
allegroai
f1de48a082 Fix Azure support delete_object() returns incorrect value 2021-01-10 12:54:15 +02:00
allegroai
365c79326a Fix argparse support to store consistent str representation of custom objects. Avoid changing default value if remote value matches. 
Fix argsparse type as function
 2021-01-10 12:53:15 +02:00
allegroai
3cb42acd12 Improve utility get_single_result() 2021-01-10 12:51:22 +02:00
allegroai
9da307730c Add Task.debug_simulate_remote_task() to simulate task execution by ClearML Agent 2021-01-10 12:50:43 +02:00
allegroai
2b9a28f908 Add warning on archived Task in pipeline (issue #274) 2021-01-10 12:49:44 +02:00
allegroai
c28f8cf65b Fix Task.archived_tag read-only property does not work (issue #274) 2021-01-10 12:49:06 +02:00
allegroai
504dfd8f32 Fix reusing task after its project was deleted (issue #274) 2021-01-10 12:48:16 +02:00
allegroai
78e892f337 Change default offline API version to 2.9 (trains-server 0.16) 2021-01-10 12:47:52 +02:00
allegroai
5608b7238c Fix PipelineController running remotely without configuration does not execute the default code pipeline (issue #273) 2021-01-10 12:47:25 +02:00
allegroai
35a8471943 Use archived task constant 2021-01-10 12:46:51 +02:00
allegroai
3c00453bd4 Add Task.delete() support 2021-01-10 12:46:02 +02:00
erezalg
c4158b55b2
fix manager.py (#275) 
Co-authored-by: jkhenning <>
 2021-01-04 12:15:36 +02:00
allegroai
0bb590b5da Fix PY2 compatibility 2020-12-30 16:50:45 +02:00
allegroai
643bc3d3ea Add support for password protected jupyter using sdk.development.jupyter_server_password configuration option 2020-12-30 16:50:17 +02:00
allegroai
80d3a02f4e Add specific error for password protected jupyter notebook (not supported) 2020-12-30 16:45:11 +02:00
allegroai
9f0d7b5eb8 Fix Task.create() standalone mode 2020-12-30 16:40:47 +02:00
allegroai
486520abfd Fix extracted archives were extracted to the running folder instead of the cached folder 
Fix support for direct access links
 2020-12-30 16:40:25 +02:00
allegroai
2b81bc33a5 Add argparse argument passing to Task.create() (issue #230) 2020-12-30 16:38:56 +02:00
allegroai
6ca4836bea Add clearml-data - Dataset Management & Versioning CLI 2020-12-30 16:38:25 +02:00
allegroai
e8af6d7e02 PEP8 2020-12-30 16:37:25 +02:00
allegroai
1484a7005e version bump 2020-12-25 02:23:10 +02:00
allegroai
7edc998824 Fix clearml-task 2020-12-24 00:29:00 +02:00
allegroai
e5e1396f66 version bump 2020-12-23 01:53:02 +02:00
allegroai
d3ccbe1d5b version bump 2020-12-23 00:17:21 +02:00
allegroai
d327f2e2b9 clearml initial version 0.17.0 2020-12-22 23:25:37 +02:00
allegroai
a460df1e68 renamed trains -> clearml 2020-12-22 21:17:56 +02:00