ClearML/clearml-server
1
0
Fork 0
mirror of https://github.com/clearml/clearml-server synced 2025-02-07 13:33:42 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
708 Commits 1 Branch 46 Tags 4.4 MiB
3f22423c3f
Commit Graph

383 Commits

Author SHA1 Message Date
allegroai
b2feafac09 Support workers filtering with tags 2022-07-08 17:37:33 +03:00
allegroai
b41ab8c550 Better support for queue metrics and queue metrics refresh on sample 2022-07-08 17:36:46 +03:00
allegroai
62d5779bd5 Count own tasks/models for projects 2022-07-08 17:35:01 +03:00
allegroai
f8b9d9802e Add support for organization.get_entities_count 2022-07-08 17:32:56 +03:00
allegroai
dd8a1503b0 Add support for navigate_current_metric in events.get_debug_image_sample 2022-07-08 17:31:44 +03:00
allegroai
cff98ae900 Add support for events.get_task_single_value_metrics, events.plots, events.get_plot_sample and events.next_plot_sample 2022-07-08 17:29:39 +03:00
allegroai
9b108740da Bump PyJWT version due to "Key confusion through non-blocklisted public key formats" vulnerability 2022-05-25 16:50:19 +03:00
allegroai
08a7bc7c9f Fix not all the event logs are returned from sharded ES 2022-05-20 15:11:05 +03:00
allegroai
fb256d7e5b Version bump to v1.5 2022-05-18 15:29:45 +03:00
allegroai
710443b078 Fix move task to trash is not thread-safe 2022-05-18 10:31:20 +03:00
allegroai
e0cde2f7c9 Add support for deleting pipeline projects 2022-05-18 10:30:21 +03:00
allegroai
60b9c8de14 Allow arbitrary task fields in project statistics filter 2022-05-18 10:29:36 +03:00
allegroai
ecffe26be4 Fix auth.edit_credentials 2022-05-18 10:28:58 +03:00
allegroai
2570bd9e26 Fix ES issue with capital letters in index name 2022-05-18 10:18:23 +03:00
allegroai
174f84514a Fix no destination when merging projects 2022-05-18 10:17:34 +03:00
allegroai
65cb8d7b43 Refactor method name 2022-05-18 10:16:41 +03:00
allegroai
5f8ef808a3 Fix ES issue with capital letters in index name 2022-05-18 10:16:19 +03:00
allegroai
4941ac70e0 Add events.clear_task_log 2022-05-17 16:09:23 +03:00
allegroai
67cd461145 Add auth.edit_credentials 2022-05-17 16:08:12 +03:00
allegroai
92b5fc6f9a Fix handling hidden sub-projects 2022-05-17 16:06:34 +03:00
allegroai
b90165b4e4 Support queue_name in tasks enqueue 2022-05-17 16:04:34 +03:00
allegroai
6c2dcb5c8a Improve error message 2022-05-17 15:56:18 +03:00
allegroai
3efed32934 Add X-Jwt-Payload to redacted headers 2022-05-17 15:55:41 +03:00
allegroai
69737308fe Version bump to v1.4.0 2022-04-18 16:38:22 +03:00
allegroai
a6dbea808a Add indices for task.last_update and task.status_changed 2022-04-18 16:37:22 +03:00
allegroai
5131b17901 Support not returning hidden sub-projects when include_stats is specified without search_hidden 2022-04-18 16:36:14 +03:00
allegroai
5f21c3a56d Add support for searching hidden projects and tasks 2022-04-18 16:34:18 +03:00
allegroai
2350ac64ed Fix internal error on count task events if there is no events index 2022-04-18 16:31:02 +03:00
allegroai
d146127c18 Add events.clear_scroll endpoint to clear event search scrolls 2022-04-18 16:29:57 +03:00
allegroai
124684f53f Version bump to v1.3.0 2022-03-15 16:34:35 +02:00
allegroai
455b5d6758 Fix pre-populate to convert model metadata from the old format 2022-03-15 16:30:14 +02:00
allegroai
c04e2e498b Support credentials label and last_used_from fields 2022-03-15 16:29:37 +02:00
allegroai
da8a45072f Add pipelines support 2022-03-15 16:28:59 +02:00
allegroai
e1992e2054 Fix queue metrics calculation 2022-03-15 16:28:49 +02:00
allegroai
b6ad8f8790 Add support for worker auto-unregister (instead of raising an error) 2022-03-15 16:25:14 +02:00
allegroai
5acc7eebc3 Set API version to 2.17 2022-03-15 16:22:51 +02:00
allegroai
02933a9c93 Support disabling response compression 
Return fixed server header
 2022-03-15 16:21:14 +02:00
allegroai
e537651f29 Better support for assets upload/download 2022-03-15 16:19:52 +02:00
allegroai
af09fba755 Add metadata dict support for models, queues 
Add more info for projects
 2022-03-15 16:18:57 +02:00
allegroai
fc4fd9e61c Version bump to v1.2.0 2022-02-14 15:26:27 +02:00
allegroai
8908c7dcf9 Update driver requirements 
Refactor ES initialization
 2022-02-13 20:27:12 +02:00
allegroai
b9996e2c1a Protect against multiple connects to the update server from different processes 
Code cleanup
 2022-02-13 20:12:12 +02:00
allegroai
afdc56f37c Use task active duration for worker task running time 2022-02-13 20:01:47 +02:00
allegroai
a25cd5dae8 Fix version conflicts when deleting task events cause an error 2022-02-13 20:01:25 +02:00
allegroai
447adb9090 Add support for credentials label 
Support no_scroll in events.get_task_plots
Support better project stats
Fix Redis required on mongodb initialization
Update tests
 2022-02-13 19:59:58 +02:00
allegroai
92fd98d5ad Add support for lists and nested fields in URL args and form 2022-02-13 19:52:05 +02:00
allegroai
c4001b4037 Add Redis cluster support 
Fix for lru_cache usage
 2022-02-13 19:48:26 +02:00
allegroai
970a32287a Add Redis password support 2022-02-13 19:37:52 +02:00
allegroai
17cd48dada Add support for override cookie domains 
Support for community invitation alarms
Remove duplicate property
Add query optimizations
 2022-02-13 19:35:35 +02:00
allegroai
ea3b6e955f Optimize nested_get() 2022-02-13 19:32:22 +02:00
allegroai
843450bb9b Fix add_or_update_artifacts should always be allowed on in_progress tasks 
Fix delete_artifacts should always be allowed on in_progress tasks
Fix query code
 2022-02-13 19:31:54 +02:00
allegroai
e149af58b1 Support for additional mata data in api call response 2022-02-13 19:30:36 +02:00
allegroai
604a38035b Add organization.update_company_name 
Fix unit-tests
 2022-02-13 19:29:46 +02:00
allegroai
cae38a365b Fix base query building 
Fix schema
Improve events.scalar_metrics_iter_raw implementation
 2022-02-13 19:28:23 +02:00
allegroai
e334246b46 Add support for project stats with children flag 2022-02-13 19:26:47 +02:00
allegroai
36e013b40c Add support for events.scalar_metrics_iter_raw 2022-02-13 19:26:03 +02:00
allegroai
f20cd6536e Add scroll support to *.get_* 2022-02-13 19:23:29 +02:00
allegroai
446bd35006 Refactor debug images response, model ORM 2022-02-13 19:21:07 +02:00
allegroai
a377a7e315 Support status_message and status_reason in tasks.delete 2022-02-13 19:20:31 +02:00
allegroai
3d046ac282 Fix project should not be merged into itself 2022-02-13 19:18:08 +02:00
allegroai
a08fa9a0e1 Add missing API Errors 2022-02-13 19:16:58 +02:00
allegroai
5856ed2836 Update Model.last_update on changes to tags and system tags 2022-02-13 19:15:37 +02:00
allegroai
d295355d99 Better logger name if called from __init__.py 2022-02-13 19:15:10 +02:00
Niels ten Boom
bc2c2ebbfd
Add connection string functionality for MongoDB access (#102) 2022-01-08 12:07:59 +02:00
Théo Mathieu
e352a6a1e7
Fix elasticsearch authentication when initializing (#98) 2021-12-05 09:55:06 +02:00
Théo Mathieu
8a3d992aaf
Support MongoDB SRV endpoints (#96) 2021-12-02 10:07:33 +02:00
allegroai
c37f3d8d5b Fix set() not supported in ConfigTree() 
Add user/pass config support
 2021-11-15 18:33:49 +02:00
allegroai
a96870e092 Add admonition in case only username or password were provided 2021-11-15 15:19:07 +02:00
Weixiao Huang
3d816c747d
Add ES http_auth credentials support (#93) 
Also update ES and MongoDB versions and fix nginx configuration bug

Co-authored-by: huangweixiao <huangweixiao@megvii.com>
 2021-11-15 15:01:27 +02:00
allegroai
1bc8529d83 Version bump 2021-08-05 16:46:29 +03:00
allegroai
083fd315e9 Fix server error when running with non-migrated v0.16 ElasticSearch data 2021-08-05 16:46:05 +03:00
allegroai
c5d6ce3e65 Version bump 2021-07-25 14:40:57 +03:00
allegroai
694dbc31c4 Fix incorrect ES query (merge issue) 2021-07-25 14:40:49 +03:00
allegroai
6488dc54e6 Better handling of stack trace report on 500 error 2021-07-25 14:39:59 +03:00
allegroai
158da9b480 Allow setting status_message in tasks.update 
Optimizations and refactoring
 2021-07-25 14:35:36 +03:00
allegroai
ec2e071ab7 Fix mongoengine cannot handle field name with leading or trailing "_" when used in fields query within get_all endpoints 2021-07-25 14:34:04 +03:00
allegroai
465e270342 Fix queued task is not dequeued on tasks.stop 2021-07-25 14:32:09 +03:00
allegroai
6705aff56f Allow requesting plots and iter_histograms for all variants 2021-07-25 14:30:38 +03:00
allegroai
9069cfe1da Support querying task events per specific metrics and variants 2021-07-25 14:29:41 +03:00
allegroai
677bb3ba6d Add force parameter to tasks.enqueue 2021-07-25 14:27:46 +03:00
allegroai
cb253cff9e Don't use special characters in secrets 2021-07-25 14:26:49 +03:00
allegroai
39ceb5ac5c Fix pre-populate logic to avoid overriding existing users 2021-07-25 14:26:31 +03:00
allegroai
d4edeaaf1b Add projects.validate_delete 2021-07-25 14:17:29 +03:00
allegroai
56aea1ffb8 Fix filtering on hyperparams (https://github.com/allegroai/clearml/issues/385, https://clearml.slack.com/archives/CTK20V944/p1626600582284700) 2021-07-25 13:55:09 +03:00
allegroai
09ab2af34c Version bump 2021-05-27 17:13:19 +03:00
allegroai
3f2304549d Move new migrations to 1_0_2 2021-05-27 16:56:47 +03:00
allegroai
ad72a435f1 Clean Task runtime on reset 2021-05-27 16:56:03 +03:00
allegroai
f34332344e Fix Task container raises validation error on null values 2021-05-27 16:55:32 +03:00
allegroai
2216bfe875 Version bump 2021-05-11 16:12:48 +03:00
allegroai
9beefa7473 Add missing login.logout endpoint 2021-05-11 16:12:27 +03:00
allegroai
8ebc334889 Fix broken config dir backwards compatibility (/opt/trains/config should still be supported) 2021-05-11 16:12:13 +03:00
allegroai
1e5163e530 Upgrade jinja2 version due to CVE-2020-28493 2021-05-03 23:23:06 +03:00
allegroai
1567774765 Version bump 2021-05-03 18:20:32 +03:00
allegroai
babfcbb707 Update migration script 2021-05-03 18:15:43 +03:00
allegroai
4fe61ee25c Fix running migration scripts calling other files 2021-05-03 18:13:49 +03:00
allegroai
e18b21639c Fix regex query for fields containing "_" 2021-05-03 18:13:00 +03:00
allegroai
1cef03b8c2 Add check_contents flag for projects.get_all_ex 2021-05-03 18:12:44 +03:00
allegroai
27d086bca2 Fix schema for Task.runtime 
Add infrastructure for API calls limits handling
 2021-05-03 18:11:46 +03:00
allegroai
add3f011a0 Add runtime to tasks.edit 2021-05-03 18:10:48 +03:00
allegroai
ee90b0b024 Remove "Auto-generated while cloning" project description 2021-05-03 18:10:32 +03:00
allegroai
9bf107866f Fix crash in models publish_many without model task 2021-05-03 18:10:09 +03:00
allegroai
4d2f282950 Add Model.last_update to schema 2021-05-03 18:09:54 +03:00
allegroai
b55fad1b59 Remove "Auto-generated during move" project description 2021-05-03 18:09:31 +03:00
allegroai
ba77ff11e9 Fix missing custom metric values turn up first in sorting 2021-05-03 18:08:39 +03:00
allegroai
b67aa05d6f Return results per task iterations in debug images request 2021-05-03 18:08:14 +03:00
allegroai
6b0c45a861 Fix batch operations results 2021-05-03 18:07:37 +03:00
allegroai
dc9623e964 Fix docker_cmd projection in backwards compatibility 
Fix support to clear input/output models and docker_cmd in backwards compatibility mode
Fix schema
 2021-05-03 18:06:39 +03:00
allegroai
3d73d60826 Better handling of invalid iterations on add_batch 2021-05-03 18:05:24 +03:00
allegroai
9f0c9c3690 Fix open ranges 2021-05-03 18:05:03 +03:00
allegroai
1a3d3494ce Fix numeric locale 2021-05-03 18:04:45 +03:00
allegroai
b99f620073 Added unarchive APIs 2021-05-03 18:04:17 +03:00
allegroai
e2f265b4bc Unify batch operations 2021-05-03 18:03:54 +03:00
allegroai
251ee57ffd Fix rapidjson dumps does not support ensure_ascii, only Encoder initialization does 
Add task enqueue status
 2021-05-03 18:03:17 +03:00
allegroai
7e03104f1c Add Model last_update field 2021-05-03 18:02:25 +03:00
allegroai
f1a258208e Disable backwards compatibility for 2.13 clients 2021-05-03 18:01:59 +03:00
allegroai
66cc49313b Fix schema 2021-05-03 18:01:29 +03:00
allegroai
9ae2943f7d Fix crash in tasks.reset 2021-05-03 17:59:44 +03:00
allegroai
54326f707b Add JSON flags support to APICall 2021-05-03 17:58:57 +03:00
allegroai
3a3b57c15f Support mongodb authentication 2021-05-03 17:57:53 +03:00
allegroai
8ea8ad34e6 Remove collecting task output models from Models collection during migration 2021-05-03 17:57:27 +03:00
allegroai
179661a0d4 Rename default input and output models 
Better handling of backwards compatibility in task models
Code cleanup
 2021-05-03 17:56:50 +03:00
allegroai
3d22ca1888 Escape task.container and task.execution.model_labels fields in DB 2021-05-03 17:56:17 +03:00
allegroai
fdf6798d0c Don't unset Task's execution.queue on dequeue 2021-05-03 17:54:16 +03:00
allegroai
9d9a44b927 Add skip_empty parameter in get_configuration_names 2021-05-03 17:53:56 +03:00
allegroai
a75534ec34 Add batch operations support 2021-05-03 17:52:54 +03:00
allegroai
eab33de97e Add bcrypt support to fixed user password 2021-05-03 17:52:25 +03:00
allegroai
29de110abb Add support for queue and model metadata 2021-05-03 17:50:25 +03:00
allegroai
2e7f418ee2 Fix Task.container backwards-compatibility 
Fix sub-projects
 2021-05-03 17:49:48 +03:00
allegroai
dadb996d22 Refactor es_factory to better support override host/port 2021-05-03 17:48:41 +03:00
allegroai
174f692edf Code cleanup 2021-05-03 17:48:24 +03:00
allegroai
f4d5168a20 Add Task.container support 2021-05-03 17:48:01 +03:00
allegroai
5a438e8435 Fix projects.move 2021-05-03 17:47:11 +03:00
allegroai
ce4814dc47 Add field override support in config (using "-" prefix) 2021-05-03 17:46:36 +03:00
allegroai
ef42d0265d Add multi-models support 2021-05-03 17:46:00 +03:00
allegroai
3c5195028e More sub-projects support and fixes 2021-05-03 17:44:54 +03:00
allegroai
0d5174c453 Support iterating over all task metrics in task debug images 2021-05-03 17:43:02 +03:00
allegroai
c034c1a986 Add sub-projects support 2021-05-03 17:42:10 +03:00
allegroai
1b49da8748 Revoke tests account in fixed mode, cleanup 2021-05-03 17:40:41 +03:00
allegroai
26bda01a28 Add missing errors 2021-05-03 17:39:49 +03:00
allegroai
f5008d80ad Optimize and improve tasks/models/projects.delete 2021-05-03 17:39:13 +03:00
allegroai
8b464e7ae6 Return file urls for tasks.delete/reset and models.delete 2021-05-03 17:38:09 +03:00
allegroai
78e4a58c91 Fix API enum fields and add last_iteration to range queries 2021-05-03 17:37:49 +03:00
allegroai
7a4a5eb03e Fix dropping index by name during the migration fails if the index does not exist 2021-05-03 17:36:49 +03:00
allegroai
d029d56508 Support active users in projects 2021-05-03 17:36:04 +03:00
allegroai
6411954002 Improve visibility for distributed lock hanging 2021-05-03 17:35:17 +03:00
allegroai
7f4ad0d1ca Support projects.get_hyperparam_values 2021-05-03 17:34:40 +03:00
allegroai
4cd4b2914d Add range queries 
Switch from sematic_version to packaging.version in db migrations
 2021-05-03 17:33:47 +03:00
allegroai
1d55710a0b Update max API version 2021-05-03 17:33:12 +03:00
allegroai
8f646043bb Allow enqueueing stopped tasks 
More clearml stuff
 2021-05-03 17:31:02 +03:00
allegroai
4b11a6efcd Move apiserver to clearml 2021-05-03 17:26:44 +03:00
allegroai
749ff4a44f Fix Tasks.reset does not mark children's parent as deleted 2021-05-03 16:57:06 +03:00
allegroai
7d6918ecb0 Fix large plots comparison 2021-05-03 16:55:59 +03:00
allegroai
47184c2833 Fix querying by task parent 2021-05-03 16:55:03 +03:00
allegroai
59fc98e0c4 Upgrade Jinja2 version (vulnerability found in older versions) 2021-01-05 20:18:09 +02:00
allegroai
a936a210e8 Version bump to 0.17.0 2021-01-05 20:08:19 +02:00
allegroai
be0cf0caa8 Unify v0.17 migrations 2021-01-05 20:07:49 +02:00
allegroai
a8d90887e2 Fix task execution queue is not cleared on clone 2021-01-05 20:07:04 +02:00
allegroai
6f3257fed3 Fix X-ClearML headers to X-Clearml 2021-01-05 20:06:36 +02:00
allegroai
286b8c3df5 Change default company name to "clearml" 2021-01-05 19:05:11 +02:00
allegroai
16430a6636 Support query by task state in projects.get_tasks_parents, return task project names 2021-01-05 19:02:48 +02:00
allegroai
d7ddfde26e Fix tasks.failed error for task that was never started 2021-01-05 19:01:43 +02:00
allegroai
e6c0f1b6d8 Add migration: remove outdated user email index from db 2021-01-05 18:55:10 +02:00
allegroai
641ed1b510 Fix basic config not throwing missing key exception when no default passed 2021-01-05 18:54:13 +02:00
allegroai
3473d2bb02 Update PyJWT requirement (2.0.0 breaks interface) 2021-01-05 18:52:32 +02:00
allegroai
ba03924cb4 Fix archived and development system tags should not be cleaned up 2021-01-05 18:52:12 +02:00
allegroai
6870d8aba9 Refactor service_repo 
Code cleanup
 2021-01-05 18:50:42 +02:00
allegroai
64c63d2560 Add projects.get_task_parents 2021-01-05 18:49:25 +02:00
allegroai
88836fae66 Allow parent query in tasks.get_all 2021-01-05 18:48:25 +02:00
allegroai
436883148b Improve schema reader 2021-01-05 18:47:32 +02:00
allegroai
f9f2f0ccf0 Add request model detection in endpoint decorator 2021-01-05 18:47:01 +02:00
allegroai
f879f6924f Better exception log 2021-01-05 18:36:25 +02:00
allegroai
b9cb587580 Support docker_init_script in Task 2021-01-05 18:35:32 +02:00
allegroai
370e92c3dd Use sparse index for email addresses 2021-01-05 18:34:46 +02:00
allegroai
03094076c8 Fix tags handling fail reloading datetime-like strings 2021-01-05 18:32:18 +02:00
allegroai
bdf6c353bd Refactor APICall and schema validation 2021-01-05 18:30:59 +02:00
allegroai
23736efbc3 Add support for ClearML headers 2021-01-05 18:29:50 +02:00
allegroai
3c8e27dc94 Unify server request handlers 2021-01-05 18:28:43 +02:00
allegroai
ca890c7ae8 Remove dependency on api_version 2021-01-05 18:27:31 +02:00
allegroai
30909df73f Fix tasks.clone schema 
Reintroduce email field uniqueness
 2021-01-05 18:26:14 +02:00
allegroai
b97a6084ce Refactor configuration infrastructure 
Remove untracked files left from previous commit
 2021-01-05 18:25:18 +02:00
allegroai
50438bd931 Refactor apierrors infrastructure and auto-generation 2021-01-05 18:22:39 +02:00
allegroai
28daf49c91 Remove unique company name constraint 2021-01-05 18:21:49 +02:00
allegroai
4707647c92 Use EVENT_TYPE enum instead of string 2021-01-05 18:21:11 +02:00
allegroai
6974aa3a99 Improve internal events implementation 2021-01-05 18:20:38 +02:00
allegroai
e2deff4eef Fix update project time on task changes 
Fix project time in non responsive tasks watchdog
 2021-01-05 18:19:45 +02:00
allegroai
59994ccf9c Fix task and model last_change handling 
Improve db model index
Improve db model infrastructure
 2021-01-05 18:17:29 +02:00
allegroai
29c792d459 Fix tasks.clone 2021-01-05 18:15:01 +02:00
allegroai
df334d083e Add error message 
Improve error handling
 2021-01-05 18:14:29 +02:00
allegroai
b548958c80 Improve login.supported_modes 
Fix schema
 2021-01-05 18:13:43 +02:00
allegroai
7bdf8fe30d Fix DictField deserialization 2021-01-05 18:12:32 +02:00
allegroai
c71c65be87 Fix crash in auth.get_credentials if any of the credentials is missing last_used 2021-01-05 18:12:13 +02:00
allegroai
1cc6a8f787 Unify API model classes 2021-01-05 18:11:22 +02:00
allegroai
e5b92f4a80 Block users management in Redis 2021-01-05 18:10:32 +02:00
allegroai
3272d0f31f Rename migration script 
Support refresh flag in debug image samples
Remove silent_dequeue_fail param to prevent status change in case task wasn't queued
Add organizations.get_user_companies
Fix reset should also reset active_duration
Add api_version to server.info
 2021-01-05 18:09:34 +02:00
allegroai
618a0b9473 Do not set Task.last_update when moving or updating (i.e. changing name, comment, tags etc.) 2021-01-05 18:06:04 +02:00
allegroai
bca3a6e556 Set default task active duration to None 
Move endpoints to new API version
Support tasks.move and models.move for moving tasks and models into projects
Support new project name in tasks.clone
Improve task active duration migration
 2021-01-05 18:05:44 +02:00
allegroai
8b0afd47a6 Set configurable and consistent limits on variants and metrics across different iterators 2021-01-05 18:02:01 +02:00
allegroai
0303c3525f API version bump 
Update internal tests
Allow edit/delete task artifacts/hyperparams/configs using force flag
Improve lists query support for get_all calls
 2021-01-05 17:57:58 +02:00
allegroai
563c451ac9 Add task active duration migration 2021-01-05 17:53:44 +02:00
allegroai
91b1b34a6b Update configuration for debug images 2021-01-05 17:53:16 +02:00