ClearML/clearml-server
1
0
Fork 0
mirror of https://github.com/clearml/clearml-server synced 2025-02-26 05:59:20 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
865 Commits 1 Branch 46 Tags 4.4 MiB
v1.17.0
Commit Graph

523 Commits

Author SHA1 Message Date
allegroai
5239755066 Support include_subprojects flag in reports.get_all_ex endpoint 2023-07-26 18:30:34 +03:00
allegroai
8061dfedbb Fix NewListBucketsHelper backwards compatibility 2023-07-26 18:27:51 +03:00
allegroai
011164ce9b Support __$and condition for excluded terms in get_all_ex endpoints list filters 2023-07-26 18:26:49 +03:00
allegroai
8135cf5258 Add include_subprojects to tasks/models.get_all endpoints 
Fix escaping metadata for tasks, models and queues
 2023-07-26 18:24:49 +03:00
allegroai
a83a932e84 Add pipelines.delete_runs endpoint 2023-07-26 18:23:05 +03:00
allegroai
db021f2863 Add workers.get_count endpoint 2023-07-26 18:21:52 +03:00
allegroai
1b650b1689 Add projects.get_user_names endpoint 2023-07-26 18:21:16 +03:00
allegroai
14d18a7aba Remove obsolete duration field 2023-07-26 18:19:41 +03:00
allegroai
452f606889 Version bump to v1.11 2023-05-25 19:40:07 +03:00
allegroai
fc47ccbf09 Add default services agent user 2023-05-25 19:39:53 +03:00
allegroai
0206811342 Improve empty database check during startup 2023-05-25 19:39:17 +03:00
allegroai
a3ac1049a3 Update ClearML SDK dependency 2023-05-25 19:38:48 +03:00
allegroai
9206a7c57d Schedule external file URLs for deletion on models deletion 2023-05-25 19:36:28 +03:00
allegroai
0c37ced2a1 Fix model Id handling when deleting models for tasks 2023-05-25 19:35:18 +03:00
allegroai
b22f26129e Update requirements 2023-05-25 19:34:19 +03:00
allegroai
d8b998ebd8 Bump API version to 2.25 2023-05-25 19:33:37 +03:00
allegroai
741fa84b52 Fix projects own_tasks does not take task state filter into account 2023-05-25 19:32:52 +03:00
allegroai
d9579891c8 Return only reports from the .reports projects in reports.get_all_ex 2023-05-25 19:31:05 +03:00
allegroai
900414d0de Add option to echo ping payload 2023-05-25 19:30:13 +03:00
allegroai
5449b332d2 Support reports from the root project in reports.get_all_ex 2023-05-25 19:29:46 +03:00
allegroai
875f4b9536 Fix task dequeue will changes status for un-queued/running tasks 2023-05-25 19:28:49 +03:00
allegroai
4058fb9ce5 Migrate to python 3.9 bullseye docker images 
Update Mongo driver version
 2023-05-25 19:27:14 +03:00
allegroai
cf8e847ed3 Switch to new redis version 2023-05-25 19:22:39 +03:00
allegroai
755cc803d9 Add remove_from_all_queues parameter to tasks.dequeue/dequeue_many endpoints 2023-05-25 19:22:10 +03:00
allegroai
3729afe014 Optimize queues.get_next_task to retrieve required task fields only 2023-05-25 19:21:24 +03:00
allegroai
dff2ed34e8 Support receiving mixed events for both locked and unlocked tasks and models events.add_batch 2023-05-25 19:20:35 +03:00
allegroai
de9651d761 Allow mixing Model and task events in the same events batch 2023-05-25 19:19:45 +03:00
allegroai
818496236b Support filtering by children tags in projects.get_all_ex 2023-05-25 19:19:10 +03:00
allegroai
e99817b28b Task reports can now return single value metrics 2023-05-25 19:18:24 +03:00
allegroai
58465fbc17 Model events are fully supported 2023-05-25 19:17:40 +03:00
allegroai
2e4e060a82 Task move forward/backwards in queue is now atomic 2023-05-25 19:16:33 +03:00
allegroai
5c5d9b6434 Fix numeric hyperparam values are not sorted lexicographically with descending sort order 2023-05-25 19:15:59 +03:00
allegroai
4291ad682a Support filtering by task name in projects.get_task_parent 2023-05-25 19:15:26 +03:00
allegroai
4c22757002 Fix task that is not in queue but has 'queued' status can't be dequeued 2023-05-25 19:14:25 +03:00
allegroai
6e777e80b8 Cleaned up unit tests 2023-05-25 19:13:10 +03:00
dependabot[bot]
b51aa5c29b
Bump redis from 3.5.3 to 4.4.4 in /apiserver (#190) 
Bumps [redis](https://github.com/redis/redis-py) from 3.5.3 to 4.4.4.
- [Release notes](https://github.com/redis/redis-py/releases)
- [Changelog](https://github.com/redis/redis-py/blob/master/CHANGES)
- [Commits](https://github.com/redis/redis-py/compare/3.5.3...v4.4.4)

---
updated-dependencies:
- dependency-name: redis
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-04-01 08:59:02 +03:00
allegroai
e7c9daa42b Fix get_task_events to correctly use last_iters for model events 2023-03-28 16:45:44 +03:00
allegroai
7357654249 Version bump to v1.10 2023-03-23 19:17:00 +02:00
allegroai
17a8b440bd Fix only last event of each type is stored per model (all should be stored) 2023-03-23 19:16:30 +02:00
allegroai
eb2b9cbd9a Fix project count for datasets and pipelines 2023-03-23 19:15:42 +02:00
allegroai
30cfdac8f2 Fix project preview completed_tasks_24h should not count tasks that are marked as failed or running 2023-03-23 19:13:52 +02:00
allegroai
24bb87aaee Turn on mongo sorting using disk usage by default for sorting in *.get_all* apis 2023-03-23 19:12:52 +02:00
allegroai
dd49ba180a Improve statistics on projects children 2023-03-23 19:11:45 +02:00
allegroai
bda903d0d8 Set API version to 2.24 2023-03-23 19:11:13 +02:00
allegroai
9739eb2d5a Add report_assets field to report tasks 2023-03-23 19:09:03 +02:00
allegroai
cfbb37238f Add default workers timeout to the server's configuration 2023-03-23 19:08:11 +02:00
allegroai
6664c6237e Support querying by children_type in projects.get_all_ex 2023-03-23 19:07:42 +02:00
allegroai
74200a24bd Add filtering on child projects in projects.get_all_ex 2023-03-23 19:06:49 +02:00
allegroai
3a2675abe1 Version bump to v1.9.2 2023-01-24 16:11:21 +02:00
allegroai
f0d68b1ce9 Make sure model label values are integer 2023-01-24 16:11:12 +02:00
allegroai
15db9cdaef Allow updating comments on published reports 2023-01-24 14:40:32 +02:00
allegroai
b1a50c1370 Version bump to v1.9.1 2023-01-03 12:16:07 +02:00
allegroai
22a2a02760 Allow renaming published reports 2023-01-03 12:15:44 +02:00
allegroai
ab798e4170 Allow updating tags on published reports 2023-01-03 12:15:02 +02:00
allegroai
f09ac672d2 Add pipeline test 2023-01-03 12:14:12 +02:00
allegroai
2149b76f63 Fix crash when starting pipeline 2023-01-03 12:13:48 +02:00
allegroai
d96420aa67 Version bump to v1.9 2022-12-21 18:47:03 +02:00
allegroai
ed6c7b7bcb Fix Project time is not updated when moved or merged 2022-12-21 18:46:53 +02:00
allegroai
a392bc0bd7 Bump API version to 2.23 2022-12-21 18:46:12 +02:00
allegroai
7e97ec5555 Fix events.get_task_plots endpoint 2022-12-21 18:45:17 +02:00
allegroai
9c41124b81 Add support for moving objects to projects root 2022-12-21 18:43:45 +02:00
allegroai
14ff639bb0 Removed limit on event comparison for the same company tasks only 2022-12-21 18:42:40 +02:00
allegroai
e66257761a Add support for server-side delete for AWS S3, Google Storage and Azure Blob Storage 2022-12-21 18:41:16 +02:00
allegroai
0ffde24dc2 Add min and max value iteration to last metrics 2022-12-21 18:36:50 +02:00
allegroai
d4fdcd9b32 Upgrade mongoengine version 2022-12-21 18:35:23 +02:00
allegroai
18570bfccb Add project_id response field to reports.create endpoint 2022-12-21 18:35:14 +02:00
allegroai
54ce6c34c6 Fix bad field values might cause ugly server exception to be returned 2022-12-21 18:33:28 +02:00
allegroai
ae4c33fa0e Add support for allow_public flag in get_all_ex endpoint 
Add `last_changed_by` field on task updates
Fix reports support
 2022-12-21 18:32:56 +02:00
allegroai
c7cd949fd0 Add reports support 
Fix schema
 2022-12-21 18:30:54 +02:00
allegroai
1ce4058157 Change tasks comparison limit to 100 2022-12-21 18:29:49 +02:00
allegroai
7b6f24b24d Version bump to 1.8.0 2022-11-29 17:50:32 +02:00
allegroai
d03a931d84 Remove buggy debug code 2022-11-29 17:50:17 +02:00
allegroai
5cc7199661 Fix clearing ES scroll 2022-11-29 17:44:31 +02:00
allegroai
6537e9ef69 Add active_users and search_hidden options to get_entities_count endpoint 2022-11-29 17:44:19 +02:00
allegroai
930aaff791 Fix test 2022-11-29 17:43:43 +02:00
allegroai
9db14cc31d Update endpoint API versions 2022-11-29 17:42:19 +02:00
allegroai
e3cc689528 Enhance async_urls_delete feature with max_async_deleted_events_per_sec setting and fileserver timeout and prefixes 2022-11-29 17:41:49 +02:00
allegroai
9e0adc77dd Make sure id field is included in ordering 2022-11-29 17:40:09 +02:00
allegroai
58d9a64537 Fix endpoint name in schema 2022-11-29 17:39:35 +02:00
allegroai
d397d2ae20 Support optional async events deletion when deleting tasks 2022-11-29 17:38:41 +02:00
allegroai
2d711e1500 Collect model event URLs during task and project cleanup 2022-11-29 17:38:03 +02:00
allegroai
97992b0d9e Support returning multiple plots during history navigation 2022-11-29 17:37:30 +02:00
allegroai
bc23f1b0cf Add "queue watched" indication for tasks.enqueue and tasks.enqueue_many 2022-11-29 17:36:41 +02:00
allegroai
6b3eff1426 Support workers system tags 2022-11-29 17:35:25 +02:00
allegroai
caaf801cd0 Support plots navigation by iteration 2022-11-29 17:34:57 +02:00
allegroai
c23e8a90d0 Support model events 2022-11-29 17:34:06 +02:00
allegroai
fa5b28ca0e Support HTTP URLs when deleting fileserver files 2022-11-29 17:33:18 +02:00
allegroai
bfb55a9463 Support deleting external artifacts when deleting projects 2022-11-29 17:32:41 +02:00
allegroai
37e485e1f2 Upgrade API version to 2.22 2022-11-29 17:29:57 +02:00
allegroai
3451ff441f Fix projects.get_all_ex with active_users filtering did not work if the project id was passed as a string and not list 2022-11-29 17:27:54 +02:00
allegroai
53c9b5525e Add preliminary support for datasets under projects 2022-11-29 17:27:02 +02:00
PSL
e5230edac3
Add mongo username and password authentication (#162) 
Co-authored-by: pangshaoliang <pangshaoliang@megvii.com>
 2022-10-08 15:56:02 +03:00
allegroai
482a5c34bc Version bump to 1.7.0 2022-09-29 19:40:20 +03:00
allegroai
ee2a72c70f Add company/uri index 2022-09-29 19:40:05 +03:00
allegroai
de1f823213 Removed stub timing context 2022-09-29 19:37:15 +03:00
allegroai
0c9e2f92ee Add server-side support for deleting files from fileserver on task delete 2022-09-29 19:34:24 +03:00
allegroai
6c49e96ff0 Update API version to 2.21 2022-09-29 19:31:42 +03:00
allegroai
81e3fc6577 Improve utilities 2022-09-29 19:30:57 +03:00
allegroai
e6dc4b7557 Set cloned task parent to original task if original task has no parent 2022-09-29 19:30:13 +03:00
allegroai
238a47a197 Add created field to backend.user 2022-09-29 19:29:36 +03:00
allegroai
04e7076628 Add support for specifying a specific task ID in queues.get_next_task 2022-09-29 19:27:42 +03:00
allegroai
0531612bf4 Fix deleting a queue should dequeue all enqueued tasks 2022-09-29 19:27:09 +03:00
allegroai
3ae410a1e9 Remove the ThreadsManager.terminating flag 2022-09-29 19:23:26 +03:00
allegroai
98ed3075dd Added exclude support when converting mongo objects to dictionary 2022-09-29 19:21:28 +03:00
allegroai
b871bf4224 Fix projects.get_all_ex 2022-09-29 19:20:50 +03:00
allegroai
8d4c02fc3c Add support for hidden internal queues 2022-09-29 19:20:24 +03:00
allegroai
b986980c75 Use correct attrs version, remove related dependency 2022-09-29 19:18:38 +03:00
allegroai
a4fa567be2 Fix task stats update 2022-09-29 19:18:22 +03:00
allegroai
ddb91f226a Add Task Unique Metrics to task object 2022-09-29 19:16:56 +03:00
allegroai
7772f47773 Support datetime ranges in field queries 2022-09-29 19:15:50 +03:00
allegroai
9c118d14e0 Add missing last_update field to models.get_all APIs 2022-09-29 19:14:13 +03:00
allegroai
efd56e085e Fix threaded jobs management (invoke only from AppSequence) 2022-09-29 19:13:22 +03:00
allegroai
4dff163af4 Improve examples pre-population code 2022-09-29 19:11:21 +03:00
allegroai
242a78a0fe Add request logging using the CLEARML_SERVER_DEBUG_REQUESTS env var 2022-09-29 19:10:55 +03:00
allegroai
78989fea91 Add better mongodb connection string verbosity 2022-09-29 19:10:13 +03:00
allegroai
5de7c12062 Version bump to v1.6 2022-07-08 18:05:43 +03:00
allegroai
3f79c19079 Add v1.6.0 mongodb migration 2022-07-08 18:05:33 +03:00
allegroai
fe29743c54 Add support for new IDs generation when importing projects 2022-07-08 18:04:40 +03:00
allegroai
d760cf5835 Remove use of dpath in query projection 2022-07-08 18:04:02 +03:00
allegroai
3695f25a5f Fix internal error returned to clients 2022-07-08 18:03:38 +03:00
allegroai
c6f1beafdd Update API version to 2.20 2022-07-08 18:02:44 +03:00
allegroai
68a54c34f3 Add user creation time to users.get_current_user 2022-07-08 17:59:45 +03:00
allegroai
ab495ae586 Fix archived projects handling 2022-07-08 17:55:02 +03:00
allegroai
b058770af1 Fix handling of empty hyperparam/configuration keys 2022-07-08 17:54:19 +03:00
allegroai
f7e833bf6f Fix loading expired worker entries from Redis 2022-07-08 17:53:12 +03:00
allegroai
36b9ab0453 Fix handling of empty keys in query 2022-07-08 17:51:49 +03:00
allegroai
ec0436d0da Fix task cleanup 2022-07-08 17:50:49 +03:00
allegroai
0f6c4e75b7 Fix debug images URL handling and task routing 2022-07-08 17:50:26 +03:00
allegroai
a41ae112a1 Fix backward compatibility when importing old projects 2022-07-08 17:49:36 +03:00
allegroai
c28f478ea8 Fix worker Id is used instead of worker key when processing report 2022-07-08 17:48:17 +03:00
allegroai
c18eb99d06 Return getting_started_info in users.get_current_user 2022-07-08 17:45:33 +03:00
allegroai
3a60f00d93 Add support for Dataset projects 2022-07-08 17:45:03 +03:00
allegroai
ee87778548 Better support for PyJWT 2.4 2022-07-08 17:44:17 +03:00
allegroai
52c0c4d438 Add model task cleanup on tasks.reset 2022-07-08 17:43:54 +03:00
allegroai
d117a4f022 Support max_task_entries option in queues.get_by_id/get_all 
Add queues.peek_task and queues.get_num_entries
 2022-07-08 17:42:20 +03:00
allegroai
6683d2d7a9 Fix task cleanup 2022-07-08 17:40:55 +03:00
allegroai
05357fe25e Support publish option in tasks.completed 2022-07-08 17:40:43 +03:00
allegroai
adc1825843 Add support for model statistics 2022-07-08 17:39:41 +03:00
allegroai
0c15169668 Improve tests 2022-07-08 17:38:31 +03:00
allegroai
123dc1dcfb Improve query error handling 2022-07-08 17:38:22 +03:00
allegroai
b2feafac09 Support workers filtering with tags 2022-07-08 17:37:33 +03:00
allegroai
b41ab8c550 Better support for queue metrics and queue metrics refresh on sample 2022-07-08 17:36:46 +03:00
allegroai
62d5779bd5 Count own tasks/models for projects 2022-07-08 17:35:01 +03:00
allegroai
f8b9d9802e Add support for organization.get_entities_count 2022-07-08 17:32:56 +03:00
allegroai
dd8a1503b0 Add support for navigate_current_metric in events.get_debug_image_sample 2022-07-08 17:31:44 +03:00
allegroai
cff98ae900 Add support for events.get_task_single_value_metrics, events.plots, events.get_plot_sample and events.next_plot_sample 2022-07-08 17:29:39 +03:00
allegroai
9b108740da Bump PyJWT version due to "Key confusion through non-blocklisted public key formats" vulnerability 2022-05-25 16:50:19 +03:00
allegroai
08a7bc7c9f Fix not all the event logs are returned from sharded ES 2022-05-20 15:11:05 +03:00
allegroai
fb256d7e5b Version bump to v1.5 2022-05-18 15:29:45 +03:00
allegroai
710443b078 Fix move task to trash is not thread-safe 2022-05-18 10:31:20 +03:00
allegroai
e0cde2f7c9 Add support for deleting pipeline projects 2022-05-18 10:30:21 +03:00
allegroai
60b9c8de14 Allow arbitrary task fields in project statistics filter 2022-05-18 10:29:36 +03:00
allegroai
ecffe26be4 Fix auth.edit_credentials 2022-05-18 10:28:58 +03:00
allegroai
2570bd9e26 Fix ES issue with capital letters in index name 2022-05-18 10:18:23 +03:00
allegroai
174f84514a Fix no destination when merging projects 2022-05-18 10:17:34 +03:00
allegroai
65cb8d7b43 Refactor method name 2022-05-18 10:16:41 +03:00
allegroai
5f8ef808a3 Fix ES issue with capital letters in index name 2022-05-18 10:16:19 +03:00
allegroai
4941ac70e0 Add events.clear_task_log 2022-05-17 16:09:23 +03:00
allegroai
67cd461145 Add auth.edit_credentials 2022-05-17 16:08:12 +03:00
allegroai
92b5fc6f9a Fix handling hidden sub-projects 2022-05-17 16:06:34 +03:00
allegroai
b90165b4e4 Support queue_name in tasks enqueue 2022-05-17 16:04:34 +03:00
allegroai
6c2dcb5c8a Improve error message 2022-05-17 15:56:18 +03:00
allegroai
3efed32934 Add X-Jwt-Payload to redacted headers 2022-05-17 15:55:41 +03:00
allegroai
69737308fe Version bump to v1.4.0 2022-04-18 16:38:22 +03:00
allegroai
a6dbea808a Add indices for task.last_update and task.status_changed 2022-04-18 16:37:22 +03:00
allegroai
5131b17901 Support not returning hidden sub-projects when include_stats is specified without search_hidden 2022-04-18 16:36:14 +03:00
allegroai
5f21c3a56d Add support for searching hidden projects and tasks 2022-04-18 16:34:18 +03:00
allegroai
2350ac64ed Fix internal error on count task events if there is no events index 2022-04-18 16:31:02 +03:00
allegroai
d146127c18 Add events.clear_scroll endpoint to clear event search scrolls 2022-04-18 16:29:57 +03:00
allegroai
124684f53f Version bump to v1.3.0 2022-03-15 16:34:35 +02:00
allegroai
455b5d6758 Fix pre-populate to convert model metadata from the old format 2022-03-15 16:30:14 +02:00
allegroai
c04e2e498b Support credentials label and last_used_from fields 2022-03-15 16:29:37 +02:00
allegroai
da8a45072f Add pipelines support 2022-03-15 16:28:59 +02:00
allegroai
e1992e2054 Fix queue metrics calculation 2022-03-15 16:28:49 +02:00
allegroai
b6ad8f8790 Add support for worker auto-unregister (instead of raising an error) 2022-03-15 16:25:14 +02:00
allegroai
5acc7eebc3 Set API version to 2.17 2022-03-15 16:22:51 +02:00
allegroai
02933a9c93 Support disabling response compression 
Return fixed server header
 2022-03-15 16:21:14 +02:00
allegroai
e537651f29 Better support for assets upload/download 2022-03-15 16:19:52 +02:00
allegroai
af09fba755 Add metadata dict support for models, queues 
Add more info for projects
 2022-03-15 16:18:57 +02:00
allegroai
fc4fd9e61c Version bump to v1.2.0 2022-02-14 15:26:27 +02:00
allegroai
8908c7dcf9 Update driver requirements 
Refactor ES initialization
 2022-02-13 20:27:12 +02:00
allegroai
b9996e2c1a Protect against multiple connects to the update server from different processes 
Code cleanup
 2022-02-13 20:12:12 +02:00
allegroai
afdc56f37c Use task active duration for worker task running time 2022-02-13 20:01:47 +02:00
allegroai
a25cd5dae8 Fix version conflicts when deleting task events cause an error 2022-02-13 20:01:25 +02:00
allegroai
447adb9090 Add support for credentials label 
Support no_scroll in events.get_task_plots
Support better project stats
Fix Redis required on mongodb initialization
Update tests
 2022-02-13 19:59:58 +02:00
allegroai
92fd98d5ad Add support for lists and nested fields in URL args and form 2022-02-13 19:52:05 +02:00
allegroai
c4001b4037 Add Redis cluster support 
Fix for lru_cache usage
 2022-02-13 19:48:26 +02:00
allegroai
970a32287a Add Redis password support 2022-02-13 19:37:52 +02:00
allegroai
17cd48dada Add support for override cookie domains 
Support for community invitation alarms
Remove duplicate property
Add query optimizations
 2022-02-13 19:35:35 +02:00
allegroai
ea3b6e955f Optimize nested_get() 2022-02-13 19:32:22 +02:00
allegroai
843450bb9b Fix add_or_update_artifacts should always be allowed on in_progress tasks 
Fix delete_artifacts should always be allowed on in_progress tasks
Fix query code
 2022-02-13 19:31:54 +02:00
allegroai
e149af58b1 Support for additional mata data in api call response 2022-02-13 19:30:36 +02:00
allegroai
604a38035b Add organization.update_company_name 
Fix unit-tests
 2022-02-13 19:29:46 +02:00
allegroai
cae38a365b Fix base query building 
Fix schema
Improve events.scalar_metrics_iter_raw implementation
 2022-02-13 19:28:23 +02:00
allegroai
e334246b46 Add support for project stats with children flag 2022-02-13 19:26:47 +02:00
allegroai
36e013b40c Add support for events.scalar_metrics_iter_raw 2022-02-13 19:26:03 +02:00
allegroai
f20cd6536e Add scroll support to *.get_* 2022-02-13 19:23:29 +02:00
allegroai
446bd35006 Refactor debug images response, model ORM 2022-02-13 19:21:07 +02:00
allegroai
a377a7e315 Support status_message and status_reason in tasks.delete 2022-02-13 19:20:31 +02:00
allegroai
3d046ac282 Fix project should not be merged into itself 2022-02-13 19:18:08 +02:00