ClearML/clearml-serving
1
0
Fork 0
mirror of https://github.com/clearml/clearml-serving synced 2025-06-26 18:16:00 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
160 Commits 2 Branches 8 Tags 2.1 MiB
8ecb51f1db
Commit Graph

21 Commits

Author SHA1 Message Date
IlyaMescheryakov1402
8ecb51f1db add models endpoint 2025-03-12 01:09:50 +03:00
IlyaMescheryakov1402
77e1f95dbd fix response import 2025-03-09 22:53:44 +03:00
IlyaMescheryakov1402
cadd48f672 add openai_serving and openai_serving_models 2025-03-09 15:12:05 +03:00
IlyaMescheryakov1402
428be76642 major vllm engine update 2025-03-09 01:46:05 +03:00
IlyaMescheryakov1402
5b73bdf085 fix suffix and add router 2025-02-27 22:56:39 +03:00
IlyaMescheryakov1402
2685d2a0e5 Merge branch 'main' into feature/multimodel 2025-02-27 13:41:54 +03:00
clearml
aff27c62b8 Fix gRPC errors print stack traces and full verbose details. Add support for controlling error printouts using CLEARML_SERVING_AIO_RPC_IGNORE_ERRORS and CLEARML_SERVING_AIO_RPC_VERBOSE_ERRORS (pass a whitespace-separated list of error codes or error names) 2024-12-12 23:57:21 +02:00
IlyaMescheryakov1402
724c99c605
Add clearml_serving_inference restart on CUDA OOM (#75) 
* initial commit

* add OOM handler for MIG profiles

---------

Co-authored-by: Meshcheryakov Ilya <i.meshcheryakov@mts.ai>
 2024-07-07 15:54:08 +03:00
Meshcheryakov Ilya
4796d77ad7 fix shash processing 2024-05-30 15:52:06 +03:00
Meshcheryakov Ilya
b8f5d81636 initial commit 2024-05-30 00:30:30 +03:00
Meshcheryakov Ilya
64daef23ba initial commit 2024-05-29 21:18:39 +03:00
Meshcheryakov Ilya
6859920848 initial commit 2024-04-16 00:54:35 +03:00
allegroai
71c104c9df Add exception prints to serving session Task and inference Task, for better debugging capabilities 
Add report instance ID when reporting back to the main serving session task
 2024-03-01 13:13:48 +02:00
allegroai
368a03dc70 Fix internal ValueError exception should return 422 (not 404 as before) 2024-01-06 17:55:49 +02:00
allegroai
395a547c04 Optimize async processing for increased speed 2022-10-08 02:12:04 +03:00
Aleksandar Ivanovski
09ed480bc2 [DEV] feature/bytes-payload | Add bytes as payload 2022-10-06 13:31:54 +02:00
allegroai
48f720ac91 Optimize request serving statistics reporting 2022-06-07 00:20:33 +03:00
allegroai
4355c1b1f4 Add model metric logging 2022-03-21 01:00:19 +02:00
allegroai
d684169367 Add model ensemble and model pipelines support 2022-03-09 04:02:03 +02:00
allegroai
34e5a0b2c8 Fix routing 2022-03-06 03:14:06 +02:00
allegroai
b4cb27b27d ClearML-Serving v2 initial working commit 2022-03-06 01:25:56 +02:00