ClearML/clearml-serving
1
0
Fork 0
mirror of https://github.com/clearml/clearml-serving synced 2025-06-26 18:16:00 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
166 Commits 2 Branches 8 Tags 2.1 MiB
1c3bcc7532
Commit Graph

166 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
IlyaMescheryakov1402
1c3bcc7532 fix request_body 2025-03-20 15:21:40 +03:00
IlyaMescheryakov1402
34c4a9df91 update readme and fix docker-compose-gpu.yml 2025-03-20 02:26:54 +03:00
IlyaMescheryakov1402
db3d4539ee update readme 2025-03-12 15:46:43 +03:00
IlyaMescheryakov1402
42d8738074 add empty string 2025-03-12 14:28:11 +03:00
IlyaMescheryakov1402
a2817e38da small changes for pr 2025-03-12 14:24:00 +03:00
IlyaMescheryakov1402
10f887d449 add some sugar 2025-03-12 02:20:19 +03:00
IlyaMescheryakov1402
8ecb51f1db add models endpoint 2025-03-12 01:09:50 +03:00
IlyaMescheryakov1402
25e2940596 fix jsonresponse 2025-03-11 22:44:32 +03:00
IlyaMescheryakov1402
fedfcdadeb add getattr for process methods 2025-03-11 22:42:59 +03:00
IlyaMescheryakov1402
9bb0dbb182 fix imports 2025-03-11 11:45:52 +03:00
IlyaMescheryakov1402
9441ae8473 move engine init in separate class 2025-03-10 23:52:14 +03:00
IlyaMescheryakov1402
1c591f2d15 fix openai testing 2025-03-10 00:21:24 +03:00
IlyaMescheryakov1402
77e1f95dbd fix response import 2025-03-09 22:53:44 +03:00
IlyaMescheryakov1402
cadd48f672 add openai_serving and openai_serving_models 2025-03-09 15:12:05 +03:00
IlyaMescheryakov1402
428be76642 major vllm engine update 2025-03-09 01:46:05 +03:00
IlyaMescheryakov1402
32d72bcd1c add vllm example 2025-02-28 22:36:14 +03:00
IlyaMescheryakov1402
f51bf2e081 revert some old changes 2025-02-27 23:13:47 +03:00
IlyaMescheryakov1402
5b73bdf085 fix suffix and add router 2025-02-27 22:56:39 +03:00
IlyaMescheryakov1402
2685d2a0e5 Merge branch 'main' into feature/multimodel 2025-02-27 13:41:54 +03:00
clearml
1def0a6901 Update github repo link 2025-01-13 18:40:02 +02:00
clearml
9f51a9334f Fix torch import 2024-12-16 18:51:58 +02:00
clearml
aff27c62b8 Fix gRPC errors print stack traces and full verbose details. Add support for controlling error printouts using CLEARML_SERVING_AIO_RPC_IGNORE_ERRORS and CLEARML_SERVING_AIO_RPC_VERBOSE_ERRORS (pass a whitespace-separated list of error codes or error names) 2024-12-12 23:57:21 +02:00
IlyaMescheryakov1402
724c99c605
Add clearml_serving_inference restart on CUDA OOM (#75) 
* initial commit

* add OOM handler for MIG profiles

---------

Co-authored-by: Meshcheryakov Ilya <i.meshcheryakov@mts.ai>
 2024-07-07 15:54:08 +03:00
stephanbertl
666ce26ab2
Add exit-on-error option for tritonserver (#76) 
This fixes #60 
Co-authored-by: = <s.bertl@iaea.org>
 2024-07-07 15:51:23 +03:00
Meshcheryakov Ilya
4796d77ad7 fix shash processing 2024-05-30 15:52:06 +03:00
Meshcheryakov Ilya
b8f5d81636 initial commit 2024-05-30 00:30:30 +03:00
Meshcheryakov Ilya
64daef23ba initial commit 2024-05-29 21:18:39 +03:00
Meshcheryakov Ilya
6859920848 initial commit 2024-04-16 00:54:35 +03:00
allegroai
7ba356efc9 Update README 2024-03-11 16:54:08 +02:00
allegroai
047a120100 Fix broken requirements 2024-03-01 13:13:48 +02:00
allegroai
71c104c9df Add exception prints to serving session Task and inference Task, for better debugging capabilities 
Add report instance ID when reporting back to the main serving session task
 2024-03-01 13:13:48 +02:00
allegroai
c658780d97 Update README 2024-03-01 00:02:56 +02:00
allegroai
8df521b949 Fix python < 3.10 support 
Fix custom_async engine
Suppress warning
 2024-02-27 09:45:32 +02:00
allegroai
bca162810b Add async pipeline example (--engine custom_async) 2024-02-27 09:44:53 +02:00
allegroai
9488ead080 Add async pipeline version 2024-02-27 09:44:20 +02:00
allegroai
8f6feef938 Fix broken link 2024-02-27 09:43:47 +02:00
allegroai
3611a040f7 Fix requirements 2024-02-27 09:43:18 +02:00
allegroai
dc6fd46a46 Update requirements 2024-02-26 11:34:21 +02:00
allegroai
0f4122247d Fix ping serving session task to make sure everyone knows we are alive 2024-02-26 11:31:58 +02:00
allegroai
f2ba37c8d4 Fix version enabled endpoints on Triton engine were not called 2024-02-26 11:27:12 +02:00
allegroai
4ac13d5287 Fix requirements issue 2024-01-11 15:21:13 +02:00
allegroai
4335ebd340 Improve preprocess template docstring 2024-01-06 17:58:20 +02:00
allegroai
b42a0b0cfc Fix requirements 2024-01-06 17:57:50 +02:00
allegroai
368a03dc70 Fix internal ValueError exception should return 422 (not 404 as before) 2024-01-06 17:55:49 +02:00
allegroai
02fef01657 Add python 3.11 to support matrix 2024-01-06 17:54:56 +02:00
Jake Henning
6c4bece663
Fix Pillow vulnerability "libwebp: OOB write in BuildHuffmanTable" 2023-10-04 13:23:42 +03:00
Jake Henning
c20bbd66b9
Fix Pillow vulnerability "libwebp: OOB write in BuildHuffmanTable" 2023-10-04 13:22:46 +03:00
allegroai
05cbfade2a Update requirements 2023-09-23 18:03:24 +03:00
allegroai
cc5823cfc6 Update README 2023-09-23 17:48:25 +03:00
allegroai
82ade1e24a Fix check triton config.pbtxt for missing values or colliding specifications (#62) 2023-09-23 17:42:57 +03:00