ClearML/clearml-serving
1
0
Fork 0
mirror of https://github.com/clearml/clearml-serving synced 2025-06-26 18:16:00 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
151 Commits 2 Branches 8 Tags 2.1 MiB
fd8b7604a9
Commit Graph

27 Commits

Author SHA1 Message Date
Eugen Ajechiloae
fd8b7604a9 call gc on remove as well 2024-08-15 16:18:41 +03:00
Eugen Ajechiloae
1f4e1599e3 let gc handle unload 2024-08-15 15:23:29 +03:00
Eugen Ajechiloae
5d5188de40 add debugging comment 2024-08-14 21:20:13 +03:00
Eugen Ajechiloae
79b987ac0c more dbg prints 2024-08-14 21:03:25 +03:00
Eugen Ajechiloae
67325df4d5 change dbg print 2024-08-14 20:50:57 +03:00
Eugen Ajechiloae
f95939cffe more debug prints 2024-08-14 20:46:45 +03:00
Eugen Ajechiloae
6a8e616256 not tested - exit on cuda oom 2024-08-14 19:43:44 +03:00
Eugen Ajechiloae
6ef1f67ad0 initial commit for oom issues - allow for model unload and restart serving on crash 2024-08-14 17:01:29 +03:00
IlyaMescheryakov1402
724c99c605
Add clearml_serving_inference restart on CUDA OOM (#75) 
* initial commit

* add OOM handler for MIG profiles

---------

Co-authored-by: Meshcheryakov Ilya <i.meshcheryakov@mts.ai>
 2024-07-07 15:54:08 +03:00
allegroai
71c104c9df Add exception prints to serving session Task and inference Task, for better debugging capabilities 
Add report instance ID when reporting back to the main serving session task
 2024-03-01 13:13:48 +02:00
allegroai
0f4122247d Fix ping serving session task to make sure everyone knows we are alive 2024-02-26 11:31:58 +02:00
allegroai
82ade1e24a Fix check triton config.pbtxt for missing values or colliding specifications (#62) 2023-09-23 17:42:57 +03:00
allegroai
78a03cc166 Register models on serving session 2023-04-12 23:34:49 +03:00
allegroai
395a547c04 Optimize async processing for increased speed 2022-10-08 02:12:04 +03:00
Aleksandar Ivanovski
d89d1370d8 [DEV] feature/bytes-payload | Add typing 2022-10-06 16:01:31 +02:00
Aleksandar Ivanovski
2aa91a3d43 [DEV] feature/bytes-payload | Handle keys when req is bytes 2022-10-06 15:13:36 +02:00
allegroai
f4eed33f10 Add Triton support for variable length requests, adds support for HuggingFace Transformers 
Add triton_grpc_compression=False (default) for grpc connection compression control
 2022-09-02 23:41:54 +03:00
allegroai
5beb077f51 Add support for update pre/post processing code to a live endpoint 2022-06-07 00:52:08 +03:00
allegroai
48f720ac91 Optimize request serving statistics reporting 2022-06-07 00:20:33 +03:00
allegroai
f2e207e2f2 Add per endpoint-variable add/remove statistics logging 2022-06-05 16:11:17 +03:00
allegroai
8778f723e6 Add pre/post processing callnack state dict, for safe per request state storage 2022-06-05 16:10:20 +03:00
Victor Sonck
e3a8ed95b5 Add task reload call that made statistics service not update correctly 2022-06-01 09:54:02 +02:00
allegroai
c3f3008868 pep8 2022-04-29 03:10:35 +03:00
allegroai
409fc156fd Add Preprocess.model_endpoint 2022-04-18 23:24:30 +03:00
allegroai
4355c1b1f4 Add model metric logging 2022-03-21 01:00:19 +02:00
allegroai
d684169367 Add model ensemble and model pipelines support 2022-03-09 04:02:03 +02:00
allegroai
b4cb27b27d ClearML-Serving v2 initial working commit 2022-03-06 01:25:56 +02:00