IlyaMescheryakov1402 cadd48f672 add openai_serving and openai_serving_models

2025-03-09 15:12:05 +03:00

..

__init__.py

ClearML-Serving v2 initial working commit

2022-03-06 01:25:56 +02:00

Dockerfile

initial commit

2024-05-29 21:18:39 +03:00

endpoints.py

Add Triton support for variable length requests, adds support for HuggingFace Transformers

2022-09-02 23:41:54 +03:00

entrypoint.sh

fix suffix and add router

2025-02-27 22:56:39 +03:00

init.py

Add clearml_serving_inference restart on CUDA OOM (#75)

2024-07-07 15:54:08 +03:00

main.py

add openai_serving and openai_serving_models

2025-03-09 15:12:05 +03:00

model_request_processor.py

add openai_serving and openai_serving_models

2025-03-09 15:12:05 +03:00

preprocess_service.py

add openai_serving and openai_serving_models

2025-03-09 15:12:05 +03:00

requirements.txt

add openai_serving and openai_serving_models

2025-03-09 15:12:05 +03:00

utils.py

Fix gRPC errors print stack traces and full verbose details. Add support for controlling error printouts using CLEARML_SERVING_AIO_RPC_IGNORE_ERRORS and CLEARML_SERVING_AIO_RPC_VERBOSE_ERRORS (pass a whitespace-separated list of error codes or error names)

2024-12-12 23:57:21 +02:00

uvicorn_mp_entrypoint.py

Optimize async processing for increased speed

2022-10-08 02:12:04 +03:00