refac: ollama tool calling support
Some checks are pending
Deploy to HuggingFace Spaces / check-secret (push) Waiting to run
Deploy to HuggingFace Spaces / deploy (push) Blocked by required conditions
Create and publish Docker images with specific build args / build-main-image (linux/amd64) (push) Waiting to run
Create and publish Docker images with specific build args / build-main-image (linux/arm64) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda-image (linux/amd64) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda-image (linux/arm64) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/amd64) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/arm64) (push) Waiting to run
Create and publish Docker images with specific build args / merge-main-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-cuda-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-ollama-images (push) Blocked by required conditions
Python CI / Format Backend (3.11) (push) Waiting to run
Frontend Build / Format & Build Frontend (push) Waiting to run
Frontend Build / Frontend Unit Tests (push) Waiting to run

Co-Authored-By: smonux <85928277+smonux@users.noreply.github.com>
This commit is contained in:
Timothy Jaeryang Baek 2025-02-04 19:22:08 -08:00
parent ec9be0d20d
commit 23df351239
2 changed files with 9 additions and 0 deletions

View File

@ -939,6 +939,7 @@ async def generate_completion(
class ChatMessage(BaseModel):
role: str
content: str
tool_calls: Optional[list[dict]] = None
images: Optional[list[str]] = None
@ -950,6 +951,7 @@ class GenerateChatCompletionForm(BaseModel):
template: Optional[str] = None
stream: Optional[bool] = True
keep_alive: Optional[Union[int, str]] = None
tools: Optional[list[dict]] = None
async def get_ollama_url(request: Request, model: str, url_idx: Optional[int] = None):

View File

@ -73,6 +73,13 @@ def get_tools(
# convert to function that takes only model params and inserts custom params
original_func = getattr(module, function_name)
callable = apply_extra_params_to_tool_function(original_func, extra_params)
if callable.__doc__ and callable.__doc__.strip() != "":
s = re.split(":(param|return)", callable.__doc__, 1)
spec["description"] = s[0]
else:
spec["description"] = function_name
# TODO: This needs to be a pydantic model
tool_dict = {
"toolkit_id": tool_id,