mirror of
https://github.com/open-webui/docs
synced 2025-05-20 03:08:56 +00:00
Update filter.mdx
This commit is contained in:
Timothy Jaeryang Baek
parent
f17a442493
commit
a44847bc11
@ -3,60 +3,238 @@ sidebar_position: 2
|
||||
title: "🪄 Filter Function"
|
||||
---
|
||||
|
||||
# 🪄 Filter Function: Modify Inputs and Outputs
|
||||
|
||||
### Filter
|
||||
Filters are used to manipulate the user input and/or the LLM output to add, remove, format, or otherwise adjust the content of the body object.
|
||||
Welcome to the comprehensive guide on Filter Functions in Open WebUI! Filters are a flexible and powerful **plugin system** for modifying data *before it's sent to the Large Language Model (LLM)* (input) or *after it’s returned from the LLM* (output). Whether you’re transforming inputs for better context or cleaning up outputs for improved readability, **Filter Functions** let you do it all.
|
||||
|
||||
Filters have a few main components:
|
||||
This guide will break down **what Filters are**, how they work, their structure, and everything you need to know to build powerful and user-friendly filters of your own. Let’s dig in, and don’t worry—I’ll use metaphors, examples, and tips to make everything crystal clear! 🌟
|
||||
|
||||
#### Inlet Function
|
||||
The inlet is user to pre-process a user input before it is send to the LLM for processing.
|
||||
---
|
||||
|
||||
#### Outlet Function
|
||||
The outlet is used to post-process the output from the LLM. It is important to note that when you perform actions such as stripping/replacing content, this will happen after the output is rendered to the UI.
|
||||
## 🌊 What Are Filters in Open WebUI?
|
||||
|
||||
<details>
|
||||
<summary>Example</summary>
|
||||
Imagine Open WebUI as a **stream of water** flowing through pipes:
|
||||
|
||||
- **User inputs** and **LLM outputs** are the water.
|
||||
- **Filters** are the **water treatment stages** that clean, modify, and adapt the water before it reaches the final destination.
|
||||
|
||||
Filters sit in the middle of the flow—like checkpoints—where you decide what needs to be adjusted.
|
||||
|
||||
Here’s a quick summary of what Filters do:
|
||||
|
||||
1. **Modify User Inputs (Inlet Function)**: Tweak the input data before it reaches the AI model. This is where you enhance clarity, add context, sanitize text, or reformat messages to match specific requirements.
|
||||
2. **Modify Model Outputs (Outlet Function)**: Adjust the AI's response **after it’s processed**, before showing it to the user. This can help refine, log, or adapt the data for a cleaner user experience.
|
||||
|
||||
> **Key Concept:** Filters are not standalone models but tools that enhance or transform the data traveling *to* and *from* models.
|
||||
|
||||
Filters are like **translators or editors** in the AI workflow: you can intercept and change the conversation without interrupting the flow.
|
||||
|
||||
---
|
||||
|
||||
## 🗺️ Structure of a Filter Function: The Skeleton
|
||||
|
||||
Let's start with the simplest representation of a Filter Function. Don't worry if some parts feel technical at first—we’ll break it all down step by step!
|
||||
|
||||
### 🦴 Basic Skeleton of a Filter
|
||||
|
||||
```python
|
||||
from pydantic import BaseModel
|
||||
from typing import Optional
|
||||
|
||||
```
|
||||
class Filter:
|
||||
# Define and Valves
|
||||
class Valves(BaseModel):
|
||||
priority: int = Field(
|
||||
default=0, description="Priority level for the filter operations."
|
||||
)
|
||||
test_valve: int = Field(
|
||||
default=4, description="A valve controlling a numberical value"
|
||||
)
|
||||
pass
|
||||
|
||||
# Define any UserValves
|
||||
class UserValves(BaseModel):
|
||||
test_user_valve: bool = Field(
|
||||
default=False, description="A user valve controlling a True/False (on/off) switch"
|
||||
)
|
||||
# Valves: Configuration options for the filter
|
||||
class Valves(BaseModel):
|
||||
pass
|
||||
|
||||
def __init__(self):
|
||||
# Initialize valves (optional configuration for the Filter)
|
||||
self.valves = self.Valves()
|
||||
pass
|
||||
|
||||
def inlet(self, body: dict) -> dict:
|
||||
# This is where you manipulate user inputs.
|
||||
print(f"inlet called: {body}")
|
||||
return body
|
||||
|
||||
def outlet(self, body: dict) -> None:
|
||||
# This is where you manipulate model outputs.
|
||||
print(f"outlet called: {body}")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 🎯 Key Components Explained
|
||||
|
||||
#### 1️⃣ **`Valves` Class (Optional Settings)**
|
||||
|
||||
Think of **Valves** as the knobs and sliders for your filter. If you want to give users configurable options to adjust your Filter’s behavior, you define those here.
|
||||
|
||||
```python
|
||||
class Valves(BaseModel):
|
||||
OPTION_NAME: str = "Default Value"
|
||||
```
|
||||
|
||||
For example:
|
||||
If you're creating a filter that converts responses into uppercase, you might allow users to configure whether every output gets totally capitalized via a valve like `TRANSFORM_UPPERCASE: bool = True/False`.
|
||||
|
||||
---
|
||||
|
||||
#### 2️⃣ **`inlet` Function (Input Pre-Processing)**
|
||||
|
||||
|
||||
The `inlet` function is like **prepping food before cooking**. Imagine you’re a chef: before the ingredients go into the recipe (the LLM in this case), you might wash vegetables, chop onions, or season the meat. Without this step, your final dish could lack flavor, have unwashed produce, or simply be inconsistent.
|
||||
|
||||
In the world of Open WebUI, the `inlet` function does this important prep work on the **user input** before it’s sent to the model. It ensures the input is as clean, contextual, and helpful as possible for the AI to handle.
|
||||
|
||||
📥 **Input**:
|
||||
- **`body`**: The raw input from Open WebUI to the model. It is in the format of a chat-completion request (usually a dictionary that includes fields like the conversation's messages, model settings, and other metadata). Think of this as your recipe ingredients.
|
||||
|
||||
🚀 **Your Task**:
|
||||
Modify and return the `body`. The modified version of the `body` is what the LLM works with, so this is your chance to bring clarity, structure, and context to the input.
|
||||
|
||||
|
||||
##### 🍳 Why Would You Use the `inlet`?
|
||||
1. **Adding Context**: Automatically append crucial information to the user’s input, especially if their text is vague or incomplete. For example, you might add "You are a friendly assistant" or "Help this user troubleshoot a software bug."
|
||||
|
||||
2. **Formatting Data**: If the input requires a specific format, like JSON or Markdown, you can transform it before sending it to the model.
|
||||
|
||||
3. **Sanitizing Input**: Remove unwanted characters, strip potentially harmful or confusing symbols (like excessive whitespace or emojis), or replace sensitive information.
|
||||
|
||||
4. **Streamlining User Input**: If your model’s output improves with additional guidance, you can use the `inlet` to inject clarifying instructions automatically!
|
||||
|
||||
|
||||
##### 💡 Example Use Cases: Build on Food Prep
|
||||
###### 🥗 Example 1: Adding System Context
|
||||
Let’s say the LLM is a chef preparing a dish for Italian cuisine, but the user hasn’t mentioned "This is for Italian cooking." You can ensure the message is clear by appending this context before sending the data to the model.
|
||||
|
||||
```python
|
||||
def inlet(self, body: dict, __user__: Optional[dict] = None) -> dict:
|
||||
# Add system message for Italian context in the conversation
|
||||
context_message = {
|
||||
"role": "system",
|
||||
"content": "You are helping the user prepare an Italian meal."
|
||||
}
|
||||
# Insert the context at the beginning of the chat history
|
||||
body.setdefault("messages", []).insert(0, context_message)
|
||||
return body
|
||||
```
|
||||
|
||||
📖 **What Happens?**
|
||||
- Any user input like "What are some good dinner ideas?" now carries the Italian theme because we’ve set the system context! Cheesecake might not show up as an answer, but pasta sure will.
|
||||
|
||||
|
||||
###### 🔪 Example 2: Cleaning Input (Remove Odd Characters)
|
||||
Suppose the input from the user looks messy or includes unwanted symbols like `!!!`, making the conversation inefficient or harder for the model to parse. You can clean it up while preserving the core content.
|
||||
|
||||
```python
|
||||
def inlet(self, body: dict, __user__: Optional[dict] = None) -> dict:
|
||||
# Clean the last user input (from the end of the 'messages' list)
|
||||
last_message = body["messages"][-1]["content"]
|
||||
body["messages"][-1]["content"] = last_message.replace("!!!", "").strip()
|
||||
return body
|
||||
```
|
||||
|
||||
📖 **What Happens?**
|
||||
- Before: `"How can I debug this issue!!!"` ➡️ Sent to the model as `"How can I debug this issue"`
|
||||
|
||||
Note: The user feels the same, but the model processes a cleaner and easier-to-understand query.
|
||||
|
||||
|
||||
##### 📊 How `inlet` Helps Optimize Input for the LLM:
|
||||
- Improves **accuracy** by clarifying ambiguous queries.
|
||||
- Makes the AI **more efficient** by removing unnecessary noise like emojis, HTML tags, or extra punctuation.
|
||||
- Ensures **consistency** by formatting user input to match the model’s expected patterns or schemas (like, say, JSON for a specific use case).
|
||||
|
||||
|
||||
💭 **Think of `inlet` as the sous-chef in your kitchen**—ensuring everything that goes into the model (your AI "recipe") has been prepped, cleaned, and seasoned to perfection. The better the input, the better the output!
|
||||
|
||||
---
|
||||
|
||||
#### 3️⃣ **`outlet` Function (Output Post-Processing)**
|
||||
|
||||
The `outlet` function is like a **proofreader**: tidy up the AI's response (or make final changes) *after it’s processed by the LLM.*
|
||||
|
||||
📤 **Input**:
|
||||
- **`body`**: This contains **all current messages** in the chat (user history + LLM replies).
|
||||
|
||||
🚀 **Your Task**: Modify this `body`. You can clean, append, or log changes, but be mindful of how each adjustment impacts the user experience.
|
||||
|
||||
💡 **Best Practices**:
|
||||
- Prefer logging over direct edits in the outlet (e.g., for debugging or analytics).
|
||||
- If heavy modifications are needed (like formatting outputs), consider using the **pipe function** instead.
|
||||
|
||||
💡 **Example Use Case**: Strip out sensitive API responses you don't want the user to see:
|
||||
```python
|
||||
def outlet(self, body: dict, __user__: Optional[dict] = None) -> dict:
|
||||
for message in body["messages"]:
|
||||
message["content"] = message["content"].replace("<API_KEY>", "[REDACTED]")
|
||||
return body
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🌟 Filters in Action: Building Practical Examples
|
||||
|
||||
Let’s build some real-world examples to see how you’d use Filters!
|
||||
|
||||
### 📚 Example #1: Add Context to Every User Input
|
||||
|
||||
Want the LLM to always know it's assisting a customer in troubleshooting software bugs? You can add instructions like **"You're a software troubleshooting assistant"** to every user query.
|
||||
|
||||
```python
|
||||
class Filter:
|
||||
def inlet(self, body: dict, __user__: Optional[dict] = None) -> dict:
|
||||
print(f"inlet:{__name__}")
|
||||
print(f"inlet:body:{body}")
|
||||
print(f"inlet:user:{__user__}")
|
||||
|
||||
# Pre-processing logic here
|
||||
|
||||
return body
|
||||
|
||||
def outlet(self, body: dict, __user__: Optional[dict] = None) -> dict:
|
||||
print(f"outlet:{__name__}")
|
||||
print(f"outlet:body:{body}")
|
||||
print(f"outlet:user:{__user__}")
|
||||
|
||||
# Post-processing logic here
|
||||
|
||||
context_message = {
|
||||
"role": "system",
|
||||
"content": "You're a software troubleshooting assistant."
|
||||
}
|
||||
body.setdefault("messages", []).insert(0, context_message)
|
||||
return body
|
||||
```
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
### 📚 Example #2: Highlight Outputs for Easy Reading
|
||||
|
||||
Returning output in Markdown or another formatted style? Use the `outlet` function!
|
||||
|
||||
```python
|
||||
class Filter:
|
||||
def outlet(self, body: dict, __user__: Optional[dict] = None) -> dict:
|
||||
# Add "highlight" markdown for every response
|
||||
for message in body["messages"]:
|
||||
if message["role"] == "assistant": # Target model response
|
||||
message["content"] = f"**{message['content']}**" # Highlight with Markdown
|
||||
return body
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚧 Potential Confusion: Clear FAQ 🛑
|
||||
|
||||
### **Q: How Are Filters Different From Pipe Functions?**
|
||||
|
||||
Filters modify data **going to** and **coming from models** but do not significantly interact with logic outside of these phases. Pipes, on the other hand:
|
||||
- Can integrate **external APIs** or significantly transform how the backend handles operations.
|
||||
- Expose custom logic as entirely new "models."
|
||||
|
||||
### **Q: Can I Do Heavy Post-Processing Inside `outlet`?**
|
||||
|
||||
You can, but **it’s not the best practice.**:
|
||||
- **Filters** are designed to make lightweight changes or apply logging.
|
||||
- If heavy modifications are required, consider a **Pipe Function** instead.
|
||||
|
||||
---
|
||||
|
||||
## 🎉 Recap: Why Build Filter Functions?
|
||||
|
||||
By now, you’ve learned:
|
||||
1. **Inlet** manipulates **user inputs** (pre-processing).
|
||||
2. **Outlet** tweaks **AI outputs** (post-processing).
|
||||
3. Filters are best for lightweight, real-time alterations to the data flow.
|
||||
4. With **Valves**, you empower users to configure Filters dynamically for tailored behavior.
|
||||
|
||||
---
|
||||
|
||||
🚀 **Your Turn**: Start experimenting! What small tweak or context addition could elevate your Open WebUI experience? Filters are fun to build, flexible to use, and can take your models to the next level!
|
||||
|
||||
Happy coding! ✨
|
||||
Loading…
Reference in New Issue
Block a user