Commit Graph

538 Commits

Author SHA1 Message Date
google-labs-jules[bot]
71fabc1579 feat: Implement AI-Powered Design Generation via Chat
This feature allows you to generate and iteratively refine visual designs
(like posters, logos, etc.) directly through the chat interface.

Key changes include:

1.  **Backend - Intent Detection & Processing:**
    *   I've introduced an `ImageGenerationIntentDetector` in `backend/open_webui/utils/intent_processors.py`. This function:
        *   Uses keyword-based intent detection for new designs and refinements.
        *   Extracts prompts from your messages.
        *   Modifies previous prompts for iterative refinements.
        *   Calls the existing `/api/v1/images/generations` endpoint using `httpx`.
        *   Formats the response (image URL or error) as a chat message, including metadata like `is_generated_design`, `original_prompt`, and `engine_used`.
    *   I've integrated this detector into the main chat processing logic in `backend/open_webui/utils/chat.py`. Design generation requests are now handled by the detector, bypassing the LLM if intent is recognized.

2.  **Frontend - Image Display:**
    *   My analysis confirmed that existing Svelte components (`MarkdownInlineTokens.svelte` using `Image.svelte`) are capable of rendering Markdown-formatted image URLs (`![alt text](url)`) sent by the backend.
    *   The `Image.svelte` component also provides an image preview feature.

3.  **Design Management (MVP Approach):**
    *   For the MVP, generated images are saved via the existing file upload mechanism.
    *   The chat history, with messages containing image URLs and generation metadata, serves as the primary way for you to access and track their designs and refinements. No new database models for explicit design management were added.

4.  **Dependencies:**
    *   I've added `httpx>=0.25.0` to `backend/requirements.txt` to ensure the HTTP client for the intent detector is explicitly listed.

5.  **Documentation:**
    *   I've drafted updates for `README.md` to highlight the new "AI-Powered Design Generation" feature, replacing the previous, more basic "Image Generation Integration" description.

**Testing Plan:**
*   I've prepared detailed manual end-to-end test cases, unit test cases for the `ImageGenerationIntentDetector`, and a frontend visual review checklist to guide developer testing.

This set of changes provides the core functionality for you to conversationally create and refine designs within Open WebUI.
2025-05-25 08:39:03 +00:00
Tim Jaeryang Baek
fbcc80485b Merge pull request #13972 from jarrod-lowe/otel-attrs
Some checks failed
Deploy to HuggingFace Spaces / check-secret (push) Waiting to run
Deploy to HuggingFace Spaces / deploy (push) Blocked by required conditions
Create and publish Docker images with specific build args / build-main-image (linux/amd64) (push) Waiting to run
Create and publish Docker images with specific build args / build-main-image (linux/arm64) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda-image (linux/amd64) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda-image (linux/arm64) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/amd64) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/arm64) (push) Waiting to run
Create and publish Docker images with specific build args / merge-main-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-cuda-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-ollama-images (push) Blocked by required conditions
Python CI / Format Backend (3.11.x) (push) Has been cancelled
Python CI / Format Backend (3.12.x) (push) Has been cancelled
feat: Add user details to opentelemetry spans
2025-05-17 21:54:56 +04:00
Timothy Jaeryang Baek
08e4c163ea feat: local/external connections 2025-05-17 01:47:48 +04:00
Jarrod Lowe
df853246f3 Add user details to otel span 2025-05-17 09:11:26 +12:00
Timothy Jaeryang Baek
359bcb837d refac 2025-05-17 00:08:03 +04:00
Timothy Jaeryang Baek
ecae944fd0 refac 2025-05-16 23:59:24 +04:00
Timothy Jaeryang Baek
1f38350128 feat: toggle filter middleware 2025-05-16 23:33:02 +04:00
Timothy Jaeryang Baek
b61efcf54a feat: toggle filter 2025-05-16 22:59:02 +04:00
Timothy Jaeryang Baek
7bbeaec93b revert 2025-05-16 22:43:42 +04:00
Timothy Jaeryang Baek
363ac35c09 refac 2025-05-16 22:17:47 +04:00
Tim Jaeryang Baek
50bc6d9b12 Merge pull request #13919 from dongfangzan/main
feat: add switching thinking/non-thinking modes to models
2025-05-16 22:08:04 +04:00
Timothy Jaeryang Baek
07b5e84221 refac 2025-05-16 21:29:50 +04:00
Timothy Jaeryang Baek
528a2cf96a refac 2025-05-16 18:19:28 +04:00
Timothy Jaeryang Baek
2aa5e73728 refac: filter details 2025-05-16 18:01:55 +04:00
Timothy Jaeryang Baek
dec9dd1ac0 refac: web search queries 2025-05-16 17:11:42 +04:00
dongfangzan
9b93b81f84 feat: add switching thinking/non-thinking modes to models 2025-05-15 19:56:10 +08:00
Timothy Jaeryang Baek
b143c71da2 refac: AIOHTTP_CLIENT_SESSION_SSL 2025-05-14 23:33:52 +04:00
Timothy Jaeryang Baek
04287eb6d6 refac 2025-05-14 23:27:34 +04:00
hwzhuhao
fb2950201a refactor: replace print statements with logging 2025-05-12 22:18:47 +08:00
Timothy Jaeryang Baek
c61790b355 chore: format 2025-05-10 19:00:01 +04:00
Timothy Jaeryang Baek
34ec10a78c refac: web search performance
Co-Authored-By: Mabeck <64421281+mmabeck@users.noreply.github.com>
2025-05-10 17:54:41 +04:00
Anthony Uk
337ece4d50 code_interpreter.py: allow non-root jupyter base_url 2025-05-09 20:32:01 +02:00
Alexander Grimm
a655bb5a63 skip checks in title gen 2025-05-08 06:55:43 +00:00
Alexander Grimm
cf5201aada remove files from message for tasks 2025-05-08 06:50:21 +00:00
Tim Jaeryang Baek
410af53eca Merge pull request #13581 from kaytwo/refreshOauthPfp
feat: refresh oauth profile picture
2025-05-07 22:06:43 +04:00
Serkan Sakar
d62353c140 enh: add presence_penalty parameter to openai 2025-05-07 00:56:20 +02:00
Timothy Jaeryang Baek
c667d27c38 refac 2025-05-07 02:37:23 +04:00
Timothy Jaeryang Baek
6359cb55fe chore: format 2025-05-07 02:01:03 +04:00
Chris Kanich
500f4d73e1 refresh oauth profile picture 2025-05-06 11:00:35 -05:00
Tim Jaeryang Baek
ed210517c2 Merge pull request #13533 from TheRauch1/main
Some checks are pending
Deploy to HuggingFace Spaces / check-secret (push) Waiting to run
Deploy to HuggingFace Spaces / deploy (push) Blocked by required conditions
Create and publish Docker images with specific build args / build-main-image (linux/amd64) (push) Waiting to run
Create and publish Docker images with specific build args / build-main-image (linux/arm64) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda-image (linux/amd64) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda-image (linux/arm64) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/amd64) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/arm64) (push) Waiting to run
Create and publish Docker images with specific build args / merge-main-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-cuda-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-ollama-images (push) Blocked by required conditions
Python CI / Format Backend (3.11.x) (push) Waiting to run
Python CI / Format Backend (3.12.x) (push) Waiting to run
feat: handle user OAuth groups as list or string
2025-05-06 09:57:14 +04:00
therauch1
c36acd46cc enh: handle user OAuth groups as list or string 2025-05-05 21:38:31 +02:00
Timothy Jaeryang Baek
db0a0b395f fix: oauth 2025-05-05 19:38:36 +04:00
Timothy Jaeryang Baek
512d475ecc refac: api usage 2025-05-05 17:43:51 +04:00
Timothy Jaeryang Baek
4cfb99248d chore: format 2025-05-03 23:48:24 +04:00
Timothy Jaeryang Baek
754f631a07 feat: OAUTH_BLOCKED_GROUPS support 2025-05-02 14:47:02 +04:00
Tim Jaeryang Baek
11d599c55d Merge pull request #13111 from tth37/perf_multi_thread_web_searching
perf: Concurrent processing for web search queries
2025-05-02 03:15:49 -07:00
Tim Jaeryang Baek
2d5643fb17 Merge pull request #13405 from yak1ex/fix_multiple_actions_in_an_action_module_do_not_work_as_intended
fix: Multiple actions in an action module do not work as intended.
2025-05-02 03:06:05 -07:00
Timothy Jaeryang Baek
6d81eef425 refac: reasoning detection 2025-05-02 14:04:12 +04:00
Yak!
b5cfa2b992 fix: Multiple actions in an action module do not work as intended. 2025-05-01 16:41:06 +09:00
Timothy Jaeryang Baek
079af5fffe fix: send webhook notification when user is not active 2025-05-01 09:41:07 +04:00
Tim Jaeryang Baek
f3b38e9777 Merge pull request #13372 from therealmichaelberna/main
Some checks are pending
Deploy to HuggingFace Spaces / check-secret (push) Waiting to run
Deploy to HuggingFace Spaces / deploy (push) Blocked by required conditions
Create and publish Docker images with specific build args / build-main-image (linux/amd64) (push) Waiting to run
Create and publish Docker images with specific build args / build-main-image (linux/arm64) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda-image (linux/amd64) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda-image (linux/arm64) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/amd64) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/arm64) (push) Waiting to run
Create and publish Docker images with specific build args / merge-main-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-cuda-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-ollama-images (push) Blocked by required conditions
Python CI / Format Backend (3.11.x) (push) Waiting to run
Python CI / Format Backend (3.12.x) (push) Waiting to run
Frontend Build / Format & Build Frontend (push) Waiting to run
Frontend Build / Frontend Unit Tests (push) Waiting to run
**fix** convert_function_to_pydantic_model returning empty descriptions
2025-04-30 13:36:22 -07:00
Michael Berna
4a6b964a51 Fixed convert_function_to_pydantic_model
convert_function_to_pydantic_model was returning empty function descriptions. I have renamed the variables more precisely which helps to avoid confusion and also resolved this issue which was caused by a variable naming conflict.
2025-04-30 14:22:38 -04:00
Timothy Jaeryang Baek
eb68b21e0f refac 2025-04-30 16:01:23 +04:00
Alexander Grimm
e80ed32aeb fix: install external requirements of admin functions and tools on startups 2025-04-30 11:14:38 +00:00
tth37
cf2d7de873 refac: Implicit asyncio create_task 2025-04-29 00:20:55 +08:00
Tim Jaeryang Baek
dedc45183f Merge pull request #13258 from choket/dev
Fix: Fixed a bug that caused user notification webhooks to never be sent
2025-04-28 06:14:24 -07:00
Timothy Jaeryang Baek
7d0a78a43a refac: aiohttp trust_env=True 2025-04-28 16:47:34 +04:00
root
8a9de0a62f Fix bug that never triggers user notification webhooks 2025-04-27 21:31:34 +02:00
Timothy Jaeryang Baek
5030041683 chore: format
Some checks are pending
Deploy to HuggingFace Spaces / check-secret (push) Waiting to run
Deploy to HuggingFace Spaces / deploy (push) Blocked by required conditions
Create and publish Docker images with specific build args / build-main-image (linux/amd64) (push) Waiting to run
Create and publish Docker images with specific build args / build-main-image (linux/arm64) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda-image (linux/amd64) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda-image (linux/arm64) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/amd64) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/arm64) (push) Waiting to run
Create and publish Docker images with specific build args / merge-main-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-cuda-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-ollama-images (push) Blocked by required conditions
Python CI / Format Backend (3.11.x) (push) Waiting to run
Python CI / Format Backend (3.12.x) (push) Waiting to run
Frontend Build / Format & Build Frontend (push) Waiting to run
Frontend Build / Frontend Unit Tests (push) Waiting to run
2025-04-23 16:05:15 +09:00
Timothy Jaeryang Baek
f2314596ba enh: failed login attempts audit log
Some checks are pending
Deploy to HuggingFace Spaces / check-secret (push) Waiting to run
Deploy to HuggingFace Spaces / deploy (push) Blocked by required conditions
Create and publish Docker images with specific build args / build-main-image (linux/amd64) (push) Waiting to run
Create and publish Docker images with specific build args / build-main-image (linux/arm64) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda-image (linux/amd64) (push) Waiting to run
Create and publish Docker images with specific build args / build-cuda-image (linux/arm64) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/amd64) (push) Waiting to run
Create and publish Docker images with specific build args / build-ollama-image (linux/arm64) (push) Waiting to run
Create and publish Docker images with specific build args / merge-main-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-cuda-images (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge-ollama-images (push) Blocked by required conditions
Python CI / Format Backend (3.11.x) (push) Waiting to run
Python CI / Format Backend (3.12.x) (push) Waiting to run
2025-04-23 00:06:44 +09:00