From 802cf8c643177ed3da35dca90bfae843a54904aa Mon Sep 17 00:00:00 2001 From: 1dom <48624549+1dom@users.noreply.github.com> Date: Sat, 23 Mar 2024 18:32:09 +0000 Subject: [PATCH 1/3] Add RAG feature page --- docs/features/rag.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 docs/features/rag.md diff --git a/docs/features/rag.md b/docs/features/rag.md new file mode 100644 index 0000000..3d223f1 --- /dev/null +++ b/docs/features/rag.md @@ -0,0 +1,26 @@ +# Retrieval Augmented Generation (RAG) + +Retrieval Augmented Generation (RAG) allows context from other diverse sources to be included in chats. Text from different sources is combined with the RAG template and prefixed to the user's prompt. + +## Including external sources in chats +Activate RAG by starting the prompt with a `#` symbol. A list of sources will appear. + +Once selected, a document icon appears at the top of the prompt, indicating successful retrieval. + +### Local sources + +Local documents must first be uploaded via the Documents section. + +### Sourcing from the Web + +Remote sources acheived with `#` followed by the target URL. Open WebUI fetches and parses the URL. + +> **Tip:** Webpages often include extraneous information such as navigation and footer. Link to a raw or reader-friendly version of the page for better results. + +## Document parsing + +Local and remote documents have content extract with a variety of parsers. For more, see [https://github.com/open-webui/open-webui/blob/2fa94956f4e500bf5c42263124c758d8613ee05e/backend/apps/rag/main.py#L328] + +## RAG Template Customisation + +Modify the RAG template by accessing the 'settings' icon housed within the 'Documents' section. From ebaa918b9c3def8dc05a73b01ca20ba9bc49d05b Mon Sep 17 00:00:00 2001 From: 1dom <48624549+1dom@users.noreply.github.com> Date: Sat, 23 Mar 2024 18:41:00 +0000 Subject: [PATCH 2/3] Link to RAG feature from intro. --- docs/intro.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/intro.md b/docs/intro.md index 30a581d..58ae2bb 100644 --- a/docs/intro.md +++ b/docs/intro.md @@ -30,8 +30,7 @@ hide_title: true - 🚀 **Effortless Setup**: Install seamlessly using Docker or Kubernetes (kubectl, kustomize or helm) for a hassle-free experience. - 💻 **Code Syntax Highlighting**: Enjoy enhanced code readability with our syntax highlighting feature. - ✒️🔢 **Full Markdown and LaTeX Support**: Elevate your LLM experience with comprehensive Markdown and LaTeX capabilities for enriched interaction. -- 📚 **Local RAG Integration**: Dive into the future of chat interactions with the groundbreaking Retrieval Augmented Generation (RAG) support. This feature seamlessly integrates document interactions into your chat experience. You can load documents directly into the chat or add files to your document library, effortlessly accessing them using `#` command in the prompt. In its alpha phase, occasional issues may arise as we actively refine and enhance this feature to ensure optimal performance and reliability. -- 🌐 **Web Browsing Capability**: Seamlessly integrate websites into your chat experience using the `#` command followed by the URL. This feature allows you to incorporate web content directly into your conversations, enhancing the richness and depth of your interactions. +- 📚🌐 **[Local and remote RAG Integration](features/rag)**: Dive into the future of chat interactions with the groundbreaking Retrieval Augmented Generation (RAG) support. Seamlessly integrate local and web based content into your chats by starting a prompt with `#`. This feature is still in its alpha phase, occasional issues may arise! - 📜 **Prompt Preset Support**: Instantly access preset prompts using the `/` command in the chat input. Load predefined conversation starters effortlessly and expedite your interactions. Effortlessly import prompts through [Open WebUI Community](https://openwebui.com/) integration. - 👍👎 **RLHF Annotation**: Empower your messages by rating them with thumbs up and thumbs down, facilitating the creation of datasets for Reinforcement Learning from Human Feedback (RLHF). Utilize your messages to train or fine-tune models, all while ensuring the confidentiality of locally saved data. - 🏷️ **Conversation Tagging**: Effortlessly categorize and locate specific chats for quick reference and streamlined data collection. From 9e3621d67e8627ea447eb0aa5f91720aeb0295b3 Mon Sep 17 00:00:00 2001 From: "Timothy J. Baek" Date: Sun, 31 Mar 2024 02:14:30 -0700 Subject: [PATCH 3/3] rag moved to tutorial --- docs/intro.md | 2 +- docs/{features => tutorial}/rag.md | 0 2 files changed, 1 insertion(+), 1 deletion(-) rename docs/{features => tutorial}/rag.md (100%) diff --git a/docs/intro.md b/docs/intro.md index 58ae2bb..6594e0a 100644 --- a/docs/intro.md +++ b/docs/intro.md @@ -30,7 +30,7 @@ hide_title: true - 🚀 **Effortless Setup**: Install seamlessly using Docker or Kubernetes (kubectl, kustomize or helm) for a hassle-free experience. - 💻 **Code Syntax Highlighting**: Enjoy enhanced code readability with our syntax highlighting feature. - ✒️🔢 **Full Markdown and LaTeX Support**: Elevate your LLM experience with comprehensive Markdown and LaTeX capabilities for enriched interaction. -- 📚🌐 **[Local and remote RAG Integration](features/rag)**: Dive into the future of chat interactions with the groundbreaking Retrieval Augmented Generation (RAG) support. Seamlessly integrate local and web based content into your chats by starting a prompt with `#`. This feature is still in its alpha phase, occasional issues may arise! +- 📚🌐 **[Local and remote RAG Integration](tutorial/rag)**: Dive into the future of chat interactions with the groundbreaking Retrieval Augmented Generation (RAG) support. Seamlessly integrate local and web based content into your chats by starting a prompt with `#`. This feature is still in its alpha phase, occasional issues may arise! - 📜 **Prompt Preset Support**: Instantly access preset prompts using the `/` command in the chat input. Load predefined conversation starters effortlessly and expedite your interactions. Effortlessly import prompts through [Open WebUI Community](https://openwebui.com/) integration. - 👍👎 **RLHF Annotation**: Empower your messages by rating them with thumbs up and thumbs down, facilitating the creation of datasets for Reinforcement Learning from Human Feedback (RLHF). Utilize your messages to train or fine-tune models, all while ensuring the confidentiality of locally saved data. - 🏷️ **Conversation Tagging**: Effortlessly categorize and locate specific chats for quick reference and streamlined data collection. diff --git a/docs/features/rag.md b/docs/tutorial/rag.md similarity index 100% rename from docs/features/rag.md rename to docs/tutorial/rag.md