{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ff858f10-9827-4ed4-b388-f34b8776153b",
   "metadata": {},
   "source": [
    "# 🌟 Effortlessly Manage Knowledge Base with WebUI and Python\n",
    "\n",
    "> **Disclaimer:** This guide is written for **Open WebUI version 0.5**. Future versions might introduce changes that could affect compatibility. Please refer to the latest documentation for updates. 🙏\n",
    "\n",
    "In this guide, we'll show you how to programmatically upload and manage files in your Knowledge Base using WebUI's API. Plus, you’ll learn how to build an **auto-synced knowledge base** that detects file changes in your directory and syncs them. Let’s dive in! 🦾📂✨\n",
    "\n",
    "\n",
    "---\n",
    "\n",
    "## 📚 Step 1: Create Your Knowledge in WebUI\n",
    "\n",
    "1️⃣ Open your WebUI and create a new Knowledge base.  \n",
    "2️⃣ Add a name and description for your Knowledge.  \n",
    "👏 Example: We'll call ours \"Open WebUI Knowledge\"!  \n",
    "\n",
    "![image.png](./images/create-knowledge.png)\n",
    "\n",
    "\n",
    "### ✏️ Note Your Knowledge ID  \n",
    "\n",
    "![image.png](./images/knowledge-id.png)\n",
    "Once created, you'll need the `knowledge_id`. For example:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "cf52a400-61c0-4f48-841b-26ecf73da08e",
   "metadata": {},
   "outputs": [],
   "source": [
    "knowledge_id = \"835121e1-d02d-4e13-b5bc-d54434f6c8b0\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e342596-db2f-46e1-84a8-d405d610deee",
   "metadata": {},
   "source": [
    "## 🧰 Step 2: Set Up Your Environment\n",
    "\n",
    "🔧 Import the necessary tools and set up environment variables in Python. You can use `.env` for storing your configuration like `WEBUI_URL`, `TOKEN`, and `MODEL`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "80b25efc-88f2-48d9-aadf-acfaffd9e890",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import requests\n",
    "\n",
    "\n",
    "try:\n",
    "    from dotenv import load_dotenv\n",
    "\n",
    "    load_dotenv(\".env\")\n",
    "except ImportError:\n",
    "    print(\"dotenv not installed, skipping...\")\n",
    "\n",
    "\n",
    "WEBUI_URL = os.getenv(\"WEBUI_URL\", \"http://localhost:8080\") \n",
    "TOKEN = os.getenv(\"TOKEN\", \"your-webui-token-here\")\n",
    "MODEL = os.getenv(\"MODEL\", \"llama3.2:latest\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b05e360a-3379-4d19-b246-491f8c810df7",
   "metadata": {},
   "source": [
    "## 💬 Step 3: Chat with the Model (No Files Yet)\n",
    "\n",
    "Use the `chat` function to interact with the model even without any files 🚫📄. Here's an example:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "dd674fdd-9a52-4254-aff0-7e199d651d1e",
   "metadata": {},
   "outputs": [],
   "source": [
    "def chat(query, file_id=None, collection_id=None):\n",
    "    url = f'{WEBUI_URL}/api/chat/completions'\n",
    "    headers = {\n",
    "        'Authorization': f'Bearer {TOKEN}',\n",
    "        'Content-Type': 'application/json'\n",
    "    }\n",
    "\n",
    "    files = []\n",
    "    if file_id:\n",
    "        files.append({'type': 'file', 'id': file_id})\n",
    "\n",
    "    if collection_id:\n",
    "        files.append({'type': 'collection', 'id': collection_id})\n",
    "        \n",
    "    payload = {\n",
    "        'model': MODEL,\n",
    "        'messages': [{'role': 'user', 'content': query}],\n",
    "        'files': files\n",
    "    }\n",
    "    response = requests.post(url, headers=headers, json=payload)\n",
    "    return response.json()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "462cd88a-e528-4318-ba39-56ca6c016ac4",
   "metadata": {},
   "source": [
    "👉 **Chat Example**  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "441807e2-3ff0-40df-a2b5-082368450c03",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'id': 'llama3.2:latest-fbc97f17-e228-4459-a6e3-0c918264172a', 'created': 1735372215, 'model': 'llama3.2:latest', 'choices': [{'index': 0, 'logprobs': None, 'finish_reason': 'stop', 'message': {'content': \"OpenWebUI (OWUI) is an open-source web-based user interface that provides a simple and intuitive way to manage and monitor various Linux systems, including servers, virtual machines, and containers. It was created by Thomas Koenig and has been around since 2010.\\n\\nOpenWebUI offers a range of features, including:\\n\\n1. Console access: allows administrators to remotely access the console output of a system.\\n2. File management: enables users to upload and download files between systems.\\n3. Network monitoring: provides information about network connections and bandwidth usage.\\n4. CPU and memory usage monitoring: displays real-time data on CPU and memory utilization.\\n5. Disk space monitoring: shows available disk space and usage patterns.\\n6. System information: provides detailed system information, including kernel version, operating system, and hardware specifications.\\n7. Backup and restore: allows users to create and manage backups of their systems.\\n\\nOpenWebUI is designed to be easy to use and requires minimal configuration, making it a great option for small businesses, individuals, and IT teams who need to manage multiple Linux systems. It can be accessed using a web browser and is compatible with most modern browsers.\\n\\nSome of the benefits of OpenWebUI include:\\n\\n* Easy to set up and use\\n* Provides real-time monitoring and control\\n* Scalable and flexible\\n* Secure and reliable\\n* Supports multiple operating systems\\n\\nHowever, it's worth noting that OpenWebUI has a relatively small user base compared to other remote desktop solutions, which might limit its feature set and customization options.\", 'role': 'assistant'}}], 'object': 'chat.completion'}\n"
     ]
    }
   ],
   "source": [
    "response = chat(\"What is Open WebUI?\")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbc11af9-f403-438a-92b3-005c1c5f8b86",
   "metadata": {},
   "source": [
    "## 📤 Step 4: Upload a File to WebUI\n",
    "\n",
    "Let’s upload a file (e.g., `whitepaper.pdf`) to WebUI! 📄\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "514b9bb4-411b-4264-99be-4d5327588f13",
   "metadata": {},
   "outputs": [],
   "source": [
    "def upload_file(file_path):\n",
    "    url = f'{WEBUI_URL}/api/v1/files/'\n",
    "    headers = {\n",
    "        'Authorization': f'Bearer {TOKEN}',\n",
    "        'Accept': 'application/json'\n",
    "    }\n",
    "    files = {'file': open(file_path, 'rb')}\n",
    "    response = requests.post(url, headers=headers, files=files)\n",
    "    return response.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "41101cdf-1d91-404b-869b-67edb1be4497",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "uploaded_file = upload_file(\"./whitepaper.pdf\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "916d4701-92d8-48fe-a0ff-555ec5331de5",
   "metadata": {},
   "source": [
    "👉 **Chat w/ File Example**  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "7f49df6a-3d1e-48e4-828a-e888b206f08d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'id': 'llama3.2:latest-e944bc15-dff8-439e-a22a-1e90d3e0060f', 'created': 1735372233, 'model': 'llama3.2:latest', 'choices': [{'index': 0, 'logprobs': None, 'finish_reason': 'stop', 'message': {'content': \"Open WebUI is an open-source interface for local and/or private Large Language Models (LLMs). It offers a flexible way to interact with multiple LLMs in various configurations, enabling users to evaluate the models' performance across different settings and use cases. This flexibility allows for comparing different models under identical conditions or the same model under varying conditions, providing a rich dataset for nuanced analysis.\\n\\nThe primary strength of open-source UIs like Open WebUI lies in their ability to obtain data from real-world LLMs that perform on diverse hardware. It enables a more holistic analysis, aiding in the optimization of LLMs for a wide range of applications.\\n\\nOne notable advantage of Open WebUI is its streamlined data exportability, which significantly facilitates the use of its existing userbase for data collection in evaluations.\", 'role': 'assistant'}}], 'object': 'chat.completion'}\n"
     ]
    }
   ],
   "source": [
    "response = chat(\"What is Open WebUI?\",file_id=uploaded_file['id'])\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ecd5cc63-fec4-450d-9d82-a8ba297226fa",
   "metadata": {},
   "source": [
    "## 🧠 Step 5: Add a File to Your Knowledge\n",
    "\n",
    "Now, let’s add your uploaded file to the knowledge base. 📂🔗"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "2ea61016-1d13-4960-ae52-0bc34b1feebb",
   "metadata": {},
   "outputs": [],
   "source": [
    "def add_file_to_knowledge(knowledge_id, file_id):\n",
    "    url = f'{WEBUI_URL}/api/v1/knowledge/{knowledge_id}/file/add'\n",
    "    headers = {\n",
    "        'Authorization': f'Bearer {TOKEN}',\n",
    "        'Content-Type': 'application/json'\n",
    "    }\n",
    "    data = {'file_id': file_id}\n",
    "    response = requests.post(url, headers=headers, json=data)\n",
    "    return response.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "169acf7d-3d0e-4ed2-9f87-cacd249de9de",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "knowledge = add_file_to_knowledge(knowledge_id, uploaded_file['id'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "691dce68-0a67-4a63-889b-59c202fd676f",
   "metadata": {},
   "source": [
    "🎉 **Success**: Your file is now part of the knowledge base!  \n",
    "Open your WebUI and refresh–you’ll see your file listed there. ✅🎊  \n",
    "![Knowledge File Added Screenshot](./images/knowledge-with-file.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec41f78d-d9a3-431a-8a57-94f719ef91d2",
   "metadata": {},
   "source": [
    "👉 **Chat w/ Collection Example**  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "eafba0b9-bbe2-48c3-b54b-7e148dff5219",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'id': 'llama3.2:latest-52e675f8-2fc9-4961-8a92-0dec3d1a81ee', 'created': 1735372264, 'model': 'llama3.2:latest', 'choices': [{'index': 0, 'logprobs': None, 'finish_reason': 'stop', 'message': {'content': 'Open WebUI is an open-source Local Model UI (LMUI) that operates entirely locally, allowing users to interact with various Large Language Models (LLMs) in a similar experience to popular platforms like ChatGPT. This local deployment capability enables its use in a variety of settings, including high-security environments and remote locations with limited internet access [8].', 'role': 'assistant'}}], 'object': 'chat.completion'}\n"
     ]
    }
   ],
   "source": [
    "response = chat(\"What is Open WebUI?\",collection_id=knowledge['id'])\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c45805af-89c4-4633-9734-6d4471424e70",
   "metadata": {},
   "source": [
    "## ❌ Step 6: Remove a File from Knowledge (If Needed)\n",
    "\n",
    "If you change your mind, removing a file from a knowledge base is just as easy! 🧹\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "9f156aaf-5abf-4264-8b5c-21a6d834be2a",
   "metadata": {},
   "outputs": [],
   "source": [
    "def remove_file_from_knowledge(knowledge_id, file_id):\n",
    "    url = f'{WEBUI_URL}/api/v1/knowledge/{knowledge_id}/file/remove'\n",
    "    headers = {\n",
    "        'Authorization': f'Bearer {TOKEN}',\n",
    "        'Content-Type': 'application/json'\n",
    "    }\n",
    "    data = {'file_id': file_id}\n",
    "    response = requests.post(url, headers=headers, json=data)\n",
    "    return response.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "6f0bfcea-3afe-4628-9135-514861679340",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'id': '835121e1-d02d-4e13-b5bc-d54434f6c8b0', 'user_id': 'a750f3d8-586a-48b9-b6d5-508463a18ad5', 'name': 'Open WebUI', 'description': 'Open WebUI Knowledge', 'data': {'file_ids': []}, 'meta': None, 'access_control': None, 'created_at': 1735371354, 'updated_at': 1735372303, 'files': []}\n"
     ]
    }
   ],
   "source": [
    "response = remove_file_from_knowledge(knowledge_id, uploaded_file['id'])\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e32cd211-0761-4629-b228-563f947824b6",
   "metadata": {},
   "source": [
    "## 💡 Bonus: Build a Remotely Synced Knowledge Base 🤖📂\n",
    "\n",
    "With these steps, you can create a **remotely synced knowledge base** that detects file changes in your local directories and syncs them to WebUI. 🚀 Here's how to get started:\n",
    "\n",
    "1️⃣ Use the [Python `watchdog` library](https://pypi.org/project/watchdog/) to monitor changes in a directory.  \n",
    "2️⃣ Detect new, updated, or deleted files.  \n",
    "3️⃣ Automatically add/upload new files or remove deleted ones in the knowledge base.\n",
    "\n",
    "### Example Idea\n",
    "\n",
    "```python\n",
    "from watchdog.observers import Observer\n",
    "from watchdog.events import FileSystemEventHandler\n",
    "import time\n",
    "\n",
    "class KnowledgeSync(FileSystemEventHandler):\n",
    "    def __init__(self, knowledge_id):\n",
    "        self.knowledge_id = knowledge_id\n",
    "\n",
    "    def on_created(self, event):\n",
    "        print(f'New file detected: {event.src_path}')\n",
    "        file_info = upload_file(event.src_path)\n",
    "        add_file_to_knowledge(self.knowledge_id, file_info['id'])\n",
    "\n",
    "    def on_deleted(self, event):\n",
    "        print(f'File deleted: {event.src_path}')\n",
    "        # Logic to remove the file from knowledge\n",
    "\n",
    "# Monitor directory\n",
    "if __name__ == \"__main__\":\n",
    "    path_to_watch = \"./sync_directory\"\n",
    "    sync_handler = KnowledgeSync(knowledge_id)\n",
    "    observer = Observer()\n",
    "    observer.schedule(sync_handler, path_to_watch, recursive=True)\n",
    "    observer.start()\n",
    "\n",
    "    try:\n",
    "        while True:\n",
    "            time.sleep(1)\n",
    "    except KeyboardInterrupt:\n",
    "        observer.stop()\n",
    "    observer.join()\n",
    "```\n",
    "\n",
    "🛠️ With this script, you can sync files in real-time! Use it to **automate workflows**, keep remote knowledge bases up to date, or collaborate with your teammates. 🤝\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a83c6223-4c1d-45ac-9bc3-7f7a98cab52e",
   "metadata": {},
   "source": [
    "## 📝 Summary \n",
    "\n",
    "Here’s what you’ve learned:\n",
    "1. 🛠️ Set up your environment.  \n",
    "2. 📤 Upload files to WebUI.  \n",
    "3. 🔗 Add files to your Knowledge base.  \n",
    "4. ❌ Easily remove files when needed.  \n",
    "5. 🤖 BONUS: Build a synced knowledge base with Python!\n",
    "\n",
    "👏 You're now a pro at managing files programmatically in WebUI! 🎉 Use this newfound power to automate your workflows and create smarter systems.\n",
    "\n",
    "--- \n",
    "\n",
    "🚀 Go forth and build amazing things with your Knowledge base! 🦾✨  "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}