OpenWebUI/docs
1
0
Fork 0
mirror of https://github.com/open-webui/docs synced 2025-05-20 11:18:42 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Add tutorial draft: Local LLM Setup with IPEX-LLM on Intel GPU

Browse Source
This commit is contained in:
Yuwen Hu 2024-05-31 16:23:30 +08:00
parent 073ed0c4ba
commit b637d412e2
1 changed files with 24 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

24
docs/tutorial/ipex_llm.md Normal file
View File

@ -0,0 +1,24 @@
---
sidebar_position: 10
title: "Local LLM Setup with IPEX-LLM on Intel GPU"
---
:::note
This guide is verified with Open WebUI setup through [Mannual Installation](../getting-started/index.mdx#manual-installation).
:::
# Local LLM Setup with IPEX-LLM on Intel GPU
:::info
[**IPEX-LLM**](https://github.com/intel-analytics/ipex-llm) is a PyTorch library for running LLM on Intel CPU and GPU (e.g., local PC with iGPU, discrete GPU such as Arc A-Series, Flex and Max) with very low latency.
:::
This tutorial demonstrates how to setup Open WebUI with **IPEX-LLM accelerated Ollama backend hosted on Intel GPU**. By following this guide, you will be able to setup Open WebUI even on a low-cost PC (i.e. only with integrated GPU), and achieve a smooth experience.
## Start Ollama Serve on Intel GPU
Refer to [this guide](https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/ollama_quickstart.html) from IPEX-LLM official documentation about how to install and run Ollama serve accelerated by IPEX-LLM on Intel GPU.
:::tip
If you would like to reach the Ollama serve from another machine, make sure you set or export the environment variable `OLLAMA_HOST=0.0.0.0` before executing the command `ollama serve`.
:::