Commit Graph

613 Commits

Author SHA1 Message Date
lucasew
5b26d2a686 backend: make the data directory and the artifacts from the frontend customizable using environment variables
Signed-off-by: lucasew <lucas59356@gmail.com>
2024-01-19 17:13:09 -03:00
Marclass
35ace57784 add rst document for RAG 2024-01-19 10:48:04 -07:00
Dave Bauman
f559068186
feat: Add epub support 2024-01-19 12:23:59 -05:00
Timothy Jaeryang Baek
f079cb6b56
Merge pull request #524 from Marclass/rag-arbitrary-files
feat: Allow RAG on XML and arbitrary text files including source code
2024-01-19 00:09:04 -08:00
Marclass
aa1d386042 Allow any file to be used for RAG.
Changed RAG parser to prefer file extensions over MIME content types. If the type of file is not recognized assume it's a text file.
2024-01-18 20:41:14 -07:00
Timothy J. Baek
220530c450 feat: convo tag filtering 2024-01-18 02:55:25 -08:00
Timothy J. Baek
1eec176313 feat: save tags to chat data 2024-01-18 02:17:31 -08:00
Timothy J. Baek
987685dbf9 feat: convo tagging full integration 2024-01-18 02:10:07 -08:00
Timothy J. Baek
d5ed119687 feat: convo tagging api added 2024-01-18 01:04:24 -08:00
Timothy J. Baek
077f1fa34b feat: convo tagging backend support 2024-01-18 00:58:45 -08:00
Timothy J. Baek
442e3d978a feat: terminate request on user stop 2024-01-17 19:19:44 -08:00
Marclass
6070e6bcd1 add svelte type to RAG 2024-01-17 20:10:34 -07:00
Marclass
8dacc86ab0
Merge branch 'ollama-webui:main' into main 2024-01-17 16:20:52 -07:00
Timothy J. Baek
b6ab357e8c fix: more edge cases 2024-01-17 14:47:56 -08:00
Marclass
cf6b3fa48a remove html type and add js/css 2024-01-17 00:34:22 -07:00
Marclass
43d8466677 feat: Add RAG support for various programming languages
Enables RAG for golang, python, java, sh, bat, powershell, cmd, js, css, c/c++/c#, sql, logs, ini, perl, r, dart, docker, env, php, haskell, lua, conf, plsql, ruby, db2, scalla, bash, swift, vue, html, xml, and other arbitrary text files.
2024-01-17 00:09:47 -07:00
goecho
74f91bc74d Fix bug: Header attributes (Host, Authorization, Origin, Referer) not sanitized
- Resolved an issue where header attributes Host, Authorization, Origin, and Referer were not being sanitized, resulting in two major issues:
  1. Ollama requests inadvertently exposed user information, leading to data leakage.
  2. When Ollama is deployed on different servers, and the intermediary proxy layer uses the host header to locate downstream services, it fails to find them.

Root Cause:
- In FastAPI, when accessing request.headers, all header names are converted to lowercase. This is because FastAPI, and its underlying framework Starlette, adhere to the HTTP/2 standard, which mandates lowercase header field names for performance and consistency.
- In HTTP/2, enforcing lowercase header field names reduces complexity in header processing as case sensitivity is no longer a concern. Thus, regardless of the case used in client-sent header fields, the server processes them uniformly in lowercase.
- This practice is adopted in FastAPI and other modern HTTP frameworks, even in an HTTP/1.1 context, to maintain consistency with HTTP/2 and improve overall performance. As a result, header field names are always presented in lowercase in FastAPI, even if the original request used capitalization or mixed case.
2024-01-11 14:36:34 +08:00
Timothy J. Baek
c1ec604f21 feat: rag md support 2024-01-09 15:24:53 -08:00
Timothy J. Baek
76d37393ee feat: gguf upload 2024-01-09 13:25:42 -08:00
Timothy J. Baek
1ff3eb8038 feat: doc upload error handling 2024-01-08 01:54:03 -08:00
Timothy J. Baek
eddb6fc7b7 feat: documents backend integration 2024-01-08 01:49:20 -08:00
Timothy J. Baek
54c4e0761a feat: documents file upload 2024-01-08 01:26:15 -08:00
Timothy J. Baek
cc3f84f916 feat: # to import doc 2024-01-07 23:43:32 -08:00
Timothy J. Baek
57c050326c feat: docx support 2024-01-07 13:56:01 -08:00
Timothy J. Baek
9a63376e55 feat: file upload error handling 2024-01-07 09:33:34 -08:00
Timothy J. Baek
b37b157638 feat: reset vectordb storage support 2024-01-07 09:15:45 -08:00
Timothy J. Baek
d4b2578f6e feat: rag csv support 2024-01-07 09:05:52 -08:00
Timothy J. Baek
d6a1bf1406 refac: file upload 2024-01-07 09:00:30 -08:00
Timothy J. Baek
ffd0a5a2a0 Update main.py 2024-01-07 08:34:05 -08:00
Timothy J. Baek
c68bb3b950 docker: slim 2024-01-07 08:28:35 -08:00
Timothy Jaeryang Baek
34e0f64fb3
Merge pull request #333 from ollama-webui/rag
feat: RAG support
2024-01-07 02:50:32 -08:00
Timothy J. Baek
464d0fb016 fix: update langchain.document_loaders 2024-01-07 02:49:13 -08:00
Timothy J. Baek
70d2571be1 feat: rag backend auth 2024-01-07 02:46:12 -08:00
Timothy J. Baek
142269374f feat: vectordb query error handling 2024-01-07 01:59:00 -08:00
Timothy J. Baek
ad3d69be30 refac 2024-01-07 01:54:58 -08:00
Timothy J. Baek
9634e2da3e feat: full integration 2024-01-07 01:40:36 -08:00
Timothy J. Baek
fef4725d56 feat: frontend file upload support 2024-01-07 00:57:10 -08:00
Timothy J. Baek
cd86c36953 feat: pdf data load 2024-01-06 23:40:51 -08:00
Timothy J. Baek
784b369cc9 feat: chromadb vector store api 2024-01-06 22:59:22 -08:00
Timothy J. Baek
b2c9f6dff8 feat: rag api endpoint 2024-01-06 22:07:20 -08:00
Timothy J. Baek
82114c45ee fix: allow command named create 2024-01-06 17:55:41 -08:00
Timothy J. Baek
73b88a8c17 fix: update user email issue 2024-01-06 02:51:57 -08:00
Timothy J. Baek
e441875af7 fix: update role 2024-01-05 21:02:49 -08:00
Timothy J. Baek
fb0c64379d feat: edit user support 2024-01-05 20:59:56 -08:00
Timothy J. Baek
bb2971260d fix: backend proxy 2024-01-05 17:16:35 -08:00
Timothy J. Baek
78477baa0e feat: proxy logging 2024-01-05 15:51:33 -08:00
Timothy J. Baek
a37b9c126f refac 2024-01-05 01:29:04 -08:00
Timothy J. Baek
47dc3b5fb2 feat: async reverse proxy 2024-01-05 01:25:34 -08:00
ThatOneCalculator
699fbefb8a
chore: 🎨 format 2024-01-04 20:05:19 -08:00
Timothy Jaeryang Baek
1a93191259
Merge branch 'main' into bun 2024-01-04 22:57:11 -05:00
Timothy J. Baek
c0b099da4f feat: openai frontend refac 2024-01-04 18:54:00 -08:00
Timothy J. Baek
17c66fde0f feat: openai compatible api support 2024-01-04 18:38:03 -08:00
Timothy J. Baek
5e4dc98f44 feat: openai backend support 2024-01-04 16:49:34 -08:00
ThatOneCalculator
05fade1426
chore: 🎨 format backend 2024-01-04 13:30:47 -08:00
ThatOneCalculator
8480a9b418
Merge branch 'main' into bun 2024-01-04 13:30:30 -08:00
Timothy J. Baek
30aff2db53 feat: enable backend ollama url update 2024-01-04 13:06:31 -08:00
ThatOneCalculator
07cc7f15d5
chore: 🚨 lint and format 2024-01-03 14:33:57 -08:00
Timothy J. Baek
7fc1d7c2c7 feat: prompt crud 2024-01-02 21:35:47 -08:00
Timothy J. Baek
69ff596045 feat: prompts backend frontend integration 2024-01-02 21:00:50 -08:00
Timothy J. Baek
247414743b feat: prompts backend support 2024-01-02 20:51:19 -08:00
Timothy J. Baek
22c210e8f6 feat: custom prompt support 2024-01-02 20:41:37 -08:00
Timothy J. Baek
09e1458d59 fix: default models value should be None 2024-01-02 16:48:49 -08:00
Timothy J. Baek
7bc0c09b25 fix: openai issue 2024-01-02 16:48:10 -08:00
Timothy J. Baek
d8754b4486 feat/fix: email format validation 2024-01-02 16:22:48 -08:00
Timothy J. Baek
bb5bf3964e feat: toggle signup enable from admin panel 2024-01-01 12:32:28 -08:00
Anuraag Jain
77323d9b25 refac: remove the verify_token and use get-current user for auth+user 2024-01-01 10:55:50 +02:00
Anuraag Jain
60c801ee22 remove unused import in main 2023-12-30 13:08:03 +02:00
Anuraag Jain
2532cb1e70 Merge branch 'main' of github.com:ollama-webui/ollama-webui into refac/auth-middleware 2023-12-30 13:03:18 +02:00
Anuraag Jain
08c0d7a9ec fix: merge conflicts 2023-12-30 13:00:21 +02:00
Anuraag Jain
613053e46c Merge branch 'refac/auth-middleware' of https://github.com/anuraagdjain/ollama-webui into refac/auth-middleware 2023-12-30 12:53:35 +02:00
Anuraag Jain
bdd153d8f5 refac: use dependencies to verify token
- feat: added new util to get the current user when needed. Middleware was adding authentication logic to all the routes. let's revisit if we can move the non-auth endpoints to a separate route.
- refac: update the routes to use new helpers for verification and retrieving user
- chore: added black for local formatting of py code
2023-12-30 12:53:33 +02:00
Timothy J. Baek
d5dc36b2a9 feat: delete all user chats 2023-12-30 00:15:37 -08:00
Timothy Jaeryang Baek
39b8e4430e
Merge branch 'main' into refac/auth-middleware 2023-12-29 12:45:46 -05:00
Timothy J. Baek
d5bc54b8f8 fix: update password 2023-12-29 00:31:23 -08:00
Timothy J. Baek
500f61b7ee chore: update password refac 2023-12-29 00:29:18 -08:00
Timothy J. Baek
d8bb19fd8a feat: change password frontend added 2023-12-29 00:26:47 -08:00
Timothy J. Baek
9bd48ffd93 feat: change password support 2023-12-29 00:12:30 -08:00
Timothy J. Baek
5b4bf45ad2 fix: delete auth with user 2023-12-28 23:24:51 -08:00
Timothy J. Baek
48cc2c5053 chore: delete user backend refac 2023-12-28 23:17:58 -08:00
Timothy J. Baek
ad1cb5fc25 fix: disable admin self user delete 2023-12-28 23:07:46 -08:00
Timothy J. Baek
b61bb77950 feat: delete user backend support 2023-12-28 23:02:49 -08:00
Anuraag Jain
a01b112f7f feat(auth): add auth middleware
- refactored chat routes to use request.user instead of doing authentication in every route
2023-12-28 22:15:54 +02:00
Timothy J. Baek
c31ff1f679 fix: save email as lowercase 2023-12-27 19:10:56 -08:00
Timothy J. Baek
7c553f1287 feat: improved error message for signup 2023-12-27 12:06:22 -08:00
Timothy J. Baek
1d458db316 fix: print removed 2023-12-27 11:14:46 -08:00
Timothy J. Baek
7bdef56192 fix: docker container volume mount location 2023-12-27 00:11:23 -08:00
Timothy J. Baek
21c7f50790 feat: modelfiles migration support 2023-12-26 23:58:40 -08:00
Timothy J. Baek
032d7c7440 feat: modelfile backend & ollama version 0.0.0 whitelisted 2023-12-26 22:51:52 -08:00
Timothy J. Baek
4221594778 feat: chat export 2023-12-26 22:10:22 -08:00
Timothy J. Baek
a2b1e3756b feat: modelfiles backend 2023-12-26 22:02:17 -08:00
Timothy J. Baek
d78df83453 feat: delete idb after migration 2023-12-26 18:44:08 -08:00
Timothy J. Baek
a696698ac8 feat: chat import to backend added 2023-12-26 16:35:01 -08:00
Timothy J. Baek
8f2b691ee0 fix: headers issue 2023-12-26 13:43:32 -08:00
Timothy J. Baek
0ae57b5ae1 fix: cors & #281 2023-12-26 13:40:03 -08:00
Timothy J. Baek
0c30a08596 fix: non existent chat page issue 2023-12-26 13:21:47 -08:00
Timothy J. Baek
1303407f53 feat: update chat 2023-12-26 10:41:55 -08:00
Timothy J. Baek
2cb0bf4439 fix: chat return type to dict 2023-12-26 01:27:43 -08:00
Timothy J. Baek
6350d86bde fix: chat model schema 2023-12-25 23:43:21 -08:00
Timothy J. Baek
540b50e176 feat: wip chat route 2023-12-25 21:55:29 -08:00
Timothy J. Baek
9174331025 feat: db migration to sqlite 2023-12-25 21:44:28 -08:00
Timothy J. Baek
ee83791596 feat: return file_name for model name 2023-12-24 00:40:59 -08:00
Timothy J. Baek
232401a042 feat: gguf file upload status 2023-12-24 00:27:04 -08:00
Timothy J. Baek
0fe7d89f0b feat: create blob from huggingface url 2023-12-23 23:40:14 -08:00
Timothy J. Baek
a640652a9b fix: large file upload issue 2023-12-23 23:05:52 -08:00
Timothy J. Baek
e539cf5c28 feat: gguf upload 2023-12-23 15:38:52 -08:00
Timothy J. Baek
d10e367664 feat: improved backend error message 2023-12-14 17:05:46 -08:00
Timothy J. Baek
25987fe3c9 feat: better error handling for ollama reverse proxy 2023-12-13 17:37:29 -08:00
Timothy J. Baek
ebddb8e47d fix: backend get users output type 2023-11-21 09:39:53 -08:00
Timothy J. Baek
83ff1d77ea feat: set first user to admin by default 2023-11-19 00:41:29 -08:00
Timothy J. Baek
07d2c9871f feat: admin panel added 2023-11-19 00:13:59 -08:00
Timothy J. Baek
8547b7807d feat: basic RBAC support 2023-11-18 21:41:43 -08:00
Timothy J. Baek
921eef03b3 feat: multi-user support w/ RBAC 2023-11-18 16:47:12 -08:00
Timothy J. Baek
6a9bef755b feat: backend reverse proxy 2023-11-14 16:28:51 -08:00