Commit Graph

1902 Commits

Author SHA1 Message Date
Timothy Jaeryang Baek
d05c2f56ea
Merge pull request #4715 from Peter-De-Ath/ollama-batch-embeddings
feat: support ollama batch processing embeddings
2024-10-07 18:32:08 -07:00
Timothy J. Baek
f099b277c8 enh: youtube watch param support 2024-10-07 18:19:13 -07:00
Peter De-Ath
b242460874 formatting 2024-10-08 00:04:35 +01:00
Peter De-Ath
3fcdca2f28 backwards comp: use RAG_EMBEDDING_OPENAI_BATCH_SIZE 2024-10-08 00:04:35 +01:00
Peter De-Ath
885b9f1ece refactor: Update GenerateEmbeddingsForm to support batch processing
refactor: Update embedding batch size handling in RAG configuration

refactor: add query_doc query caching

refactor: update logging statements in generate_chat_completion function

change embedding_batch_size to Optional
2024-10-08 00:04:35 +01:00
Timothy J. Baek
48e7f47558 refac 2024-10-07 14:03:42 -07:00
Timothy J. Baek
ee22ba9676 fix: web, youtube attachment issue 2024-10-06 19:44:02 -07:00
Timothy J. Baek
05c15b017d fix: milvus 2024-10-06 17:58:09 -07:00
Timothy J. Baek
da1e88a427 enh: __task_body__ param 2024-10-06 14:56:49 -07:00
Timothy J. Baek
d23f05fe0a refac 2024-10-06 14:52:12 -07:00
Sylvere Richard
9706b76b36 feat: db pool config #5935 2024-10-06 17:20:10 +02:00
smonux
c17c9e2558 fix: correct variable name in nonstreaming response handler 2024-10-06 11:14:01 +02:00
smonux
83191dbe08 refactor: remove unused imports in main.py 2024-10-06 11:11:52 +02:00
smonux (aider)
6bcfdaaaf0 fix: resolve undefined names 'call_next' and 'response' in handle_nonstreaming_response 2024-10-06 11:11:00 +02:00
smonux (aider)
2a51781633 feat: Complete handle_nonstreaming_response with tool call handling 2024-10-06 11:09:34 +02:00
smonux
bfc5f6b2f6 feat: Add tool handling and response modification for non-streaming responses 2024-10-06 11:09:32 +02:00
Timothy J. Baek
55d0ecc879 refac 2024-10-05 12:41:18 -07:00
Timothy J. Baek
614f3d5f80 fix 2024-10-05 12:40:13 -07:00
Timothy J. Baek
378223aedb refac: comments 2024-10-05 10:08:48 -07:00
Timothy J. Baek
61b147441c refac 2024-10-05 10:05:12 -07:00
Timothy J. Baek
1f9b5b6456 refac: retain metadata for collection 2024-10-05 09:58:46 -07:00
Timothy J. Baek
0bd88090bb refac: DOCS_DIR deprecated 2024-10-05 01:45:22 -07:00
Timothy J. Baek
493745a70b refac 2024-10-04 19:32:33 -07:00
Timothy J. Baek
a909aa1c20 enh: sync directory 2024-10-04 18:44:57 -07:00
Timothy J. Baek
e3889522d6 refac 2024-10-04 18:28:48 -07:00
Timothy J. Baek
79c005a041 refac: deprecate docs_dir 2024-10-04 18:22:55 -07:00
Timothy J. Baek
a6c797d4c2 refac: process docs dir 2024-10-04 17:22:00 -07:00
Timothy J. Baek
5017ca90ff refac 2024-10-04 16:16:16 -07:00
Timothy J. Baek
8f22e911e0 fix: empty knowledge delete issue 2024-10-04 11:11:53 -07:00
Timothy J. Baek
d1b8af6220 chore: format 2024-10-04 01:04:04 -07:00
Timothy Jaeryang Baek
0876c9b5ef
Merge pull request #5829 from jannikstdl/query-embedding-perf-fix
fix:  performance issues on large collections
2024-10-04 10:01:17 +02:00
Timothy Jaeryang Baek
ebc7da6f82
Merge pull request #5861 from open-webui/projects
feat: knowledge/projects
2024-10-04 10:00:47 +02:00
Timothy J. Baek
1fe1c27220 refac: legacy support 2024-10-04 00:59:19 -07:00
Timothy J. Baek
8013c152d0 feat: edit file content support 2024-10-04 00:46:32 -07:00
Timothy J. Baek
630a78cead fix 2024-10-04 00:24:51 -07:00
Timothy J. Baek
17c772831d refac 2024-10-04 00:23:14 -07:00
Timothy J. Baek
05970157f6 refac 2024-10-03 23:06:47 -07:00
Timothy J. Baek
d834bd2a18 refac 2024-10-03 22:35:43 -07:00
Timothy J. Baek
e9b68524e8 refac 2024-10-03 22:24:05 -07:00
Timothy J. Baek
b291271df3 refac 2024-10-03 22:22:22 -07:00
Timothy J. Baek
b862dff185 refac 2024-10-03 21:10:33 -07:00
Timothy J. Baek
6747478f67 refac 2024-10-03 21:05:55 -07:00
Timothy J. Baek
124a17e826 refac 2024-10-03 20:58:56 -07:00
Timothy J. Baek
57360b7a61 refac 2024-10-03 20:51:21 -07:00
Timothy J. Baek
2fc07fd6a2 enh: vector db hash collision check 2024-10-03 06:53:21 -07:00
Timothy J. Baek
78413d0c2e enh: add/remove file from knowledge 2024-10-03 06:46:20 -07:00
Timothy J. Baek
15a3c6b171 refac 2024-10-03 06:45:15 -07:00
Timothy J. Baek
d394f8b7be enh: add to vector db support 2024-10-03 06:44:17 -07:00
Timothy J. Baek
325ca98773 enh: vector db delete filter support 2024-10-03 06:43:50 -07:00
Timothy J. Baek
351b1dbf31 refac 2024-10-02 21:14:58 -07:00
Timothy J. Baek
a2eadb30f5 refac 2024-10-02 20:42:10 -07:00
Timothy Jaeryang Baek
2e7e346e19
Merge pull request #5871 from tcgumus/dev
fix: Unsupported JSON schema type {type_}
2024-10-03 05:07:30 +02:00
Jun Siang Cheah
318f61161e refac: lazy load big dependencies to reduce min memory usage 2024-10-02 22:18:42 +01:00
Timothy J. Baek
4aca1e86ad refac 2024-10-02 06:19:09 -07:00
t
bdb2ce9448 fix: Unsupported JSON schema type {type_} 2024-10-02 15:34:48 +03:00
Timothy J. Baek
08969ecf89 refac: rename projects -> knowledge 2024-10-01 22:45:04 -07:00
Timothy J. Baek
3e012f0219 refac 2024-10-01 21:48:15 -07:00
Timothy J. Baek
5933d7a216 enh: sort by descending order 2024-10-01 21:35:18 -07:00
Timothy J. Baek
1b7d363d32 refac 2024-10-01 21:32:59 -07:00
Timothy J. Baek
c2732a0990 refac 2024-10-01 17:46:56 -07:00
Timothy J. Baek
c5eb0a9732 refac: documents -> projects 2024-10-01 17:35:35 -07:00
Timothy J. Baek
bf57dd808e feat: project migration 2024-10-01 17:35:10 -07:00
Timothy J. Baek
fb083237cd refac 2024-10-01 14:00:19 -07:00
Timothy J. Baek
a0fb4a9b84 refac 2024-10-01 13:13:39 -07:00
Timothy J. Baek
5c9dd25459 refac: files migration 2024-10-01 11:01:26 -07:00
Timothy J. Baek
0907c32e10 refac 2024-10-01 09:51:58 -07:00
Timothy J. Baek
4752df9bd8 refac 2024-10-01 09:40:08 -07:00
Timothy J. Baek
7b9b29253f Merge branch 'dev' of https://github.com/open-webui/open-webui into dev 2024-10-01 09:18:15 -07:00
Timothy Jaeryang Baek
126aff7a7d
Merge pull request #5839 from open-webui/dependabot/pip/backend/dev/langchain-chroma-0.1.4
chore(deps): bump langchain-chroma from 0.1.2 to 0.1.4 in /backend
2024-10-01 17:58:14 +02:00
Timothy Jaeryang Baek
6ca1de3601
Merge pull request #5838 from open-webui/dependabot/pip/backend/dev/duckduckgo-search-approx-eq-6.2.13
chore(deps): update duckduckgo-search requirement from ~=6.2.11 to ~=6.2.13 in /backend
2024-10-01 17:57:59 +02:00
Timothy Jaeryang Baek
fb8af637e1
Merge pull request #5837 from open-webui/dependabot/pip/backend/dev/aiohttp-3.10.8
chore(deps): bump aiohttp from 3.10.5 to 3.10.8 in /backend
2024-10-01 17:57:51 +02:00
Timothy Jaeryang Baek
ab983887cc
Merge pull request #5836 from open-webui/dependabot/pip/backend/dev/pandas-2.2.3
chore(deps): bump pandas from 2.2.2 to 2.2.3 in /backend
2024-10-01 17:57:40 +02:00
dependabot[bot]
736cbb961f
chore(deps): bump langchain-chroma from 0.1.2 to 0.1.4 in /backend
Bumps [langchain-chroma](https://github.com/langchain-ai/langchain) from 0.1.2 to 0.1.4.
- [Release notes](https://github.com/langchain-ai/langchain/releases)
- [Commits](https://github.com/langchain-ai/langchain/compare/langchain-chroma==0.1.2...langchain-chroma==0.1.4)

---
updated-dependencies:
- dependency-name: langchain-chroma
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-10-01 02:22:25 +00:00
dependabot[bot]
8d8c2752dc
chore(deps): update duckduckgo-search requirement in /backend
Updates the requirements on [duckduckgo-search](https://github.com/deedy5/duckduckgo_search) to permit the latest version.
- [Release notes](https://github.com/deedy5/duckduckgo_search/releases)
- [Commits](https://github.com/deedy5/duckduckgo_search/compare/v6.2.12...v6.2.13)

---
updated-dependencies:
- dependency-name: duckduckgo-search
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-10-01 02:22:21 +00:00
dependabot[bot]
d78a7d431d
chore(deps): bump aiohttp from 3.10.5 to 3.10.8 in /backend
Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.10.5 to 3.10.8.
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst)
- [Commits](https://github.com/aio-libs/aiohttp/compare/v3.10.5...v3.10.8)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-10-01 02:22:18 +00:00
dependabot[bot]
91ca5c424b
chore(deps): bump pandas from 2.2.2 to 2.2.3 in /backend
Bumps [pandas](https://github.com/pandas-dev/pandas) from 2.2.2 to 2.2.3.
- [Release notes](https://github.com/pandas-dev/pandas/releases)
- [Commits](https://github.com/pandas-dev/pandas/compare/v2.2.2...v2.2.3)

---
updated-dependencies:
- dependency-name: pandas
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-10-01 02:22:08 +00:00
dependabot[bot]
7e1c55d2ed
chore(deps): bump pydantic from 2.8.2 to 2.9.2 in /backend
Bumps [pydantic](https://github.com/pydantic/pydantic) from 2.8.2 to 2.9.2.
- [Release notes](https://github.com/pydantic/pydantic/releases)
- [Changelog](https://github.com/pydantic/pydantic/blob/main/HISTORY.md)
- [Commits](https://github.com/pydantic/pydantic/compare/v2.8.2...v2.9.2)

---
updated-dependencies:
- dependency-name: pydantic
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-10-01 02:22:04 +00:00
Timothy J. Baek
c7a0e45bea refac 2024-09-30 16:32:38 +02:00
Jannik Streidl
6bff5a4d09 fix: do not embed the query every single time 2024-09-30 16:18:02 +02:00
Timothy J. Baek
1c4b6b9cd9 refac 2024-09-30 01:00:13 +02:00
Timothy J. Baek
3899405864 refac 2024-09-30 00:39:30 +02:00
Timothy J. Baek
7152af949b feat: compress audio
Co-Authored-By: Beck Bekmyradov <47065940+bekmuradov@users.noreply.github.com>
2024-09-30 00:30:12 +02:00
Timothy J. Baek
8206c47a47 refac 2024-09-29 23:20:37 +02:00
Timothy J. Baek
6afc686e17 refac 2024-09-29 23:08:55 +02:00
Timothy J. Baek
677c36c3aa refac 2024-09-29 22:55:53 +02:00
Timothy J. Baek
6d764ee55e feat: retrieval whole document mode 2024-09-29 22:52:27 +02:00
Timothy J. Baek
92dd173b27 refac 2024-09-29 18:55:26 +02:00
Timothy J. Baek
c93a10388b refac 2024-09-28 19:51:28 +02:00
Timothy J. Baek
276ce3374d refac 2024-09-28 19:17:21 +02:00
Timothy J. Baek
e77c3ab043 refac 2024-09-28 19:16:52 +02:00
Timothy J. Baek
90ec458c4c enh: show extracted file content 2024-09-28 10:53:25 +02:00
smonux
e039b4ec54
Merge branch 'open-webui:dev' into dev 2024-09-28 06:01:26 +02:00
Timothy J. Baek
9d2ed3d2be refac 2024-09-28 02:56:56 +02:00
Timothy J. Baek
b8b994a820 refac 2024-09-28 02:49:18 +02:00
Timothy J. Baek
00eb022450 refac 2024-09-28 02:38:59 +02:00
Timothy J. Baek
2428878f42 refac 2024-09-28 02:29:08 +02:00
Timothy J. Baek
af57a2c153 refac 2024-09-28 02:23:09 +02:00
Timothy J. Baek
a52e8cd537 refac 2024-09-28 01:35:31 +02:00
Timothy J. Baek
5b7cf88915 refac 2024-09-28 01:28:45 +02:00
Timothy J. Baek
e1103305f5 refac: "rag" endpoints renamed to "retrieval" 2024-09-28 01:27:46 +02:00