342 Commits

Author	SHA1	Message	Date
Timothy J. Baek	0feaf6f649	refac	2024-10-08 13:39:27 -07:00
Timothy J. Baek	dcab991d44	fix: model dump issue	2024-10-08 11:05:57 -07:00
Peter De-Ath	885b9f1ece	refactor: Update GenerateEmbeddingsForm to support batch processing ... refactor: Update embedding batch size handling in RAG configuration refactor: add query_doc query caching refactor: update logging statements in generate_chat_completion function change embedding_batch_size to Optional	2024-10-08 00:04:35 +01:00
Timothy J. Baek	48e7f47558	refac	2024-10-07 14:03:42 -07:00
Jannik Streidl	9fc813cfa6	fix: only append if distances are available	2024-10-07 21:13:13 +02:00
Jannik Streidl	b105efa05f	enh: append citations with distance scores	2024-10-07 21:11:04 +02:00
Timothy J. Baek	ee22ba9676	fix: web, youtube attachment issue	2024-10-06 19:44:02 -07:00
Timothy J. Baek	05c15b017d	fix: milvus	2024-10-06 17:58:09 -07:00
Timothy J. Baek	da1e88a427	enh: __task_body__ param	2024-10-06 14:56:49 -07:00
Sylvere Richard	9706b76b36	feat: db pool config #5935	2024-10-06 17:20:10 +02:00
Timothy J. Baek	378223aedb	refac: comments	2024-10-05 10:08:48 -07:00
Timothy J. Baek	61b147441c	refac	2024-10-05 10:05:12 -07:00
Timothy J. Baek	1f9b5b6456	refac: retain metadata for collection	2024-10-05 09:58:46 -07:00
Timothy J. Baek	0bd88090bb	refac: DOCS_DIR deprecated	2024-10-05 01:45:22 -07:00
Timothy J. Baek	493745a70b	refac	2024-10-04 19:32:33 -07:00
Timothy J. Baek	a909aa1c20	enh: sync directory	2024-10-04 18:44:57 -07:00
Timothy J. Baek	e3889522d6	refac	2024-10-04 18:28:48 -07:00
Timothy J. Baek	79c005a041	refac: deprecate docs_dir	2024-10-04 18:22:55 -07:00
Timothy J. Baek	a6c797d4c2	refac: process docs dir	2024-10-04 17:22:00 -07:00
Timothy J. Baek	5017ca90ff	refac	2024-10-04 16:16:16 -07:00
Timothy J. Baek	8f22e911e0	fix: empty knowledge delete issue	2024-10-04 11:11:53 -07:00
Timothy J. Baek	d1b8af6220	chore: format	2024-10-04 01:04:04 -07:00
Timothy Jaeryang Baek	0876c9b5ef	Merge pull request #5829 from jannikstdl/query-embedding-perf-fix ... fix: performance issues on large collections	2024-10-04 10:01:17 +02:00
Timothy Jaeryang Baek	ebc7da6f82	Merge pull request #5861 from open-webui/projects ... feat: knowledge/projects	2024-10-04 10:00:47 +02:00
Timothy J. Baek	1fe1c27220	refac: legacy support	2024-10-04 00:59:19 -07:00
Timothy J. Baek	8013c152d0	feat: edit file content support	2024-10-04 00:46:32 -07:00
Timothy J. Baek	630a78cead	fix	2024-10-04 00:24:51 -07:00
Timothy J. Baek	17c772831d	refac	2024-10-04 00:23:14 -07:00
Timothy J. Baek	05970157f6	refac	2024-10-03 23:06:47 -07:00
Timothy J. Baek	d834bd2a18	refac	2024-10-03 22:35:43 -07:00
Timothy J. Baek	e9b68524e8	refac	2024-10-03 22:24:05 -07:00
Timothy J. Baek	b291271df3	refac	2024-10-03 22:22:22 -07:00
Timothy J. Baek	b862dff185	refac	2024-10-03 21:10:33 -07:00
Timothy J. Baek	6747478f67	refac	2024-10-03 21:05:55 -07:00
Timothy J. Baek	124a17e826	refac	2024-10-03 20:58:56 -07:00
Timothy J. Baek	57360b7a61	refac	2024-10-03 20:51:21 -07:00
Willnow, Patrick	dc92178641	Fix missing key mapping	2024-10-03 20:55:32 +02:00
Timothy J. Baek	2fc07fd6a2	enh: vector db hash collision check	2024-10-03 06:53:21 -07:00
Timothy J. Baek	78413d0c2e	enh: add/remove file from knowledge	2024-10-03 06:46:20 -07:00
Timothy J. Baek	15a3c6b171	refac	2024-10-03 06:45:15 -07:00
Timothy J. Baek	d394f8b7be	enh: add to vector db support	2024-10-03 06:44:17 -07:00
Timothy J. Baek	325ca98773	enh: vector db delete filter support	2024-10-03 06:43:50 -07:00
Timothy J. Baek	351b1dbf31	refac	2024-10-02 21:14:58 -07:00
Timothy J. Baek	a2eadb30f5	refac	2024-10-02 20:42:10 -07:00
Jun Siang Cheah	318f61161e	refac: lazy load big dependencies to reduce min memory usage	2024-10-02 22:18:42 +01:00
Timothy J. Baek	4aca1e86ad	refac	2024-10-02 06:19:09 -07:00
Timothy J. Baek	08969ecf89	refac: rename projects -> knowledge	2024-10-01 22:45:04 -07:00
Timothy J. Baek	5933d7a216	enh: sort by descending order	2024-10-01 21:35:18 -07:00
Timothy J. Baek	1b7d363d32	refac	2024-10-01 21:32:59 -07:00
Timothy J. Baek	c5eb0a9732	refac: documents -> projects	2024-10-01 17:35:35 -07:00
Timothy J. Baek	fb083237cd	refac	2024-10-01 14:00:19 -07:00
Timothy J. Baek	a0fb4a9b84	refac	2024-10-01 13:13:39 -07:00
Timothy J. Baek	5c9dd25459	refac: files migration	2024-10-01 11:01:26 -07:00
Timothy J. Baek	0907c32e10	refac	2024-10-01 09:51:58 -07:00
Timothy J. Baek	c7a0e45bea	refac	2024-09-30 16:32:38 +02:00
Jannik Streidl	6bff5a4d09	fix: do not embed the query every single time	2024-09-30 16:18:02 +02:00
Timothy J. Baek	1c4b6b9cd9	refac	2024-09-30 01:00:13 +02:00
Timothy J. Baek	3899405864	refac	2024-09-30 00:39:30 +02:00
Timothy J. Baek	7152af949b	feat: compress audio ... Co-Authored-By: Beck Bekmyradov <47065940+bekmuradov@users.noreply.github.com>	2024-09-30 00:30:12 +02:00
Timothy J. Baek	8206c47a47	refac	2024-09-29 23:20:37 +02:00
Timothy J. Baek	6afc686e17	refac	2024-09-29 23:08:55 +02:00
Timothy J. Baek	677c36c3aa	refac	2024-09-29 22:55:53 +02:00
Timothy J. Baek	6d764ee55e	feat: retrieval whole document mode	2024-09-29 22:52:27 +02:00
Timothy J. Baek	92dd173b27	refac	2024-09-29 18:55:26 +02:00
Timothy J. Baek	c93a10388b	refac	2024-09-28 19:51:28 +02:00
Timothy J. Baek	276ce3374d	refac	2024-09-28 19:17:21 +02:00
Timothy J. Baek	e77c3ab043	refac	2024-09-28 19:16:52 +02:00
Timothy J. Baek	90ec458c4c	enh: show extracted file content	2024-09-28 10:53:25 +02:00
Timothy J. Baek	9d2ed3d2be	refac	2024-09-28 02:56:56 +02:00
Timothy J. Baek	b8b994a820	refac	2024-09-28 02:49:18 +02:00
Timothy J. Baek	00eb022450	refac	2024-09-28 02:38:59 +02:00
Timothy J. Baek	2428878f42	refac	2024-09-28 02:29:08 +02:00
Timothy J. Baek	af57a2c153	refac	2024-09-28 02:23:09 +02:00
Timothy J. Baek	a52e8cd537	refac	2024-09-28 01:35:31 +02:00
Timothy J. Baek	5b7cf88915	refac	2024-09-28 01:28:45 +02:00
Timothy J. Baek	e1103305f5	refac: "rag" endpoints renamed to "retrieval"	2024-09-28 01:27:46 +02:00
Timothy J. Baek	4ead3c5b80	chore: format	2024-09-27 19:43:40 +02:00
kivvi	e13614e11b	Fix: O1 does not support the system parameter	2024-09-27 22:47:24 +08:00
Timothy J. Baek	719f4da1dc	fix: milvus collection creation issue	2024-09-26 22:59:09 +02:00
Hwang In Tak	4fe1f2487d	fix: Fix OpenAI batch embedding	2024-09-26 20:48:14 +09:00
Hwang In Tak	a4bc0b2829	fix: Fix OpenAI batch embedding	2024-09-26 20:39:40 +09:00
Timothy Jaeryang Baek	cc19b8049a	Merge pull request #5710 from kivvi3412/fix_o1_max_tokens ... Fix: o1 input parameter must be max_completion_tokens	2024-09-26 03:03:15 +02:00
kivvi	405d0561df	Fix: o1 input parameter must be max_completion_tokens	2024-09-26 08:49:59 +08:00
Timothy J. Baek	ee33b4e2a3	fix: ollama /embed form_data	2024-09-25 22:34:02 +02:00
Timothy J. Baek	e19406cdd7	fix	2024-09-24 17:43:43 +02:00
Timothy J. Baek	3cee507687	fix: dev2	2024-09-24 16:19:24 +02:00
Timothy J. Baek	a44e9a8dda	refac	2024-09-24 15:41:23 +02:00
Timothy J. Baek	504d910557	fix: no running event loop issue	2024-09-24 15:31:55 +02:00
Timothy J. Baek	e268ee5675	enh: audio/x-m4a support	2024-09-24 11:00:47 +02:00
Etienne Perot	fdd27aa321	fix: close temporary file after creating it. ... This fixes "The process cannot access the file because it is being used by another process" errors on Windows. The file is still automatically deleted by the `os.unlink` call later in the function. Updates #5606 Fixes #5642	2024-09-23 23:22:47 -07:00
Timothy J. Baek	c292fd89f9	refac	2024-09-22 02:55:45 +02:00
Timothy J. Baek	768717aaf9	refac	2024-09-22 02:35:25 +02:00
Timothy J. Baek	d055e1f888	refac	2024-09-22 02:14:59 +02:00
Timothy J. Baek	5f84145a2d	enh: socket full redis support	2024-09-22 02:12:55 +02:00
Timothy J. Baek	e35883ca9c	fix: openai payload issue	2024-09-21 21:35:00 +02:00
Timothy J. Baek	00f6b4bf09	refac	2024-09-21 15:35:35 +02:00
Timothy J. Baek	9126ceac08	fix: WEBUI_AUTH=False not working issue	2024-09-21 15:33:34 +02:00
Timothy Jaeryang Baek	273787fe78	Merge pull request #5541 from kivvi3412/fix-title-generate ... [fix] Fix title generate	2024-09-21 03:58:28 +02:00
Timothy J. Baek	ee6b1376c3	fix: rag duplicate collection issue	2024-09-21 03:53:53 +02:00
kivvi	e1ea0c23eb	Fixed the issue of being unable to generate titles	2024-09-21 09:06:28 +08:00
Timothy J. Baek	657d443a3e	fix	2024-09-21 02:24:30 +02:00
Timothy J. Baek	ffd7d74f77	enh: websocket redis support	2024-09-20 23:43:22 +02:00
Timothy J. Baek	b7ad82757d	refac	2024-09-20 03:18:14 +02:00
Timothy J. Baek	d12b6cda4e	refac: openai image async gen	2024-09-20 03:16:08 +02:00
Timothy J. Baek	eb9ad47ef8	refac: temp tools & functions files	2024-09-20 00:12:52 +02:00
Timothy J. Baek	39c57c0e94	refac	2024-09-19 22:40:06 +02:00
Timothy J. Baek	9be73ea94a	refac	2024-09-19 22:17:32 +02:00
Timothy J. Baek	cd117f5b67	refac	2024-09-19 21:51:45 +02:00
Timothy J. Baek	2dad9b9432	refac	2024-09-19 20:56:13 +02:00
Timothy J. Baek	cba2d31175	refac	2024-09-19 18:40:23 +02:00
Timothy J. Baek	a2b77fd072	refac	2024-09-19 18:08:52 +02:00
Timothy J. Baek	9b83e57372	refac	2024-09-19 17:46:11 +02:00
Timothy J. Baek	687cae9b79	refac	2024-09-19 17:31:59 +02:00
Timothy J. Baek	7b330d1490	refac	2024-09-19 17:05:49 +02:00
Timothy J. Baek	8426874426	fix	2024-09-19 16:54:34 +02:00
Timothy J. Baek	ff737a9e25	enh: openai error message handling	2024-09-19 16:20:00 +02:00
Timothy Jaeryang Baek	d52eaf19d6	Merge pull request #5511 from EtiennePerot/import-file ... fix: restore `__file__` variable for imported toolkits and functions	2024-09-19 15:29:34 +02:00
Timothy Jaeryang Baek	5d3a89dd25	Merge pull request #5496 from pawel-ochman/azure-tts ... Integrate Azure Speech service for TTS	2024-09-19 02:42:45 +02:00
Timothy J. Baek	afa42dd2e4	refac	2024-09-19 02:40:54 +02:00
Pawel Ochman	4d9677e808	Update configuration page, expose all Azure settings through ENV variables	2024-09-18 14:13:42 +01:00
Pawel Ochman	eacb69074e	remove dependency and migrate to raw rest calls	2024-09-18 12:24:55 +01:00
Etienne Perot	6477bf37fe	fix: restore __file__ variable for imported toolkits and functions ... Commit cf86ba7786 changed the way toolkits and functions were imported to use `exec`, whereas they previously were written to files and `import`ed. The use of `exec` means that module-global variables such as `__file__` are no longer defined. This breaks https://github.com/EtiennePerot/open-webui-code-execution (code execution tool for Open WebUI), as the module needs to re-execute its own code in a subprocess in order to properly sandbox itself. This is done using `__file__` in order to know where the module's code is located. This PR creates a temporary in-memory file that contains the imported toolkit or function's code and exists only during the import process. Then it injects the path to this in-memory file as the `__file__` variable in the `exec` context. This restores the ability for the toolkit or function being imported to rely on `__file__`.	2024-09-18 01:08:30 -07:00
Timothy J. Baek	7a9c0946a4	refac	2024-09-17 23:13:51 +02:00
Timothy J. Baek	7e0a26ef4e	refac	2024-09-17 23:07:04 +02:00
Timothy J. Baek	67f95ddfdc	refac	2024-09-17 22:58:06 +02:00
Pawel Ochman	d6b68f405e	added azure speech service support	2024-09-17 09:13:10 +01:00
Timothy J. Baek	06debb322b	refac: colbert cuda support	2024-09-16 12:42:48 +02:00
Timothy J. Baek	b7f0759485	refac	2024-09-16 12:36:43 +02:00
Timothy J. Baek	bc6f23f82f	fix	2024-09-16 12:33:55 +02:00
Timothy J. Baek	cb9e76c7f9	refac: default rag template	2024-09-16 12:01:04 +02:00
Timothy J. Baek	b38986a0aa	enh: colbert rerank support	2024-09-16 11:46:39 +02:00
Timothy J. Baek	db0c576f48	refac	2024-09-16 07:14:17 +02:00
Timothy J. Baek	d0df2cbe53	fix: disable "enable new sign ups" not working issue	2024-09-14 23:13:52 +01:00
Timothy Jaeryang Baek	27baa00afb	Merge pull request #5419 from cheahjs/fix/remove-doc-support ... fix: remove unsupported .doc file ingest	2024-09-14 22:52:40 +01:00
Timothy J. Baek	869063c743	refac: better reranking model error handling	2024-09-14 20:58:42 +01:00
Jun Siang Cheah	3199e26500	fix: remove unsupported .doc file ingest	2024-09-14 12:44:51 +01:00
Timothy Jaeryang Baek	d75d638b9a	Merge pull request #5406 from open-webui/main ... refac	2024-09-13 19:26:51 +01:00
Timothy J. Baek	b64c9d966a	refac	2024-09-13 14:26:32 -04:00
Timothy Jaeryang Baek	8a928e5356	Merge pull request #5405 from open-webui/main ... refac: filter non chat completions models	2024-09-13 19:23:02 +01:00
Timothy J. Baek	adf958559b	refac: filter non chat completions models	2024-09-13 14:20:27 -04:00
Timothy Jaeryang Baek	c1d3481c41	Merge pull request #5399 from open-webui/main ... fix: allow o1	2024-09-13 15:01:48 +01:00
Timothy J. Baek	bc0baa35e6	fix: allow o1	2024-09-13 09:59:41 -04:00
Timothy J. Baek	f549cb1f87	fix	2024-09-13 01:30:30 -04:00
Timothy J. Baek	823093eea6	fix: hybrid search	2024-09-13 01:21:47 -04:00
Timothy J. Baek	939bfd153e	refac	2024-09-13 01:18:20 -04:00
Timothy J. Baek	b943b7d337	refac	2024-09-13 01:08:02 -04:00
Timothy J. Baek	8a4b3e6bc9	refac: allow multiple [context] in prompt	2024-09-13 01:06:51 -04:00
Timothy J. Baek	7df997c992	chore: format	2024-09-13 00:49:23 -04:00
Timothy J. Baek	bebc0d2073	chore: format	2024-09-13 00:48:54 -04:00
Timothy J. Baek	47e9c12fc2	refac	2024-09-13 00:41:20 -04:00
Timothy Jaeryang Baek	5970dadead	Merge pull request #5256 from Bazsalanszky/main ... feat: More options for AUTOMATIC1111	2024-09-13 05:40:09 +01:00
Timothy Jaeryang Baek	7dc4cb30b2	Merge pull request #5378 from thiswillbeyourgithub/fix_RAG_and_web ... fix: RAG and Web Search + RAG enhancements	2024-09-13 05:38:53 +01:00
Peter De-Ath	ca9874757f	fix: change metadata to metadatas	2024-09-12 20:14:14 +01:00
thiswillbeyourgithub	65d5545cf0	added a few type hints	2024-09-12 17:22:22 +02:00
thiswillbeyourgithub	e872f5dc78	log: added a debug log if detecting a potential prompt injection attack	2024-09-12 17:22:22 +02:00
thiswillbeyourgithub	9661fee554	fix: handle case where [query] happens in the RAG context	2024-09-12 17:22:22 +02:00
thiswillbeyourgithub	adf26789b8	logs: crash if rag_template would be wrong	2024-09-12 17:22:22 +02:00
thiswillbeyourgithub	ed2a1e7db9	enh: use non hybrid search as fallback if hybrid search failed	2024-09-12 17:22:22 +02:00
thiswillbeyourgithub	53f03f6556	fix: log exception when issues of collection querying	2024-09-12 16:51:37 +02:00
Timothy J. Baek	62f1933e3c	refac: audio input (audio/ogg support)	2024-09-12 09:18:20 -04:00
Timothy J. Baek	0b30dc357c	refac: remove prints	2024-09-12 09:13:21 -04:00
Balazs Toldi	d05ba042c0	Make the optional AUTOMATIC1111 values nullable ... This commit makes the optional AUTOMATIC1111 options default to None, and if the value is removed, it resets to None. Signed-off-by: Balazs Toldi <balazs@toldi.eu>	2024-09-12 14:00:24 +02:00
Timothy Jaeryang Baek	675403d26d	Merge pull request #5364 from open-webui/multiple-vector-dbs ... refac: vector db clients	2024-09-12 07:00:55 +01:00
Timothy J. Baek	8be6e16513	refac: vector db clients	2024-09-12 02:00:31 -04:00
Timothy Jaeryang Baek	c7fc17da69	Merge pull request #5312 from open-webui/multiple-vector-dbs ... feat: various vector db support	2024-09-12 06:53:47 +01:00
Timothy J. Baek	4775fe43d8	feat: milvus support	2024-09-12 01:52:19 -04:00
Timothy Jaeryang Baek	61ee4bd629	Merge pull request #5319 from Liuzhch1/fix-openai_timeout ... fix: get OpenAI models only if OpenAI API Enabled; timeout for querying OpenAI models	2024-09-10 23:03:20 +01:00
Liuzhch1	26700ac4ac	fix: get OpenAI models only if OpenAI Enabled;timeout for query OpenAI models	2024-09-10 19:22:05 +08:00
Timothy J. Baek	0886b3a0a4	refac: comments	2024-09-10 04:46:40 +01:00
Timothy J. Baek	522afbb0a0	refac	2024-09-10 04:37:06 +01:00
Timothy Jaeryang Baek	3d6d8c91dd	Merge branch 'multiple-vector-dbs' into dev	2024-09-10 02:29:55 +01:00
Timothy J. Baek	4354f270ce	refac	2024-09-10 02:27:50 +01:00
Timothy J. Baek	28087ccf40	refac	2024-09-10 01:37:36 +01:00
Timothy Jaeryang Baek	1023ff8454	Merge pull request #5270 from cheahjs/fix/websocket-take-2 ... fix: socket.io connections failing when websockets are not available	2024-09-09 23:19:05 +01:00
Jun Siang Cheah	827c419251	feat: add ENABLE_WEBSOCKET_SUPPORT to force socket.io to ignore websocket upgrades	2024-09-09 23:17:17 +01:00
Timothy J. Baek	f1fae805a2	fix: separate /embed and /embedding ollama endpoint	2024-09-09 23:02:26 +01:00
Jun Siang Cheah	698976add0	feat: add ENABLE_WEBSOCKET_SUPPORT to force socket.io to ignore websocket upgrades	2024-09-08 12:00:36 +01:00
Balazs Toldi	7f6dae41f0	More options for AUTOMATIC1111 ... This commit adds 3 new options to the AUTOMATIC1111 settings: - CFG Scale - Sampler - Scheduler These options allow users to configure these parameters directly through the admin settings, without needing to modify the source code, which was previously required to change the default values in AUTOMATIC1111. Signed-off-by: Balazs Toldi <balazs@toldi.eu>	2024-09-07 17:21:17 +02:00
Timothy J. Baek	1c20db775c	refac: enable /api/embed	2024-09-07 05:12:46 +01:00
Timothy J. Baek	5c8fb4b3d5	refac: web search	2024-09-07 04:50:29 +01:00
Timothy J. Baek	ff46fe2b4a	refac	2024-09-07 03:09:57 +01:00
Timothy J. Baek	02d5bca44d	fix: tools & function not installing requirements	2024-09-06 18:35:43 +02:00
Timothy J. Baek	14eda1bf5b	fix: pdf download FONTS_DIR issue	2024-09-06 15:52:23 +02:00
Timothy J. Baek	b35bbaade2	fix: extract_frontmatter issue	2024-09-05 20:35:58 +02:00
Timothy J. Baek	0c1fab09ff	refac	2024-09-05 18:55:31 +02:00
Timothy J. Baek	92a88df484	refac	2024-09-04 20:00:47 +02:00
Timothy J. Baek	f2f713023d	refac	2024-09-04 19:57:41 +02:00
Timothy J. Baek	cf86ba7786	refac: tools & functions	2024-09-04 19:55:20 +02:00
Timothy J. Baek	d6fc0ccf65	fix	2024-09-04 19:03:51 +02:00
Timothy J. Baek	3afd66d50f	fix: tools & functions import	2024-09-04 18:59:50 +02:00
Timothy J. Baek	1779e6fecc	chore: format	2024-09-04 16:58:28 +02:00
Timothy J. Baek	03d5a670f6	refac: mv backend files to /open_webui dir	2024-09-04 16:54:48 +02:00