UniqAI/open-webui
1
0
Fork 0
mirror of https://github.com/open-webui/open-webui synced 2024-11-06 08:56:39 +00:00
Code Issues Actions 5 Packages Projects Releases Wiki Activity

refac

Browse Source
This commit is contained in:
Timothy J. Baek 2024-10-25 22:12:46 -07:00
parent 21a28e3bf0
commit 47e4250f58
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
backend/open_webui/apps/retrieval/main.py
View File

@ -667,7 +667,7 @@ def save_docs_to_vector_db(
add_start_index=True,
)
elif app.state.config.TEXT_SPLITTER == "token":
tiktoken.get_encoding(app.state.config.TIKTOKEN_ENCODING_NAME)
text_splitter = TokenTextSplitter(
encoding_name=str(app.state.config.TIKTOKEN_ENCODING_NAME),
chunk_size=app.state.config.CHUNK_SIZE,