Timothy J. Baek
b1d83fc42c
chore: format
2024-06-17 14:32:23 -07:00
Que Nguyen
c487385980
Set filter_list as optional param in serpstack.py
2024-06-17 14:38:11 +07:00
Que Nguyen
bcb84235b1
Set filter_list as optional param in serply.py
2024-06-17 14:37:52 +07:00
Que Nguyen
6b8290fa6d
Set filter_list as optional param in serper.py
2024-06-17 14:37:26 +07:00
Que Nguyen
9c446d9fb4
Set filter_list as optional param in searxng.py
2024-06-17 14:36:56 +07:00
Que Nguyen
3cc0e3ecb6
Refactor rag/main.py
...
Renamed function get_filtered_results
2024-06-17 14:36:26 +07:00
Que Nguyen
d8beed13b4
Set filter_list as optional param in google_pse.py
2024-06-17 14:35:27 +07:00
Que Nguyen
7d2ad8c4bf
Set filter_list as optional param in duckduckgo.py
2024-06-17 14:34:59 +07:00
Que Nguyen
a02139ba9d
Set filter_list as optional param in brave.py
2024-06-17 14:34:17 +07:00
Que Nguyen
a3ac9ee774
Refactor main.py
...
Rename RAG_WEB_SEARCH_WHITE_LIST_DOMAINS to RAG_WEB_SEARCH_DOMAIN_FILTER_LIST
2024-06-17 14:31:44 +07:00
Que Nguyen
a02ba52de8
Merge branch 'dev' into searxng
2024-06-15 23:44:31 +07:00
Yash-1511
b9da72560a
feat: add tavily web search in web search provider
2024-06-14 20:44:11 +05:30
Que Nguyen
7b5f434a07
Implement domain whitelisting for web search results
2024-06-13 07:14:48 +07:00
Timothy J. Baek
c794d59fd5
revert: do not change the default
2024-06-12 11:47:19 -07:00
Timothy Jaeryang Baek
90dadf0bec
Merge pull request #3073 from que-nguyen/searxng
...
Set searxng language to auto and enable safesearch (moderate).
2024-06-12 11:26:10 -07:00
Timothy J. Baek
1163745a03
revert
2024-06-12 11:08:05 -07:00
Que Nguyen
305ec59d76
Set searxng language as 'auto' and enable safesearch (moderate).
...
Configure searxng with language param set to auto and add "safesearch": 1 (moderate) for safer web results.
2024-06-12 21:33:33 +07:00
Timothy J. Baek
c0ca447041
chore: format
2024-06-12 01:37:53 -07:00
Timothy Jaeryang Baek
5d3db15eca
Merge pull request #3049 from que-nguyen/dev
...
Refactor URL validation function
2024-06-12 01:36:34 -07:00
Timothy J. Baek
e8fc522eba
chore: format
2024-06-12 00:18:22 -07:00
Que Nguyen
eb7bba81fe
Refactor URL validation function
...
- The check for private IP addresses often did not yield the expected results, especially with errors like: `[Errno -2] Name or service not known`.
- Removed the check for private IP addresses in the URL validation process.
- Simplified the `validate_url` function to focus on validating the URL format and checking the existence of the URL using a HEAD request.
2024-06-12 08:15:04 +07:00
Timothy Jaeryang Baek
d709038b5b
Merge pull request #3029 from Yash-1511/main
...
feat: add DuckDuckGo search functionality using duckduckgo_search library
2024-06-11 09:53:26 -07:00
Que Nguyen
3bec60b80c
Fixed the issue where a single URL error disrupts the data loading process in Web Search mode
...
To address the unresolved issue in the LangChain library where a single URL error disrupts the data loading process, the lazy_load method in the WebBaseLoader class has been modified. The enhanced method now handles exceptions appropriately, logging errors and continuing with the remaining URLs.
2024-06-11 22:06:14 +07:00
Yash-1511
83f9475584
feat: add DuckDuckGo search functionality using duckduckgo_search library
2024-06-11 19:49:08 +05:30
Timothy J. Baek
bd5a8567ef
refac: tools & rag
2024-06-11 01:10:24 -07:00
Timothy J. Baek
644f0fe6c3
chore: version bump
2024-06-10 13:52:35 -07:00
teampen
14d33f0fcc
Merge branch 'add-serply' into dev
2024-06-09 21:40:50 -04:00
teampen
efb4a710c8
adding Serply as an alternative web search
2024-06-09 20:44:34 -04:00
Timothy J. Baek
f2b9a5f5bf
refac: rag
2024-06-09 03:01:25 -07:00
mindspawn
6f9148ac4c
Update main.py
2024-06-07 21:41:30 -07:00
mindspawn
4ecc1c06d3
Update main.py
2024-06-07 21:18:04 -07:00
Timothy J. Baek
0495f01acb
feat: reset upload dir
2024-06-03 21:45:36 -07:00
Timothy J. Baek
61867c1545
Update searxng.py
2024-06-03 17:02:50 -07:00
Timothy J. Baek
4068a421bf
fix
2024-06-03 17:00:35 -07:00
Timothy Jaeryang Baek
768941bded
Merge pull request #2785 from cheahjs/feat/openai-embeddings-batch
...
feat: add RAG_EMBEDDING_OPENAI_BATCH_SIZE to batch multiple embeddings
2024-06-03 13:50:14 -07:00
Jun Siang Cheah
7fefbb316d
fix: add backwards compat with older searxng urls
2024-06-03 21:13:10 +01:00
Timothy Jaeryang Baek
92d9b38110
Merge branch 'dev' into feat/openai-embeddings-batch
2024-06-03 12:39:09 -07:00
Lyuboslav Petrov
08443b3c55
Revert log level to debug
2024-06-03 12:48:40 +03:00
Lyuboslav Petrov
7e761a69a7
FIX searxng URL construction using params for arg passing
...
Accept additional parameters such as language, time_range, and categories to tailor the search results.
Raise an exception if a request error occurs during the search process.
Use params argument to construct the query string
Sort by relevance
Expand docstring
2024-06-03 12:44:46 +03:00
Jun Siang Cheah
0cb8163321
feat: add RAG_EMBEDDING_OPENAI_BATCH_SIZE to batch multiple embeddings
2024-06-02 15:34:31 +01:00
Timothy J. Baek
a53796270f
refac: web search config
2024-06-01 20:08:08 -07:00
Timothy J. Baek
fbdfb7e4fa
refac: web search
2024-06-01 19:57:00 -07:00
Timothy J. Baek
999d2bc21b
refac: web search
2024-06-01 19:52:12 -07:00
Timothy J. Baek
912a704fdc
refac: web search settings
2024-06-01 19:40:48 -07:00
Timothy J. Baek
ea6b8984ab
refac: web search
2024-06-01 19:03:56 -07:00
Timothy J. Baek
74a8deb19f
refac
2024-05-27 14:25:36 -07:00
Timothy J. Baek
4685f523b6
refac
2024-05-27 12:48:08 -07:00
Jun Siang Cheah
276b7b90b8
Merge remote-tracking branch 'upstream/dev' into feat/backend-web-search
2024-05-26 11:31:23 +01:00
Timothy J. Baek
84bfebd05e
fix
2024-05-26 01:17:57 -07:00
Jun Siang Cheah
224a578e6b
Merge remote-tracking branch 'upstream/dev' into feat/backend-web-search
2024-05-20 19:53:23 +01:00
