UniqueSoft
/
firecrawl
mirror of https://github.com/mendableai/firecrawl
3
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
529 Commits 223 Branches 1 Tag 45 MiB
Commit Graph

529 Commits

Author SHA1 Message Date
Nicolas c20c38721d Update index.test.ts 2024-05-28 17:17:20 -07:00
Nicolas 0f43a12906 Update index.test.ts 2024-05-28 17:17:12 -07:00
Nicolas f53d25efac Merge branch 'main' into nsc/wait-for-param 2024-05-28 12:56:28 -07:00
Nicolas 1b3547dcf2 Nick: 2024-05-28 12:56:24 -07:00
Nicolas 9b5e5b8722
Merge pull request #183 from mendableai/test-sdks 
[Test] Added sdk e2e tests
 2024-05-28 11:32:29 -07:00
Nicolas 41b98f15a8
Update README.md 2024-05-27 22:57:13 -07:00
rafaelsideguide 71187b03a2 added timeout 2024-05-27 16:48:08 -03:00
rafaelsideguide d5c83803cd fixing idempotency test 2024-05-27 16:35:01 -03:00
rafaelsideguide 41c4ef6a82 dotenv was missing 2024-05-27 16:23:57 -03:00
rafaelsideguide 6b58da1c96 jest 2024-05-27 16:01:48 -03:00
rafaelsideguide e87d39e6ec typo 2024-05-27 15:55:48 -03:00
rafaelsideguide 127d2db1dd added js/ts sdk tests 2024-05-27 15:54:09 -03:00
rafaelsideguide d0c4b24a0e missing redis 2024-05-27 14:36:32 -03:00
rafaelsideguide 952ccd8755 envs 2024-05-27 14:34:39 -03:00
rafaelsideguide f32c16258a missing node setup 2024-05-27 14:32:34 -03:00
rafaelsideguide a9b68d95d8 Update test.py 2024-05-27 14:28:44 -03:00
rafaelsideguide c410dbe5bd Update python-tests.yml 2024-05-27 14:24:30 -03:00
rafaelsideguide 667d3e4c4f Merge branch 'test-sdks' of https://github.com/mendableai/firecrawl into test-sdks 2024-05-27 14:23:39 -03:00
rafaelsideguide 19decd1062 fixing workflow 2024-05-27 14:21:33 -03:00
Rafael Miller 3c8edf683c
Merge branch 'main' into test-sdks 2024-05-27 14:15:18 -03:00
rafaelsideguide 63772ea711 added github action workflow 2024-05-27 14:14:00 -03:00
Nicolas 1ef307cb6f Nick: checks 2024-05-27 10:01:12 -07:00
Nicolas a9e45cdb15
Merge pull request #191 from mattjoyce/main 
Various PyPi Metadata
 2024-05-26 18:15:19 -07:00
Nicolas 1de53cc4d0 Nick: fixes 2024-05-26 18:15:05 -07:00
Nicolas efb821d63b
Merge branch 'main' into main 2024-05-26 18:12:23 -07:00
Nicolas ed4226fd1f
Update setup.py 2024-05-26 18:11:54 -07:00
Nicolas 1bbfb98d7e
Merge pull request #186 from Keredu/main 
Limit on /search is not deterministic
 2024-05-26 18:08:16 -07:00
Nicolas 67a53a9ae0
Merge pull request #190 from simonha9/simonha9/improve-rate-limit-error-msg 
Feat: Provide more details for 429 error msg
 2024-05-26 18:07:42 -07:00
Nicolas 7e2df7bd5e Update auth.ts 2024-05-26 18:07:21 -07:00
Nicolas 7948c6cee2 Nick: fixed pip issues 2024-05-26 18:03:37 -07:00
Matt Joyce b061e12030 added python versions requirement 
this is inline with requests module, a critical dependency
 2024-05-26 11:37:47 +10:00
Matt Joyce f00dffbbb1 added misc PyPi keys 
help potential users find and understand the purpose and status of the project.
 2024-05-26 11:36:29 +10:00
Matt Joyce cd7f260288 Added PyPi classifiers 
These classifiers will help potential users find and understand the purpose and status of the project.  use python 3.8 as the base, because that's what module 'requests' needs.
 2024-05-26 11:33:28 +10:00
Matt Joyce e5c6ac23fe Added long description to PyPi 
https://packaging.python.org/en/latest/guides/making-a-pypi-friendly-readme/
 2024-05-26 10:01:35 +10:00
Simon H 115204e6b6 Feat: Provide more details for 429 error msg 
- Added better error code for when rate limit exceeded including
consumed/remaining points, reset date and retry-after seconds
 2024-05-25 12:03:20 -04:00
Keredu 2192978f91 Limit on /search is not deterministic 2024-05-25 00:12:26 +02:00
Nicolas e5c8719554 Update blocklist.ts 2024-05-24 14:53:04 -07:00
rafaelsideguide 397769c7e3 added python sdk e2e tests with pytest 
some of them are still missing though
 2024-05-24 17:56:27 -03:00
Rafael Miller 4ce28593b2
Merge pull request #132 from mendableai/feat/idempotency-key 
[Feat] Added idempotency key to crawl route
 2024-05-24 14:22:50 -03:00
rafaelsideguide d39860c08b Merge branch 'main' into feat/idempotency-key 2024-05-24 14:15:37 -03:00
Nicolas 605ba4c031
Merge pull request #178 from mattjoyce/main 
Fix FIRECRAWL_API_URL bug, also various PyLint fixes
 2024-05-24 09:49:42 -07:00
Nicolas 8c380d70a5
Update firecrawl.py 2024-05-24 09:48:48 -07:00
Nicolas 65fe9c4f80
Merge branch 'main' into main 2024-05-24 09:47:12 -07:00
Rafael Miller 53a7ec0f6e
Removed hard coded timeout 2024-05-24 13:46:16 -03:00
Nicolas e0d979edad
Merge pull request #176 from mendableai/bug/data-check-in-python-sdk 
[Bug] Added data check for python SDK
 2024-05-24 09:45:39 -07:00
Nicolas 53a214cefb
Merge pull request #168 from mendableai/nsc/allowed-keywords-in-blocklist 
feat: Allow privacy/legal/ other pages in social media websites
 2024-05-24 09:43:15 -07:00
Nicolas e166c07690
Merge pull request #170 from qyou/fix-hardcode-timeout 
update: wait until body attached in playwright-service
 2024-05-24 09:41:27 -07:00
Rafael Miller c00580eabd
Merge pull request #181 from JakobStadlhuber/feat/proxy-support 
[Feat] Added proxy and media blocking support for Playwright
 2024-05-24 13:34:34 -03:00
Jakob Stadlhuber 9fc5a0ff98 Update comment in .env.example for proxy settings 
This commit modifies the comment in .env.example to specify that proxy settings are for Playwright. This clarification aims to provide users a more clear context about when and why these proxy settings are used.
 2024-05-24 17:45:59 +02:00
Jakob Stadlhuber b001aded46 Add proxy and media blocking configurations 
Updated environment variables and application settings to include proxy configurations and media blocking option. The proxy settings allow users to use a proxy service, while the media blocking is an optional feature that can help save bandwidth. Changes have been made in the .env.example, docker-compose.yaml, and main.py files.
 2024-05-24 17:41:34 +02:00