Commit Graph

1743 Commits

Author SHA1 Message Date
Nicolas 9da3432596 Update map.ts 2024-09-16 12:03:14 -04:00
Nicolas af4804e13b Merge branch 'main' into nsc/map-pagination 2024-09-16 11:04:44 -04:00
Nicolas 98de36feb2
Merge pull request #664 from mendableai/nsc/manual-rate-limit
Manual Rate Limiter for select team ids
2024-09-13 18:12:41 -04:00
Nicolas 030ecab6ee Update rate-limiter.ts 2024-09-13 18:09:59 -04:00
Gergo Moricz 000a316cc3 fix(fire-engine): poll more frequently 2024-09-13 16:41:27 +02:00
Gergo Moricz f7c4cee404 fix(queue-worker): don't send LLM extract hallucination error to Sentry 2024-09-13 14:02:49 +02:00
Gergő Móricz 0d1b46d476 fix(js-sdk): improve error logging 2024-09-12 21:30:17 +02:00
Nicolas e6ac90c1a0 Update package.json 2024-09-12 14:01:47 -04:00
Gergő Móricz d30356a22c fix(js-sdk): infer keyword collision 2024-09-12 19:57:33 +02:00
Gergő Móricz d497284b40 feat(api/queue): auto-remove completed jobs after 25 hours 2024-09-12 19:47:22 +02:00
Nicolas 620b02f9ca Nick: 2024-09-12 12:51:14 -04:00
Gergő Móricz a2903e75cf feat(js-sdk): type-safe LLM extract 2024-09-12 18:48:19 +02:00
Nicolas eec22a56d3 Nick: self host issue template 2024-09-12 11:43:40 -04:00
Nicolas ee38273ff9
Merge pull request #653 from mendableai/mog/fix-status-job-get
fix(v0/crawl-status): don't crash on big crawls when requesting jobs from supa
2024-09-12 11:39:42 -04:00
Nicolas e1171ade1b Merge branch 'main' into mog/fix-status-job-get 2024-09-12 11:35:36 -04:00
Nicolas 503c8b3efa Update package-lock.json 2024-09-12 11:35:26 -04:00
Nicolas e19dc68718 Merge branch 'main' of https://github.com/mendableai/firecrawl 2024-09-11 14:59:40 -04:00
Nicolas 99c1af0a9f Update package.json 2024-09-11 14:59:36 -04:00
Gergő Móricz 5adfd74cc5 feat(js-sdk/test): add API_URL env var 2024-09-11 20:32:34 +02:00
Gergő Móricz ad70c30be5 fix(js-sdk): check at bad if 2024-09-11 20:31:58 +02:00
Gergő Móricz 6e1cf2f40d feat(js-sdk): fixes, update tests 2024-09-11 20:15:43 +02:00
Nicolas 5791d3a0e5 Merge branch 'main' of https://github.com/mendableai/firecrawl 2024-09-11 14:03:36 -04:00
Nicolas 4cd1065ae2 Update rate-limiter.ts 2024-09-11 14:03:34 -04:00
Gergő Móricz c92065ea24
Merge pull request #611 from MonsterDeveloper/fix-attw
build(js-sdk): simplify build process
2024-09-11 19:45:49 +02:00
Gergő Móricz 51da038405
Merge branch 'main' into fix-attw 2024-09-11 19:45:13 +02:00
Gergő Móricz 75f4bcd2ad
Merge pull request #614 from MonsterDeveloper/dependencies
perf(js-sdk): move `dotenv` and `uuid` to `devDependencies`, fix `zod` import
2024-09-11 19:34:50 +02:00
Andrei Bobkov f6fc71b46a
fix(js-sdk): bring back cjs exports 2024-09-11 17:53:17 +03:00
Gergő Móricz 97ffabff3a fix(v1): converting bad docs always gives null 2024-09-10 21:21:20 +02:00
Gergő Móricz 83a165db0f fix(v0/scrape): ensure url is string 2024-09-10 21:18:53 +02:00
Gergő Móricz ad1a6fbc74 fix(v1/map): handle invalid URLs gracefully 2024-09-10 19:41:01 +02:00
Gergő Móricz e19f7a102e feat(js-sdk): paginate next on checkCrawlStatus + better types for CSR 2024-09-10 19:29:38 +02:00
Gergő Móricz 4ebc35c9dd fix(crawl-status): add success: true 2024-09-10 18:59:09 +02:00
rafaelsideguide f855ad3436 bumping py-sdk version 2024-09-10 10:29:44 -03:00
Nicolas 93b7b313d6
Merge pull request #654 from mendableai/fix/issue-644
fix(py-sdk): removed asyncio package
2024-09-10 10:27:24 -03:00
rafaelsideguide ee8a54213c fix(py-sdk): removed asyncio package
tested websocket with example.py without asyncio and it works with no problem.
2024-09-10 10:25:27 -03:00
rafaelsideguide 1074e976cd Merge branch 'main' of https://github.com/mendableai/firecrawl 2024-09-10 09:09:43 -03:00
rafaelsideguide 45237a29dd updated js-sdk examples 2024-09-10 09:09:39 -03:00
Nicolas a7406031fa Merge branch 'main' of https://github.com/mendableai/firecrawl 2024-09-10 06:53:30 -03:00
Nicolas a17e1cac92 Rate bump 2024-09-10 06:53:24 -03:00
Gergo Moricz b4dbf75537 fix(v1): check if url is string in blocklistMiddleware
Fixes FIRECRAWL-SCRAPER-JS-9Z
2024-09-10 10:25:14 +02:00
Gergo Moricz 26f2095de6 fix(v1): proper Invalid URL handling 2024-09-10 09:24:23 +02:00
Gergo Moricz f8fbc71f91 fix(supabase-jobs): do not use RPCs
RPCs are more failure-prone for this use case than regular queries are.
2024-09-10 09:20:18 +02:00
Gergo Moricz a6bcf7b438 fix(v0/crawl-status): don't crash on big crawls when requesting jobs from supabase 2024-09-10 08:53:40 +02:00
Nicolas 2cbc4c59ce
Merge pull request #649 from mendableai/nsc/chrome-cdp-for-params
Better engine for wait + other params
2024-09-09 21:12:46 -03:00
Nicolas 17e419a7fb Nick: 2024-09-09 21:06:23 -03:00
Nicolas 31e973d915
Merge pull request #648 from mendableai/nsc/fix-json-error-handler
Fixed 500 errors when JSON is badly formatted
2024-09-09 12:28:18 -03:00
Nicolas ca9a781eb7 Update index.ts 2024-09-09 12:27:55 -03:00
Nicolas 22a5e85899 Update index.ts 2024-09-09 12:26:55 -03:00
Nicolas 60a15d00eb Update types.ts 2024-09-08 16:39:12 -03:00
Nicolas 84fb00ec3f
Merge pull request #640 from mendableai/nsc/improved-billing-notifications
Improved team credits check and billing notifications
2024-09-08 13:07:45 -03:00