Gergő Móricz
|
85ff0c311e
|
Add worker ID to job attribute
|
2024-08-21 19:21:29 +02:00 |
|
Gergő Móricz
|
3ad9bf7ac0
|
Update GH Actions deployment
|
2024-08-21 19:15:25 +02:00 |
|
Gergő Móricz
|
920702cdde
|
Update builder to handle uploading sourcemaps
|
2024-08-21 19:08:03 +02:00 |
|
Gergő Móricz
|
86942728e3
|
Add metadata for queue-worker and Express
|
2024-08-21 17:58:27 +02:00 |
Nicolas
|
35decb1af2
|
Nick:
|
2024-08-21 12:35:03 -03:00 |
rafaelsideguide
|
52abec41c2
|
fixing delete
|
2024-08-21 10:35:50 -03:00 |
|
Nicolas
|
db8c84ff0f
|
Update requests.http
|
2024-08-21 10:19:37 -03:00 |
|
rafaelsideguide
|
b66553867e
|
reverting delete, fixed express bug on checkCredits
|
2024-08-21 09:28:20 -03:00 |
|
rafaelsideguide
|
138437d616
|
commenting out delete, crashing on fire-engine
|
2024-08-21 08:11:24 -03:00 |
|
rafaelsideguide
|
5e48bec1fd
|
commenting out delete, crashing on fire-engine
|
2024-08-21 08:10:46 -03:00 |
|
Nicolas
|
90b32f16c8
|
Nick: fixes
|
2024-08-20 21:38:11 -03:00 |
|
Nicolas
|
819ad50af3
|
Update fireEngine.ts
|
2024-08-20 21:16:33 -03:00 |
|
Nicolas
|
1b3ad60a2c
|
Reapply "Merge pull request #561 from mendableai/bug/dealing-with-dns-error"
This reverts commit ffe11a5bf7.
|
2024-08-20 19:22:09 -03:00 |
|
Nicolas
|
441628998f
|
Reapply "Merge pull request #561 from mendableai/bug/dealing-with-dns-error"
This reverts commit ffe11a5bf7.
|
2024-08-20 19:16:48 -03:00 |
|
Nicolas
|
ffe11a5bf7
|
Revert "Merge pull request #561 from mendableai/bug/dealing-with-dns-error"
This reverts commit 2030ec6031, reversing
changes made to f494d2b707.
|
2024-08-20 18:16:11 -03:00 |
|
Nicolas
|
2030ec6031
|
Merge pull request #561 from mendableai/bug/dealing-with-dns-error
[Bug] Added a way for dealing with DNS without IP resolution
|
2024-08-20 16:59:09 -03:00 |
|
rafaelsideguide
|
f98be7d94e
|
Update fireEngine.ts
|
2024-08-20 16:53:01 -03:00 |
|
Gergő Móricz
|
f494d2b707
|
prioritize search lower
|
2024-08-20 21:18:31 +02:00 |
|
Nicolas
|
59eb552df5
|
Merge pull request #560 from mendableai/feat/cancel-fire-engine
[Feat] Added check job and cancel to fire-engine requests
|
2024-08-20 15:43:31 -03:00 |
|
rafaelsideguide
|
1f27182a13
|
added try catch
|
2024-08-20 15:42:39 -03:00 |
|
Gergő Móricz
|
efb91f9ca5
|
fix(search): delete jobs after done
|
2024-08-20 20:29:08 +02:00 |
|
Nicolas
|
2b221fd4a9
|
Merge pull request #559 from mendableai/nsc/Nick-set-the-crawl-limit-to-the-remaining-credits
Set the crawl limit to the remaining credits
|
2024-08-20 14:38:10 -03:00 |
|
rafaelsideguide
|
e326249a57
|
added check job and cancel to fire-engine requests
|
2024-08-20 14:26:42 -03:00 |
|
Nicolas
|
5abd26a267
|
Nick: set the crawl limit to the remaining credits
|
2024-08-20 14:16:54 -03:00 |
|
rafaelsideguide
|
e1c9cbf709
|
bug fixed. crawl should not stop if sitemap url is invalid
|
2024-08-20 09:11:58 -03:00 |
Thomas Kosmas
|
0dce57832d
|
Merge pull request #555 from mendableai/feat/beta-customers
added variables to beta customers
|
2024-08-20 02:20:23 +03:00 |
|
rafaelsideguide
|
ecd472356b
|
added variables to beta customers
|
2024-08-19 16:41:54 -03:00 |
|
Nicolas
|
5a44191344
|
Merge pull request #554 from mendableai/nsc/check-team-credits-limit
Check team credits based on the crawl limit
|
2024-08-19 11:54:47 -03:00 |
|
Nicolas
|
8e4ca86463
|
Update crawl.ts
|
2024-08-19 11:02:24 -03:00 |
|
Nicolas
|
36b35dbc67
|
Update crawl.ts
|
2024-08-19 11:01:26 -03:00 |
|
rafaelsideguide
|
4ffc60596a
|
Update queue-worker.ts
|
2024-08-19 09:29:23 -03:00 |
|
rafaelsideguide
|
b8170aaa47
|
Update blocklist.ts
|
2024-08-19 08:51:48 -03:00 |
|
Nicolas
|
3fe82b4f12
|
Update queue-worker.ts
|
2024-08-17 03:09:31 -04:00 |
|
Nicolas
|
f797380112
|
Nick:
|
2024-08-16 22:17:38 -04:00 |
|
Nicolas
|
47123be783
|
Nick: weird activity block
|
2024-08-16 22:01:56 -04:00 |
|
Gergő Móricz
|
c281fe62c0
|
fix(crawl): propagate db fix to preview endpoint
|
2024-08-16 23:43:54 +02:00 |
|
Gergő Móricz
|
e6738abf96
|
fix(crawl-status): retrieve from DB in bulk
|
2024-08-16 23:39:39 +02:00 |
|
Nicolas
|
78ca94251c
|
Merge pull request #480 from mendableai/nsc/hyper-v81
Reduce metrics ingestion w/ HyperDX v0.8.1
|
2024-08-16 14:34:14 -04:00 |
|
Gergő Móricz
|
fd6432e7fd
|
fix(queue-worker): correct job success
|
2024-08-16 19:16:08 +02:00 |
|
Gergő Móricz
|
6e54942265
|
fix(queue-worker): add cancelled to crawl log
|
2024-08-16 19:11:53 +02:00 |
|
Gergő Móricz
|
d0a8382a5b
|
fix(queue-worker): crawl finishing race condition
|
2024-08-16 18:48:52 +02:00 |
|
Gergő Móricz
|
6bd52e63bf
|
fix(queue-worker): fix linksOnPage undefined error
|
2024-08-16 18:42:24 +02:00 |
|
Gergő Móricz
|
5a6570cba2
|
fix(webhooks): call back with parent crawl ID
|
2024-08-16 17:42:42 +02:00 |
|
Nicolas
|
ec361609d2
|
Nick: added growth-2x plan
|
2024-08-15 18:37:19 -04:00 |
|
Nicolas
|
32c6b1f136
|
Nick: remove active job alerts
|
2024-08-15 14:50:30 -04:00 |
|
Gergő Móricz
|
0c14366720
|
fix: add checkandupdateurl to crawlPreview
|
2024-08-15 20:30:25 +02:00 |
|
Nicolas
|
81b2479db3
|
Merge pull request #459 from mendableai/feat/queue-scrapes
feat: Move scraper to queue
|
2024-08-15 14:19:55 -04:00 |
|
Gergő Móricz
|
fc08ff450d
|
search port
|
2024-08-15 20:10:59 +02:00 |
|
Nicolas
|
86326f34e9
|
Update single_url.test.ts
|
2024-08-15 13:48:42 -04:00 |
|
Gergő Móricz
|
129a882bcc
|
fix(scrape): give scrapes their real job id
|
2024-08-15 19:29:47 +02:00 |