Commit Graph

32 Commits

Author SHA1 Message Date
Nicolas 74565a9da3
Merge pull request #639 from yekkhan/main
feat: kubernetes example optimization
2024-09-19 18:59:42 -04:00
Eric Ciarla 2619522fe7
Merge branch 'main' into o1-crawler 2024-09-16 16:06:15 -04:00
Eric Ciarla e58144798f Update o1_web_crawler.py 2024-09-16 16:04:32 -04:00
Eric Ciarla 8c05aed6e9 Finishing o1 crawler example 2024-09-16 11:30:25 -04:00
Eric Ciarla 3900603a28 Almost done 2024-09-16 11:18:57 -04:00
Eric Ciarla 2ee7d1d0e4 init 2024-09-13 15:08:23 -04:00
y5n 4278fae51e
Update README.md 2024-09-09 10:55:31 +08:00
y5n 1ea9131e63 feat: Update redis deployment to run redis with password if REDIS_PASSWORD is configured 2024-09-07 16:00:32 +08:00
Eric Ciarla e89e6d89a8
Merge pull request #637 from mendableai/ETL-unicorn-example-app
Add Docs to API Spec Example
2024-09-06 15:28:41 -04:00
Eric Ciarla 2d245a35f2 Delete combined_api_spec.json 2024-09-06 15:27:58 -04:00
Eric Ciarla 2044e71fcf Docs to API Spec 2024-09-06 15:26:33 -04:00
Nicolas d872bf0c4c Merge branch 'main' into v1-webscraper 2024-08-28 12:42:23 -03:00
Eric Ciarla 51d1a2e5f2 Add new example Web Scraping and Extraction with Firecrawl and Claude 2024-08-28 09:35:43 -04:00
Nicolas 4d0acc9722 Merge branch 'main' into v1-webscraper 2024-08-26 16:22:05 -03:00
Eric Ciarla 0566e54d85 init 2024-08-26 15:16:50 -04:00
Eric Ciarla 732e6af8b9 Add internal link opportunities example 2024-08-23 15:49:41 -04:00
rafaelsideguide 7a61325500 map + search + scrape markdown bug 2024-08-16 17:57:11 -03:00
Jakob Stadlhuber 2dc7be3869 Remove liveness and readiness probes from worker.yaml
This commit removes the liveness and readiness probes configuration from the Kubernetes worker manifest. Additionally, a Service definition for the worker application has been removed. These changes might be necessary to update the deployment strategy or simplify the configuration.
2024-07-24 19:38:54 +02:00
Jakob Stadlhuber d68f349109 Update Kubernetes YAMLs and add worker service
Refactored container configurations in worker, api, and playwright-service YAMLs to streamline syntax and add missing fields. Added a service definition for the worker component and included a new environment variable in the configmap for rate-limiting. These changes enhance configuration clarity and ensure proper resource definitions.
2024-07-24 19:31:37 +02:00
Jakob Stadlhuber f26bda2477 Update Docker build paths in Kubernetes setup README
Corrected relative paths for Docker build commands to ensure the appropriate directories are targeted. This fix is crucial for successful image builds and deployment consistency in the Kubernetes cluster setup.
2024-07-24 19:06:19 +02:00
Jakob Stadlhuber 895e80caa4 Add liveness and readiness probes to Kubernetes configs
Introduced liveness and readiness probes for the Playwright service, API, and worker components. This ensures that Kubernetes can better manage the health and availability of these services by periodically checking their endpoints. This enhancement will improve the robustness and reliability of the deployed applications.
2024-07-24 19:00:23 +02:00
Jakob Stadlhuber 497aa5d25e Update Kubernetes configs for playwright-service, api, and worker
Added new ConfigMap for playwright-service and adjusted existing references.
Applied imagePullPolicy: Always to ensure all images are updated promptly.
Updated README to include --no-cache for Docker build instructions.
2024-07-24 17:55:45 +02:00
Eric Ciarla d80046d17c Gemini caching example 2024-06-26 09:48:15 -04:00
Eric Ciarla 22541362d7 Reduce web example bloat 2024-06-22 08:40:26 -04:00
Eric Ciarla 8e39083d8c Update examples section 2024-06-21 15:40:46 -04:00
Eric Ciarla 06b0d01fd4 Update examples 2024-06-11 12:23:36 -04:00
Eric Ciarla 00c23855b1 Update examples 2024-06-11 11:46:35 -04:00
Jakob Stadlhuber 3cca2e3bfe Update Docker image names in README
The Docker image names in README.md in the k8n example scripts were updated to ensure they accurately reflect the intended container images. This change primarily involves switching the names of the API and Playwright service Docker images.
2024-06-04 21:09:43 +02:00
Jakob Stadlhuber 07246d0e1c Update README file in k8n directory
Removed a redundant list item and updated section title for deleting Firecrawl. The "Delete Firecrawl" section is now more concise and does not refer specifically to an environment, making it more generally applicable.
2024-06-04 20:59:04 +02:00
Jakob Stadlhuber 078d4c8d41 Add Kubernetes configuration for Firecrawl deployment
Added new files for setting up Firecrawl on a Kubernetes Cluster. The files include Kubernetes manifests for deploying API, worker, playwright service, and Redis with associated ConfigMap and Secret associated resources. Also, updated the self-host documentation to include instructions for Kubernetes deployment.
2024-06-04 20:52:08 +02:00
Eric Ciarla 3ea801d9dd Commit Roast My Website 2024-06-02 20:40:19 -07:00
Eric Ciarla ea04fe2e3f Add Roast My Website Example 2024-06-02 20:38:05 -07:00