Jakob Stadlhuber d68f349109 Update Kubernetes YAMLs and add worker service
Refactored container configurations in worker, api, and playwright-service YAMLs to streamline syntax and add missing fields. Added a service definition for the worker component and included a new environment variable in the configmap for rate-limiting. These changes enhance configuration clarity and ensure proper resource definitions.
2024-07-24 19:31:37 +02:00
..
2024-06-21 15:40:46 -04:00
2024-06-21 15:40:46 -04:00

Install Firecrawl on a Kubernetes Cluster (Simple Version)

Before installing

  1. Set secret.yaml and configmap.yaml and do not check in secrets
  2. Build Docker images, and host it in your Docker Registry (replace the target registry with your own)
    1. API (which is also used as a worker image)
      1. docker build --no-cache -t ghcr.io/winkk-dev/firecrawl:latest ../../../apps/api
        docker push ghcr.io/winkk-dev/firecrawl:latest
        
    2. Playwright
      1.    docker build --no-cache -t ghcr.io/winkk-dev/firecrawl-playwright:latest ../../../apps/playwright-service
           docker push ghcr.io/winkk-dev/firecrawl-playwright:latest
        
  3. Replace the image in worker.yaml, api.yaml and playwright-service.yaml

Install

kubectl apply -f configmap.yaml
kubectl apply -f secret.yaml
kubectl apply -f playwright-service.yaml
kubectl apply -f api.yaml
kubectl apply -f worker.yaml
kubectl apply -f redis.yaml

Port Forwarding for Testing

kubectl port-forward svc/api 3002:3002 -n dev

Delete Firecrawl

kubectl delete -f configmap.yaml
kubectl delete -f secret.yaml
kubectl delete -f playwright-service.yaml
kubectl delete -f api.yaml
kubectl delete -f worker.yaml
kubectl delete -f redis.yaml