3537 Commits

Author SHA1 Message Date
Gergő Móricz
05c29e8299 js-sdk: bump 2025-02-28 12:40:39 +01:00
Gergő Móricz
856ec37d4c fix(ci/js-sdk): properly build SDK before publishing 2025-02-28 12:34:53 +01:00
Gergő Móricz
904e69bfbc
feat(supabase): add read replica routing (#1274) 2025-02-28 09:52:26 +01:00
Nicolas
39b61132bf Nick: fixed js sdk 2025-02-27 20:39:53 -03:00
Gergő Móricz
44bf59229a fix(acuc): cache for 1 hour 2025-02-27 21:36:33 +01:00
Nicolas
b72e21a697
Nick: batch billing (#1264) 2025-02-27 20:18:03 +01:00
Nicolas
289e351c14
(feat/deep-research-alpha) Added Max Urls, Sources and Fixes (#1271)
* Nick: fixes

* Nick:

* Update deep-research-status.ts
2025-02-27 13:24:40 -03:00
Gergő Móricz
1d3757b391 bump map to 30k 2025-02-27 12:44:23 +01:00
Benjy
78334e4ef4
feat(self-host/ai): pass in the ollama envs into docker compose (#1269) 2025-02-27 11:56:31 +01:00
Grass Huang
7bf04d409a
fix(scraper): improve charset detection regex to accurately parse meta tags (#1265) 2025-02-26 17:31:06 +01:00
ceewaigithub
75ac980fe4 Add groq_web_crawler example and dependencies 2025-02-26 22:24:38 +08:00
Nicolas
bf1a79588e Merge branch 'feat/fire-index' 2025-02-25 21:01:30 -03:00
Nicolas
31df234127 Update log_job.ts 2025-02-25 21:01:05 -03:00
Nicolas
ec90aaffd6 Update log_job.ts 2025-02-25 21:01:00 -03:00
Nicolas
0f0520391a
Update log_job.ts (#1263) 2025-02-25 19:35:57 -03:00
Nicolas
59d09f5c45 Update log_job.ts 2025-02-25 19:32:16 -03:00
Gergő Móricz
115b6b61c4 add initial codeowners 2025-02-25 14:28:09 +01:00
Gergő Móricz
51bc7757ef
fix(self-host/compose): pass SearXNG params 2025-02-25 11:19:27 +01:00
Eric Ciarla
2da6d7bdb4
Merge pull request #1257 from aparupganguly/feature/claude3.7-crawler
Implemented claude 3.7 web crawler
2025-02-24 15:46:52 -05:00
Gergő Móricz
8c42b08b7e
feat(v1/crawl-status-ws): update behavior to ignore errors like regular crawl-status (#1234) 2025-02-24 21:44:29 +01:00
Aparup Ganguly
6508afc624 Implemented claude 3.7 2025-02-25 01:54:25 +05:30
Gergő Móricz
15489be542
feat(self-host/ai): use any OpenAI-compatible API (#1245) 2025-02-23 09:07:32 +01:00
Gergő Móricz
b88b573909 docker: force host to 0.0.0.0 to fix env precedence issues
Fixes #1225
2025-02-23 08:49:31 +01:00
Nicolas
b24ac0f6b5
Nick: done (#1237) 2025-02-22 20:18:46 -03:00
Nicolas
5ab86b8b43
(fix/token-slicer) Fixes extract token limit issues (#1236)
* Nick: fixes extract token limit errors

* Update llmExtract.ts

* Update llmExtract.ts
2025-02-21 20:44:42 +01:00
Gergő Móricz
76e1f29ae8
Update Dockerfile (#1231) (#1232)
* Update Dockerfile (#1231)

* Dockerfile: re-add prod-deps stage and fix copies

---------

Co-authored-by: Loris <loris.rion@gmail.com>
2025-02-21 17:44:01 +01:00
Gergő Móricz
bfe6a0ab32 feat(self-host/docker-compose): add option to use Valkey
Fixes #1228
2025-02-21 16:19:37 +01:00
Gergő Móricz
82adf81b15 feat(self-host/docker-compose): allow configuring the internal port of the api 2025-02-21 16:19:37 +01:00
Eric Ciarla
9671e68512
Merge pull request #1229 from aparupganguly/feature/gemini-github-analyzer
Implemented github analyzer
2025-02-21 09:52:42 -05:00
Aparup Ganguly
448b44cdd9 Implemented github analyzer 2025-02-21 20:08:37 +05:30
Nicolas
6c51ef401e Update rate-limiter.ts 2025-02-20 22:31:43 -03:00
Nicolas
25d9bdb1f6
(feat/ai-sdk) Migrate to AI-SDK (#1220)
* Nick: init

* Update llmExtract.ts

* Update llmExtract.ts

* Nick rename

* fix(v1/types): extract json schema validation

* Update url-processor.ts

* feat(ai-sdk): ollama support

* feat(ai-sdk): further ollama support

* Nick: it is broken btw

* feat(ai-sdk): abstract model adapter

* Update pnpm-lock.yaml

* Update analyzeSchemaAndPrompt.ts

* Nick:

* feat(ai-sdk): ollama support

* doc(SELF_HOST): update with embedding param

* Nick:

* Update ranker.ts

* Nick:

* feat(ai-sdk): fixes

* Update llmExtract.ts

* feat: remove zod-to-json-schema

* fix

* Update llmExtract.ts

* use openai

* fixes

---------

Co-authored-by: Gergő Móricz <mo.geryy@gmail.com>
2025-02-20 22:48:58 +01:00
rafaelmmiller
943eb775c6 Update eval_run.py 2025-02-20 17:11:42 -03:00
rafaelmmiller
570dc28da2 bugfix eval run 2025-02-20 16:10:36 -03:00
rafaelmmiller
d211240f58 Update eval-prod.yml 2025-02-20 16:02:45 -03:00
rafaelmmiller
04127cca64 Update eval-prod.yml 2025-02-20 15:59:30 -03:00
Rafael Miller
a13863638a
Merge pull request #1224 from mendableai/feat/eval-run-workflow
Feat/added eval run after deploy workflow
2025-02-20 15:23:57 -03:00
Gergő Móricz
16c305775e
fix(crawl-redis): ignore empty includes/excludes (#1223)
* fix(crawl-redis): ignore empty includes/excludes

* fix(snips/scrape): bump timeouts
2025-02-20 19:06:02 +01:00
rafaelmmiller
d7db58e477 Feat/added eval run after deploy workflow 2025-02-20 15:01:26 -03:00
Gergő Móricz
283a3bfef3
fix(scrapeURL/engines/fetch): discover charset and re-decode (#1221)
* fix(scrapeURL/engines/fetch): discover charset and re-decode

* fix(snips/scrape): allow more time for stealth proxy
2025-02-20 18:56:15 +01:00
Gergő Móricz
e417f83c28
feat(self-host): ollama support (#1219) v1.5.0 2025-02-20 16:59:19 +01:00
Eric Ciarla
192f0031cf
Merge pull request #1206 from aparupganguly/feature/gemini-extractor
gemini extractor Implementation
2025-02-20 09:48:10 -05:00
Gergő Móricz
e84c7325d9 chore: remove dead code
Fixes #1149
Fixes #1150
2025-02-20 15:44:34 +01:00
Gergő Móricz
62d11f21fc self-host: remove old playwright-service 2025-02-20 15:39:19 +01:00
Gergő Móricz
d533718312 feat(self-host): rework docker-compose.yaml 2025-02-20 15:37:19 +01:00
Gergő Móricz
305fbb37e7
temp (#1218) 2025-02-20 15:08:32 +01:00
Gergő Móricz
387cc60668 fix(ci/test-server): clean up old envs 2025-02-20 15:06:37 +01:00
Gergő Móricz
fca349f345
fix(ci/js-sdk): bad sed command 2025-02-20 15:02:56 +01:00
Nicolas
2151ca846c Merge branch 'main' of https://github.com/mendableai/firecrawl 2025-02-20 10:50:32 -03:00
Nicolas
7db2d25efa Nick: 2025-02-20 10:50:22 -03:00