yujunjun/firecrawl
1
0
Fork 0
mirror of https://github.com/mendableai/firecrawl.git synced 2025-12-26 06:35:52 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,222 Commits 1,045 Branches 31 Tags
Commit Graph

2532 Commits

Author SHA1 Message Date
Gergő Móricz
b9dde3fc3d temp: move more to main instance 2025-03-29 18:18:55 +01:00
Gergő Móricz
4f0510e71d temp: switch over crawl fetches to main instance 2025-03-29 18:05:50 +01:00
Gergő Móricz
f0e0d3e2e3 fix(api): crawl origin tracking (FIR-1499) 2025-03-28 12:47:37 +01:00
Gergő Móricz
46048bc94d
feat(scrapeURL): return js returns from f-e (FIR-1535) (#1385) 
* feat(scrapeURL): return js returns from f-e

* feat(js-sdk): handle new results
 2025-03-28 12:42:25 +01:00
Gergő Móricz
e799cf206b increase manual rl 2025-03-26 20:13:32 +01:00
Gergő Móricz
2b39788d73 manual rl 2025-03-26 20:00:25 +01:00
Gergő Móricz
42236ef0f0 feat(admin/check-fire-engine): better logging 2025-03-25 20:51:40 +01:00
Gergő Móricz
e8f27bef08 feat(rate-limiter): manual_etier2c 2025-03-25 16:24:01 +01:00
Nicolas
555dab50e7 Nick: bump 2025-03-24 20:17:22 +04:00
Nicolas
a50dc106ef
(feat/deep-research) Deep Research Alpha v1 - Structured Outputs + Customizability (#1365) 
* Nick:

* Nick: structured output support

* Nick: support for zod and pydantic
 2025-03-24 20:13:52 +04:00
Gergő Móricz
5e35782b5d update acuc 2025-03-20 17:47:29 +01:00
Nicolas
867e545511 Update rate-limiter.ts 2025-03-19 23:25:05 -04:00
Nicolas
3e0d3db90e Update rate-limiter.ts 2025-03-19 23:20:18 -04:00
Gergő Móricz
0bdaa97bfb acuc fix 2025-03-19 21:10:52 +01:00
Gergő Móricz
6dc5b1cd64 feat(auth): acuc update 2025-03-19 17:39:59 +01:00
Nicolas
723c16499d Merge branch 'nsc/deep-research-prompts' 2025-03-18 15:53:02 -04:00
Nicolas
4fc5e6f6ca Nick: added analysis prompt to the sdks 2025-03-18 15:52:55 -04:00
Nicolas
e97a279efe
Nick: let user format the analysis (#1351) 2025-03-18 15:51:40 -04:00
Gergő Móricz
d0b468ee7b
feat(scrape/actions/click): add all parameter (FIR-1443) (#1342) 
* feat(scrape/actions/click): add all parameter

* bump sdk
 2025-03-17 20:47:17 +01:00
Rafael Miller
6d3c639f58
added 403s to sdk error handlers (#1357) 2025-03-17 18:06:29 +01:00
Gergő Móricz
d12feaea52 fix(crawl): allow execution time longer than 24h 2025-03-17 18:04:23 +01:00
Nicolas
010c8750d4 Nick: let user format the analysis 2025-03-16 23:14:44 -04:00
Nicolas
7e7b7e10fe Nick: fixes py sdk 2025-03-16 23:02:16 -04:00
Nicolas
c514d14176 Merge branch 'nsc/feat/extract-no-urls' 2025-03-16 22:56:07 -04:00
Nicolas
6d250360c2 Nick: bump 2025-03-16 22:45:16 -04:00
Nicolas
20c93db43f
(feat/extract) URLs can now be optional in /extract (#1346) 
* Nick: urls optional on extract

* Update index.ts
 2025-03-16 22:29:25 -04:00
Nicolas
0fb9c1f32e Update index.ts 2025-03-16 22:28:47 -04:00
Gergő Móricz
200de9e7e7
feat(scrape): add warning to document if it was concurrency limited (#1348) 
* feat(scrape): add warning to document if it was concurrency limited

* phrasing + test fix
 2025-03-16 19:57:27 +01:00
Gergő Móricz
670ca84ae9
fix(v1/checkCredits): snap crawl limit to remaining credits if over without erroring out (#1350) 2025-03-16 19:57:08 +01:00
Gergő Móricz
87ad53e727 fix(api/tests/snips/billing): bump timeout 2025-03-16 16:37:11 +01:00
Eric Ciarla
180770f1a3
init and final (#1349) 2025-03-16 16:16:35 +01:00
Gergő Móricz
bad822421f fix(v1/scrape): make log show up in queries 2025-03-16 16:04:25 +01:00
Gergő Móricz
611c2d9c83 feat(v1/scrape): add further logging to document scrape bugs better 2025-03-16 16:02:53 +01:00
Nicolas
f1206e4870 Nick: urls optional on extract 2025-03-15 22:50:19 -04:00
Gergő Móricz
ca93ba6c6d
fix(js-sdk/crawl,batch-scrape): retry status call if it returns an error up to 3 times (#1343) 2025-03-15 12:42:53 +01:00
Gergő Móricz
c3ebfafba7
fix(llmExtract): remove unsupported JSON schema properties (#1335) 2025-03-13 20:10:39 +01:00
Gergő Móricz
387dd3aa38 fix(tests/snips/billing): don't wait 40s for nothing when self hosted 2025-03-13 20:07:52 +01:00
Gergő Móricz
c7ae50d2d0
fix(crawler): sitemaps poisoning crawls with unrelated links (#1334) 2025-03-13 19:31:05 +01:00
Dominik Farr
da6b7505f3 Meaningful log message for high resource usage errors 2025-03-13 11:08:04 -07:00
Nicolas
c6cad942ab Nick: errors -> warn 2025-03-13 13:07:51 -04:00
Nicolas
7ec278a908 Nick: fixes 2025-03-13 13:05:12 -04:00
Nicolas
134de67a3b
(fix/map) Map failed to filter by path if indexed (#1333) 
* Nick:

* Update map.ts

* Update map.ts
 2025-03-13 12:48:56 -04:00
Gergő Móricz
f87e11712c fix: don't log bull secret 2025-03-13 17:30:37 +01:00
Loris
c53df8650a
Update searxng.ts (#1319) 
remove default categories. otherwise we cannot use engines
 2025-03-13 17:21:44 +01:00
Nicolas
d54af97754 Update rate-limiter.ts 2025-03-12 21:57:50 -04:00
Nicolas
b87b0d9b7f Merge branch 'nsc/concurrent-browsers' 2025-03-12 20:21:26 -04:00
Nicolas
e886189394 Update search.ts 2025-03-12 20:21:19 -04:00
Nicolas
1e6b484add
(feat/pricing) Concurrent Browsers - Improve rate limits (#1331) 
* Update rate-limiter.ts

* Update rate-limiter.ts
 2025-03-12 15:23:09 -04:00
Nicolas
d438b235f6 Update rate-limiter.ts 2025-03-12 15:22:28 -04:00
Gergő Móricz
71b6b83ec2
tally rework api switchover (#1328) 
* tally rework api switchover

* fix and send logs

* temp: force main instance while RPCs propagate

* Revert "temp: force main instance while RPCs propagate"

This reverts commit 4c93379cfa64efd60eb4767dd8eced1bdd302531.
 2025-03-12 20:10:33 +01:00