1453 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Nicolas	7f553fea70	Merge branch 'v1-webscraper' of https://github.com/mendableai/firecrawl into v1-webscraper	2024-08-26 18:29:06 -03:00
Nicolas	59dcc70d34	Delete temp-37564.rdb	2024-08-26 18:29:00 -03:00
rafaelsideguide	72454de18d	Merge branch 'v1-webscraper' of https://github.com/mendableai/firecrawl into v1-webscraper	2024-08-26 18:21:54 -03:00
rafaelsideguide	04556ded40	tests: e2e for crawl and crawl status	2024-08-26 18:21:52 -03:00
Nicolas	8c37ea6d96	Merge branch 'v1-webscraper' of https://github.com/mendableai/firecrawl into v1-webscraper	2024-08-26 18:17:17 -03:00
Nicolas	f277a0e2bb	Update package.json	2024-08-26 18:17:09 -03:00
rafaelsideguide	f2f6f78dcf	fix(url validation): sub paths	2024-08-26 18:12:03 -03:00
Nicolas	0bbb8bb24e	Nick:	2024-08-26 17:17:12 -03:00
Nicolas	98a770f38f	Nick: rm wip	2024-08-26 17:16:44 -03:00
Nicolas	6f68678b5d	Nick:	2024-08-26 17:13:00 -03:00
Nicolas	b0bd71a3a9	Merge branch 'main' into v1-webscraper	2024-08-26 16:58:00 -03:00
Nicolas	2d78c20d68	Nick:	2024-08-26 16:56:27 -03:00
Nicolas	fa7dc5b10b	Update rate-limiter.ts	2024-08-26 16:33:34 -03:00
Nicolas	4d0acc9722	Merge branch 'main' into v1-webscraper	2024-08-26 16:22:05 -03:00
Nicolas	5606fe5870	Nick:	2024-08-26 16:05:11 -03:00
rafaelsideguide	1baba3ce0a	fix(go-sdk): submodules	2024-08-26 11:11:34 -03:00
Gergo Moricz	d591e0f51c	block corterix.com for performance issues	2024-08-25 20:06:12 +02:00
rafaelsideguide	6f9a2687ae	fixed turndown bug	2024-08-25 15:04:32 -03:00
Gergo Moricz	96e91ab9ec	convert webhook call to v1	2024-08-25 14:05:46 +02:00
Nicolas	4e196a9146	Delete autoscale.yml	2024-08-25 01:48:51 -03:00
Nicolas	1f99bfd3c8	Update queue.ts	2024-08-23 22:47:12 -03:00
Nicolas	b80277d4de	Update queue.ts	2024-08-23 22:46:44 -03:00
Nicolas	d87b62fed9	Nick:	2024-08-23 22:33:17 -03:00
Nicolas	b9e06e27f4	Update queue.ts	2024-08-23 22:17:27 -03:00
Nicolas	8e78511ed4	Update queue.ts	2024-08-23 22:15:47 -03:00
Nicolas	28d7a637c2	Update queue.ts	2024-08-23 22:07:49 -03:00
Nicolas	b23bf2eef9	Update autoscale.yml	2024-08-23 20:14:35 -03:00
Nicolas	0dc592b3e4	Update autoscale.yml	2024-08-23 20:11:30 -03:00
Nicolas	173f4ee1bf	Nick: chrome cdp main | simple autoscaler	2024-08-23 20:09:59 -03:00
Eric Ciarla	732e6af8b9	Add internal link opportunities example	2024-08-23 15:49:41 -04:00
Gergő Móricz	064ebfc54d	fix websocket	2024-08-23 19:55:41 +02:00
Gergő Móricz	05c250d3b8	Merge branch 'main' into v1-webscraper	2024-08-23 19:38:57 +02:00
Gergő Móricz	2ab0dd2e15	fix(scrape): add further llm extraction catch	2024-08-23 19:20:17 +02:00
Gergő Móricz	1054a1397b	Merge branch 'main' into v1-webscraper	2024-08-23 19:14:49 +02:00
Nicolas	3d53f4e213	Nick: unblocking pin	2024-08-23 13:56:05 -03:00
Gergő Móricz	5ef3926d2a	fix(scrape,search): handle failed jobs	2024-08-23 18:47:56 +02:00
Gergő Móricz	866e71910c	further fixes	2024-08-23 18:27:00 +02:00
Gergő Móricz	eea530e0ad	feat(v1): update for sentry	2024-08-23 17:29:42 +02:00
Gergő Móricz	e7f267b6fe	Merge branch 'main' into v1-webscraper	2024-08-23 17:21:54 +02:00
Gergő Móricz	52a05b8c6e	rename "dragonfly" to "redis"	2024-08-23 17:05:59 +02:00
Gergő Móricz	64e9be0cd4	feat(redis): use bitnami image	2024-08-22 23:38:04 +02:00
Gergő Móricz	8d9ff90bcb	feat(fire-engine): propagate sentry trace	2024-08-22 23:38:04 +02:00
rafaelsideguide	74ea820bc6	fix: url and check for metadata	2024-08-22 18:32:19 -03:00
Nicolas	1f0abacadf	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-08-22 18:30:54 -03:00
Nicolas	1f779e261a	Update rate-limiter.ts	2024-08-22 18:30:45 -03:00
Gergő Móricz	8e3c2b2855	fix(crawler): verify URL	2024-08-22 23:30:19 +02:00
Gergő Móricz	e690a6fda7	fix: remove QueueEvents	2024-08-22 22:38:39 +02:00
Gergő Móricz	76c8e9f996	fix	2024-08-22 22:24:24 +02:00
Gergő Móricz	ad82175fb8	fix(scrape): poll	2024-08-22 22:12:02 +02:00
rafaelsideguide	5f60a55967	workflow and npm now running v1 tests	2024-08-22 15:28:49 -03:00