3537 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Nicolas	9d187bf3bb	Merge pull request #529 from mendableai/nsc/redlock-cache-auth ... Redlock cache in auth	2024-08-12 15:14:07 -04:00
Nicolas	74a5125185	Nick: removed redlock	2024-08-12 15:07:30 -04:00
Nicolas	0bd1a820ee	Update auth.ts	2024-08-12 13:42:09 -04:00
Nicolas	25a899eae3	Nick: redlock cache in auth	2024-08-12 13:37:47 -04:00
rafaelsideguide	bbed6ef23d	added validation on every USE_DB_AUTHENTICATION call	2024-08-12 14:20:41 -03:00
Rafael Miller	36e4b2cf49	Update .env.example	2024-08-12 10:37:00 -03:00
Quan Ming	fe179d0cb1	Update redis troubleshooting in self host guide	2024-08-10 12:39:22 +08:00
Quan Ming	a96ad4b0e2	Update redis url to use comment	2024-08-10 12:33:26 +08:00
Kevin Swiber	ba2af74adf	Ensuring USE_DB_AUTHENTICATION is true in single URL scraper.	2024-08-09 15:29:18 -07:00
Nicolas	e28c415cf4	Nick:	2024-08-09 14:07:46 -04:00
Gergo Moricz	5a778f2c22	fix(js-sdk): add type metadata to exports	2024-08-09 20:05:36 +02:00
Rafael Miller	6a78f6fe78	Merge pull request #497 from KentHsu/feat/add-go-sdk ... [Feat] Add Go SDK implementation	2024-08-09 14:58:20 -03:00
rafaelsideguide	0591000b64	bugfix includes excludes	2024-08-09 14:30:41 -03:00
Kent (Chia-Hao), Hsu	1fda882983	Merge branch 'mendableai:main' into feat/add-go-sdk	2024-08-10 00:46:15 +08:00
Quan Ming	0221872a70	Update redis urls in example .env	2024-08-10 00:40:11 +08:00
rafaelsideguide	b802ea02a1	small improvements ... - wait for getting results on crawl: sometimes crawl takes some a second to save the data on the db and this causes response.data to be empty - added timeout value to test script - increased http client timeout (llm extract was failing on e2e tests) - fixed env path on test script	2024-08-09 11:13:14 -03:00
rafaelsideguide	0b8df5e264	python sdk and tests	2024-08-08 14:25:09 -03:00
Nicolas	bf1d325aa9	Merge pull request #512 from mendableai/nsc/digikey-tls ... Digikey support	2024-08-08 12:34:18 -04:00
Nicolas	f1f5605010	Update website_params.ts	2024-08-08 12:31:58 -04:00
rafaelsideguide	cf9d77d889	typescript fixes	2024-08-08 11:41:13 -03:00
Nicolas	b0abad07da	Merge pull request #496 from tak-s/improve-logging-level ... Improve logs	2024-08-07 22:01:12 -04:00
rafaelsideguide	c16437e933	fixed bunch of types	2024-08-07 17:05:18 -03:00
Gergo Moricz	920b7f2f44	fix(runWebScraper): don't filter empty docs	2024-08-07 21:00:22 +02:00
Gergo Moricz	55ec96c23f	fix(queue-worker): bad job lock extension time	2024-08-07 20:24:16 +02:00
Gergo Moricz	ab7a35c581	fix(queue-worker): log lock extensions	2024-08-07 19:49:48 +02:00
Gergo Moricz	a1c2ee5aa9	fix: always complete job, no try	2024-08-07 19:39:09 +02:00
Gergo Moricz	191dfbd9ca	fix: move to completed in one place	2024-08-07 18:49:58 +02:00
Nicolas	457c082ba1	Nick: fixed tests	2024-08-07 11:08:53 -04:00
Nicolas	8a992b1596	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-08-07 10:40:06 -04:00
Nicolas	b12e1157cc	Nick: v35 bump	2024-08-07 10:40:00 -04:00
Gergő Móricz	5fc7fcb77c	Merge branch 'main' into feat/queue-scrapes	2024-08-07 16:35:44 +02:00
Gergo Moricz	fe9fdb578b	revert bad hotfixes	2024-08-07 16:34:25 +02:00
Gergo Moricz	b7c01dcb9b	fix(webScraperQueue): reduce retries to 2	2024-08-07 16:31:50 +02:00
Gergo Moricz	cdf7bad5b4	fix(runWebScraper): don't move to completed	2024-08-07 15:20:56 +02:00
Gergo Moricz	9df8719efa	fix(queue-worker): raise queue log level to info	2024-08-07 14:56:04 +02:00
Gergo Moricz	7bb922071c	fix(queue-worker): manually renew lock (testing)	2024-08-07 14:35:20 +02:00
Gergo Moricz	8216266d16	fix(scrape_log): display error properly	2024-08-07 14:19:20 +02:00
Gergo Moricz	2e2e80d679	fix(scrape-events): updateScrapeResult fix	2024-08-07 14:17:50 +02:00
Gergo Moricz	b5ec47fd96	fix(runWebScraper): don't fetch next job	2024-08-07 13:53:04 +02:00
Nicolas	44c9a22731	Merge pull request #508 from mendableai/mog/js-sdk-cjs ... fix(js-sdk): build both CJS and EJS versions	2024-08-06 19:46:20 -04:00
Gergo Moricz	020a5efdb7	Revert "Revert "Merge pull request #432 from mendableai/mog/js-sdk-cjs"" ... This reverts commit 5da4472842497dbe4d462609293917e0b2199fc8.	2024-08-07 01:27:26 +02:00
Gergő Móricz	7380d7799f	Merge branch 'main' into mog/js-sdk-cjs	2024-08-07 01:12:36 +02:00
Gergo Moricz	5f7724205f	fix(js-sdk): re-add types	2024-08-07 01:06:21 +02:00
Nicolas	f294d3922c	Nick: revert	2024-08-06 18:44:45 -04:00
Nicolas	5da4472842	Revert "Merge pull request #432 from mendableai/mog/js-sdk-cjs" ... This reverts commit bb90e03dea5dc2f475dbf8dc70425af9b3dfe246, reversing changes made to 3321ca9398b5a129ffc1689ec2a88b2468cf16e9.	2024-08-06 18:41:06 -04:00
Nicolas	a67a5c04c9	Revert "Merge pull request #432 from mendableai/mog/js-sdk-cjs" ... This reverts commit bb90e03dea5dc2f475dbf8dc70425af9b3dfe246, reversing changes made to 3321ca9398b5a129ffc1689ec2a88b2468cf16e9.	2024-08-06 18:02:56 -04:00
Nicolas	bb90e03dea	Merge pull request #432 from mendableai/mog/js-sdk-cjs ... fix(js-sdk): build both CommonJS and ESM versions	2024-08-06 17:38:57 -04:00
rafaelsideguide	3fb2307010	Update index.ts	2024-08-06 17:34:13 -03:00
rafaelsideguide	d599d31e63	wip	2024-08-06 17:33:39 -03:00
rafaelsideguide	6cdf4c68ec	wip: map, crawl, scrape mockups	2024-08-06 15:24:45 -03:00