1978 Commits

Author	SHA1	Message	Date
Nicolas	21bf89b6cc	Update search.ts	2025-01-02 19:57:51 -03:00
Nicolas	22ae1730bd	Update search.ts	2025-01-02 19:57:41 -03:00
Nicolas	a0dbf20c40	Update types.ts	2025-01-02 19:55:28 -03:00
Nicolas	25da20efd2	Nick: e2e	2025-01-02 19:53:54 -03:00
Nicolas	eae393afb5	Nick: fixed js sdk	2025-01-02 19:52:50 -03:00
Nicolas	07a6ba5d91	Nick:	2025-01-02 19:34:37 -03:00
Nicolas	35d7202894	Update search.ts	2025-01-02 19:33:21 -03:00
Nicolas	d2742bec4d	Nick: v1 search	2025-01-02 19:31:03 -03:00
Nicolas	c822e34d37	Nick: fixed extract schema	2025-01-02 14:03:23 -03:00
Nicolas	07f4b714af	Update removeUnwantedElements.ts	2024-12-31 15:23:02 -03:00
Nicolas	0847a6038e	Merge pull request #1014 from mendableai/nsc/extract-url-trace ... /extract URL trace	2024-12-30 19:00:58 -03:00
Gergő Móricz	71a8f7452c	fix(WebScraper/sitemap): await urlsHandler to fix race condition	2024-12-30 16:09:22 +01:00
Nicolas	8ae34a0d31	Nick: rm .xml from isFile	2024-12-30 11:57:01 -03:00
Gergő Móricz	9005757de3	fix(queue-worker): do not follow redirect URLs if they are not allowed by the crawl options	2024-12-30 14:41:31 +01:00
Gergő Móricz	4d1f92f4c8	fix(scrapeURL/fetch): block loopback and link-local IPs	2024-12-29 17:35:14 +01:00
Nicolas	e255301005	Update index.ts	2024-12-27 21:31:29 -03:00
Nicolas	1eca61bffb	Update index.ts	2024-12-27 20:59:18 -03:00
Nicolas	f9d55efba8	Update index.ts	2024-12-27 20:54:26 -03:00
Nicolas	b8d7f9f257	Nick: we are using runpod	2024-12-27 19:59:05 -03:00
Nicolas	5fcf3fa97e	Merge branch 'main' into mog/mineru	2024-12-27 19:53:09 -03:00
Nicolas	a431cafa47	Merge pull request #991 from RutamBhagat/rust-sdk-conditionally-enforce-api-key ... feat(rust-sdk): Make API key optional for self-hosted instances	2024-12-27 19:07:01 -03:00
Nicolas	65cf4cd74e	Merge pull request #1013 from yujunhui/main ... fix: merge mock success data	2024-12-27 19:04:04 -03:00
Nicolas	05d5f84d87	Merge pull request #1018 from mendableai/feat/add-favicon-metadata ... [FIR-37] feat: extract and return favicon URL during scraping	2024-12-27 17:44:03 -03:00
Nicolas	eba5fda9a1	Merge pull request #955 from mendableai/rafa/fix-default-on-schema-llm-extract ... fixed optional+default bug on llm schema	2024-12-27 16:33:04 -03:00
Ademílson F. Tonato	a4cf814f70	feat: return favicon url when scraping	2024-12-27 19:18:53 +00:00
Gergő Móricz	0421f81020	Sitemap fixes (#1010) ... * sitemap fixes iter 1 * feat(sitemap): dedupe improvements --------- Co-authored-by: Nicolas <nicolascamara29@gmail.com>	2024-12-27 19:59:26 +01:00
Nicolas	6851281beb	Update __init__.py	2024-12-27 15:46:00 -03:00
Nicolas	cd08be7f37	Merge pull request #990 from RutamBhagat/python-sdk-conditionally-enforce-api-key ... feat(python-sdk): Make API key optional for self-hosted instances	2024-12-27 15:43:37 -03:00
Nicolas	c5b6495e48	Merge pull request #1015 from mendableai/nsc/improves-sitemap-fetching ... Improves sitemap fetching	2024-12-27 14:41:04 -03:00
Nicolas	e8f0a22ebe	Update v1-openapi.json	2024-12-27 13:59:43 -03:00
Nicolas	f7cfbba651	Merge branch 'main' into pr/1003	2024-12-27 13:59:24 -03:00
Nicolas	1abb544e3e	Update index.test.ts	2024-12-27 13:59:09 -03:00
Gergő Móricz	4772951313	feat(scrapeURL/fire-engine): explicitly delete job after scrape	2024-12-27 16:44:41 +01:00
Gergő Móricz	0b55fb836b	feat(scrapeURL/pdf): switch to MinerU	2024-12-27 16:37:32 +01:00
Nicolas	ece95e97f4	Merge branch 'main' into nsc/extract-url-trace	2024-12-26 21:28:51 -03:00
Gergő Móricz	c543f4f76c	feat(scrapeURL/pdf): update mock Blob implementation to pass TypeScript	2024-12-26 20:31:51 +01:00
Gergő Móricz	f15ef0e758	feat(scrapeURL/fire-engine/chrome-cdp): handle file downloads	2024-12-26 20:29:09 +01:00
Nicolas	4451c4f671	Nick:	2024-12-26 13:51:20 -03:00
Nicolas	bcc18e1c07	Merge branch 'main' into fix-sdk/next-in-when-502	2024-12-26 12:53:10 -03:00
Nicolas	4f65d350a3	Update package.json	2024-12-26 12:52:52 -03:00
Nicolas	4332f18a8f	Nick: making it optional for the user	2024-12-26 12:43:58 -03:00
Nicolas	233f347f5e	Nick: refactor	2024-12-26 12:41:37 -03:00
Nicolas	f467a3ae6c	Nick: init	2024-12-26 12:21:46 -03:00
yujunhui	2f39bdddd9	fix: merge mock success data	2024-12-26 17:56:30 +08:00
Nicolas	c911aad228	Update package.json	2024-12-23 18:48:03 -03:00
Nicolas	b1a5625b22	Revert "Merge pull request #997 from mendableai/feat/sdk-without-ws" ... This reverts commit 53cda5f81c53d3de35925c610ce083923ca09fbe, reversing changes made to 51f79b55efadc53243a8c22d86bb2d08d878d524.	2024-12-23 18:45:51 -03:00
Nicolas	18ceaf10a5	Update .gitignore	2024-12-23 18:42:05 -03:00
Nicolas	0c1c4f2ede	Merge branch 'main' into feat/sdk-without-ws	2024-12-23 18:41:31 -03:00
Nicolas	67c643ad1c	Merge pull request #989 from RutamBhagat/js-sdk-conditionally-enforce-api-key ... feat(js-sdk): Make API key optional for self-hosted instances	2024-12-23 12:56:38 -03:00
RutamBhagat	ca2d3dc6d2	docs(credit-usage-api): add new endpoint documentation for credit usage	2024-12-21 06:24:53 -08:00