Nicolas
05d5f84d87
Merge pull request #1018 from mendableai/feat/add-favicon-metadata
...
[FIR-37] feat: extract and return favicon URL during scraping
2024-12-27 17:44:03 -03:00
Nicolas
eba5fda9a1
Merge pull request #955 from mendableai/rafa/fix-default-on-schema-llm-extract
...
fixed optional+default bug on llm schema
2024-12-27 16:33:04 -03:00
Ademílson F. Tonato
a4cf814f70
feat: return favicon url when scraping
2024-12-27 19:18:53 +00:00
Gergő Móricz
0421f81020
Sitemap fixes ( #1010 )
...
* sitemap fixes iter 1
* feat(sitemap): dedupe improvements
---------
Co-authored-by: Nicolas <nicolascamara29@gmail.com>
2024-12-27 19:59:26 +01:00
Nicolas
6851281beb
Update __init__.py
2024-12-27 15:46:00 -03:00
Nicolas
cd08be7f37
Merge pull request #990 from RutamBhagat/python-sdk-conditionally-enforce-api-key
...
feat(python-sdk): Make API key optional for self-hosted instances
2024-12-27 15:43:37 -03:00
Nicolas
c5b6495e48
Merge pull request #1015 from mendableai/nsc/improves-sitemap-fetching
...
Improves sitemap fetching
2024-12-27 14:41:04 -03:00
Nicolas
e8f0a22ebe
Update v1-openapi.json
2024-12-27 13:59:43 -03:00
Nicolas
f7cfbba651
Merge branch 'main' into pr/1003
2024-12-27 13:59:24 -03:00
Nicolas
1abb544e3e
Update index.test.ts
2024-12-27 13:59:09 -03:00
Gergő Móricz
4772951313
feat(scrapeURL/fire-engine): explicitly delete job after scrape
2024-12-27 16:44:41 +01:00
Gergő Móricz
0b55fb836b
feat(scrapeURL/pdf): switch to MinerU
2024-12-27 16:37:32 +01:00
Nicolas
ece95e97f4
Merge branch 'main' into nsc/extract-url-trace
2024-12-26 21:28:51 -03:00
Gergő Móricz
c543f4f76c
feat(scrapeURL/pdf): update mock Blob implementation to pass TypeScript
2024-12-26 20:31:51 +01:00
Gergő Móricz
f15ef0e758
feat(scrapeURL/fire-engine/chrome-cdp): handle file downloads
2024-12-26 20:29:09 +01:00
Nicolas
4451c4f671
Nick:
2024-12-26 13:51:20 -03:00
Nicolas
bcc18e1c07
Merge branch 'main' into fix-sdk/next-in-when-502
2024-12-26 12:53:10 -03:00
Nicolas
4f65d350a3
Update package.json
2024-12-26 12:52:52 -03:00
Nicolas
4332f18a8f
Nick: making it optional for the user
2024-12-26 12:43:58 -03:00
Nicolas
233f347f5e
Nick: refactor
2024-12-26 12:41:37 -03:00
Nicolas
f467a3ae6c
Nick: init
2024-12-26 12:21:46 -03:00
yujunhui
2f39bdddd9
fix: merge mock success data
2024-12-26 17:56:30 +08:00
Nicolas
c911aad228
Update package.json
2024-12-23 18:48:03 -03:00
Nicolas
b1a5625b22
Revert "Merge pull request #997 from mendableai/feat/sdk-without-ws"
...
This reverts commit 53cda5f81c53d3de35925c610ce083923ca09fbe, reversing
changes made to 51f79b55efadc53243a8c22d86bb2d08d878d524.
2024-12-23 18:45:51 -03:00
Nicolas
18ceaf10a5
Update .gitignore
2024-12-23 18:42:05 -03:00
Nicolas
0c1c4f2ede
Merge branch 'main' into feat/sdk-without-ws
2024-12-23 18:41:31 -03:00
Nicolas
67c643ad1c
Merge pull request #989 from RutamBhagat/js-sdk-conditionally-enforce-api-key
...
feat(js-sdk): Make API key optional for self-hosted instances
2024-12-23 12:56:38 -03:00
RutamBhagat
ca2d3dc6d2
docs(credit-usage-api): add new endpoint documentation for credit usage
2024-12-21 06:24:53 -08:00
Thomas Kosmas
199bd2d1f4
Merge branch 'main' into feat/sdk-without-ws
2024-12-21 02:32:09 +02:00
Thomas Kosmas
a9d31c8e42
Merge branch 'main' into feat/sdk-without-ws
2024-12-21 02:30:40 +02:00
Thomas Kosmas
f043f5fd61
Enhance error handling in E2E tests and introduce CrawlWatcher tests
...
- Updated error messages in E2E tests to provide clearer feedback for blocked URLs and invalid API keys.
- Added new test suite for CrawlWatcher to ensure proper instantiation and error handling when WebSocket is unavailable.
- Improved test conditions for URL scraping and crawling to reflect updated error responses.
2024-12-21 02:27:22 +02:00
Nicolas
d1f3e26f9e
Nick: blocklist string
2024-12-20 18:09:49 -03:00
Nicolas
ba95df96b1
Update rate-limiter.ts
2024-12-20 15:45:44 -03:00
Nicolas
6222152249
Nick: credit usage endpoint
2024-12-20 15:44:17 -03:00
rafaelmmiller
63bbeadbfc
Added try catch to message handler
2024-12-20 08:05:13 -03:00
RutamBhagat
f47e3114d6
feat(rust-sdk): improve API key handling for cloud vs self-hosted services in FirecrawlApp
2024-12-20 02:04:02 -08:00
RutamBhagat
2b488cac3d
chore: remove pytest dependency from pyproject.toml
2024-12-20 01:54:29 -08:00
Nicolas
3e60f175bb
Nick: prompt should be optional on /extract sdks
2024-12-19 16:14:49 -03:00
Nicolas
8063474c85
Update __init__.py
2024-12-19 16:14:17 -03:00
Nicolas
525a71d789
Update __init__.py
2024-12-19 16:10:42 -03:00
Nicolas
4fddc86e66
Update package.json
2024-12-19 16:09:08 -03:00
Nicolas
d67db99791
Merge pull request #994 from mendableai/feat/added-id-to-ws-sdks
...
feat-SDK/added crawl id to ws
2024-12-19 16:08:08 -03:00
Nicolas
ed24853ca6
Merge pull request #996 from mendableai/fix/title-extra-info
...
[BUG] fixed title extra info
2024-12-19 16:05:49 -03:00
Thomas Kosmas
c8cd0148dd
refactor: remove error logging for 'isows' module import in WebSocket initialization
2024-12-19 20:39:30 +02:00
Gergő Móricz
071b9a01c3
fix(scrapeURL/fire-engine): pass geolocation
2024-12-19 18:23:21 +01:00
Thomas Kosmas
6002bf3228
feat: dynamically import WebSocket module with error handling
2024-12-19 14:52:43 +02:00
rafaelmmiller
cf2ec77131
fixed title extra info
2024-12-19 08:32:10 -03:00
Nicolas
066071cd54
Update llmExtract.ts
2024-12-18 23:45:43 -03:00
Nicolas
05605112bb
Update extract.ts
2024-12-18 23:34:07 -03:00
Nicolas
2d37dca9dc
Nick: introduced system prompt to /extract
2024-12-18 22:10:41 -03:00
