yujunjun/firecrawl
1
0
Fork 0
mirror of https://github.com/mendableai/firecrawl.git synced 2025-11-04 03:53:17 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,451 Commits 899 Branches 29 Tags
Commit Graph

1451 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Nicolas
854d5b3cb3 Update single_url.ts 2024-06-03 23:32:55 -07:00
Nicolas
99059814a8 Nick: 2024-06-03 21:32:48 -07:00
Nicolas
918059ee9e Merge branch 'main' into nsc/improvemnts-fixes-misc 2024-06-03 16:46:02 -07:00
Nicolas
93bb53271e Merge branch 'nsc/improved-blocklist' 2024-06-03 16:44:33 -07:00
Nicolas
38e583f66c Update socialBlockList.test.ts 2024-06-03 16:44:23 -07:00
Nicolas
b26c5f1588
Merge pull request #185 from mendableai/nsc/improved-blocklist 
Improvements to the blocklist regex
 2024-06-03 16:43:34 -07:00
Nicolas
c69c89f838 Nick: 2024-06-03 16:42:42 -07:00
Nicolas
48d1ec05b2 Merge branch 'main' into nsc/improved-blocklist 2024-06-03 16:38:03 -07:00
Nicolas
d30ced4394
Merge pull request #221 from mendableai/nsc/fwd-header-auth 
feat: Ability to forward headers to reliable providers for auth etc...
 2024-06-03 16:33:40 -07:00
Nicolas
d865b0c5c8
Merge pull request #229 from rombru/main 
Use @ instead of # for default BULL_AUTH_KEY. Hash mark is reserved for URI fragments.
 2024-06-03 12:38:34 -07:00
Romain Bruyère
4987f901d1
Merge branch 'mendableai:main' into main 2024-06-03 21:29:33 +02:00
rafaelsideguide
4100cc9223 Update index.test.ts 2024-06-03 16:29:16 -03:00
rombru
3ff91ddd1f fix: use @ instead of # for default BULL_AUTH_KEY. hash mark is reserved for URI fragments. 2024-06-03 21:28:25 +02:00
rafaelsideguide
c1aed1360e Update index.test.ts 2024-06-03 15:51:07 -03:00
Nicolas
30a0c5de1a
Merge pull request #228 from mendableai/bugfix/fire-engine-content 
Fixed fire-engine content bug
 2024-06-03 11:42:03 -07:00
rafaelsideguide
1fc3a15149 Update single_url.ts 2024-06-03 15:24:40 -03:00
Eric Ciarla
3ea801d9dd Commit Roast My Website 2024-06-02 20:40:19 -07:00
Eric Ciarla
ea04fe2e3f Add Roast My Website Example 2024-06-02 20:38:05 -07:00
Nicolas
fde522c3e1 Update single_url.ts 2024-06-02 20:23:45 -07:00
Matt Joyce
deefe65cbe Change the way the playwright response is parsed 
Was failing with a Type Error, but actually looked ok.
This fixes the type error, and stop scraper fallback.
 2024-06-01 19:16:56 +10:00
Matt Joyce
14896a9fdd Fix PLAYWRIGHT_MICROSERVICE_URL 
It needs to end in html, otherwise scrape will 404
 2024-06-01 19:03:16 +10:00
Matt Joyce
1eacad4ef3 Clarifying wait type and name 2024-06-01 18:53:03 +10:00
Matt Joyce
c516140bfb Various Linting 
Pylint
C0114: Missing module docstring
C0115: Missing class docstring
C0116: Missing function or method docstring
C0303: Trailing whitespace
Import ordering
 2024-06-01 18:53:03 +10:00
Matt Joyce
2a39b5382b Add timeout to class and provide default. 2024-06-01 18:52:42 +10:00
Nicolas
c7d5a9ad48 Merge branch 'main' into nsc/fwd-header-auth 2024-05-31 18:19:20 -07:00
Nicolas
8cb62dde92 Update website_params.ts 2024-05-31 16:09:39 -07:00
Nicolas
3b8059edb6 Update single_url.ts 2024-05-31 15:43:06 -07:00
Nicolas
6bea803120 Nick: 2024-05-31 15:39:54 -07:00
Nicolas
03ba4a9fa7
Merge pull request #218 from mendableai/nsc/python-sdk-waiting-fixes 
[sdk] Fixes waiting status not being present on check status
 2024-05-31 11:54:27 -07:00
Nicolas
2139129296 Nick: v12 2024-05-31 11:39:55 -07:00
Nicolas
260e31c68b Merge branch 'nsc/new-pricing' 2024-05-30 16:08:31 -07:00
Nicolas
aa8133ca7f Update load-testing-example.ts 2024-05-30 16:07:14 -07:00
Nicolas
0c115c6181
Merge pull request #216 from mendableai/nsc/new-pricing 
feat: New pricing/limits changes
 2024-05-30 15:36:59 -07:00
Nicolas
d486d7da1c
Merge pull request #207 from mendableai/feat/screenshot-support 
Feat/screenshot support
 2024-05-30 15:36:09 -07:00
Nicolas
6860ace4af Nick: 2024-05-30 15:07:49 -07:00
Nicolas
6ceb7ff50a Nick: 2024-05-30 14:46:55 -07:00
Nicolas
33f10a7f91 Nick: fixes 2024-05-30 14:42:32 -07:00
Nicolas
ace46f340b Nick: new limits, new pricing 2024-05-30 14:31:36 -07:00
Matt Joyce
677102e676 Script to check local vs published versions 
This script is for use with Github workflows
 2024-05-30 22:05:21 +10:00
Matt Joyce
9f8792f00c Script to check local vs published versions 
This script is for use with Github workflows
 2024-05-30 21:48:40 +10:00
Matt Joyce
5c4b3e8f8a Initial pyproject.toml 
This will enable building using 'python -m build', without impacting the utility of setup.py, also provide a base for other build tools and automation.
 2024-05-30 21:48:40 +10:00
Matt Joyce
dec225d368 Move version to __init__.py 
Setup.py does not need to be edited when building the package.
 2024-05-30 21:48:40 +10:00
rafaelsideguide
2b763d848b improved js response and test for getting partial_data 2024-05-30 08:44:38 -03:00
rafaelsideguide
5b8b6902e7 Update index.ts 2024-05-30 08:25:13 -03:00
Nicolas
6c939d534d Nick: small refactor 2024-05-29 19:43:51 -07:00
Eric Ciarla
37915e11e8 Final push 2024-05-29 21:18:24 -04:00
Eric Ciarla
a0e404f94e init commit 2024-05-29 18:56:57 -04:00
Nicolas
51b0b88cd4
Merge pull request #204 from mendableai/feat/custom-scraping-readme 
[Feat] Added custom scraping conditions for readme docs
 2024-05-29 10:00:24 -07:00
rafaelsideguide
ee9a2184e2 Added custom scraping conditions for readme docs 2024-05-29 13:39:43 -03:00
Nicolas
8911ddf10c Merge branch 'nsc/wait-for-param' 2024-05-28 17:17:29 -07:00