UncleCode d024749633 refactor(deep-crawl): add max_pages limit and improve crawl control ...

Add max_pages parameter to all deep crawling strategies to limit total pages crawled.
Add score_threshold parameter to BFS/DFS strategies for quality control.
Remove legacy parameter handling in AsyncWebCrawler.
Improve error handling and logging in crawl strategies.

BREAKING CHANGE: Removed support for legacy parameters in AsyncWebCrawler.run_many()

2025-03-03 21:51:11 +08:00

..

browser-crawler-config.md

Feat/llm config (#724)

2025-02-21 15:41:37 +08:00

cache-modes.md

refactor(docs): reorganize documentation structure and update styles

2025-01-07 20:49:50 +08:00

cli.md

feat(cli): add command line interface with comprehensive features

2025-02-10 16:58:52 +08:00

content-selection.md

Feat/llm config (#724)

2025-02-21 15:41:37 +08:00

crawler-result.md

Release prep (#749)

2025-02-28 19:53:35 +08:00

deep-crawling.md

refactor(deep-crawl): add max_pages limit and improve crawl control

2025-03-03 21:51:11 +08:00

docker-deployment.md

Release prep (#749)

2025-02-28 19:53:35 +08:00

fit-markdown.md

Release prep (#749)

2025-02-28 19:53:35 +08:00

installation.md

refactor(docs): reorganize documentation structure and update styles

2025-01-07 20:49:50 +08:00

link-media.md

refactor(docs): reorganize documentation structure and update styles

2025-01-07 20:49:50 +08:00

local-files.md

refactor(docs): reorganize documentation structure and update styles

2025-01-07 20:49:50 +08:00

markdown-generation.md

Release prep (#749)

2025-02-28 19:53:35 +08:00

page-interaction.md

refactor(docs): reorganize documentation structure and update styles

2025-01-07 20:49:50 +08:00

quickstart.md

Release prep (#749)

2025-02-28 19:53:35 +08:00

simple-crawling.md

Release prep (#749)

2025-02-28 19:53:35 +08:00