161 Commits

Author SHA1 Message Date
Leyang
3d102ab4b5
feat(web-integration): override timeout from fixture (#880)
* feat(web-integration): override timeout from fixture

* docs(site): api update
2025-07-03 10:43:01 +08:00
Leyang
b486f5d5d4
fix(web-integration): unify agent options for Puppeteer and Playwright, adding waitForNavigationTimeout and waitForNetworkIdleTimeout parameters (#872) 2025-07-01 22:13:14 +08:00
Leyang
51f017521d
docs(site): 0.20 changelog (#867) 2025-06-25 15:20:21 +08:00
Leyang
1f4afcc1fe
feat(core): add scale for aiLocate api (#864)
* feat(core): add scale for aiLocate api

* refactor(web-integration): rename var

* refactor(web-integration): rename var
2025-06-24 19:52:06 +08:00
Leyang
c3b6c991cf
docs(site): add availability notes for web-only methods (#848) 2025-06-20 15:08:44 +08:00
yuyutaotao
b7c252f729
docs(core): update index page (#800)
* docs(core): update index page

* docs(core): update index page
2025-06-19 11:52:01 +08:00
Leyang
09c6aaaa8b
feat(web-integration): support user expected xpath option for locate methods (#844)
* feat(web-integration): support user expected xpath option for locate methods

* docs(site): xpath option for api

* fix(ci): nx config

* docs(site): update docs
2025-06-19 10:55:14 +08:00
Leyang
bee1c40637
feat(web-integration): support aiAsk for agent (#841)
* feat(web-integration): support aiAsk for agent

* docs(core): update docs

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-06-19 10:55:07 +08:00
Leyang
2965e3cf10
docs(site): 0.19 changelog (#842) 2025-06-18 16:56:37 +08:00
Leyang
5f79ef8c6b
feat(android): support ime strategy for android device (#839)
* feat(android): support ime strategy for android device

* docs(site): update
2025-06-17 09:33:03 +08:00
65fc4ead23
docs(site): fix code block formatting in playwright integration guide (#830) 2025-06-13 15:55:08 +08:00
Leyang
32f503f69e
fix(shared): reduce null element node (#827)
* fix(shared): reduce null element node

* feat(web-integration): delete content and locator, support visible-only for domIncluded

* feat(web-integration): remove logScreenshot documentation and update tests to use treeToList for content extraction

* feat(web-integration): add trimContextByViewport function to filter execution tasks by visible elements

* release: v0.18.1

* fix(workflow): update default release version to 'prepatch'

* fix(web-integration): node detect empty

* release: v0.18.2

* fix(evaluation): restore generateExtractData

* fix(web-integration): reporter export

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-06-13 11:25:31 +08:00
Leyang
199ab17504
feat(cli): support dotenv-override argv for cli (#819)
* feat(cli): support dotenv-override argv for cli

* feat(cli): support dotenv-debug argv for cli

* docs(core): update doc of dotenv

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-06-11 17:09:17 +08:00
Leyang
a84100e7d8
feat(mcp, shared, visualizer): add license (#818)
* feat(mcp, shared, visualizer): add license

* docs(core): update playwright reporter doc

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-06-11 17:09:05 +08:00
Leyang
3510a8c736
feat(web-integration): support unstableLogContent api for agent (#826)
* feat(web-integration): support unstableLogContent api for agent

* fix(web-integration): use sync mehtod

* docs(core): update API doc

* fix(web-integration): playwright

* docs(site): update

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-06-11 16:50:11 +08:00
Leyang
aeab83c777
docs: add previous changelog (#816)
* docs(site): add previous changelog

* docs(site): add recently changelog

* docs(site): summary for previous changlog
2025-06-09 17:44:00 +08:00
Leyang
7ced918451
docs(site): add changelog for 0.18 (#815) 2025-06-09 11:55:25 +08:00
Leyang
37ae71241e
feat(web-integration): support logScreenshot api for agent (#813)
* feat(web-integration): support logScreenshot api for agent

* docs(core): update api description

* docs(site): fix error

* feat(web-integration): title is also optional

* docs(site): fix

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-06-06 18:58:49 +08:00
Leyang
738251481c
docs(site): update gemini example (#811) 2025-06-06 13:37:36 +08:00
Leyang
a3fa0e23eb
support android auto hide keyborad for aiInput (#808)
* fix(android): modify automatically closing the keyboard after typed text (#793)

* fix(android): modify automatically closing the keyboard after typed text

* fix(android): rename to autoDismissKeyboard

* feat(android): support autoDismissKeyboard option for aiInput

* docs(core): update doc

* refactor(android): agent no need support autoDismissKeyboard parameter

* refactor(android): build parameter in build plan

* chore(core): update android docs

---------

Co-authored-by: welkang <welkang@qq.com>
Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-06-06 10:56:05 +08:00
Leyang
b17b3b3ded
docs(site): changelog for 0.17.4 (#807)
* docs(site): changelog for 0.17.4

* docs(core): update change log

* docs(core): update change log

* docs(site): update sidebar title

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-06-05 13:04:15 +08:00
Leyang
548a9801b8
feat: support customizing adb host (#801)
* feat(android): support custom host and port of adb

* feat(android): add options for custom ADB path, host, and port in AndroidDevice constructor

* docs(android): custom remote host and port for adb
2025-06-04 14:37:41 +08:00
Leyang
fd48cbd8c9
docs(core): add query right click and query from html docs (#787)
* docs(site): add right click docs

* docs(site): add ai query opts docs

* fix(web-integration): test

* docs(core): update api doc

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-05-30 14:25:53 +08:00
yuyutaotao
ddc60d0416
docs(core): update blog of workflow (#776) 2025-05-28 22:20:09 +08:00
Leyang
8f4e722239
feat(core): support error message for ai assert in yaml (#774) 2025-05-27 19:15:25 +08:00
yuyutaotao
5451c27db4
fix(core): fix planning test (#773)
* fix(core): fix planning test

* fix(report): report insight view

* chore(core): rename workflow

* fix(core): report replay scripts

* fix(core): test case

* docs(core): update api doc for ui-tars

* chore(core): fix test case
2025-05-27 19:13:15 +08:00
Leyang
8c8e5d236a
fix(web-integration): add cacheable option to YAML actions and pass the cacheable parameter (#764) 2025-05-26 10:27:48 +08:00
Leyang
d07c6a78e3
feat(web-integration): support full xpath for cache (#754)
* feat(web-integration): support full xpath for cache

* refactor(web-integration): simplify getElementXPath to improve clarity and performance

* feat(web-integration): add support for 'A' node type and related extraction logic

* feat(web-integration): merge children rects of button

* feat(web-integration): enhance caching documentation and tests

* test(web-integration): update snapshot

* docs(core): update caching

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-05-23 17:51:04 +08:00
yuyutaotao
08466cac1e
feat(core): add element describer (#750) 2025-05-21 21:05:47 +08:00
Thulof Qu
1a4eabd577
docs(common): fix broken relative links (#746) 2025-05-21 19:24:11 +08:00
yuyutaotao
1589d10f35
docs(core): update blog (#735)
* docs(core): update blog

* docs(blog): update workflow blog

* docs(site): add details and pic

* docs(site): update pic

* docs(site): update pic 2

---------

Co-authored-by: quanruzhuoxiu <quanruzhuoxiu@gmail.com>
2025-05-21 16:49:26 +08:00
Leyang
5a1a3ba18a
feat(web-integration): support disable cache for a single api call (#740)
* feat(web-integration): support disable cache for a single api call

* feat(workflow): version mismatch

* feat(web-integration): cache rename to cacheable

* feat(web-integration): add cacheable option to multiple API methods and update caching documentation

* docs(site): update cacheable option descriptions to reference caching feature documentation

* docs(core): update caching doc

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-05-21 16:46:20 +08:00
89fd77b96a
docs(llm): add doubao-1.5-vl-pro model (#719)
* docs(llm): add doubao-1.5-vl-pro model

* docs(site): move model comparison to choose a model doc

* docs(core): update doubao model

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-05-19 15:56:28 +08:00
Wang Fanghua
c407921449
fix(site): add aiScroll import for playwright example docs. (#726) 2025-05-19 10:45:34 +08:00
yuyutaotao
b261ed7f2a
feat(web): use xpath and yaml as cache (#711)
* feat(web-integration): use xpath for cache instead of id

* feat(web-integration): enhance TaskCache to support xpaths for cache matching and add new test cases

* feat(web-integration): add debug log for unknown page types in TaskCache

* feat(web-integration): update caching logic and cache hit conditions for Plan and Locate tasks

* chore(core): update debug log

* feat(web-integration): update rspress.config and enhance TaskCache structure with new properties

* feat(web-integration): recalculate id when hit cache

* fix(web-integration): update mock implementation in task-cache test to use evaluate method

* feat(web-integration): enhance element caching by adding XPath support and improving cache hit logic

* chore(core): lint

* feat(web-integration): improve XPath handling in web-extractor

* test(web-integration): fix tests

* feat(core, web-integration): add attributes to LocateResultElement and enhance element handling

* fix(core): lint

* feat(web-integration): add midsceneVersion to TaskCache and update cache validation logic

* fix(core): test

* fix(web-integration): update cache validation logic to prevent reading outdated midscene cache files

* feat(web-integration): enhance TaskCache to track used cache items and improve cache retrieval logic

* fix(core): xpath logic (#710)

* feat(core): resue context for locate

* feat(core): build yamlFlow from aiAction

* feat(core): refine task-cache

* feat(core): update cache

* feat(core): refine task-cache

* feat(core): refine task-cache

* feat(core): remove unused checkElementExistsByXPath

* feat(core): use yaml file as cache

* chore(core): fix lint

* chore(core): print warning for previous cache

* refactor(core): remove quickAnswer references and improve element matching logic

* fix(core): update import path for buildYamlFlowFromPlans

* chore(web-integration): update output image and skip task error test

* fix(web-integration): update test snapshots to handle beta versions

* fix(web-integration): adjust test snapshots for version consistency

* fix(web-integration): track original cache length and adjust matching logic in tests

* fix(web-integration): update test URLs to reflect new target site and enable previously skipped test

* chore(core): update cache docs

* fix(core): test

* feat(core): try to match element from plan

* fix(web-integration): cache id stable when retry in palywright

* fix(web-integration): typo

* style(web-integration): lint

* fix(web-integration): stable cacheid in tests

* fix(web-integration): cache id

---------

Co-authored-by: quanruzhuoxiu <quanruzhuoxiu@gmail.com>
2025-05-16 17:16:56 +08:00
Leyang
ab4fcfe220
docs(site): blog: Key to Automation Stability: Programming Practices Using Structured API (#709)
* docs(site): blog: Key to Automation Stability: Programming Practices Using Structured API

* docs(core): update blog

* docs(site): add blog post on optimizing automation code with structured API

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-05-16 14:20:11 +08:00
f99582ad66
docs(llm): add llm.txt (#694)
* docs(llm): add llm.txt

* docs(llm): delete unless code

* docs: modify sidebar
2025-05-08 17:36:38 +08:00
iyaozhen
cd05fb3a98
docs(site): fix volcengine version USE_VLM_UI_TARS value and modify 火山云 to 火山引擎 (#692)
Co-authored-by: yaozhen.00 <yaozhen.00@bytedance.com>
2025-05-07 19:54:23 +08:00
Leyang
8661c97dcf
docs(site): update timeout parameters in API to include options for disabling timeouts (#686)
* docs(site): update timeout parameters in API to include options for disabling timeouts

* docs(core): update docs for timeout

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-05-06 11:31:06 +08:00
yuyutaotao
c1bc73c78b
feat(android): customize adb path (#684)
* feat(shared): add custom adb path
feat(android): add custom adb path

* feat(android): add docs for custom adb path

---------

Co-authored-by: HBLADEH <1012582116@qq.com>
2025-04-30 17:16:38 +08:00
yuyutaotao
b8f29e8e66
fix(core): use unified config for doubao-ui-tars model (#678) 2025-04-29 21:39:58 +08:00
yuyutaotao
2c13c082e5
docs(core): update docs for aiQuery (#673) 2025-04-29 19:57:51 +08:00
yuyutaotao
3506d8f1e3
docs(core): update profiling config (#677) 2025-04-29 19:57:30 +08:00
Leyang
4718066bb1
feat(web-integration): add support for new agent method(aiBoolean, aiString, aiNumber, aiLocate) (#658)
* feat(web-integration): add support for new agent method(aiBoolean, aiString, aiNumber, aiLocate)

* fix(core, web-integration): update data extraction logic and refine return types for agent methods

* feat(site): add detailed descriptions and examples

* fix(web-integration): remove unused properties from aiLocate return type and update UI titles

* fix(core): enhance agent methods for better extraction and querying

* fix(core): update test scripts and refine data extraction prompts

* fix(core): refine return type in ExecutionTaskApply and remove unused import in inspect

* fix(core): update data extraction logic and enhance type handling in Insight class

* fix(web-integration): update import path for puppeteerAgentForTarget in player test file

* fix(core): tests fix

* fix(core): fix tests

* fix(shared): enhance imageInfo tests by validating individual properties and updating snapshots

* fix(site): remove waitForNavigationTimeout from Playwright integration examples

* fix(site): update timeout configuration examples in FAQ for clarity

* test(web-integration): remove unused client-extractor test and related snapshots

* chore(core): fix query implementation

* fix(report): improve data handling in DetailSide component and enhance output display

* fix(core): refine data extraction prompts and improve error handling in PageTaskExecutor

* fix(core): update data extraction prompt for clarity and type validation

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-04-29 13:36:49 +08:00
cd358bdac3
docs(mcp): optimize mcp showcase and add how to get report (#662)
* docs(mcp): optimize mcp showcase and add how to get report

* docs: optimize docs
2025-04-28 14:18:09 +08:00
yuyutaotao
58c5cc0989
docs(core): update docs for doubao vl model (#661) 2025-04-28 13:33:56 +08:00
yuyutaotao
5fb208a08c
feat(core): adapt UI tars 1.5 (#616)
* feat(core): adapt ui-tars 1.5

* chore(core): adaptr ui-tars-1.5

* chore(core): fix lint

* fix(core): env building issue

* fix(core): update import for uiTarsModelVersion from shared env

* feat(core): ui-tars hotkey event

* chore(core): move @ui-tars/action-parser to devDependencies

* fix(core): adapting new model
2025-04-28 08:42:43 +08:00
d02452d879
docs(mcp): add mcp doc (#656) 2025-04-27 22:35:00 +08:00
Leyang
ca644d8914
feat(core): allow custom midscene_run dir (#631)
* feat(core): support custom midscene_run dir

* feat(report): add search functionality to PlaywrightCaseSelector component

* refactor(shared): simplify base directory resolution and remove unused environment variable

* feat(shared): integrate shared environment variables across multiple packages

* refactor(shared): update base directory resolution to use dynamic midscene_run directory

* fix(puppeteer): increase screenshot timeout from 3s to 10s for improved reliability
2025-04-24 22:54:52 +08:00
Leyang
f85cd6cd1b
feat(playwright): enhance report filename generation with unique identifiers and support split mode (#629)
* feat(playwright): enhance report filename generation with unique identifiers and support split mode

* fix(playwright): update reporter type from "single" to "merged" and adjust related configurations

* chore(playwright): improve filename sanitization in reporter

* feat(web-integration): add function to replace illegal path characters

* feat(web-integration): update path sanitization function to replace spaces and illegal characters
2025-04-24 22:42:08 +08:00