561 Commits

Author SHA1 Message Date
quanruzhuoxiu
f15a355d87 docs(site): update pic 2025-05-21 19:49:21 +08:00
quanruzhuoxiu
fb6ac68f6d docs(site): add details and pic 2025-05-21 19:49:10 +08:00
Thulof Qu
1a4eabd577
docs(common): fix broken relative links (#746) 2025-05-21 19:24:11 +08:00
Leyang
4c8d881614
fix(web-integration): handle undefined aiInput case in ScriptPlayer (#743)
* fix(web-integration): handle undefined aiInput case in ScriptPlayer

* fix(workflow): version

* fix(core): yaml player

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-05-21 19:18:29 +08:00
yuyutaotao
1589d10f35
docs(core): update blog (#735)
* docs(core): update blog

* docs(blog): update workflow blog

* docs(site): add details and pic

* docs(site): update pic

* docs(site): update pic 2

---------

Co-authored-by: quanruzhuoxiu <quanruzhuoxiu@gmail.com>
2025-05-21 16:49:26 +08:00
Leyang
5a1a3ba18a
feat(web-integration): support disable cache for a single api call (#740)
* feat(web-integration): support disable cache for a single api call

* feat(workflow): version mismatch

* feat(web-integration): cache rename to cacheable

* feat(web-integration): add cacheable option to multiple API methods and update caching documentation

* docs(site): update cacheable option descriptions to reference caching feature documentation

* docs(core): update caching doc

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-05-21 16:46:20 +08:00
github-actions[bot]
049d9f6565 release: v0.17.1 v0.17.1 2025-05-20 09:11:17 +00:00
dependabot[bot]
9c3d0f7fe7
chore(deps): bump semver from 7.3.7 to 7.5.2 in /packages/web-integration (#738)
* chore(deps): bump semver in /packages/web-integration

Bumps [semver](https://github.com/npm/node-semver) from 7.3.7 to 7.5.2.
- [Release notes](https://github.com/npm/node-semver/releases)
- [Changelog](https://github.com/npm/node-semver/blob/main/CHANGELOG.md)
- [Commits](https://github.com/npm/node-semver/compare/v7.3.7...v7.5.2)

---
updated-dependencies:
- dependency-name: semver
  dependency-version: 7.5.2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: resolve lock file

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-05-20 15:58:34 +08:00
bac9051d2d
feat(evaluation): add screenspot v2 evaluation (#737)
* feat(evaluation): add screenspot v2 evaluation

* style(evaluation): format files array in package.json
2025-05-20 15:52:03 +08:00
yuyutaotao
5a041bcba3
fix(core): dump page context when locate fails (#736)
* fix(core): dump page context when locate fails

* fix(core): dump page context when locate fails

* fix(core): dump page context when locate fails
2025-05-20 15:08:14 +08:00
Leyang
fb2580616c
fix cache bugs(scroll instant, text node info, cache hit condition) (#732)
* fix(web-integration): cache hit when element.id exist and scroll element instantly

* fix(web-integration): use parent xpath for text node

* fix(web-integration): only scroll in to view when element is not completly visible

* fix(shared): distinct text node

* test(web-integration): getElementInfoByXpath

* test(web-integration): rename desc

* test(web-integration): fix

* test(web-integration): snapshot attributes only

* test(web-integration): fix test
2025-05-20 13:19:32 +08:00
89fd77b96a
docs(llm): add doubao-1.5-vl-pro model (#719)
* docs(llm): add doubao-1.5-vl-pro model

* docs(site): move model comparison to choose a model doc

* docs(core): update doubao model

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-05-19 15:56:28 +08:00
781b758f58
docs(site): add DeepWiki link to README (#728) 2025-05-19 11:42:29 +08:00
Wang Fanghua
c407921449
fix(site): add aiScroll import for playwright example docs. (#726) 2025-05-19 10:45:34 +08:00
github-actions[bot]
39ddc57df3 release: v0.17.0 v0.17.0 2025-05-16 14:32:01 +00:00
Leyang
b10a7751d4
fix(report): show highlight elment when cache hit (#723)
* fix(report): show highlight elment when cache hit

* fix(core): replay animation

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-05-16 22:28:29 +08:00
yuyutaotao
b261ed7f2a
feat(web): use xpath and yaml as cache (#711)
* feat(web-integration): use xpath for cache instead of id

* feat(web-integration): enhance TaskCache to support xpaths for cache matching and add new test cases

* feat(web-integration): add debug log for unknown page types in TaskCache

* feat(web-integration): update caching logic and cache hit conditions for Plan and Locate tasks

* chore(core): update debug log

* feat(web-integration): update rspress.config and enhance TaskCache structure with new properties

* feat(web-integration): recalculate id when hit cache

* fix(web-integration): update mock implementation in task-cache test to use evaluate method

* feat(web-integration): enhance element caching by adding XPath support and improving cache hit logic

* chore(core): lint

* feat(web-integration): improve XPath handling in web-extractor

* test(web-integration): fix tests

* feat(core, web-integration): add attributes to LocateResultElement and enhance element handling

* fix(core): lint

* feat(web-integration): add midsceneVersion to TaskCache and update cache validation logic

* fix(core): test

* fix(web-integration): update cache validation logic to prevent reading outdated midscene cache files

* feat(web-integration): enhance TaskCache to track used cache items and improve cache retrieval logic

* fix(core): xpath logic (#710)

* feat(core): resue context for locate

* feat(core): build yamlFlow from aiAction

* feat(core): refine task-cache

* feat(core): update cache

* feat(core): refine task-cache

* feat(core): refine task-cache

* feat(core): remove unused checkElementExistsByXPath

* feat(core): use yaml file as cache

* chore(core): fix lint

* chore(core): print warning for previous cache

* refactor(core): remove quickAnswer references and improve element matching logic

* fix(core): update import path for buildYamlFlowFromPlans

* chore(web-integration): update output image and skip task error test

* fix(web-integration): update test snapshots to handle beta versions

* fix(web-integration): adjust test snapshots for version consistency

* fix(web-integration): track original cache length and adjust matching logic in tests

* fix(web-integration): update test URLs to reflect new target site and enable previously skipped test

* chore(core): update cache docs

* fix(core): test

* feat(core): try to match element from plan

* fix(web-integration): cache id stable when retry in palywright

* fix(web-integration): typo

* style(web-integration): lint

* fix(web-integration): stable cacheid in tests

* fix(web-integration): cache id

---------

Co-authored-by: quanruzhuoxiu <quanruzhuoxiu@gmail.com>
2025-05-16 17:16:56 +08:00
Leyang
ab4fcfe220
docs(site): blog: Key to Automation Stability: Programming Practices Using Structured API (#709)
* docs(site): blog: Key to Automation Stability: Programming Practices Using Structured API

* docs(core): update blog

* docs(site): add blog post on optimizing automation code with structured API

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-05-16 14:20:11 +08:00
yuyutaotao
fba2c00aec
chore(core): update bbox preprocessing of ui-tars (#716) 2025-05-16 13:22:57 +08:00
Leyang
b9ff80a0db
implement repeat function for scrolling until actions (#713)
* feat(android): implement repeat function for scrolling until actions

* fix(shared): fix potential error in getAIConfig by ensuring trim is called correctly

* feat(android): update scrolling behavior with adjustable duration and added sleep

* feat(android): refine scrolling durations with new constants for fast and normal scroll
2025-05-14 18:28:10 +08:00
Leyang
388bbb6a34
fix(android): update yadb (#714) 2025-05-14 18:26:54 +08:00
github-actions[bot]
571ec07829 release: v0.16.10 v0.16.10 2025-05-09 09:00:22 +00:00
yuyutaotao
34e15c7247
fix(core): empty report issue (#702)
* fix(core): empty report issue

* fix(core): empty report issue
2025-05-09 16:51:30 +08:00
github-actions[bot]
7f982f7cb7 release: v0.16.9 v0.16.9 2025-05-09 03:44:51 +00:00
yuyutaotao
73188fa43f
fix(core): bundle template content into core package (#699)
* fix(core): bundle template content into core package

* fix(core): lint

* docs: add bundler mode error msg

* fix(core): ci

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-05-09 11:41:16 +08:00
yutao
de99bee2f4 Merge branch 'neewbee-main' 2025-05-09 11:06:00 +08:00
yutao
99b4bf4638 fix(core): lint 2025-05-09 11:05:04 +08:00
yutao
6224154bdc Merge branch 'main' of https://github.com/neewbee/midscene into neewbee-main 2025-05-09 10:55:12 +08:00
Leyang
a63e018fc9
feat(site): update clarity script ID and Google site verification token (#697) 2025-05-09 09:50:31 +08:00
f99582ad66
docs(llm): add llm.txt (#694)
* docs(llm): add llm.txt

* docs(llm): delete unless code

* docs: modify sidebar
2025-05-08 17:36:38 +08:00
3f39f42e36
docs(mcp): add MCP to README (#695) 2025-05-08 16:44:52 +08:00
Leyang
57f1a7b7d2
feat(site): seo optimization (#693)
* feat(site): seo optimization

* feat(android, web-integration): update package descriptions and add keywords

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-05-08 10:54:52 +08:00
iyaozhen
cd05fb3a98
docs(site): fix volcengine version USE_VLM_UI_TARS value and modify 火山云 to 火山引擎 (#692)
Co-authored-by: yaozhen.00 <yaozhen.00@bytedance.com>
2025-05-07 19:54:23 +08:00
HBLADEH
7762e5d04e
feat(android): fix custom adb path no effect (#688)
* feat(android): fix custom adb path no effect

* feat(android): fix custom adb, removed redundant `this.adb =` assignment
2025-05-06 17:29:13 +08:00
Leyang
2b502df712
style(chrome-extension): update padding for popup layout with responsive design (#687) 2025-05-06 14:14:58 +08:00
Leyang
8661c97dcf
docs(site): update timeout parameters in API to include options for disabling timeouts (#686)
* docs(site): update timeout parameters in API to include options for disabling timeouts

* docs(core): update docs for timeout

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-05-06 11:31:06 +08:00
github-actions[bot]
c5a88931f0 release: v0.16.8 v0.16.8 2025-04-30 14:31:14 +00:00
yuyutaotao
c1bc73c78b
feat(android): customize adb path (#684)
* feat(shared): add custom adb path
feat(android): add custom adb path

* feat(android): add docs for custom adb path

---------

Co-authored-by: HBLADEH <1012582116@qq.com>
2025-04-30 17:16:38 +08:00
yuyutaotao
6a8ccbbde4
fix(core): id conflict in quick answer (#679)
* fix(core): id conflict in quick answer

* fix(core): lint

* docs(core): update docs for qwen

* fix(core): test case

* refactor(core): refactor elementById args

* refactor(core): refactor elementById args

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-04-30 16:51:31 +08:00
yuyutaotao
80b4e7b609
fix(mcp): mcp error connection (#676)
* fix(mcp): mcp error connection

* fix(mcp): lint

* fix(core): add kill signal for server

* chore(core): force close conflict port in bridge

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>

* chore(core): merge main
2025-04-30 16:07:59 +08:00
yuyutaotao
b6bf0a2323
fix(core): ui-tars test case (#683) 2025-04-30 16:07:31 +08:00
yuyutaotao
0c6dbec5bd
fix(core): use unified config for doubao-ui-tars model (#678) 2025-04-29 22:52:45 +08:00
yuyutaotao
b8f29e8e66
fix(core): use unified config for doubao-ui-tars model (#678) 2025-04-29 21:39:58 +08:00
yuyutaotao
2c13c082e5
docs(core): update docs for aiQuery (#673) 2025-04-29 19:57:51 +08:00
yuyutaotao
3506d8f1e3
docs(core): update profiling config (#677) 2025-04-29 19:57:30 +08:00
github-actions[bot]
7ce8225369 release: v0.16.7 v0.16.7 2025-04-29 08:36:53 +00:00
Leyang
9c2abf7729
feat(core): yaml support aiBoolean, aiNumber, aiString, aiLocate (#674) 2025-04-29 15:46:48 +08:00
github-actions[bot]
aa2e767b79 release: v0.16.6 v0.16.6 2025-04-29 05:54:48 +00:00
2e738f71cd
feat(mcp): make mcp navigated faster (#672)
* Revert "feat(mcp): make mcp navigated faster (#671)"

This reverts commit 81bcc5ec1a6338123ee87dfee76f41bdc97d578d.

* chore: use fkill replace kill-port
2025-04-29 13:51:04 +08:00
Leyang
4718066bb1
feat(web-integration): add support for new agent method(aiBoolean, aiString, aiNumber, aiLocate) (#658)
* feat(web-integration): add support for new agent method(aiBoolean, aiString, aiNumber, aiLocate)

* fix(core, web-integration): update data extraction logic and refine return types for agent methods

* feat(site): add detailed descriptions and examples

* fix(web-integration): remove unused properties from aiLocate return type and update UI titles

* fix(core): enhance agent methods for better extraction and querying

* fix(core): update test scripts and refine data extraction prompts

* fix(core): refine return type in ExecutionTaskApply and remove unused import in inspect

* fix(core): update data extraction logic and enhance type handling in Insight class

* fix(web-integration): update import path for puppeteerAgentForTarget in player test file

* fix(core): tests fix

* fix(core): fix tests

* fix(shared): enhance imageInfo tests by validating individual properties and updating snapshots

* fix(site): remove waitForNavigationTimeout from Playwright integration examples

* fix(site): update timeout configuration examples in FAQ for clarity

* test(web-integration): remove unused client-extractor test and related snapshots

* chore(core): fix query implementation

* fix(report): improve data handling in DetailSide component and enhance output display

* fix(core): refine data extraction prompts and improve error handling in PageTaskExecutor

* fix(core): update data extraction prompt for clarity and type validation

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-04-29 13:36:49 +08:00