611 Commits

Author SHA1 Message Date
lsustc
bd0bf46e9a
fix click event should move the mouse to the click area first (#535)
* fix click event should move the mouse to the click area first

* fix reusing the existing function
2025-04-06 21:53:21 +08:00
Zhou Xiao
c4d4eb7cdf
workflow(issue): add issue template (#540) 2025-04-05 10:44:35 +08:00
Zhou Xiao
2c3c540cb1
fix(playwright): make test.describe work correctly with ai fixtures (#537)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-04-04 16:14:04 +08:00
github-actions[bot]
92e4d7ed61 release: v0.14.1 v0.14.1 2025-04-03 08:40:18 +00:00
yuyutaotao
c6cd10ebb2
fix: filename (#534) 2025-04-03 16:37:00 +08:00
yuyutaotao
5e7f82c357
docs: update blog (#530) 2025-04-03 14:36:45 +08:00
yuyutaotao
767ffcd873
docs: blog for instant action and deep think (#527) 2025-04-03 14:26:30 +08:00
github-actions[bot]
00dc2f9f50 release: v0.14.0 v0.14.0 2025-04-03 06:01:06 +00:00
yuyutaotao
0171b5e2de
fix: ci testing (#529)
* fix: ci

* fix: ci
2025-04-03 10:51:50 +08:00
Leyang
bc645c3d0c
refactor: android api (#521)
* refactor: android api

* refactor: update the Android agent and device connection logic

* refactor: enhance Android agent to accept options for device connection

* fix: type error

* feat: use first device when no deviceId pass

* fix: import from agent.ts

* fix: click after clearInput

* fix: click before clearInput
2025-04-03 10:28:40 +08:00
yuyutaotao
bcdf90b997
feat: add context for aiAction (#528) 2025-04-02 20:34:23 +08:00
yuyutaotao
0c0675ba4b
feat: update deepThink interface (#517)
* feat: update deepThink interface

* fix: CI

* chore: update error message

* feat: update search area config

* fix: deepthink ui

---------

Co-authored-by: Zhou Xiao <zhouxiao.shaw@bytedance.com>
2025-04-02 19:26:56 +08:00
Zhou Xiao
814f99ca48
refactor(visualizer): optimize extension logic (#525) 2025-04-02 19:12:52 +08:00
Zhou Xiao
5ef9202c79
refactor(report): optimize visualizer common logic (#519)
* chore: delete unless visualizer logic

* chore: delete unless code

* chore: delete unless code

* chore: delete unless code

* chore: delete unless common logic

* chore: merge main branch

* chore: update lock file
2025-04-01 17:25:40 +08:00
yuyutaotao
cf4acf30d9
feat: print stats in debug (#515)
Co-authored-by: Zhou Xiao <zhouxiao.shaw@bytedance.com>
2025-04-01 17:20:37 +08:00
Leyang
882a2db9ee
fix: force screenshot when the app doesn't allow it (#518)
* fix: force screenshot when the app doesn't allow it

* fix: support start point in scroll method

* refactor: rename to ensureYadb
2025-04-01 17:05:47 +08:00
Zhou Xiao
ca28ae0b29
fix(report): resolve report script error (#516) 2025-04-01 15:03:42 +08:00
yuyutaotao
adc3fd9e0d
feat: use aiTap(prompt, opt) style interface (#512) 2025-04-01 10:41:31 +08:00
Zhou Xiao
d1b232586b
chore: delete unless visualizer logic (#514) 2025-03-31 19:51:43 +08:00
Zhou Xiao
6468bb0206
refactor(logger): use log file output instead of command line output in Node (#509)
* chore: add element detail info

* chore: add element detail info

* chore: add element detail info

* chore: optimize logger logic

* chore: optimize logger logic
2025-03-31 19:22:39 +08:00
Zhou Xiao
16d6efa1f0
refactor(report): make the report page an app type to improve the development experience (#513) 2025-03-31 18:46:35 +08:00
Zhou Xiao
0e4b94fc9d
chore(report): init report project (#511) 2025-03-31 11:34:38 +08:00
Leyang
bf9b4e06e7
feat: migrate android code to @midscene/android (#505) 2025-03-26 18:59:50 +08:00
Zhou Xiao
375a5b0152
chore: support new ui-tars model json format (#506) 2025-03-26 18:57:24 +08:00
neverland
dcf6296dc2
docs: lazy loading report.gif (#503) 2025-03-26 11:40:04 +08:00
Leyang
1f3fc5b580
chore: not run android tests in ci && update tests for android settig case (#502)
* chore: not run android tests in ci && update tests for android setting case

* ci: add lint for json
2025-03-26 11:39:39 +08:00
yuyutaotao
a15fa3d47f
docs: updat faq (#501) 2025-03-26 10:34:57 +08:00
github-actions[bot]
00de5d6478 release: v0.13.1 v0.13.1 2025-03-25 14:53:44 +00:00
Leyang
934a1e2b5d
use adb instead of appium (#483)
* feat: enable search area for locate

* fix: update evaluation

* fix: locator

* feat: show searchArea in report

* chore: add yaml support for aiTap

* feat: use adb instead appium

* feat: Adds debugging information and reconstructs input text capabilities

* feat: refactoring Android related functions and adding android modules

* feat: update the image scaling algorithm, adjust the Android page class to support device scaling, and remove test files that are no longer needed

* feat: adjust the Android page class to support device scaling, and remove test files that are no longer needed

* feat: use appium-adb instead of bare command

* fix: update entry for @midscene/android

* feat: optimize the screenshot processing logic, add a backup mechanism when screenshots fail, and update test cases to accommodate new features

* fix: rethrow error

* feat: add Android debug configuration options and update documentation

* chore: fix code style in #483 (#492)

* fix: remove try for error handle by outside

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
Co-authored-by: linyibing <linyibing@bytedance.com>
Co-authored-by: yuyutaotao <167746126+yuyutaotao@users.noreply.github.com>
2025-03-25 22:45:05 +08:00
yuyutaotao
010ee92d62
fix: playwright integration (#497)
* fix: playwright integration

* chore: fix types

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-03-25 17:22:25 +08:00
yuyutaotao
f5049f556d
docs: docs for new API (#499) 2025-03-25 17:16:07 +08:00
Zhou Xiao
45ddfa5ee6
chore: use cdn replace netlify (#496) 2025-03-25 11:10:23 +08:00
yuyutaotao
e94c31f228
fix: ai test (#494)
* fix: ai test

* fix: planning ci
2025-03-25 10:04:03 +08:00
github-actions[bot]
291f8e1a4b release: v0.13.0 v0.13.0 2025-03-24 12:32:05 +00:00
yuyutaotao
999e52ebc1
feat: update prompt for qwen (#491) 2025-03-24 19:47:19 +08:00
Zhou Xiao
36d47e4aef
feat(playwright): add aiTap/aiInput and other AI actions for Playwright integration (#489)
* feat(playwright): add aiTap/aiInput and other AI actions for Playwright integration

* chore: update doc

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-03-24 19:30:53 +08:00
yuyutaotao
649aeceb43
feat: enable search area for locate (#473)
* feat: enable search area for locate

* fix: update evaluation

* fix: build error

* fix: ci

* fix: locator

* feat: show searchArea in report

* chore: add yaml support for aiTap

* feat: update status tip

* fix: #473 (#484)

* chore: optimize unit test list

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-03-24 09:50:27 +08:00
Zhou Xiao
5bd917fa28
feat: export overrideAIConfig in chrome extension (#482) 2025-03-20 16:23:25 +08:00
github-actions[bot]
df16339e08 release: v0.12.8 v0.12.8 2025-03-19 08:16:01 +00:00
Zhou Xiao
73bf80261c
chore: fix lint error (#477)
* chore: fix lint error

* chore: fix lint error

* chore: fix lint error
2025-03-19 16:11:04 +08:00
Zhou Xiao
47cb015c90
refactor(chrome-devtool): extract the chrome-devtool logic into an application, support local development debugging, and add contribution guidelines. (#476)
* chore: add chrome devtools app

* chore: resolve import error

* chore: support visualizer css

* add build logic

* chore: add build extension zip file script

* chore: migrate part of chrome extension content to app

* chore: delete unless file

* chore: optimize chrome devtool build script

* chore: optimize chrome devtool build script

* fix: resolve bridge mode test issues

* chore: optimize chrome devtool build script

* chore: optimize chrome devtool build script

* chore: optimize chrome devtool build script

* chore: update chrome devtools build process

* chore: optimize chrome devtool build script

* chore: optimize chrome devtool build script

* chore: optimize chrome devtool build script

* chore: optimize chrome devtool build script
2025-03-19 15:22:17 +08:00
yuyutaotao
8e1ba565d0
feat: optimize locator (#456)
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-03-17 19:19:54 +08:00
Zhou Xiao
8ea630aa4b
fix(web-integration): resolve waitForNavigation timeout (#468) 2025-03-14 22:16:01 +08:00
Vincent G
66a97164ac
feat: faster screenshotBase64 (#449)
Co-authored-by: Zhou Xiao <zhouxiao.shaw@bytedance.com>
2025-03-14 17:30:20 +08:00
yuyutaotao
ae73b6801d
chore: vince ops feat/faster screenshot base64 in web integration (#450)
* feat: faster screenshotBase64

* fix: use a more native way to do screenshot

---------

Co-authored-by: VinceOPS <10739306+VinceOPS@users.noreply.github.com>
2025-03-14 17:28:08 +08:00
github-actions[bot]
4b9736b4cd release: v0.12.7 v0.12.7 2025-03-12 15:59:54 +00:00
yuyutaotao
220a8c3171
fix: package shared as cjs (#463)
* fix: package shared as cjs

* fix: package shared as cjs

* fix: cjs export

* fix: makes web-integration a pure cjs
2025-03-12 23:17:11 +08:00
github-actions[bot]
b4d2181a44 release: v0.12.6 v0.12.6 2025-03-12 06:11:58 +00:00
yuyutaotao
2c5ea87131
fix: ai test (#460)
* fix: ai test

* fix: ci test

* fix: evaluation test

* fix: test

* fix: test

* fix: ai test

* fix: ai test
2025-03-12 13:49:50 +08:00
yuyutaotao
55d02961f7
fix: add an field for CoT in qwen (#453)
* fix: add an field for CoT in qwen

* chore: update prompt

* fix: scroll in puppeteer

* fix: multiple action in qwen

* fix: multiple action in qwen

* fix: multiple action in qwen

* chore: update prompt

* chore: change fields order to enhance the CoT
2025-03-11 22:43:59 +08:00