156 Commits

Author SHA1 Message Date
yutao
54268eb08d docs(core): instruction for override config 2025-04-21 16:59:14 +08:00
yuyutaotao
86bd961542
docs: update docs (#574)
* docs: release android automation

* chore: update docs

* chore: update readme

* docs: append docs for doubao-1.5-ui-tars

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>

* chore(docs): update doubao docs

* chore(docs): merge docs for doubao

* docs(android): update

* docs(llm): upgrade open source model

---------

Co-authored-by: 晓 <zhouxiao.shaw@bytedance.com>
2025-04-21 14:18:55 +08:00
Leyang
e1b8fa45a5
fix(android): update styles and transitions in playground components (#596)
* fix(android): update styles and transitions in playground components

* fix(android): optimize the style of the input box and the automatic scrolling function

* fix(android): remove max-width constraint from player component styles
2025-04-21 14:10:12 +08:00
01f46a202f
refactor(android): improve logic for copying Android playground (#600) 2025-04-19 15:59:11 +08:00
github-actions[bot]
0a3bc1e230 release: v0.15.0 2025-04-18 11:26:30 +00:00
Leyang
2c66902677
fix(core): player title max width (#595) 2025-04-18 19:16:17 +08:00
Leyang
76eb4e1417
fix(report): ai tap in report (#594)
* fix: ai tap in report

* chore(playwright): update lockfile

* fix(core): lint

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-04-18 18:42:47 +08:00
Leyang
06ce3c889b
android playground compatible with windows (#592)
* fix(playground): simplify class property definitions and add root redirect for static file serving

* fix(scrcpy-player): update empty state rendering logic to handle server validity more accurately
2025-04-18 17:03:00 +08:00
Leyang
cf621a564e
fix(visualizer): update type references and improve tooltip handling in components (#589) 2025-04-18 15:18:03 +08:00
yuyutaotao
7e43a29d02
chore: update extension style (#585)
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-04-18 12:53:33 +08:00
Leyang
bcfcb3c4d0
fix: style of chrome (#586) 2025-04-18 11:33:20 +08:00
Leyang
7596da2ab7
feat: support tap in prompt (#584) 2025-04-18 10:27:56 +08:00
yuyutaotao
ad457a33a8
feat(yaml): allow running javascript in yaml (#555)
* feat: allow running javascript in yaml

* feat: change the output dir

* fix: CI

* docs: update docs about evaluate javascript

* chore: merge main

* chore: merge main
2025-04-18 09:58:51 +08:00
Leyang
b76211bd5d
feat: android playground (#542)
* refactor: android api

* refactor: enhance Android agent to accept options for device connection

* fix: type error

* fix: click after clearInput

* fix: click before clearInput

* feat: android playground

* feat: support npx package name

* feat: android playground joint

* fix: git ignore conflicts

* feat: ensure adb server is running before initializing adb client

* fix: deps consistency

* ci: add android playground

* feat: integrate shared constants and improve server configuration in android playground

* feat: android playground style

* feat: style opt

* feat: add @rsbuild/plugin-svgr dependency and improve URI handling in adb

* feat: remove unused water flow scripts and update comments to English

* feat: download report file

* feat: standalone android playground

* feat: use dynamic import

* feat: migrate CSS to LESS and remove unused styles in chrome extension and report

* feat: enhance Android playground with ScrcpyPlayer ref integration and device management improvements

* feat: optimize styles and layout in Android playground and visualizer components

* chore: add bin back

* chore: update build script to exclude documentation generation

* feat: add not ready message to PlaygroundResult for improved user guidance

* feat: add error handling for screenshot capture in Android page

* docs: update readme

* feat: add PNG validation for screenshot buffer in Android page

* feat: enhance UI components with improved styling and tooltips in ScrcpyPlayer and PromptInput

* docs: update uri parameter description in integrate-with-android documentation and improve uri handling in launch function

* style: update primary color to #2B83FF across multiple components and adjust margin in App.less

* refactor: replace userConfig with globalConfig for environment configuration management and update related functions

* feat: integrate server validation logic in App, AdbDevice, and ScrcpyPlayer components for improved connection handling

* style: enhance player component layout with overflow handling and margin adjustments

* style: refine player component layout with flex adjustments and improved spacing

* feat: add midscene model name display and improve layout in EnvConfig component

* feat: integrate ShinyText component for enhanced loading progress display in PlaygroundResult

* test: add test for isValidPNGImageBuffer

* style: remove background color from App.less and adjust AI config override behavior in env.ts

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-04-17 17:44:11 +08:00
Leyang
3793e91899
docs: add android playground docs (#563)
* docs: add android playground docs

* chore: update pic

* docs: distinct android and web

* chore: update docs

* chore: update docs

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-04-15 11:31:12 +08:00
yuyutaotao
edf48da198
docs: update caching docs (#559)
* docs: update caching docs

* docs: update caching docs

* docs: update docs

* chore: update docs
2025-04-14 14:50:48 +08:00
Leyang
570c2d7294
fix: rethrow error with android faq when adb error occured (#554) 2025-04-10 20:51:56 +08:00
github-actions[bot]
1bd873b360 release: v0.14.3 2025-04-10 09:10:39 +00:00
yuyutaotao
9be848b651
feat: allow insecure https in yaml (#553) 2025-04-10 16:57:46 +08:00
Leyang
202f54ca78
docs: add docs for android (#546)
* docs: add docs for android

* chore: update docs

* docs: add extra api for android

* chore: update docs

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-04-09 17:04:56 +08:00
github-actions[bot]
4d77207c0f release: v0.14.2 2025-04-09 02:30:47 +00:00
github-actions[bot]
92e4d7ed61 release: v0.14.1 2025-04-03 08:40:18 +00:00
yuyutaotao
5e7f82c357
docs: update blog (#530) 2025-04-03 14:36:45 +08:00
yuyutaotao
767ffcd873
docs: blog for instant action and deep think (#527) 2025-04-03 14:26:30 +08:00
github-actions[bot]
00dc2f9f50 release: v0.14.0 2025-04-03 06:01:06 +00:00
yuyutaotao
bcdf90b997
feat: add context for aiAction (#528) 2025-04-02 20:34:23 +08:00
yuyutaotao
0c0675ba4b
feat: update deepThink interface (#517)
* feat: update deepThink interface

* fix: CI

* chore: update error message

* feat: update search area config

* fix: deepthink ui

---------

Co-authored-by: Zhou Xiao <zhouxiao.shaw@bytedance.com>
2025-04-02 19:26:56 +08:00
Zhou Xiao
814f99ca48
refactor(visualizer): optimize extension logic (#525) 2025-04-02 19:12:52 +08:00
Zhou Xiao
5ef9202c79
refactor(report): optimize visualizer common logic (#519)
* chore: delete unless visualizer logic

* chore: delete unless code

* chore: delete unless code

* chore: delete unless code

* chore: delete unless common logic

* chore: merge main branch

* chore: update lock file
2025-04-01 17:25:40 +08:00
yuyutaotao
cf4acf30d9
feat: print stats in debug (#515)
Co-authored-by: Zhou Xiao <zhouxiao.shaw@bytedance.com>
2025-04-01 17:20:37 +08:00
Zhou Xiao
ca28ae0b29
fix(report): resolve report script error (#516) 2025-04-01 15:03:42 +08:00
yuyutaotao
adc3fd9e0d
feat: use aiTap(prompt, opt) style interface (#512) 2025-04-01 10:41:31 +08:00
Zhou Xiao
d1b232586b
chore: delete unless visualizer logic (#514) 2025-03-31 19:51:43 +08:00
Zhou Xiao
16d6efa1f0
refactor(report): make the report page an app type to improve the development experience (#513) 2025-03-31 18:46:35 +08:00
Zhou Xiao
0e4b94fc9d
chore(report): init report project (#511) 2025-03-31 11:34:38 +08:00
neverland
dcf6296dc2
docs: lazy loading report.gif (#503) 2025-03-26 11:40:04 +08:00
yuyutaotao
a15fa3d47f
docs: updat faq (#501) 2025-03-26 10:34:57 +08:00
github-actions[bot]
00de5d6478 release: v0.13.1 2025-03-25 14:53:44 +00:00
Leyang
934a1e2b5d
use adb instead of appium (#483)
* feat: enable search area for locate

* fix: update evaluation

* fix: locator

* feat: show searchArea in report

* chore: add yaml support for aiTap

* feat: use adb instead appium

* feat: Adds debugging information and reconstructs input text capabilities

* feat: refactoring Android related functions and adding android modules

* feat: update the image scaling algorithm, adjust the Android page class to support device scaling, and remove test files that are no longer needed

* feat: adjust the Android page class to support device scaling, and remove test files that are no longer needed

* feat: use appium-adb instead of bare command

* fix: update entry for @midscene/android

* feat: optimize the screenshot processing logic, add a backup mechanism when screenshots fail, and update test cases to accommodate new features

* fix: rethrow error

* feat: add Android debug configuration options and update documentation

* chore: fix code style in #483 (#492)

* fix: remove try for error handle by outside

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
Co-authored-by: linyibing <linyibing@bytedance.com>
Co-authored-by: yuyutaotao <167746126+yuyutaotao@users.noreply.github.com>
2025-03-25 22:45:05 +08:00
yuyutaotao
f5049f556d
docs: docs for new API (#499) 2025-03-25 17:16:07 +08:00
Zhou Xiao
45ddfa5ee6
chore: use cdn replace netlify (#496) 2025-03-25 11:10:23 +08:00
github-actions[bot]
291f8e1a4b release: v0.13.0 2025-03-24 12:32:05 +00:00
Zhou Xiao
36d47e4aef
feat(playwright): add aiTap/aiInput and other AI actions for Playwright integration (#489)
* feat(playwright): add aiTap/aiInput and other AI actions for Playwright integration

* chore: update doc

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-03-24 19:30:53 +08:00
yuyutaotao
649aeceb43
feat: enable search area for locate (#473)
* feat: enable search area for locate

* fix: update evaluation

* fix: build error

* fix: ci

* fix: locator

* feat: show searchArea in report

* chore: add yaml support for aiTap

* feat: update status tip

* fix: #473 (#484)

* chore: optimize unit test list

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-03-24 09:50:27 +08:00
github-actions[bot]
df16339e08 release: v0.12.8 2025-03-19 08:16:01 +00:00
Zhou Xiao
73bf80261c
chore: fix lint error (#477)
* chore: fix lint error

* chore: fix lint error

* chore: fix lint error
2025-03-19 16:11:04 +08:00
Zhou Xiao
47cb015c90
refactor(chrome-devtool): extract the chrome-devtool logic into an application, support local development debugging, and add contribution guidelines. (#476)
* chore: add chrome devtools app

* chore: resolve import error

* chore: support visualizer css

* add build logic

* chore: add build extension zip file script

* chore: migrate part of chrome extension content to app

* chore: delete unless file

* chore: optimize chrome devtool build script

* chore: optimize chrome devtool build script

* fix: resolve bridge mode test issues

* chore: optimize chrome devtool build script

* chore: optimize chrome devtool build script

* chore: optimize chrome devtool build script

* chore: update chrome devtools build process

* chore: optimize chrome devtool build script

* chore: optimize chrome devtool build script

* chore: optimize chrome devtool build script

* chore: optimize chrome devtool build script
2025-03-19 15:22:17 +08:00
yuyutaotao
8e1ba565d0
feat: optimize locator (#456)
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-03-17 19:19:54 +08:00
yuyutaotao
2c5ea87131
fix: ai test (#460)
* fix: ai test

* fix: ci test

* fix: evaluation test

* fix: test

* fix: test

* fix: ai test

* fix: ai test
2025-03-12 13:49:50 +08:00
yuyutaotao
212e4e3725
fix: planning prompt (#448)
* feat: add more case for llm planning

* fix: ai e2e

* chore: use debug to print log

* chore: fix error in gpt mode
2025-03-10 16:50:43 +08:00