24 Commits

Author SHA1 Message Date
yuyutaotao
a6ac78aee3
chore(core): update prompt for section locator (#588) 2025-04-18 13:35:58 +08:00
Leyang
b76211bd5d
feat: android playground (#542)
* refactor: android api

* refactor: enhance Android agent to accept options for device connection

* fix: type error

* fix: click after clearInput

* fix: click before clearInput

* feat: android playground

* feat: support npx package name

* feat: android playground joint

* fix: git ignore conflicts

* feat: ensure adb server is running before initializing adb client

* fix: deps consistency

* ci: add android playground

* feat: integrate shared constants and improve server configuration in android playground

* feat: android playground style

* feat: style opt

* feat: add @rsbuild/plugin-svgr dependency and improve URI handling in adb

* feat: remove unused water flow scripts and update comments to English

* feat: download report file

* feat: standalone android playground

* feat: use dynamic import

* feat: migrate CSS to LESS and remove unused styles in chrome extension and report

* feat: enhance Android playground with ScrcpyPlayer ref integration and device management improvements

* feat: optimize styles and layout in Android playground and visualizer components

* chore: add bin back

* chore: update build script to exclude documentation generation

* feat: add not ready message to PlaygroundResult for improved user guidance

* feat: add error handling for screenshot capture in Android page

* docs: update readme

* feat: add PNG validation for screenshot buffer in Android page

* feat: enhance UI components with improved styling and tooltips in ScrcpyPlayer and PromptInput

* docs: update uri parameter description in integrate-with-android documentation and improve uri handling in launch function

* style: update primary color to #2B83FF across multiple components and adjust margin in App.less

* refactor: replace userConfig with globalConfig for environment configuration management and update related functions

* feat: integrate server validation logic in App, AdbDevice, and ScrcpyPlayer components for improved connection handling

* style: enhance player component layout with overflow handling and margin adjustments

* style: refine player component layout with flex adjustments and improved spacing

* feat: add midscene model name display and improve layout in EnvConfig component

* feat: integrate ShinyText component for enhanced loading progress display in PlaygroundResult

* test: add test for isValidPNGImageBuffer

* style: remove background color from App.less and adjust AI config override behavior in env.ts

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-04-17 17:44:11 +08:00
yuyutaotao
a1b5a54d89
fix(report): do not call mkdir in browser (#577)
* fix: log dir

* fix: log dir

* fix: gitignore config
2025-04-17 15:09:59 +08:00
yuyutaotao
dd233ef25b
chore: adapting new model (#570)
* chore: adapting new model

* feat: adapt bbox format

* chore: adapt ui-tars
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-04-17 10:53:19 +08:00
yuyutaotao
66a5d16d37
chore: update evaluation fixture (#552) 2025-04-10 16:54:21 +08:00
yuyutaotao
732f605144
fix(llm): coords offset in vl locator (#545) 2025-04-08 17:56:15 +08:00
github-actions[bot]
00dc2f9f50 release: v0.14.0 2025-04-03 06:01:06 +00:00
yuyutaotao
bcdf90b997
feat: add context for aiAction (#528) 2025-04-02 20:34:23 +08:00
yuyutaotao
0c0675ba4b
feat: update deepThink interface (#517)
* feat: update deepThink interface

* fix: CI

* chore: update error message

* feat: update search area config

* fix: deepthink ui

---------

Co-authored-by: Zhou Xiao <zhouxiao.shaw@bytedance.com>
2025-04-02 19:26:56 +08:00
Zhou Xiao
ca28ae0b29
fix(report): resolve report script error (#516) 2025-04-01 15:03:42 +08:00
yuyutaotao
adc3fd9e0d
feat: use aiTap(prompt, opt) style interface (#512) 2025-04-01 10:41:31 +08:00
yuyutaotao
999e52ebc1
feat: update prompt for qwen (#491) 2025-03-24 19:47:19 +08:00
yuyutaotao
649aeceb43
feat: enable search area for locate (#473)
* feat: enable search area for locate

* fix: update evaluation

* fix: build error

* fix: ci

* fix: locator

* feat: show searchArea in report

* chore: add yaml support for aiTap

* feat: update status tip

* fix: #473 (#484)

* chore: optimize unit test list

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-03-24 09:50:27 +08:00
yuyutaotao
8e1ba565d0
feat: optimize locator (#456)
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-03-17 19:19:54 +08:00
yuyutaotao
2c5ea87131
fix: ai test (#460)
* fix: ai test

* fix: ci test

* fix: evaluation test

* fix: test

* fix: test

* fix: ai test

* fix: ai test
2025-03-12 13:49:50 +08:00
yuyutaotao
55d02961f7
fix: add an field for CoT in qwen (#453)
* fix: add an field for CoT in qwen

* chore: update prompt

* fix: scroll in puppeteer

* fix: multiple action in qwen

* fix: multiple action in qwen

* fix: multiple action in qwen

* chore: update prompt

* chore: change fields order to enhance the CoT
2025-03-11 22:43:59 +08:00
yuyutaotao
65afa009ee
chore: update prompt of qwen (#451) 2025-03-10 22:10:55 +08:00
yuyutaotao
212e4e3725
fix: planning prompt (#448)
* feat: add more case for llm planning

* fix: ai e2e

* chore: use debug to print log

* chore: fix error in gpt mode
2025-03-10 16:50:43 +08:00
yuyutaotao
3218111c26
fix: enhance CoT of qwen (#446)
* chore: change fields order to enhance the CoT

* chore: change fields order to enhance the CoT
2025-03-10 09:14:00 +08:00
yuyutaotao
372b67f16c
fix: race condition in bridge mode (#440) 2025-03-07 17:38:46 +08:00
Zhou Xiao
5d63ef9151
refactor: switch bundle type to bundleless (#437) 2025-03-07 17:20:18 +08:00
yuyutaotao
9c779d4cd3
chore: tuning the prompt of qwen (#433)
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-03-03 11:39:28 +08:00
yuyutaotao
6591420757
chore: tuning the prompt of qwen (#432)
* chore: tuning the prompt of qwen

* chore: tuning the prompt of qwen

* fix: ci

* fix: CI
2025-02-28 10:59:22 +08:00
yuyutaotao
6a08dd7151
chore: rename core package (#431)
* chore: rename core package

* fix: lint

* fix: ci

* fix: ci
2025-02-27 22:47:06 +08:00