557 Commits

Author SHA1 Message Date
yuyutaotao
649aeceb43
feat: enable search area for locate (#473)
* feat: enable search area for locate

* fix: update evaluation

* fix: build error

* fix: ci

* fix: locator

* feat: show searchArea in report

* chore: add yaml support for aiTap

* feat: update status tip

* fix: #473 (#484)

* chore: optimize unit test list

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-03-24 09:50:27 +08:00
Zhou Xiao
5bd917fa28
feat: export overrideAIConfig in chrome extension (#482) 2025-03-20 16:23:25 +08:00
github-actions[bot]
df16339e08 release: v0.12.8 2025-03-19 08:16:01 +00:00
Zhou Xiao
73bf80261c
chore: fix lint error (#477)
* chore: fix lint error

* chore: fix lint error

* chore: fix lint error
2025-03-19 16:11:04 +08:00
Zhou Xiao
47cb015c90
refactor(chrome-devtool): extract the chrome-devtool logic into an application, support local development debugging, and add contribution guidelines. (#476)
* chore: add chrome devtools app

* chore: resolve import error

* chore: support visualizer css

* add build logic

* chore: add build extension zip file script

* chore: migrate part of chrome extension content to app

* chore: delete unless file

* chore: optimize chrome devtool build script

* chore: optimize chrome devtool build script

* fix: resolve bridge mode test issues

* chore: optimize chrome devtool build script

* chore: optimize chrome devtool build script

* chore: optimize chrome devtool build script

* chore: update chrome devtools build process

* chore: optimize chrome devtool build script

* chore: optimize chrome devtool build script

* chore: optimize chrome devtool build script

* chore: optimize chrome devtool build script
2025-03-19 15:22:17 +08:00
yuyutaotao
8e1ba565d0
feat: optimize locator (#456)
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-03-17 19:19:54 +08:00
Zhou Xiao
8ea630aa4b
fix(web-integration): resolve waitForNavigation timeout (#468) 2025-03-14 22:16:01 +08:00
yuyutaotao
ae73b6801d
chore: vince ops feat/faster screenshot base64 in web integration (#450)
* feat: faster screenshotBase64

* fix: use a more native way to do screenshot

---------

Co-authored-by: VinceOPS <10739306+VinceOPS@users.noreply.github.com>
2025-03-14 17:28:08 +08:00
github-actions[bot]
4b9736b4cd release: v0.12.7 2025-03-12 15:59:54 +00:00
yuyutaotao
220a8c3171
fix: package shared as cjs (#463)
* fix: package shared as cjs

* fix: package shared as cjs

* fix: cjs export

* fix: makes web-integration a pure cjs
2025-03-12 23:17:11 +08:00
github-actions[bot]
b4d2181a44 release: v0.12.6 2025-03-12 06:11:58 +00:00
yuyutaotao
2c5ea87131
fix: ai test (#460)
* fix: ai test

* fix: ci test

* fix: evaluation test

* fix: test

* fix: test

* fix: ai test

* fix: ai test
2025-03-12 13:49:50 +08:00
yuyutaotao
55d02961f7
fix: add an field for CoT in qwen (#453)
* fix: add an field for CoT in qwen

* chore: update prompt

* fix: scroll in puppeteer

* fix: multiple action in qwen

* fix: multiple action in qwen

* fix: multiple action in qwen

* chore: update prompt

* chore: change fields order to enhance the CoT
2025-03-11 22:43:59 +08:00
Peterwub
3bb10c0269
feat(core): fix MIDSCENE_CACHE=true not work (#452) 2025-03-11 13:48:06 +08:00
yuyutaotao
65afa009ee
chore: update prompt of qwen (#451) 2025-03-10 22:10:55 +08:00
github-actions[bot]
9dad7f3e7f release: v0.12.5 2025-03-10 08:56:47 +00:00
yuyutaotao
212e4e3725
fix: planning prompt (#448)
* feat: add more case for llm planning

* fix: ai e2e

* chore: use debug to print log

* chore: fix error in gpt mode
2025-03-10 16:50:43 +08:00
yuyutaotao
3218111c26
fix: enhance CoT of qwen (#446)
* chore: change fields order to enhance the CoT

* chore: change fields order to enhance the CoT
2025-03-10 09:14:00 +08:00
Zhou Xiao
d128745e31
fix(esm): resolve cli can't load esm module (#445)
* fix(esm): resolve cli can't load esm module

* chore: resolve deps error

* chore: resolve deps error

* chore: resolve deps error

* chore: resolve deps error

* chore: resolve deps error

* chore: resolve deps error

* chore: resolve deps error

* chore: resolve deps error

* chore: resolve deps error

* chore: resolve deps error

* chore: resolve deps error

* chore: resolve deps error
2025-03-09 21:50:20 +08:00
Zhou Xiao
88945c34e5
chore: resolve visualizer build error (#444) 2025-03-07 17:53:17 +08:00
yuyutaotao
372b67f16c
fix: race condition in bridge mode (#440) 2025-03-07 17:38:46 +08:00
Zhou Xiao
5d63ef9151
refactor: switch bundle type to bundleless (#437) 2025-03-07 17:20:18 +08:00
yuyutaotao
726a3a70dd
fix: always use the connected tab in extension (#439)
* fix: always use the connected tab in extension

* fix: always use the connected tab in extension
2025-03-06 17:08:11 +08:00
github-actions[bot]
0d5f88a456 release: v0.12.4 2025-03-03 06:26:00 +00:00
yuyutaotao
9c779d4cd3
chore: tuning the prompt of qwen (#433)
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-03-03 11:39:28 +08:00
github-actions[bot]
cbb850bb01 release: v0.12.3 2025-02-28 03:04:37 +00:00
yuyutaotao
6591420757
chore: tuning the prompt of qwen (#432)
* chore: tuning the prompt of qwen

* chore: tuning the prompt of qwen

* fix: ci

* fix: CI
2025-02-28 10:59:22 +08:00
yuyutaotao
6a08dd7151
chore: rename core package (#431)
* chore: rename core package

* fix: lint

* fix: ci

* fix: ci
2025-02-27 22:47:06 +08:00
github-actions[bot]
e35f277935 release: v0.12.2 2025-02-27 10:38:20 +00:00
yuyutaotao
a6ffbd07b7
fix: over planning for Qwen in page with form (#429) 2025-02-27 16:44:01 +08:00
Zhou Xiao
881adab4fc
feat(yaml): Add cache support for yaml files, allowing file-level caching to improve execution speed and stability. Cache ID is automatically set to yaml filename. (#430) 2025-02-27 11:33:49 +08:00
github-actions[bot]
bb56bbc248 release: v0.12.1 2025-02-26 09:15:45 +00:00
yuyutaotao
d0a3d86e22
docs: update instruction for bridge mode (#428)
* docs: update instruction for bridge mode

* fix: ai test
2025-02-26 17:06:14 +08:00
yuyutaotao
724a2c7230
fix(core): the prompt of qwen (#427)
* fix: fix the  prompt of qwen

* docs: update debugging config

* fix: sleep in qwen

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-02-26 15:04:28 +08:00
yuyutaotao
8be082e308
feat(chrome-devtool): allow longer connection of chrome bridge (#416) 2025-02-26 14:20:27 +08:00
Zhaopeng XUAN
111cf25d31
feat(core/ui-tars): support aiAssert() & aiWaitFor for UI-Tars model (#412)
* Issue-41 Support aiAssert() for ui-tars model

---------
Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-02-25 14:49:14 +08:00
yuyutaotao
29a6c33f80
fix: print warning for oversized input (#414) 2025-02-24 16:53:06 +08:00
Zhou Xiao
1eefad4422
docs: optimize agent api doc (#415)
* docs: optimize agent api doc

* docs: optimize runyaml link

* docs: optimize prompt
2025-02-24 14:29:17 +08:00
github-actions[bot]
9f1e491aa8 release: v0.12.0 2025-02-21 08:49:04 +00:00
yuyutaotao
3f438f6c0c
fix: ci of qwen model (#410) 2025-02-21 16:15:51 +08:00
yuyutaotao
bbe9874e78
fix: coord offset of qwen model (#407)
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-02-21 10:30:20 +08:00
yuyutaotao
59ce2d0140
feat: locate by coord (#383)
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-02-21 09:56:09 +08:00
yuyutaotao
e459fae751
fix: partial scroll issue in chrome extension (#401)
* fix: partial scroll issue in chrome extension

* fix: update tips for self-hosted model
2025-02-19 11:14:19 +08:00
github-actions[bot]
9dbb4b6b04 release: v0.11.3 2025-02-17 08:10:50 +00:00
Zhou Xiao
b2d183d5e3
fix(report): reduce playwright report size (#396) 2025-02-17 16:01:59 +08:00
Zhou Xiao
f661a1882e
fix(report): resolve no locate generate error (#392) 2025-02-15 12:00:57 +08:00
github-actions[bot]
dd2163a0d1 release: v0.11.2 2025-02-15 02:22:04 +00:00
myselfsaurabh
243f8d0c7e
feat(reporting): Add test ID to test data attributes (#380)
* feat(reporting): Add test ID to test data attributes

* Fixed lint issue
2025-02-15 09:49:57 +08:00
Zhou Xiao
c32a212c7b
fix(browser-event): optimize limit open new tab script and add yaml test (#391) 2025-02-15 09:14:21 +08:00
Zhou Xiao
edf0871fea
feat(browser): Add the forceSameTabNavigation configuration to prevent AI from opening new pages during operations, thus avoiding task interruptions. (#389)
* feat(tab-control): enhance the configuration to limit AI from opening new tabs during operations, preventing failures.

* chore: optimize evaluate error

* chore: resolve navigation error

* fix(browser): add forceSameTabNavigation config toe limit open new tab

* chore: upgrade vitest version

* fix: typo (#390)

---------

Co-authored-by: yuyutaotao <167746126+yuyutaotao@users.noreply.github.com>
2025-02-14 21:54:47 +08:00