384 Commits

Author SHA1 Message Date
Zhou Xiao
e3eb6f6fbc
fix: resolve shared exports error (#335) 2025-01-27 16:16:42 +08:00
Zhou Xiao
b4a6b6f44d
chore: change default model name (#333) 2025-01-27 14:05:42 +08:00
Zhou Xiao
02ad84cf6f
chore: fix ai test (#332) 2025-01-27 10:22:56 +08:00
github-actions[bot]
e6fc69eb81 release: v0.10.2 2025-01-26 12:39:17 +00:00
Zhou Xiao
9c88186540
feat(ui-tars): enhance the UI-TARS keyboard event handling and optimize parser logic (#330) 2025-01-26 20:34:56 +08:00
yuyutaotao
6d924e267f
feat(puppeteer): allow tracking new tab in puppeteer agent (#310) 2025-01-26 16:49:32 +08:00
Zhou Xiao
839dc6c799
feat(browser-event): support drag event (#321) 2025-01-26 12:00:55 +08:00
Zhou xiao
2811f15da0
chore: refine AI test for failures (#323) 2025-01-25 11:14:15 +08:00
Zhou xiao
4c82eff526
chore: fix e2e test (#316)
* chore: fix e2e test

* chore: ignore todo test

* chore: fix e2e test

* chore: upgrade gpt api version

* chore: fix e2e test

* chore: fix e2e test

* chore: fix direction

* chore: fix direction

* chore: split test

* chore: split test

* chore: modify input info

* fix: scroll param in ci

* fix: lint

* fix: ai test

* fix: ai test

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-01-24 15:02:50 +08:00
github-actions[bot]
0f92941f54 release: v0.10.1 2025-01-23 07:15:56 +00:00
yuyutaotao
57f6786628
feat(extract-data): extract data from same-origin iframe (#258)
* feat: extract data from same-origin iframe

* fix: ci snapshot

* fix: extracting timeout error

* fix: ci timeout

* fix: update assets

* feat: set default size of yaml as 1920x1080

* chore: update default viewport size

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-01-22 20:52:11 +08:00
Zhou xiao
6c188d09e8
fix(e2e): optimize e2e error (#307)
* fix(e2e): optimize e2e error

* chore: ignore unless e2e
2025-01-22 13:51:40 +08:00
yuyutaotao
3d95460f44
docs(ai-model): update docs for ui-tars (#305)
* feat: update docs for ui-tars

* doc: update

* doc: update

* doc: update

* chore: update readme

* fix: ci

* docs: upgrade video

* chore: modify huagging face icon

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-01-22 09:24:29 +08:00
yuyutaotao
ed45cfab6c
fix(action): keyboard and div extracting (#304)
* fix: clearInput in browser

* fix: input issue on mac

* fix: playwright input

* fix: issue when collecting fixed item
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>

* fix: extract fixed style content

* fix: extract fixed style content

* fix: customize window size in yaml
2025-01-21 19:21:09 +08:00
yuyutaotao
3aa1b33955
feat: use jpeg as default image format (#301)
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-01-20 20:02:49 +08:00
github-actions[bot]
9b86067ea6 release: v0.10.0 2025-01-17 10:31:04 +00:00
yuyutaotao
a114e707d1
feat: show pointer position in chrome extension (#286)
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-01-17 18:19:22 +08:00
yuyutaotao
f68b6f3139
feat(chrome-devtool): allow tracking active tab in bridge mode (#282)
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-01-15 19:54:03 +08:00
yuyutaotao
fb2b9d123f
feat(chrome-devtool): add 'stop' button in extension (#281)
* feat: add 'stop' to playground

* feat: make extension stopable

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-01-15 19:18:26 +08:00
Zhou xiao
ff9ef70941
chore(prompt): switch thought language according to timezone in vlm mode (#283)
* chore: switch Thought Language to english

* chore: use timezone to select language
2025-01-15 18:54:23 +08:00
github-actions[bot]
d580bd02fe release: v0.9.2 2025-01-15 03:22:10 +00:00
Zhou xiao
beb74f12e8
chore: optimize devtool tab connect (#279) 2025-01-14 21:31:33 +08:00
Zhou xiao
cb52da2ffd
chore: Avoid obscuring the entire page content while optimizing the execution process of DevTools (#276) 2025-01-14 17:05:56 +08:00
zhouxiao.shaw
ea4ee5f976 chore: enhance the execution spped of chrome devtools 2025-01-14 15:04:24 +08:00
Zhou xiao
0cf9a71940
feat(chrome-devtool): water flow animation on the target page (#271)
* feat(chrome-devtool): support water flow animation

* chore: optimize debugger cause window resize

* chore: optimize dynamic script inject

* chore: optimize report error
2025-01-14 14:13:50 +08:00
yuyutaotao
857c411d67
feat: allow tracking newly-opened tabs in Chrome extension (#272) 2025-01-14 11:22:20 +08:00
Zhou xiao
56845b75ba
chore: fix planning typo (#269) 2025-01-13 14:52:15 +08:00
Zhou xiao
2b18ed55de
feat(ai-model): support vlm (#262)
* feat(ai-model): support plan to target

* chore: modify test

* chore: modify test

* chore: fix env config

* chore: unify the action logic

* chore: optimize type hint

* chore: optimize type hint

* chore: fix type hint

* chore: fix build type error

* chore: support open new tab

* feat: support auto complete element

* chore: add sleep event

* chore: add ai cost time

* chore: optimize prompt

* chore: optimize report prompt

* chore: optimize vlm name

* chore: fix command

* chore: optimize error handle and use check debugger list replace last tab id

* chore: fix chrome debugger attach logic

* chore: fix unit test
2025-01-13 14:32:17 +08:00
github-actions[bot]
d92bfa8987 release: v0.9.1 2025-01-11 05:17:05 +00:00
yuyutaotao
88fd4ed52a
fix: a more friendly error code for extension (#260) 2025-01-10 12:45:10 +08:00
George Lei
fe5f7eed20
feat: add environment variable interpolation to YAML script parser (#226)
* feat: add environment variable interpolation to YAML script parser

* Update environment variable interpolation syntax in YAML utils

---------

Co-authored-by: George Lei <george.lei@lesmills.com>
2025-01-10 12:03:56 +08:00
yuyutaotao
8479bcb652
feat: bridge mode - phase 2 (#257)
* feat: allow auto reconnect in bridge mode after disconnect

* feat: show agent progress in extension

* feat: allow using bridge mode in yaml

* doc: add doc for bridge mode

* doc: add doc for bridge mode

* docs: update screenshot
2025-01-10 10:55:41 +08:00
github-actions[bot]
bc9542d8df release: v0.9.0 2025-01-09 07:06:20 +00:00
yuyutaotao
c91da56431
feat: allow continue-on-error in yaml (#254)
* feat: allow continue on error in yaml #252

* doc: update doc for new param

* fix: ignore yaml file in node-modules
2025-01-08 20:59:54 +08:00
github-actions[bot]
de92f3fc13 release: v0.8.18 2025-01-07 11:53:17 +00:00
yuyutaotao
13069875a1
fix: lazy close for puppeteer (#255) 2025-01-07 19:48:20 +08:00
yuyutaotao
fb222b19db
fix: yaml scripts issue in windows (#251) 2025-01-07 15:30:10 +08:00
github-actions[bot]
4fd930340c release: v0.8.17 2025-01-07 03:54:08 +00:00
yuyutaotao
ae49685348
feat: add bridge mode for extension (#228) 2025-01-07 11:10:28 +08:00
Zhou xiao
691eb6ef0a
feat(ai-model): support Image positioning and integrate langchain (#230)
* feat: add point img logic

* feat: migrate prompt to langchain

* chore: delete unless log

* chore: optimize test case

* chore: fix lint error

* chore: delete httpAgent logic

* chore: delete unless fn

* chore: fix some comment

* chore: fix ci error

* chore: delete unless fn

* chore: update prompt

* chore: delete unless language
2025-01-02 21:23:30 +08:00
Zhou xiao
6af7d21c3d
docs: update the midscene introduction (#236) 2025-01-02 10:58:43 +08:00
github-actions[bot]
1e7c9caf1a release: v0.8.16 2025-01-02 02:38:32 +00:00
yuyutaotao
3c9418c264
fix: use some default param in scroll (#232)
* fix: use some default param in scroll

* fix: update param description
2025-01-02 10:19:22 +08:00
github-actions[bot]
85cb87f1ec release: v0.8.15 2025-01-01 13:15:36 +00:00
github-actions[bot]
3b696033e8 release: v0.8.14 2024-12-31 14:04:23 +00:00
github-actions[bot]
dea3479242 release: v0.8.13 2024-12-27 02:25:47 +00:00
github-actions[bot]
a39684a8ff release: v0.8.12 2024-12-26 05:28:37 +00:00
yuyutaotao
4a82e9bda9
feat: let max_tokens configurable (#212)
* feat: let max_tokens configurable

* fix: update ci test case
2024-12-26 13:24:21 +08:00
github-actions[bot]
471e684a5d release: v0.8.11 2024-12-26 02:47:23 +00:00
github-actions[bot]
461a6a9dff release: v0.8.10 2024-12-25 16:11:46 +00:00