170 Commits

Author SHA1 Message Date
yuyutaotao
3aa1b33955
feat: use jpeg as default image format (#301)
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-01-20 20:02:49 +08:00
github-actions[bot]
9b86067ea6 release: v0.10.0 2025-01-17 10:31:04 +00:00
yuyutaotao
a114e707d1
feat: show pointer position in chrome extension (#286)
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-01-17 18:19:22 +08:00
yuyutaotao
f68b6f3139
feat(chrome-devtool): allow tracking active tab in bridge mode (#282)
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-01-15 19:54:03 +08:00
yuyutaotao
fb2b9d123f
feat(chrome-devtool): add 'stop' button in extension (#281)
* feat: add 'stop' to playground

* feat: make extension stopable

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-01-15 19:18:26 +08:00
Zhou xiao
ff9ef70941
chore(prompt): switch thought language according to timezone in vlm mode (#283)
* chore: switch Thought Language to english

* chore: use timezone to select language
2025-01-15 18:54:23 +08:00
github-actions[bot]
d580bd02fe release: v0.9.2 2025-01-15 03:22:10 +00:00
Zhou xiao
beb74f12e8
chore: optimize devtool tab connect (#279) 2025-01-14 21:31:33 +08:00
Zhou xiao
cb52da2ffd
chore: Avoid obscuring the entire page content while optimizing the execution process of DevTools (#276) 2025-01-14 17:05:56 +08:00
zhouxiao.shaw
ea4ee5f976 chore: enhance the execution spped of chrome devtools 2025-01-14 15:04:24 +08:00
Zhou xiao
0cf9a71940
feat(chrome-devtool): water flow animation on the target page (#271)
* feat(chrome-devtool): support water flow animation

* chore: optimize debugger cause window resize

* chore: optimize dynamic script inject

* chore: optimize report error
2025-01-14 14:13:50 +08:00
yuyutaotao
857c411d67
feat: allow tracking newly-opened tabs in Chrome extension (#272) 2025-01-14 11:22:20 +08:00
Zhou xiao
56845b75ba
chore: fix planning typo (#269) 2025-01-13 14:52:15 +08:00
Zhou xiao
2b18ed55de
feat(ai-model): support vlm (#262)
* feat(ai-model): support plan to target

* chore: modify test

* chore: modify test

* chore: fix env config

* chore: unify the action logic

* chore: optimize type hint

* chore: optimize type hint

* chore: fix type hint

* chore: fix build type error

* chore: support open new tab

* feat: support auto complete element

* chore: add sleep event

* chore: add ai cost time

* chore: optimize prompt

* chore: optimize report prompt

* chore: optimize vlm name

* chore: fix command

* chore: optimize error handle and use check debugger list replace last tab id

* chore: fix chrome debugger attach logic

* chore: fix unit test
2025-01-13 14:32:17 +08:00
github-actions[bot]
d92bfa8987 release: v0.9.1 2025-01-11 05:17:05 +00:00
yuyutaotao
88fd4ed52a
fix: a more friendly error code for extension (#260) 2025-01-10 12:45:10 +08:00
George Lei
fe5f7eed20
feat: add environment variable interpolation to YAML script parser (#226)
* feat: add environment variable interpolation to YAML script parser

* Update environment variable interpolation syntax in YAML utils

---------

Co-authored-by: George Lei <george.lei@lesmills.com>
2025-01-10 12:03:56 +08:00
yuyutaotao
8479bcb652
feat: bridge mode - phase 2 (#257)
* feat: allow auto reconnect in bridge mode after disconnect

* feat: show agent progress in extension

* feat: allow using bridge mode in yaml

* doc: add doc for bridge mode

* doc: add doc for bridge mode

* docs: update screenshot
2025-01-10 10:55:41 +08:00
github-actions[bot]
bc9542d8df release: v0.9.0 2025-01-09 07:06:20 +00:00
yuyutaotao
c91da56431
feat: allow continue-on-error in yaml (#254)
* feat: allow continue on error in yaml #252

* doc: update doc for new param

* fix: ignore yaml file in node-modules
2025-01-08 20:59:54 +08:00
github-actions[bot]
de92f3fc13 release: v0.8.18 2025-01-07 11:53:17 +00:00
yuyutaotao
13069875a1
fix: lazy close for puppeteer (#255) 2025-01-07 19:48:20 +08:00
yuyutaotao
fb222b19db
fix: yaml scripts issue in windows (#251) 2025-01-07 15:30:10 +08:00
github-actions[bot]
4fd930340c release: v0.8.17 2025-01-07 03:54:08 +00:00
yuyutaotao
ae49685348
feat: add bridge mode for extension (#228) 2025-01-07 11:10:28 +08:00
Zhou xiao
691eb6ef0a
feat(ai-model): support Image positioning and integrate langchain (#230)
* feat: add point img logic

* feat: migrate prompt to langchain

* chore: delete unless log

* chore: optimize test case

* chore: fix lint error

* chore: delete httpAgent logic

* chore: delete unless fn

* chore: fix some comment

* chore: fix ci error

* chore: delete unless fn

* chore: update prompt

* chore: delete unless language
2025-01-02 21:23:30 +08:00
Zhou xiao
6af7d21c3d
docs: update the midscene introduction (#236) 2025-01-02 10:58:43 +08:00
github-actions[bot]
1e7c9caf1a release: v0.8.16 2025-01-02 02:38:32 +00:00
yuyutaotao
3c9418c264
fix: use some default param in scroll (#232)
* fix: use some default param in scroll

* fix: update param description
2025-01-02 10:19:22 +08:00
github-actions[bot]
85cb87f1ec release: v0.8.15 2025-01-01 13:15:36 +00:00
github-actions[bot]
3b696033e8 release: v0.8.14 2024-12-31 14:04:23 +00:00
github-actions[bot]
dea3479242 release: v0.8.13 2024-12-27 02:25:47 +00:00
github-actions[bot]
a39684a8ff release: v0.8.12 2024-12-26 05:28:37 +00:00
yuyutaotao
4a82e9bda9
feat: let max_tokens configurable (#212)
* feat: let max_tokens configurable

* fix: update ci test case
2024-12-26 13:24:21 +08:00
github-actions[bot]
471e684a5d release: v0.8.11 2024-12-26 02:47:23 +00:00
github-actions[bot]
461a6a9dff release: v0.8.10 2024-12-25 16:11:46 +00:00
yuyutaotao
cd00680b76
fix: make puppeteer external in web-integration (#208)
* fix: make puppeteer external in web-integration

* fix: import path of yaml player

* fix: import path of yaml player
2024-12-25 23:53:11 +08:00
yuyutaotao
05dad23bfd
chore: print yaml file in chrome extension (#205) 2024-12-25 20:23:12 +08:00
github-actions[bot]
a5066d2f58 release: v0.8.9 2024-12-24 14:45:57 +00:00
github-actions[bot]
1e76905b5f release: v0.8.8 2024-12-23 07:08:16 +00:00
yuyutaotao
e6343b63f4
fix: keypress issue in chrome extension (#201)
* fix: keypress issue in chrome extension

* fix: keypress issue in chrome extension

* fix: connectivity

* doc: update readme
2024-12-23 14:38:07 +08:00
yuyutaotao
21b3574280
fix: fix scroll action in chrome extension (#194) 2024-12-23 12:11:10 +08:00
yuyutaotao
f3d46b504d
feat: invoke anthropic SDK to call Claude (#197)
* feat: invoke anthropic SDK

* chore: set response format for extract

* fix: do not throw if waitUntilNetworkIdle failed in aiAction

* fix: timeout config for Puppeteer

* chore: add instruction for connectivity test
2024-12-23 12:03:05 +08:00
github-actions[bot]
48fa8a56a8 release: v0.8.7 2024-12-19 08:27:51 +00:00
yuyutaotao
523adab12f
feat: support the if-statement in planning prompt (#184) 2024-12-19 10:44:08 +08:00
github-actions[bot]
3dc57eef32 release: v0.8.6 2024-12-16 07:41:16 +00:00
yuyutaotao
537a5c43d0
feat: add a real-time progress indicator for playground (#177) 2024-12-16 15:04:21 +08:00
yuyutaotao
863e5a54d0
fix: sandbox issue of puppeteer in Ubuntu (#176)
* fix: disable setuid sandbox in cli

* fix: disable setuid sandbox in cli
2024-12-10 09:24:43 +08:00
github-actions[bot]
91896c421a release: v0.8.5 2024-12-09 09:11:52 +00:00
yuyutaotao
082e347da9
feat: replanning before locating (#167) 2024-12-08 20:12:17 +08:00