26 Commits

Author SHA1 Message Date
yuyutaotao
9d5f2fbcac
feat(web-extract): extract web content as a tree (#337)
* feat: extract web content as a tree

* chore: update test data

* chore: update test data

* feat: update answer of evaluation

* chore: update test cases

* chore: remove focusing on cases

* fix: ci

* fix: put rect in html tree

* fix: CI

* fix: AI test

* fix: lint

* fix: CI

* fix: static-page compatibility

* fix: CI

* fix: map by markerId

* fix: llm planning prompt

* chore: update hash length

* chore: ignore writing dump file

* fix: lint

* fix: ci snapshot

* chore: snapshot tree in web extractor

* chore: export tree utils in core

* chore: export tree utils in core

* fix: CI

* fix: update test case and evaluation

* chore: remove unused file

* refactor(extract): modify dependencies (#358)

* refactor(extract): modify dependencies

* chore: modify files config

* chore: add indexId as key for map

---------

Co-authored-by: Zhou Xiao <zhouxiao.shaw@bytedance.com>
2025-02-07 14:55:52 +08:00
github-actions[bot]
8eacc354a7 release: v0.10.3 2025-01-27 08:21:33 +00:00
Zhou Xiao
02ad84cf6f
chore: fix ai test (#332) 2025-01-27 10:22:56 +08:00
github-actions[bot]
e6fc69eb81 release: v0.10.2 2025-01-26 12:39:17 +00:00
Zhou xiao
4c82eff526
chore: fix e2e test (#316)
* chore: fix e2e test

* chore: ignore todo test

* chore: fix e2e test

* chore: upgrade gpt api version

* chore: fix e2e test

* chore: fix e2e test

* chore: fix direction

* chore: fix direction

* chore: split test

* chore: split test

* chore: modify input info

* fix: scroll param in ci

* fix: lint

* fix: ai test

* fix: ai test

---------

Co-authored-by: yutao <yutao.tao@bytedance.com>
2025-01-24 15:02:50 +08:00
github-actions[bot]
0f92941f54 release: v0.10.1 2025-01-23 07:15:56 +00:00
yuyutaotao
57f6786628
feat(extract-data): extract data from same-origin iframe (#258)
* feat: extract data from same-origin iframe

* fix: ci snapshot

* fix: extracting timeout error

* fix: ci timeout

* fix: update assets

* feat: set default size of yaml as 1920x1080

* chore: update default viewport size

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2025-01-22 20:52:11 +08:00
github-actions[bot]
d580bd02fe release: v0.9.2 2025-01-15 03:22:10 +00:00
Zhou xiao
2b18ed55de
feat(ai-model): support vlm (#262)
* feat(ai-model): support plan to target

* chore: modify test

* chore: modify test

* chore: fix env config

* chore: unify the action logic

* chore: optimize type hint

* chore: optimize type hint

* chore: fix type hint

* chore: fix build type error

* chore: support open new tab

* feat: support auto complete element

* chore: add sleep event

* chore: add ai cost time

* chore: optimize prompt

* chore: optimize report prompt

* chore: optimize vlm name

* chore: fix command

* chore: optimize error handle and use check debugger list replace last tab id

* chore: fix chrome debugger attach logic

* chore: fix unit test
2025-01-13 14:32:17 +08:00
github-actions[bot]
3dc57eef32 release: v0.8.6 2024-12-16 07:41:16 +00:00
yuyutaotao
537a5c43d0
feat: add a real-time progress indicator for playground (#177) 2024-12-16 15:04:21 +08:00
github-actions[bot]
91896c421a release: v0.8.5 2024-12-09 09:11:52 +00:00
yuyutaotao
4282e7bf07
feat: lower the cost of page description (#159) 2024-11-20 16:00:34 +08:00
github-actions[bot]
4972d22ec4 release: v0.8.1 2024-11-15 08:41:21 +00:00
yuyutaotao
26779cf082
fix: fix some extraction bug in extension (#157) 2024-11-14 19:29:22 +08:00
github-actions[bot]
3117584e57 release: v0.8.0 2024-11-11 10:39:38 +00:00
yuyutaotao
c288baa448
feat: make playground working in the browser (#135)
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2024-10-28 11:04:40 +08:00
github-actions[bot]
ab815fef56 release: v0.7.0 2024-10-21 08:34:43 +00:00
yuyutaotao
7d4f9847f5
feat: add a playground page to debug (#131)
* fix: fix the Player style when the locator is failed

* feat: add Playground UI

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2024-10-21 16:30:07 +08:00
github-actions[bot]
e2f5e114a7 release: v0.5.2 2024-10-12 06:37:01 +00:00
yuyutaotao
f9dc0f698e
feat(ai-model): merge ai planning and insight call to accelerate the aiAction (#97)
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>

* feat(ai-model): optimize AI model for element inspection

* feat(ai-model): optimize AI model and add quick answer functionality

---------

Co-authored-by: yuyutaotao <167746126+yuyutaotao@users.noreply.github.com>

* feat(ai-model): implement quick answer functionality for element inspection

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2024-10-12 12:09:25 +08:00
Zhou xiao
10757a8ba1
refacotr(ai-model): optimize model evalution method (#98) 2024-09-29 17:16:07 +08:00
github-actions[bot]
649d71d2ec release: v0.5.0 2024-09-12 03:31:37 +00:00
yuyutaotao
5bd49852b8
feat(web-extract): collect svg elements on page, trim attribute values to save token (#83) 2024-09-10 07:04:27 +08:00
yuyutaotao
30dc7a3c22
chore(web): merge the Page implementation of Puppeteer and Playwright (#87) 2024-09-09 15:57:36 +08:00
Leyang
cfa92b3980
feat(app): supports control of iOS and Android devices through appium (#82)
Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
Co-authored-by: linyibing <linyibing@bytedance.com>
2024-09-05 20:05:19 +08:00