156 Commits

Author SHA1 Message Date
Zhou xiao
3b24df1ec0
workflow(ci): fix ai test error (#113)
* chore: update element inspector prompt and schema

* test: update snapshots for planning-input tests
2024-10-12 16:01:07 +08:00
github-actions[bot]
e2f5e114a7 release: v0.5.2 2024-10-12 06:37:01 +00:00
yuyutaotao
f9dc0f698e
feat(ai-model): merge ai planning and insight call to accelerate the aiAction (#97)
---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>

* feat(ai-model): optimize AI model for element inspection

* feat(ai-model): optimize AI model and add quick answer functionality

---------

Co-authored-by: yuyutaotao <167746126+yuyutaotao@users.noreply.github.com>

* feat(ai-model): implement quick answer functionality for element inspection

---------

Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2024-10-12 12:09:25 +08:00
Zhou xiao
8ec5622342
chore: optimize ai e2e test (#99) 2024-09-29 17:51:59 +08:00
Zhou xiao
10757a8ba1
refacotr(ai-model): optimize model evalution method (#98) 2024-09-29 17:16:07 +08:00
Leyang
43bf165ddc
define appium export in package.json & encapsulated screenshot picture path generation (#95)
* feat: define appium export in package.json

* feat: encapsulated screenshot picture path generation
2024-09-23 10:57:19 +08:00
github-actions[bot]
649d71d2ec release: v0.5.0 2024-09-12 03:31:37 +00:00
Zhou xiao
6b6f37a764
workflow(ci): fix ai e2e test (#94) 2024-09-12 11:03:42 +08:00
yuyutaotao
a2e4ff61b0
fix(cache): app should not crash when it fails to get the cache id (#93) 2024-09-11 10:22:57 +08:00
Zhou xiao
d481ea4201
feat(model): support azure open ai (#90)
* feat(model): support open ai azure methods

* chore: fix e2e test

* chore: add OPENAI_USE_AZURE env config

* docs: add openai azure env
2024-09-10 14:29:01 +08:00
yuyutaotao
9e29edc0c6
fix: type error in AI test (#91) 2024-09-10 11:21:44 +08:00
yuyutaotao
5bd49852b8
feat(web-extract): collect svg elements on page, trim attribute values to save token (#83) 2024-09-10 07:04:27 +08:00
yuyutaotao
7d9792c651
fix(ci): fix build error for Cli, remove used package (#89) 2024-09-09 18:07:22 +08:00
yuyutaotao
30dc7a3c22
chore(web): merge the Page implementation of Puppeteer and Playwright (#87) 2024-09-09 15:57:36 +08:00
Zhou xiao
eafa5bfa20
feat(cache): The cache is generalized to support puppeteers and mobile terminals (#85)
* feat(cache): The cache is generalized to support Puppeteers and mobile terminals

* chore: update cache test

* chore: update cache test

* chore: update cache test

* docs: update cache doc

* chore: update ai test command

* chore: update ai test command

* chore: update ai test command

* chore: optimize cache logic

* chore: update get dir path logic

* chore: update get dir path logic
2024-09-06 17:19:35 +08:00
Leyang
cfa92b3980
feat(app): supports control of iOS and Android devices through appium (#82)
Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
Co-authored-by: linyibing <linyibing@bytedance.com>
2024-09-05 20:05:19 +08:00
Zhou xiao
1c6237ae46
chore: delete unless debug code and fix ci cache logic (#84) 2024-09-02 14:09:56 +08:00
yuyutaotao
704a0b8a52
feat(web-extract): extract some <div />s as container (#80) 2024-08-31 08:17:50 +08:00
Zhou xiao
3769a559db
refactor(plan): optimize input prompt (#81) 2024-08-29 14:59:03 +08:00
Zhou xiao
3ad8e9cea2
fix(ai-plan): optimize KeyboardPress plan type reduce hallucinations (#79) 2024-08-29 14:34:07 +08:00
Zhou xiao
84ee7130b8
feat(web): input supports additions, deletions, and changes (#78)
* feat(web): input supports additions, deletions, and changes

* chore: optimize unit test content
2024-08-28 19:31:59 +08:00
yuyutaotao
f8fdf60570
fix(web-extract): fix the extractor may fail if there is no <body/> (#76) 2024-08-28 19:21:32 +08:00
github-actions[bot]
952d13e8b0 release: v0.4.0 2024-08-27 00:29:53 +00:00
Zhou xiao
9a8be3e879
workflow(ci): optimize ci command (#75) 2024-08-27 08:18:33 +08:00
Zhou xiao
d2a5dbecba
refactor(shared): migrate sharp to jimp and migrate common img logci to shared lib (#74)
* fix(web): fix sharp deps

* chore: optimize sharp deps

* refactor(extract): migrate sharp to jimp

* refactor: migrate img common logic to shared lib

* chore: merge main branch

* chore: merge main branch

* chore: merge main branch

* chore: delete unless code

* chore: optimize code

* chore: optimize ai test branch trigger method

* chore: optimize ai test branch trigger method

* chore: optimize trigger method
2024-08-26 18:50:33 +08:00
yuyutaotao
52bda878aa
fix(web-extract): fix the extractor for form item like <input /> (#65) 2024-08-26 11:09:39 +08:00
yuyutaotao
64244e871b
fix: the flashing issue in cli headed mode (#73) 2024-08-26 11:08:50 +08:00
github-actions[bot]
0be5bbcc54 release: v0.3.2 2024-08-23 03:02:14 +00:00
Zhou xiao
2b4beaa1eb
chore: update e2e cache (#70) 2024-08-23 10:57:08 +08:00
Zhou xiao
92076288d0
workflow(ci): add playwirght cache logic (#68)
* workflow(ci): add playwirght cache logic

* chore: optimize pr-labler github action
2024-08-23 08:28:25 +08:00
github-actions[bot]
5e79f960e8 release: v0.3.1 2024-08-22 23:05:02 +00:00
Zhou xiao
65367fa25a
workflow(ci): optimize ai e2e test (#67) 2024-08-22 20:56:34 +08:00
Zhou xiao
eb82d28082
feat(playwright): optimize the visualization and extraction of playwright ai process data (#66) 2024-08-22 18:12:01 +08:00
Zhou xiao
8815edfee5
workflow(ci): optimize web extract unit test (#64) 2024-08-21 17:24:32 +08:00
Zhou xiao
2d1b326007
chore: optimize generate code (#63) 2024-08-21 16:28:58 +08:00
Zhou xiao
bc85f7cd2d
fix(html-extract): optimize the logic for extracting HTML content to improve extraction accuracy (#62)
* chore: optimize extract logic

* chore: fix word type error

* chore: optimize build error

* chore: optimize generate test data

* chore: optimize generate test data
2024-08-21 15:52:46 +08:00
yuyutaotao
6553da1906
feat: add 'aiWaitFor' (#61)
* feat: add

* feat: add

* feat: add  for playwright

* feat: add docs for 'aiWaitFor'

* feat: update docs for report

* feat: add 'wait-for' param in cli
2024-08-21 14:43:35 +08:00
Zhou xiao
b07f83dee1
workflow(ci): optimize ai unit test (#56) 2024-08-20 07:41:08 +08:00
yuyutaotao
9c5a7a123c
feat(report): optimize report content (#55)
Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2024-08-15 17:59:43 +08:00
yuyutaotao
49d19371b3
fix: scroll issue of detail panel, exclude <scripts /> from extractor (#51)
* fix: scroll issue of detail panel

* fix: CI

* fix: extractor and visualizer

* fix: lint

* fix: lint

* fix: extractor ci

* feat: update highlight switch
2024-08-15 09:54:35 +08:00
Zhou xiao
18cafeab9b
workflow(cd): optimize release script (#54) 2024-08-12 18:04:40 +08:00
yuyutaotao
af768796df
fix: Cli output, extractor, docs, visualizer (#48)
* fix: the output for --help

* fix: filter little elements

* feat: update executor and style

* feat: update extractor and docs
2024-08-10 07:57:15 +08:00
yuyutaotao
41058a015f
feat: implement the 'sleep' action (#41) 2024-08-07 20:03:13 +08:00
yuyutaotao
7edc2be46d
feat: implement .aiAssert, update some docs (#38)
* feat: implement .aiAssert, update some docs

* fix: lint

* fix: ci

* feat: update quick-start
2024-08-06 10:00:25 +08:00
Zhou xiao
3037cd1c27
docs: optimize words (#37)
* docs: optimize words

* docs: lint content
2024-08-05 20:46:49 +08:00
Zhou xiao
e030f63db9
docs: add cache introduce (#36) 2024-08-05 16:27:02 +08:00
Zhou xiao
e1ae31ac17
workflow(ci): optimize biome lint (#31)
* workflow(ci): format code

* workflow(ci): format code

* workflow(ci): format code

* workflow(ci): format code

* workflow(ci): format code
2024-08-04 08:28:19 +08:00
yuyutaotao
3350fc7c1f
feat: update visualizer, and cache info field (#24)
* fix: page size

* feat: update quick start

* feat: update quick start

* feat: update quick start

* feat: update quick start

* feat: update visualizer

* feat: add cache info into task and dump

* feat: add cache info into task and dump

* fix: typo

* fix: typo

* feat: update docs

* feat: update docs
2024-08-02 13:58:15 +08:00
Zhou xiao
1c15fd691a
fix(web-report): modified the report generation command to fix the report generation failure (#20)
* chore: update report logic

* chore: update report logic

* chore: update report logic

* chore: update report logic
2024-08-01 19:28:30 +08:00
yuyutaotao
23752c2a95
doc: update API docs (#17)
* feat: puppeteer integration

* feat: export puppeteer agent

* fix: bug of insight visualizer

* feat: merge main

* feat: update docs, and some tiny bugfix

* feat: update showcase

* feat: update docs

* feat: add Chinese docs

* feat: update visualization tool

* feat: update part of the integration doc
2024-08-01 16:07:58 +08:00