Zhou xiao
56845b75ba
chore: fix planning typo ( #269 )
2025-01-13 14:52:15 +08:00
Zhou xiao
2b18ed55de
feat(ai-model): support vlm ( #262 )
...
* feat(ai-model): support plan to target
* chore: modify test
* chore: modify test
* chore: fix env config
* chore: unify the action logic
* chore: optimize type hint
* chore: optimize type hint
* chore: fix type hint
* chore: fix build type error
* chore: support open new tab
* feat: support auto complete element
* chore: add sleep event
* chore: add ai cost time
* chore: optimize prompt
* chore: optimize report prompt
* chore: optimize vlm name
* chore: fix command
* chore: optimize error handle and use check debugger list replace last tab id
* chore: fix chrome debugger attach logic
* chore: fix unit test
2025-01-13 14:32:17 +08:00
yuyutaotao
8479bcb652
feat: bridge mode - phase 2 ( #257 )
...
* feat: allow auto reconnect in bridge mode after disconnect
* feat: show agent progress in extension
* feat: allow using bridge mode in yaml
* doc: add doc for bridge mode
* doc: add doc for bridge mode
* docs: update screenshot
2025-01-10 10:55:41 +08:00
Zhou xiao
691eb6ef0a
feat(ai-model): support Image positioning and integrate langchain ( #230 )
...
* feat: add point img logic
* feat: migrate prompt to langchain
* chore: delete unless log
* chore: optimize test case
* chore: fix lint error
* chore: delete httpAgent logic
* chore: delete unless fn
* chore: fix some comment
* chore: fix ci error
* chore: delete unless fn
* chore: update prompt
* chore: delete unless language
2025-01-02 21:23:30 +08:00
yuyutaotao
3c9418c264
fix: use some default param in scroll ( #232 )
...
* fix: use some default param in scroll
* fix: update param description
2025-01-02 10:19:22 +08:00
yuyutaotao
21b3574280
fix: fix scroll action in chrome extension ( #194 )
2024-12-23 12:11:10 +08:00
yuyutaotao
f3d46b504d
feat: invoke anthropic SDK to call Claude ( #197 )
...
* feat: invoke anthropic SDK
* chore: set response format for extract
* fix: do not throw if waitUntilNetworkIdle failed in aiAction
* fix: timeout config for Puppeteer
* chore: add instruction for connectivity test
2024-12-23 12:03:05 +08:00
yuyutaotao
523adab12f
feat: support the if-statement in planning prompt ( #184 )
2024-12-19 10:44:08 +08:00
yuyutaotao
537a5c43d0
feat: add a real-time progress indicator for playground ( #177 )
2024-12-16 15:04:21 +08:00
yuyutaotao
082e347da9
feat: replanning before locating ( #167 )
2024-12-08 20:12:17 +08:00
yuyutaotao
e7a58b3b24
faet: support yaml style scripts ( #165 )
2024-11-25 16:05:01 +08:00
Zhou xiao
08d46becf1
feat(cache): optimize cache logic id and export overrideAIConfig ( #148 )
...
* chore(cache): optimize cache logic id and export overrideAIConfig
* chore: use cache logic
2024-11-07 21:06:01 +08:00
yuyutaotao
198172dc4e
feat: optimize the speed of screenshot in browser ( #144 )
...
* feat: optimize the speed of screenshot in browser
* feat: remove unnecessary context call
* fix: CI
2024-11-05 14:28:16 +08:00
yuyutaotao
9e1eef5cfd
feat: Build a chrome extension for playground ( #140 )
2024-11-05 11:49:21 +08:00
Zhou xiao
adb9b58879
feat(ai-model): add claude computer ablity ( #136 )
...
* Add new changes
* Add computer test results and update AI evaluation tests
* chore: Update build outputs and configurations
* feat(ai-model): support claude computer ability use position replace element id
* feat: generate add and commit
* feat: implement computer ability test for Claude
* chore: fix build Lose
* chore: Add and commit changes
* chore: optimize ai position
* chore: optimize ai position
* Add AI evaluation results and update tests
* chore: optimize ai test
* chore: add and commit changes
* chore: optimize ai test content
* chore: fix test case
* chore: fix e2e test
2024-10-31 18:18:31 +08:00
yuyutaotao
c288baa448
feat: make playground working in the browser ( #135 )
...
---------
Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2024-10-28 11:04:40 +08:00
Zhou xiao
3b24df1ec0
workflow(ci): fix ai test error ( #113 )
...
* chore: update element inspector prompt and schema
* test: update snapshots for planning-input tests
2024-10-12 16:01:07 +08:00
yuyutaotao
f9dc0f698e
feat(ai-model): merge ai planning and insight call to accelerate the aiAction ( #97 )
...
---------
Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
* feat(ai-model): optimize AI model for element inspection
* feat(ai-model): optimize AI model and add quick answer functionality
---------
Co-authored-by: yuyutaotao <167746126+yuyutaotao@users.noreply.github.com>
* feat(ai-model): implement quick answer functionality for element inspection
---------
Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
2024-10-12 12:09:25 +08:00
Leyang
43bf165ddc
define appium export in package.json & encapsulated screenshot picture path generation ( #95 )
...
* feat: define appium export in package.json
* feat: encapsulated screenshot picture path generation
2024-09-23 10:57:19 +08:00
Zhou xiao
d481ea4201
feat(model): support azure open ai ( #90 )
...
* feat(model): support open ai azure methods
* chore: fix e2e test
* chore: add OPENAI_USE_AZURE env config
* docs: add openai azure env
2024-09-10 14:29:01 +08:00
Zhou xiao
eafa5bfa20
feat(cache): The cache is generalized to support puppeteers and mobile terminals ( #85 )
...
* feat(cache): The cache is generalized to support Puppeteers and mobile terminals
* chore: update cache test
* chore: update cache test
* chore: update cache test
* docs: update cache doc
* chore: update ai test command
* chore: update ai test command
* chore: update ai test command
* chore: optimize cache logic
* chore: update get dir path logic
* chore: update get dir path logic
2024-09-06 17:19:35 +08:00
Leyang
cfa92b3980
feat(app): supports control of iOS and Android devices through appium ( #82 )
...
Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
Co-authored-by: linyibing <linyibing@bytedance.com>
2024-09-05 20:05:19 +08:00
Zhou xiao
84ee7130b8
feat(web): input supports additions, deletions, and changes ( #78 )
...
* feat(web): input supports additions, deletions, and changes
* chore: optimize unit test content
2024-08-28 19:31:59 +08:00
yuyutaotao
f8fdf60570
fix(web-extract): fix the extractor may fail if there is no <body/> ( #76 )
2024-08-28 19:21:32 +08:00
Zhou xiao
e071adfa70
chore: optimize code style ( #77 )
2024-08-28 14:59:25 +08:00
Zhou xiao
d2a5dbecba
refactor(shared): migrate sharp to jimp and migrate common img logci to shared lib ( #74 )
...
* fix(web): fix sharp deps
* chore: optimize sharp deps
* refactor(extract): migrate sharp to jimp
* refactor: migrate img common logic to shared lib
* chore: merge main branch
* chore: merge main branch
* chore: merge main branch
* chore: delete unless code
* chore: optimize code
* chore: optimize ai test branch trigger method
* chore: optimize ai test branch trigger method
* chore: optimize trigger method
2024-08-26 18:50:33 +08:00
yuyutaotao
6553da1906
feat: add 'aiWaitFor' ( #61 )
...
* feat: add
* feat: add
* feat: add for playwright
* feat: add docs for 'aiWaitFor'
* feat: update docs for report
* feat: add 'wait-for' param in cli
2024-08-21 14:43:35 +08:00
yuyutaotao
af768796df
fix: Cli output, extractor, docs, visualizer ( #48 )
...
* fix: the output for --help
* fix: filter little elements
* feat: update executor and style
* feat: update extractor and docs
2024-08-10 07:57:15 +08:00
Zhou xiao
959473fbbb
feat(ai-model): support coze platfrom's model ( #50 )
...
* chore: optimize prompt structure
* feat(ai-model): add coze platform
* feat(ai-model): add coze platform
* chore: add unit test
* feat(ai-model): add extract unit test
* chore: add unit test
* chore: add unit test
* chore: optimize plan prompt
* chore: optimize plan prompt
* chore: fix lint error
* chore: fix lint error
* chore: fix unit test
* chore: ignore cache file
2024-08-09 21:37:41 +08:00
yuyutaotao
41058a015f
feat: implement the 'sleep' action ( #41 )
2024-08-07 20:03:13 +08:00
yuyutaotao
7edc2be46d
feat: implement .aiAssert, update some docs ( #38 )
...
* feat: implement .aiAssert, update some docs
* fix: lint
* fix: ci
* feat: update quick-start
2024-08-06 10:00:25 +08:00
Zhou xiao
e1ae31ac17
workflow(ci): optimize biome lint ( #31 )
...
* workflow(ci): format code
* workflow(ci): format code
* workflow(ci): format code
* workflow(ci): format code
* workflow(ci): format code
2024-08-04 08:28:19 +08:00
yuyutaotao
3350fc7c1f
feat: update visualizer, and cache info field ( #24 )
...
* fix: page size
* feat: update quick start
* feat: update quick start
* feat: update quick start
* feat: update quick start
* feat: update visualizer
* feat: add cache info into task and dump
* feat: add cache info into task and dump
* fix: typo
* fix: typo
* feat: update docs
* feat: update docs
2024-08-02 13:58:15 +08:00
yuyutaotao
23752c2a95
doc: update API docs ( #17 )
...
* feat: puppeteer integration
* feat: export puppeteer agent
* fix: bug of insight visualizer
* feat: merge main
* feat: update docs, and some tiny bugfix
* feat: update showcase
* feat: update docs
* feat: add Chinese docs
* feat: update visualization tool
* feat: update part of the integration doc
2024-08-01 16:07:58 +08:00
Zhou xiao
cd775aff41
feat(web): added ai cache capability ( #18 )
...
* chore: optimize test case
* chore: optimize test case
* chore: optimize report path
* chore: add task cache logic
* refactor(report): optimize ai report
* chore: save cache logic
* chore: modify cache logic
* chore: modify cache logic
* chore: modify cache logic
* chore: modify unit test case
* chore: modify unit test case
* chore: optimize test case
* chore: add cache logic
* chore: optimize cache logic
* chore: optimize cache logic
* chore: optimize cache logic
* chore: update cache file
* chore: update cache file
* chore: update cache file
* chore: update cache file
* chore: Added cache version determination
* chore: Added cache version determination
* chore: add cache logic
* refactor(web): use hash replace index number id
* chore: add cache logic
* chore: fix unit test snapshot
* chore: update snapshot
* chore: update snapshot
* chore: update snapshot
* chore: update snapshot
* chore: update snapshot
* chore: add cache test
* chore: optimize test command
* chore: optimize cache logic
* chore: optimize cache logic
* chore: update snapshot
* chore: update cache
* chore: delete unless file
* chore: update test data
* chore: update test data
* chore: fix build command
* chore: update cache logic
2024-08-01 15:46:40 +08:00
yuyutaotao
beeebba0fc
feat(web): export puppeteer integration ( #11 )
...
* feat: puppeteer integration
* feat: export puppeteer agent
* fix: bug of insight visualizer
2024-07-28 08:49:57 +08:00
