mirror of
https://github.com/web-infra-dev/midscene.git
synced 2025-07-24 09:21:03 +00:00
2589a9c4ca
* docs: release android automation * chore(docs): update doubao docs * chore(docs): merge docs for doubao * docs(android): update * docs(site): add more android case * docs(site): update slogan and authors * docs(site): android yaml * docs(core): instruction for override config * docs(core): update readme * Update README.md * docs(core): update readme * docs(core): update readme * docs(core): update readme * docs(core): update readme * docs(core): update README and blog for Android automation support * docs(core): update android playground doc * docs(core): enhance Android integration documentation with setup instructions * docs(core): update android playground doc * docs(core): update Android integration documentation and add setup instructions * docs(core): update bridge mode title * docs(core): update yaml docs * docs(site): chore update * docs(site): update YAML documentation with setup instructions and clarify parameters * docs(core): update instructions * chore: update docs * chore: update bridge mode docs * docs(site): translate to zh * docs(site): translate error * docs(site): remove unnecessary code block in YAML automation documentation * docs(core): update blog * docs(core): update instructions * docs(core): update instructions --------- Co-authored-by: yutao <yutao.tao@bytedance.com> Co-authored-by: yuyutaotao <167746126+yuyutaotao@users.noreply.github.com>
111 lines
3.3 KiB
Plaintext
111 lines
3.3 KiB
Plaintext
import SetupEnv from './common/setup-env.mdx';
|
|
|
|
# Integrate with Puppeteer
|
|
|
|
import { PackageManagerTabs } from '@theme';
|
|
|
|
[Puppeteer](https://pptr.dev/) is a Node.js library which provides a high-level API to control Chrome or Firefox over the DevTools Protocol or WebDriver BiDi. Puppeteer runs in the headless (no visible UI) by default but can be configured to run in a visible ("headful") browser.
|
|
|
|
:::info Demo Project
|
|
you can check the demo project of Puppeteer here: [https://github.com/web-infra-dev/midscene-example/blob/main/puppeteer-demo](https://github.com/web-infra-dev/midscene-example/blob/main/puppeteer-demo)
|
|
|
|
There is also a demo of Puppeteer with Vitest: [https://github.com/web-infra-dev/midscene-example/tree/main/puppeteer-with-vitest-demo](https://github.com/web-infra-dev/midscene-example/tree/main/puppeteer-with-vitest-demo)
|
|
:::
|
|
|
|
<SetupEnv />
|
|
|
|
## Step 1. install dependencies
|
|
|
|
<PackageManagerTabs command="install @midscene/web puppeteer tsx --save-dev" />
|
|
|
|
## Step 2. write scripts
|
|
|
|
Write and save the following code as `./demo.ts`.
|
|
|
|
```typescript title="./demo.ts"
|
|
import puppeteer from "puppeteer";
|
|
import { PuppeteerAgent } from "@midscene/web/puppeteer";
|
|
|
|
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
|
|
Promise.resolve(
|
|
(async () => {
|
|
const browser = await puppeteer.launch({
|
|
headless: false, // here we use headed mode to help debug
|
|
});
|
|
|
|
const page = await browser.newPage();
|
|
await page.setViewport({
|
|
width: 1280,
|
|
height: 800,
|
|
deviceScaleFactor: 1,
|
|
});
|
|
|
|
await page.goto("https://www.ebay.com");
|
|
await sleep(5000);
|
|
|
|
// 👀 init Midscene agent
|
|
const agent = new PuppeteerAgent(page);
|
|
|
|
// 👀 type keywords, perform a search
|
|
await agent.aiAction('type "Headphones" in search box, hit Enter');
|
|
await sleep(5000);
|
|
|
|
// 👀 understand the page content, find the items
|
|
const items = await agent.aiQuery(
|
|
"{itemTitle: string, price: Number}[], find item in list and corresponding price"
|
|
);
|
|
console.log("headphones in stock", items);
|
|
|
|
// 👀 assert by AI
|
|
await agent.aiAssert("There is a category filter on the left");
|
|
|
|
await browser.close();
|
|
})()
|
|
);
|
|
```
|
|
|
|
|
|
## Step 3. run
|
|
|
|
Using `tsx` to run, you will get the data of Headphones on eBay:
|
|
|
|
```bash
|
|
# run
|
|
npx tsx demo.ts
|
|
|
|
# it should print
|
|
# [
|
|
# {
|
|
# itemTitle: 'Beats by Dr. Dre Studio Buds Totally Wireless Noise Cancelling In Ear + OPEN BOX',
|
|
# price: 505.15
|
|
# },
|
|
# {
|
|
# itemTitle: 'Skullcandy Indy Truly Wireless Earbuds-Headphones Green Mint',
|
|
# price: 186.69
|
|
# }
|
|
# ]
|
|
```
|
|
|
|
For the agent's more APIs, please refer to [API](./API).
|
|
|
|
## Step 4: view the report
|
|
|
|
After the above command executes successfully, the console will output: `Midscene - report file updated: /path/to/report/some_id.html`. You can open this file in a browser to view the report.
|
|
|
|
## More options in PuppeteerAgent constructor
|
|
|
|
### To limit the popup to the current page
|
|
|
|
If you want to limit the popup to the current page (like clicking a link with `target="_blank"`), you can set the `forceSameTabNavigation` option to `true`:
|
|
|
|
```typescript
|
|
const mid = new PuppeteerAgent(page, {
|
|
forceSameTabNavigation: true,
|
|
});
|
|
```
|
|
|
|
## More
|
|
|
|
* For all the APIs on the Agent, please refer to [API Reference](./API).
|
|
* For more details about prompting, please refer to [Prompting Tips](./prompting-tips)
|