<ahref="https://huggingface.co/ByteDance-Seed/UI-TARS-1.5-7B"><imgsrc="https://img.shields.io/badge/%F0%9F%A4%97-UI%20TARS%20Models-yellow"alt="hugging face model"/></a>
| Use JS code to drive task orchestration, collect information about Jay Chou's concert, and write it into Google Docs (By UI-TARS model) | <videosrc="https://github.com/user-attachments/assets/75474138-f51f-4c54-b3cf-46d61d059999"height="300"/> |
| Control Maps App on Android (By Qwen-2.5-VL model) | <videosrc="https://github.com/user-attachments/assets/1f5bab0e-4c28-44e1-b378-a38809b05a00"height="300"/> |
| Using midscene mcp to browse the page (https://www.saucedemo.com/), perform login, add products, place orders, and finally generate test cases based on mcp execution steps and playwright example | <videosrc="https://github.com/user-attachments/assets/a95ca353-e50c-4091-85ba-e542f576b6be"height="300"/> |
- **Web Automation 🖥️**: Either integrate with [Puppeteer](https://midscenejs.com/integrate-with-puppeteer.html), [Playwright](https://midscenejs.com/integrate-with-playwright.html) or use [Bridge Mode](https://midscenejs.com/bridge-mode-by-chrome-extension.html) to control your desktop browser.
- **Android Automation 📱**: Use [Javascript SDK](https://midscenejs.com/integrate-with-android.html) with adb to control your local Android device.
- [**Caching for Efficiency 🔄**](https://midscenejs.com/caching.html): Replay your script with cache and get the result faster.
- [**MCP 🔗**](https://midscenejs.com/mcp.html): Allows other MCP Clients to directly use Midscene's capabilities.
### Three kinds of APIs
- [Interaction API 🔗](https://midscenejs.com/api.html#interaction-methods): interact with the user interface.
- [Data Extraction API 🔗](https://midscenejs.com/api.html#data-extraction): extract data from the user interface and dom.
- [Utility API 🔗](https://midscenejs.com/api.html#more-apis): utility functions like `aiAssert()`, `aiLocate()`, `aiWaitFor()`.
## 👉 Zero-code Quick Experience
- **[Chrome Extension](https://midscenejs.com/quick-experience.html)**: Start in-browser experience immediately through [the Chrome Extension](https://midscenejs.com/quick-experience.html), without writing any code.
- **[Android Playground](https://midscenejs.com/quick-experience-with-android.html)**: There is also a built-in Android playground to control your local Android device.
Midscene.js supports both multimodal LLMs like `gpt-4o`, and visual-language models like `Qwen2.5-VL`, `Doubao-1.5-thinking-vision-pro`, `gemini-2.5-pro` and `UI-TARS`.
Visual-language models are recommended for UI automation.
Midscene will automatically plan the steps and execute them. It may be slower and heavily rely on the quality of the AI model.
```javascript
await aiAction('click all the records one by one. If one record contains the text "completed", skip it');
```
### Workflow Style
Split complex logic into multiple steps to improve the stability of the automation code.
```javascript
const recordList = await agent.aiQuery('string[], the record list')
for (const record of recordList) {
const hasCompleted = await agent.aiBoolean(`check if the record contains the text "completed"`)
if (!hasCompleted) {
await agent.aiTap(record)
}
}
```
> For more details about the workflow style, please refer to [Blog - Use JavaScript to Optimize the AI Automation Code](https://midscenejs.com/blog-programming-practice-using-structured-api.html)
* **Debugging Experience**: You will soon realize that debugging and maintaining automation scripts is the real challenge. No matter how magical the demo looks, ensuring stability over time requires careful debugging. Midscene.js offers a visualized report file, a built-in playground, and a Chrome Extension to simplify the debugging process. These are the tools most developers truly need, and we're continually working to improve the debugging experience.
* **Open Source, Free, Deploy as you want**: Midscene.js is an open-source project. It's decoupled from any cloud service and model provider, you can choose either public or private deployment. There is always a suitable plan for your business.
