feat(chrome-devtool): allow longer connection of chrome bridge (#416)

apps/site/docs/en/automate-with-scripts-in-yaml.mdx

@@ -153,6 +153,9 @@ target:
 
   # string, the bridge mode to use, optional, default is false, can be 'newTabWithUrl' or 'currentTab'. More details see the following section
   bridgeMode: false | 'newTabWithUrl' | 'currentTab'
+
+  # boolean, if close the new tabs after the bridge is disconnected, optional, default is false
+  closeNewTabsAfterDisconnect: <boolean>
 ```
 
 The `tasks` part is an array indicates the tasks to do. Remember to write a `-` before each item which means an array item.
@@ -209,6 +212,14 @@ You can use the environment variable in the `.yaml` file like this:
 
 By using bridge mode, you can utilize YAML scripts to automate the web browser on your desktop. This is particularly useful if you want to reuse cookies, plugins, and page states, or if you want to manually interact with automation scripts.
 
+To use bridge mode, you should install the Chrome extension first, and use this configuration in the `target` section:
+
+```diff
+target:
+  url: https://www.bing.com
++ bridgeMode: newTabWithUrl
+```
+
 See [Bridge Mode by Chrome Extension](./bridge-mode-by-chrome-extension) for more details.
 
 ## Run yaml script with javascript

apps/site/docs/en/bridge-mode-by-chrome-extension.mdx

@@ -14,7 +14,7 @@ you can check the demo project of bridge mode here: [https://github.com/web-infr
 
 ## Preparation
 
-1. Config the OpenAI API key, or [config model and provider](./model-provider)
+1. Config the OpenAI API key, or [Choose a model](./choose-a-model)
 
 ```bash
 # replace with your own
@@ -69,6 +69,18 @@ After executing the script, you should see the status of the Chrome extension sw
 ⁠Whether the scripts are run before or after clicking 'Allow connection' in the browser is not significant.
 :::
 
+## Constructor
+
+```typescript
+import { AgentOverChromeBridge } from "@midscene/web/bridge-mode";
+
+const agent = new AgentOverChromeBridge();
+```
+
+Except [the normal parameters in the agent constructor](./api), `AgentOverChromeBridge` provides one more parameter:
+
+* `closeNewTabsAfterDisconnect?: boolean`: If true, the newly created tab will be closed when the bridge is destroyed. Default is false.
+
 ## API
 
 Except [the normal agent interface](./api), `AgentOverChromeBridge` provides some other interfaces to control the desktop Chrome.
@@ -140,16 +152,19 @@ await agent.connectNewTabWithUrl(
 );
 ```
 
-### `destroy`
+### `destroy()`
 
 Destroy the connection and release resources.
 
 * Type
 
 ```typescript
-function destroy(): Promise<void>;
+function destroy(closeNewTabsAfterDisconnect?: boolean): Promise<void>;
 ```
 
+* Parameters:
+  * `closeNewTabsAfterDisconnect?: boolean` - If true, the newly created tab will be closed when the bridge is destroyed. Default is false. The will override the `closeNewTabsAfterDisconnect` parameter in the constructor.
+
 * Returns:
   * Returns a Promise that resolves to void when destruction completes
 
@@ -167,12 +182,15 @@ await agent.destroy();
 
 To use bridge mode in yaml script, set the `bridgeMode` property in the `target` section. If you want to use the current tab, set it to `currentTab`, otherwise set it to `newTabWithUrl`.
 
+Set `closeNewTabsAfterDisconnect` to true if you want to close the newly created tabs when the bridge is destroyed. This is optional and the default value is false.
+
 For example, the following script will open a new tab by Chrome extension bridge: 
 
 ```diff
 target:
   url: https://www.bing.com
 + bridgeMode: newTabWithUrl
++ closeNewTabsAfterDisconnect: true
 tasks:
 ```
 

apps/site/docs/en/quick-experience.mdx

@@ -44,7 +44,7 @@ After experiencing, you may want to write some code to integrate Midscene. There
 
 ## FAQ
 
-* Extension fails to run and shows 'Cannot access a chrome-extension:// URL of different extension'
+### Extension fails to run and shows 'Cannot access a chrome-extension:// URL of different extension'
 
 It's mainly due to conflicts with other extensions injecting `<iframe />` or `<script />` into the page. Try disabling the suspicious plugins and refresh. 
 

apps/site/docs/zh/automate-with-scripts-in-yaml.mdx

@@ -153,6 +153,9 @@ target:
 
   # 桥接模式，可选，默认 false，可以为 'newTabWithUrl' 或 'currentTab'。更多详情请参阅后文
   bridgeMode: false | 'newTabWithUrl' | 'currentTab'
+
+  # 是否在桥接断开时关闭新创建的标签页，可选，默认 false
+  closeNewTabsAfterDisconnect: <boolean>
 ```
 
 `tasks` 部分是一个数组，定义了脚本执行的步骤。记得在每个步骤前添加 `-` 符号。
@@ -207,7 +210,15 @@ topic=weather today
 
 ## 使用桥接模式
 
-通过使用桥接模式，你可以利用 YAML 脚本在已有的桌面浏览器上执行自动化。这对于需要复用 Cookies、插件和页面状态，或者需要手工与自动化脚本交互的情况非常有用。
+通过使用桥接模式，你可以利用 YAML 脚本在已有的桌面浏览器上执行自动化。这对于需要复用 Cookies、插件和页面状态，或者需要人工与自动化脚本交互的情况非常有用。
+
+使用桥接模式，你需要先安装 Chrome 扩展，然后在 `target` 部分使用以下配置：
+
+```diff
+target:
+  url: https://www.bing.com
++ bridgeMode: newTabWithUrl
+```
 
 请参阅 [通过 Chrome 扩展桥接模式](./bridge-mode-by-chrome-extension) 了解更多详细信息。
 

apps/site/docs/zh/bridge-mode-by-chrome-extension.mdx

@@ -69,6 +69,18 @@ tsx demo-new-tab.ts
 执行脚本和点击插件中的 'Allow connection' 按钮没有顺序要求。
 :::
 
+## 构造器
+
+```typescript
+import { AgentOverChromeBridge } from "@midscene/web/bridge-mode";
+
+const agent = new AgentOverChromeBridge();
+```
+
+除了 [普通 Agent 构造器](./api) 的参数，`AgentOverChromeBridge` 还提供了以下参数：
+
+* `closeNewTabsAfterDisconnect?: boolean`: 如果为 true，当桥接断开时，所有新创建的标签页都将被自动关闭。默认值为 false。
+
 ## API
 
 除了 [普通的 Agent 接口](./api)，`AgentOverChromeBridge` 还提供了一些额外的接口来控制桌面 Chrome。
@@ -141,18 +153,21 @@ await agent.connectNewTabWithUrl(
 );
 ```
 
-### `destroy`
+### `destroy()`
 
 销毁连接并释放资源。
 
 * 类型
 
 ```typescript
-function destroy(): Promise<void>;
+function destroy(closeNewTabsAfterDisconnect?: boolean): Promise<void>;
 ```
 
+* 参数：
+  * `closeNewTabsAfterDisconnect?: boolean` - 如果为 true，当桥接断开时，所有新创建的标签页都将被自动关闭。默认值为 false。这个参数将覆盖构造器中的 `closeNewTabsAfterDisconnect` 参数。
+
 * 返回值：
-  * 返回一个 Promise。销毁完成后解析为 void
+  * 返回一个 Promise，销毁完成后解析为 void
 
 * 示例：
 

packages/cli/src/yaml-runner.ts

@@ -100,7 +100,10 @@ export async function playYamlFiles(
         );
       }
 
-      const agent = new AgentOverChromeBridge();
+      const agent = new AgentOverChromeBridge({
+        closeNewTabsAfterDisconnect: target.closeNewTabsAfterDisconnect,
+      });
+
       if (target.bridgeMode === 'newTabWithUrl') {
         await agent.connectNewTabWithUrl(target.url);
       } else {

packages/cli/tests/bridge.test.ts

@@ -1,9 +1,5 @@
-import { join } from 'node:path';
-import { matchYamlFiles } from '@/cli-utils';
-import { launchServer } from '@/yaml-runner';
 import { execa } from 'execa';
-import { describe, expect, test } from 'vitest';
-const serverRoot = join(__dirname, 'server_root');
+import { describe, test } from 'vitest';
 const cliBin = require.resolve('../bin/midscene');
 
 const describeIf = process.env.BRIDGE_MODE ? describe : describe.skip;
@@ -11,7 +7,7 @@ const describeIf = process.env.BRIDGE_MODE ? describe : describe.skip;
 describeIf(
   'bridge',
   {
-    timeout: 1000 * 60 * 10,
+    timeout: 1000 * 60 * 3,
   },
   () => {
     test('open new tab', async () => {

packages/cli/tests/midscene_scripts_bridge/new_tab/open-new-tab.yaml

@@ -1,7 +1,9 @@
 target:
   url: https://www.bing.com
-  bridgeMode: newTabWithUrl
   forceSameTabNavigation: true
+  bridgeMode: newTabWithUrl
+  closeNewTabsAfterDisconnect: true
+
 tasks:
   - name: search weather
     flow:

packages/midscene/src/yaml.d.ts

@@ -28,6 +28,7 @@ export interface MidsceneYamlScriptEnv {
 
   // bridge mode config
   bridgeMode?: false | 'newTabWithUrl' | 'currentTab';
+  closeNewTabsAfterDisconnect?: boolean;
 }
 
 export interface MidsceneYamlFlowItemAIAction {

packages/visualizer/src/extension/bridge.less

@@ -16,6 +16,17 @@
     text-overflow: ellipsis;
     overflow: hidden;
     white-space: nowrap;
+    display: flex;
+    flex-direction: row;
+    align-items: center;
+
+    .bridge-status-tip {
+      margin-left: 6px;
+      flex-grow: 1;
+      text-overflow: ellipsis;
+      overflow: hidden;
+      white-space: nowrap;
+    }
   }
 
   .bridge-status-btn {

packages/visualizer/src/extension/bridge.tsx

@@ -1,5 +1,5 @@
 import { LoadingOutlined } from '@ant-design/icons';
-import { ChromeExtensionPageBrowserSide } from '@midscene/web/bridge-mode-browser';
+import { ExtensionBridgePageBrowserSide } from '@midscene/web/bridge-mode-browser';
 import { Button, Spin } from 'antd';
 import { useEffect, useRef, useState } from 'react';
 import './bridge.less';
@@ -11,26 +11,97 @@ interface BridgeLogItem {
   content: string;
 }
 
-enum BridgeStatus {
-  Closed = 'closed',
-  OpenForConnection = 'open-for-connection',
-  Connected = 'connected',
+const connectRetryInterval = 300;
+
+type BridgeStatus =
+  | 'listening'
+  | 'connected'
+  | 'disconnected' /* disconnected unintentionally */
+  | 'closed';
+
+class BridgeConnector {
+  status: BridgeStatus = 'closed';
+
+  activeBridgePage: ExtensionBridgePageBrowserSide | null = null;
+
+  constructor(
+    private onMessage: (message: string, type: 'log' | 'status') => void,
+    private onBridgeStatusChange: (status: BridgeStatus) => void,
+  ) {
+    this.status = 'closed';
+  }
+
+  setStatus(status: BridgeStatus) {
+    this.status = status;
+    this.onBridgeStatusChange(status);
+  }
+
+  keepListening() {
+    if (this.status === 'listening' || this.status === 'connected') {
+      return;
+    }
+
+    this.setStatus('listening');
+
+    Promise.resolve().then(async () => {
+      while (true) {
+        if (this.status === 'connected') {
+          await new Promise((resolve) => setTimeout(resolve, 1000));
+          continue;
+        }
+
+        if (this.status === 'closed') {
+          break;
+        }
+
+        if (this.status !== 'listening' && this.status !== 'disconnected') {
+          throw new Error(`unexpected status: ${this.status}`);
+        }
+
+        let activeBridgePage: ExtensionBridgePageBrowserSide | null = null;
+        try {
+          activeBridgePage = new ExtensionBridgePageBrowserSide(() => {
+            if (this.status !== 'closed') {
+              this.setStatus('disconnected');
+              this.activeBridgePage = null;
+            }
+          }, this.onMessage);
+          await activeBridgePage.connect();
+          this.activeBridgePage = activeBridgePage;
+
+          this.setStatus('connected');
+        } catch (e) {
+          this.activeBridgePage = null;
+          console.warn('failed to setup connection', e);
+          await new Promise((resolve) =>
+            setTimeout(resolve, connectRetryInterval),
+          );
+        }
+      }
+    });
+  }
+
+  async stopConnection() {
+    if (this.status === 'closed') {
+      console.warn('Cannot stop connection if not connected');
+      return;
+    }
+
+    if (this.activeBridgePage) {
+      await this.activeBridgePage.destroy();
+      this.activeBridgePage = null;
+    }
+
+    this.setStatus('closed');
+  }
 }
 
-const connectTimeout = 60 * 1000;
-const connectRetryInterval = 300;
 export default function Bridge() {
-  const activeBridgePageRef = useRef<ChromeExtensionPageBrowserSide | null>(
-    null,
-  );
-  const allowAutoConnectionRef = useRef(false);
-
-  const [bridgeStatus, setBridgeStatus] = useState<BridgeStatus>(
-    BridgeStatus.Closed,
-  );
+  const [bridgeStatus, setBridgeStatus] = useState<BridgeStatus>('closed');
+  const [taskStatus, setTaskStatus] = useState<string>('');
 
   const [bridgeLog, setBridgeLog] = useState<BridgeLogItem[]>([]);
-  const [bridgeAgentStatus, setBridgeAgentStatus] = useState<string>('');
+
   const appendBridgeLog = (content: string) => {
     setBridgeLog((prev) => [
       ...prev,
@@ -41,132 +112,76 @@ export default function Bridge() {
     ]);
   };
 
-  const destroyBridgePage = () => {};
+  const activeBridgeConnectorRef = useRef<BridgeConnector | null>(
+    new BridgeConnector(
+      (message, type) => {
+        appendBridgeLog(message);
+        if (type === 'status') {
+          console.log('status tip changed event', type, message);
+          setTaskStatus(message);
+        }
+      },
+      (status) => {
+        console.log('status changed event', status);
+        setTaskStatus('');
+        setBridgeStatus(status);
+        if (status !== 'connected') {
+          appendBridgeLog(`Bridge status changed to ${status}`);
+        }
+      },
+    ),
+  );
 
   useEffect(() => {
     return () => {
-      destroyBridgePage();
+      activeBridgeConnectorRef.current?.stopConnection();
     };
   }, []);
 
   const stopConnection = () => {
-    if (activeBridgePageRef.current) {
-      appendBridgeLog('Bridge disconnected');
-      activeBridgePageRef.current.destroy();
-      activeBridgePageRef.current = null;
-    }
-    setBridgeStatus(BridgeStatus.Closed);
+    activeBridgeConnectorRef.current?.stopConnection();
   };
 
-  const stopListeningFlag = useRef(false);
-  const stopListening = () => {
-    allowAutoConnectionRef.current = false;
-    stopListeningFlag.current = true;
+  const startConnection = async () => {
+    activeBridgeConnectorRef.current?.keepListening();
   };
 
-  const startConnection = async (timeout = connectTimeout) => {
-    if (activeBridgePageRef.current) {
-      console.error('activeBridgePage', activeBridgePageRef.current);
-      throw new Error('There is already a connection, cannot start a new one');
-    }
-    const startTime = Date.now();
-    setBridgeAgentStatus('');
-    appendBridgeLog('Listening for connection...');
-    setBridgeStatus(BridgeStatus.OpenForConnection);
-    stopListeningFlag.current = false;
-
-    let noConnectionTip = 'No connection found within timeout';
-    console.log('startConnection');
-    while (Date.now() - startTime < timeout) {
-      try {
-        if (stopListeningFlag.current) {
-          noConnectionTip = 'Listening stopped by user';
-          break;
-        }
-        const activeBridgePage = new ChromeExtensionPageBrowserSide(
-          () => {
-            console.log('stopConnection');
-            stopConnection();
-            if (allowAutoConnectionRef.current) {
-              setTimeout(() => {
-                startConnection();
-              }, connectRetryInterval);
-            }
-          },
-          (message, type) => {
-            appendBridgeLog(message);
-            if (type === 'status') {
-              setBridgeAgentStatus(message);
-            }
-          },
-        );
-        await activeBridgePage.connect();
-        activeBridgePageRef.current = activeBridgePage;
-        setBridgeStatus(BridgeStatus.Connected);
-        return;
-      } catch (e) {
-        console.warn('failed to setup connection', e);
-      }
-      console.log('will retry...');
-      await new Promise((resolve) => setTimeout(resolve, connectRetryInterval));
-    }
-
-    setBridgeStatus(BridgeStatus.Closed);
-    appendBridgeLog(noConnectionTip);
-  };
-
-  let statusElement: any;
+  let statusIcon: any;
+  let statusTip: string;
   let statusBtn: any;
   if (bridgeStatus === 'closed') {
-    statusElement = (
-      <span>
-        {iconForStatus('closed')}
-        {'  '}
-        Closed
-      </span>
-    );
+    statusIcon = iconForStatus('closed');
+    statusTip = 'Closed';
     statusBtn = (
       <Button
         type="primary"
         onClick={() => {
-          allowAutoConnectionRef.current = true;
           startConnection();
         }}
       >
         Allow connection
       </Button>
     );
-  } else if (bridgeStatus === 'open-for-connection') {
-    statusElement = (
-      <span>
-        <Spin indicator={<LoadingOutlined spin />} size="small" />
-        {'  '}
-        <span style={{ marginLeft: '6px', display: 'inline-block' }}>
-          Listening for connection...
-        </span>
-      </span>
+  } else if (bridgeStatus === 'listening' || bridgeStatus === 'disconnected') {
+    statusIcon = (
+      <Spin
+        className="bridge-status-icon"
+        indicator={<LoadingOutlined spin />}
+        size="small"
+      />
     );
-    statusBtn = <Button onClick={stopListening}>Stop</Button>;
+    statusTip =
+      bridgeStatus === 'listening'
+        ? 'Listening for connection...'
+        : 'Disconnected, listening for a new connection...';
+    statusBtn = <Button onClick={stopConnection}>Stop</Button>;
   } else if (bridgeStatus === 'connected') {
-    statusElement = (
-      <span>
-        {iconForStatus('connected')}
-        {'  '}
-        Connected
-        <span
-          style={{
-            marginLeft: '6px',
-            display: bridgeAgentStatus ? 'inline-block' : 'none',
-          }}
-        >
-          - {bridgeAgentStatus}
-        </span>
-      </span>
-    );
+    statusIcon = iconForStatus('connected');
+    statusTip = taskStatus ? `Connected - ${taskStatus}` : 'Connected';
+
     statusBtn = (
       <Button
         onClick={() => {
-          allowAutoConnectionRef.current = false;
           stopConnection();
         }}
       >
@@ -174,17 +189,9 @@ export default function Bridge() {
       </Button>
     );
   } else {
-    statusElement = <span>Unknown Status - {bridgeStatus}</span>;
-    statusBtn = (
-      <Button
-        onClick={() => {
-          allowAutoConnectionRef.current = false;
-          stopConnection();
-        }}
-      >
-        Stop
-      </Button>
-    );
+    statusIcon = iconForStatus('failed');
+    statusTip = `Unknown Status - ${bridgeStatus}`;
+    statusBtn = null;
   }
 
   const logs = [...bridgeLog].reverse().map((log, index) => {
@@ -222,7 +229,10 @@ export default function Bridge() {
         <div className="form-part">
           <h3>Bridge Status</h3>
           <div className="bridge-status-bar">
-            <div className="bridge-status-text">{statusElement}</div>
+            <div className="bridge-status-text">
+              <span className="bridge-status-icon">{statusIcon}</span>
+              <span className="bridge-status-tip">{statusTip}</span>
+            </div>
             <div className="bridge-status-btn">{statusBtn}</div>
           </div>
         </div>

packages/web-integration/src/bridge-mode/agent-cli-side.ts

@@ -1,6 +1,10 @@
 import assert from 'node:assert';
 import { PageAgent, type PageAgentOpt } from '@/common/agent';
-import type { KeyboardAction, MouseAction } from '@/page';
+import type {
+  ChromePageDestroyOptions,
+  KeyboardAction,
+  MouseAction,
+} from '@/page';
 import {
   type BridgeConnectTabOptions,
   BridgeEvent,
@@ -10,9 +14,9 @@ import {
   MouseEvent,
 } from './common';
 import { BridgeServer } from './io-server';
-import type { ChromeExtensionPageBrowserSide } from './page-browser-side';
+import type { ExtensionBridgePageBrowserSide } from './page-browser-side';
 
-interface ChromeExtensionPageCliSide extends ChromeExtensionPageBrowserSide {
+interface ChromeExtensionPageCliSide extends ExtensionBridgePageBrowserSide {
   showStatusMessage: (message: string) => Promise<void>;
 }
 
@@ -77,11 +81,12 @@ export const getBridgePageInCliSide = (): ChromeExtensionPageCliSide => {
       }
 
       if (prop === 'destroy') {
-        return async () => {
+        return async (...args: any[]) => {
           try {
-            await bridgeCaller('destroy');
+            const caller = bridgeCaller('destroy');
+            await caller(...args);
           } catch (e) {
-            console.error('error calling destroy', e);
+            // console.error('error calling destroy', e);
           }
           return server.close();
         };
@@ -93,7 +98,7 @@ export const getBridgePageInCliSide = (): ChromeExtensionPageCliSide => {
 };
 
 export class AgentOverChromeBridge extends PageAgent<ChromeExtensionPageCliSide> {
-  constructor(opts?: PageAgentOpt) {
+  constructor(opts?: PageAgentOpt & { closeNewTabsAfterDisconnect?: boolean }) {
     const page = getBridgePageInCliSide();
     super(
       page,
@@ -103,6 +108,12 @@ export class AgentOverChromeBridge extends PageAgent<ChromeExtensionPageCliSide>
         },
       }),
     );
+
+    if (typeof opts?.closeNewTabsAfterDisconnect === 'boolean') {
+      this.page.setDestroyOptions({
+        closeTab: opts.closeNewTabsAfterDisconnect,
+      });
+    }
   }
 
   async connectNewTabWithUrl(url: string, options?: BridgeConnectTabOptions) {
@@ -123,4 +134,13 @@ export class AgentOverChromeBridge extends PageAgent<ChromeExtensionPageCliSide>
     }
     return await super.aiAction(prompt);
   }
+
+  async destroy(closeNewTabsAfterDisconnect?: boolean) {
+    if (typeof closeNewTabsAfterDisconnect === 'boolean') {
+      await this.page.setDestroyOptions({
+        closeTab: closeNewTabsAfterDisconnect,
+      });
+    }
+    await super.destroy();
+  }
 }

packages/web-integration/src/bridge-mode/browser.ts

@@ -1,3 +1,3 @@
-import { ChromeExtensionPageBrowserSide } from '../bridge-mode/page-browser-side';
+import { ExtensionBridgePageBrowserSide } from '../bridge-mode/page-browser-side';
 
-export { ChromeExtensionPageBrowserSide };
+export { ExtensionBridgePageBrowserSide };

packages/web-integration/src/bridge-mode/common.ts

@@ -11,6 +11,7 @@ export enum BridgeEvent {
   Refused = 'bridge-refused',
   ConnectNewTabWithUrl = 'connectNewTabWithUrl',
   ConnectCurrentTab = 'connectCurrentTab',
+  SetDestroyOptions = 'setDestroyOptions',
 }
 
 export interface BridgeConnectTabOptions {

packages/web-integration/src/bridge-mode/io-client.ts

@@ -29,6 +29,13 @@ export class BridgeClient {
       });
 
       const timeout = setTimeout(() => {
+        try {
+          this.socket?.offAny();
+          this.socket?.close();
+        } catch (e) {
+          console.warn('got error when closing socket', e);
+        }
+        this.socket = null;
         reject(new Error('failed to connect to bridge server after timeout'));
       }, 1 * 1000);
 

packages/web-integration/src/bridge-mode/page-browser-side.ts

@@ -1,5 +1,9 @@
 import assert from 'node:assert';
-import type { KeyboardAction, MouseAction } from '@/page';
+import type {
+  ChromePageDestroyOptions,
+  KeyboardAction,
+  MouseAction,
+} from '@/page';
 import ChromeExtensionProxyPage from '../chrome-extension/page';
 import {
   type BridgeConnectTabOptions,
@@ -12,9 +16,13 @@ import { BridgeClient } from './io-client';
 
 declare const __VERSION__: string;
 
-export class ChromeExtensionPageBrowserSide extends ChromeExtensionProxyPage {
+export class ExtensionBridgePageBrowserSide extends ChromeExtensionProxyPage {
   public bridgeClient: BridgeClient | null = null;
 
+  private destroyOptions?: ChromePageDestroyOptions;
+
+  private newlyCreatedTabIds: number[] = [];
+
   constructor(
     public onDisconnect: () => void = () => {},
     public onLogMessage: (
@@ -113,6 +121,7 @@ export class ChromeExtensionPageBrowserSide extends ChromeExtensionProxyPage {
 
     // new tab
     this.onLogMessage(`Creating new tab: ${url}`, 'log');
+    this.newlyCreatedTabIds.push(tabId);
 
     if (options?.forceSameTabNavigation) {
       this.forceSameTabNavigation = true;
@@ -136,12 +145,25 @@ export class ChromeExtensionPageBrowserSide extends ChromeExtensionProxyPage {
     }
   }
 
+  public async setDestroyOptions(options: ChromePageDestroyOptions) {
+    this.destroyOptions = options;
+  }
+
   async destroy() {
+    if (this.destroyOptions?.closeTab && this.newlyCreatedTabIds.length > 0) {
+      this.onLogMessage('Closing all newly created tabs by bridge...', 'log');
+      for (const tabId of this.newlyCreatedTabIds) {
+        await chrome.tabs.remove(tabId);
+      }
+      this.newlyCreatedTabIds = [];
+    }
+
+    await super.destroy();
+
     if (this.bridgeClient) {
       this.bridgeClient.disconnect();
       this.bridgeClient = null;
       this.onDisconnect();
     }
-    super.destroy();
   }
 }

packages/web-integration/src/chrome-extension/page.ts

@@ -8,8 +8,8 @@
 import assert from 'node:assert';
 import type { WebKeyInput } from '@/common/page';
 import { limitOpenNewTabScript } from '@/common/ui-utils';
-import type { AbstractPage } from '@/page';
-import type { BaseElement, ElementTreeNode, Point, Size } from '@midscene/core';
+import type { AbstractPage, ChromePageDestroyOptions } from '@/page';
+import type { ElementTreeNode, Point, Size } from '@midscene/core';
 import type { ElementInfo } from '@midscene/shared/extractor';
 import { treeToList } from '@midscene/shared/extractor';
 import type { Protocol as CDPTypes } from 'devtools-protocol';
@@ -69,6 +69,7 @@ export default class ChromeExtensionProxyPage implements AbstractPage {
     // Create new attaching promise
     this.attachingDebugger = (async () => {
       const url = await this.url();
+      let error: Error | null = null;
       if (url.startsWith('chrome://')) {
         throw new Error(
           'Cannot attach debugger to chrome:// pages, please use Midscene in a normal page with http://, https:// or file://',
@@ -101,6 +102,7 @@ export default class ChromeExtensionProxyPage implements AbstractPage {
         }
 
         // detach any debugger attached to the tab
+        console.log('attaching debugger', currentTabId);
         await chrome.debugger.attach({ tabId: currentTabId }, '1.3');
         // wait util the debugger banner in Chrome appears
         await sleep(500);
@@ -108,11 +110,15 @@ export default class ChromeExtensionProxyPage implements AbstractPage {
         this.tabIdOfDebuggerAttached = currentTabId;
 
         await this.enableWaterFlowAnimation();
-      } catch (error) {
-        console.error('Failed to attach debugger', error);
+      } catch (e) {
+        console.error('Failed to attach debugger', e);
+        error = e as Error;
       } finally {
         this.attachingDebugger = null;
       }
+      if (error) {
+        throw error;
+      }
     })();
 
     await this.attachingDebugger;
@@ -146,15 +152,25 @@ export default class ChromeExtensionProxyPage implements AbstractPage {
 
   private async detachDebugger(tabId?: number) {
     const tabIdToDetach = tabId || this.tabIdOfDebuggerAttached;
+    console.log('detaching debugger', tabIdToDetach);
     if (!tabIdToDetach) {
       console.warn('No tab id to detach');
       return;
     }
 
-    await this.disableWaterFlowAnimation(tabIdToDetach);
-    await sleep(200);
-    await chrome.debugger.detach({ tabId: tabIdToDetach });
+    try {
+      await this.disableWaterFlowAnimation(tabIdToDetach);
+      await sleep(200); // wait for the animation to stop
+    } catch (error) {
+      console.warn('Failed to disable water flow animation', error);
+    }
 
+    try {
+      await chrome.debugger.detach({ tabId: tabIdToDetach });
+    } catch (error) {
+      // maybe tab is closed ?
+      console.warn('Failed to detach debugger', error);
+    }
     this.tabIdOfDebuggerAttached = null;
   }
 

packages/web-integration/src/common/agent.ts

@@ -14,7 +14,6 @@ import { NodeType } from '@midscene/shared/constants';
 import { ScriptPlayer, parseYamlScript } from '@/yaml';
 import {
   MATCH_BY_POSITION,
-  MIDSCENE_USE_QWEN_VL,
   MIDSCENE_USE_VLM_UI_TARS,
   getAIConfig,
   getAIConfigInBoolean,

packages/web-integration/src/page.ts

@@ -28,6 +28,10 @@ export interface KeyboardAction {
   ) => Promise<void>;
 }
 
+export interface ChromePageDestroyOptions {
+  closeTab?: boolean; // should close the tab when the page object is destroyed
+}
+
 export abstract class AbstractPage {
   abstract pageType: string;
   // @deprecated
@@ -82,5 +86,5 @@ export abstract class AbstractPage {
     concurrency?: number;
   }): Promise<void>;
 
-  abstract destroy(): Promise<void>;
+  abstract destroy(options?: ChromePageDestroyOptions): Promise<void>;
 }