* chore: temp
* feat(report): each time, append the report individually instead of appending in bulk to avoid excessive memory usage
* test(core): fixs
* feat(web-integration): enhance report generation with memory management and append options
* fix(report): dump element loading with DOM readiness check
* fix(core): tests
* test(core): skip memory test
* feat(core): add insertScriptBeforeClosingHtml function for efficient script insertion in HTML files
* fix(reporter): update report generation to use mode for merged reports
* feat(web-integration): support user expected xpath option for locate methods
* docs(site): xpath option for api
* fix(ci): nx config
* docs(site): update docs
* fix(shared): add text content for leaf elements before text node
* fix(core): return too early
* fix(core): update page description condition for non-vl mode
* test(core): add bug test case
* chore: add record example
* chore: delete unless file
* chore: optimize common logic
* chore: optimize record logic
* chore: support horizontal scroll
* chore: optimize label input logic
* chore: delete unless file
* chore: modify record log content
* chore: add record lib
* chore: optimize record iife logic
* chore: delete unless config
* chore: modify comment
* feat(chrome-extension): add recording functionality
- Add new record UI components (record.tsx, record.less)
- Integrate recording capabilities into popup extension
- Update worker script to handle recording events
- Add content script wrapper for recording injection
- Remove EventRecorder from record-form app (moved to extension)
- Update manifest.json permissions for recording
- Update dependencies for recording functionality
- Enhance store with recording state management
* feat(chrome-extension): improve recording stability with page refresh handling
- Add tab update monitoring to automatically stop recording on page refresh/navigation
- Implement content script injection check with ping mechanism
- Enhance error handling with specific messages for Chrome internal pages
- Improve recording stop reliability when content script becomes unavailable
- Add graceful handling for missing content script during stop operation
* fix(core): align dependency versions across workspace
- Update React to ^19.1.0 and React DOM to >=19.1.0 in all packages
- Update @types/react to ^19.1.4 and @types/react-dom to ^19.1.3
- Update TypeScript to ^5.8.3 across all packages
- Update dayjs to use caret version ^1.11.11
- Align all rsbuild and rslib plugin versions
- Fix dependency version inconsistencies to pass CI checks
* refactor(chrome-extension): rename content-script-wrapper to event-recorder-bridge
- Renamed content-script-wrapper.js to event-recorder-bridge.js
- Updated references in rsbuild.config.ts and record.tsx
- Updated internal comments to match new name
- New name better describes the file's bridge functionality
* feat(chrome-extension): enhance recording session management and UI
- Introduced RecordingSession interface for better session handling
- Implemented session persistence with localStorage for recording sessions
- Added UI components for creating, editing, and deleting recording sessions
- Enhanced event handling to load and display events from selected sessions
- Improved user feedback with success messages for session actions
- Updated styles for session management and recording status indicators
* feat(chrome-extension): enhance recording UI and session management
- Simplified the Record component structure by separating list and detail views for better user experience.
- Improved styling for the recording sessions list and detail views, ensuring better responsiveness and usability.
- Added functionality to switch between list and detail views for recording sessions.
- Updated session management to allow for viewing, editing, and deleting sessions with improved user feedback.
- Enhanced event handling and display for recorded events within the detail view.
* feat(record): add screenshot capture for recorded events
- Add screenshot capture before and after each recorded event
- Implement screenshot capture in chrome extension service worker
- Add screenshot fields to RecordedEvent interface
- Create UI components for displaying screenshot thumbnails in timeline
- Add hover effects and popover for screenshot preview
- Remove unused AGENTS.md file
* feat(chrome-extension): align React and TypeScript versions across the project
- Downgrade React and React DOM to 18.3.1 in all packages for consistency
- Update @types/react and @types/react-dom to 18.3.1
- Ensure TypeScript version is consistently set to ^5.8.3
- Adjust dependencies in package.json files for various apps and packages
- Enhance Chrome extension functionality with safe API wrappers for better error handling
- Implement checks for Chrome extension environment in recording components
* feat(chrome-extension): enhance event recording with new features and dependencies
- Add @rsbuild/plugin-node-polyfill to support Node.js polyfills in the project
- Update event recording logic to include viewport dimensions and element positioning
- Improve screenshot handling by integrating new properties in RecordedEvent interface
- Enhance RecordTimeline component to display element positioning and dimensions
- Optimize event handling and UI updates for better user experience
- Clean up and refactor code for improved readability and maintainability
* feat(record): enhance event data with coordinates in event recorder
- Add x and y coordinates to the event data sent to the Chrome extension
- Update RecordTimeline component to utilize new coordinate properties for element positioning
- Ensure compatibility with existing event handling logic
* chore: optimize select box logic
* chore: delete unless config
* feat(record): optimize AI description generation to be asynchronous and non-blocking
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(record): implement smooth scroll with enhanced animations and performance
- Add requestAnimationFrame-based smooth scrolling for better performance
- Implement container-level auto-scroll to bottom when new events arrive
- Extract event optimization logic to separate eventOptimizer utility module
- Add smooth scroll CSS properties and touch optimization for mobile
- Enhance hover animations with cubic-bezier easing and GPU acceleration
- Add slideInUp and fadeInScale keyframe animations for timeline items
- Improve scrollbar styling with transition effects
- Optimize will-change properties for better rendering performance
- Add overscroll-behavior to prevent scroll propagation
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(workflow): add CLAUDE.md with project documentation and development guide
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(record): optimize session creation to auto-start recording with timestamp name
* feat(record): optimize session creation to use timestamp naming by default
- Modified session creation to auto-create with timestamp name without showing modal
- Improved record UI to display session titles more effectively
- Reorganized description display in both list and detail views
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(record): improve session management and code quality
- Optimize session creation to use timestamp naming by default without requiring modal input
- Improve UI layout to display titles and descriptions more effectively in both list and
detail views
- Apply code formatting improvements across the codebase for better readability
- Fix linting issues in scripts and component files
* feat(record): implement caching for element descriptions and screenshots to enhance performance
* feat(record): refactor stopRecording logic and add visibility change handling for extension popup
* feat(record): auto-stop recording when leaving detail view
* feat(record): improve recording state management and handle asynchronous stop actions
* chore: upgrade lock file
* feat(record): add persistence for recording state in local storage
* feat(record): add modular structure with components and hooks for improved organization
* feat(record): implement AI-generated titles and descriptions for recording sessions
* feat(record): enhance recording session management with improved layout and AI title generation
* feat(record): enhance recording control hook with session update callback
* fix: update coding standards and improve code formatting across multiple files
* feat(record): add Playwright test generation feature and export controls
* feat(record): enhance AI description generation with caching and callback handling
* feat(record): replace RecordedEvent with ChromeRecordedEvent for improved event handling
* feat(record): improve debug logging and event handling in recording sessions
* feat(record): enhance event optimization and logging, add diagnosis feature for recording chain
* feat(record): update RecordDetail and RecordList components, improve diagnostic function and type imports
* feat(record): enhance tab monitoring with event listeners for activation and updates
* feat(record): sort sessions by updated time in RecordList component
* feat(record): implement session ID handling and cleanup for previous recordings
* feat(eventOptimizer): improve element rectangle handling and callback updates in description generation
* feat(record): enhance AI title and description generation logic, improve event handling and logging
* feat(record): implement debounce mechanism for AI description generation and enhance event rectangle validation
* feat(record): enhance AI description generation with improved caching and debouncing logic, and add session ID dependency in cleanup
* feat(record): update AI title and description generation logic to prevent overwriting existing session names and descriptions, and improve event handling in scroll events
* refactor(record): clean up code formatting and improve logging consistency across components
* docs(record): update CLAUDE.md with detailed implementation and optimization strategies for the recording system
* feat(record): enhance event details display with JSON copy functionality and improved cache key generation
* chore(commitlint): increase maximum header length to 300 characters
* chore(commitlint): disable scope-enum rule enforcement
* feat(record): implement unified logging system and replace console logs with structured logger
* feat(record): add hash ID generation for recorded events and clean up elementRect handling
* feat(record): add before and after screenshot functionality to event timeline
* feat(record): remove event-recorder-bridge script and update build configuration
* feat(record): update screenshot handling in RecordTimeline component to display before and after images
* refactor(record): clean up code formatting and remove unnecessary lines
* feat(build): enhance build configuration with caching and input/output settings
* feat(record): pause active recording sessions when creating a new session
* feat(record): enhance Playwright export controls with live event handling and improved description checks
* refactor(record): streamline configuration and improve logging clarity
* feat(record): update YAML generation to follow Midscene.js conventions with enhanced action mapping and descriptive naming
* feat(record): simplify YAML generation for Midscene.js with clearer structure and updated action mapping
* feat(record): enhance YAML export functionality with improved event handling and loading state
* feat(record): add Playwright and YAML test generation utilities with shared types and export controls
* feat(record): replace PlaywrightExportControls with a unified ExportControls component
* feat(record): integrate Playwright test generation from core package and update language handling in UI components
* feat(record): add @midscene/record dependency and integrate YAML test generation from core package
* feat(record): enhance event handling by saving current recording events during session updates and cleanup
* feat(record): enhance event recording by adding final navigation event and improving error handling in event recorder bridge
* feat(record): improve event handling by conditionally clearing events during recording start
* feat(record): optimize event handling by implementing debouncing and refining fallback descriptions
* feat(record): enhance test generation by extracting navigation and viewport info for improved options
* feat(record): implement event persistence with local storage for recording sessions
* chore: clean up code formatting and improve readability across multiple files
* feat(record): add dropdown menu for code generation and streamline state management
* feat(record): remove YAML export functionality from ExportControls and related components
* feat(record): add ProgressModal component for enhanced recording feedback
* feat(record): enhance ExportControls and ProgressModal with improved UI and confetti animation
* feat(record): simplify ExportControls and RecordDetail components by removing unused functionality and improving session details display
* feat(record): improve error handling and logging in content script checks and enhance tab URL validation
* feat(record): add generatedCode field to RecordingSession and improve error handling in content script checks
* feat(record): add canvas-confetti for celebratory effects in ProgressModal and update dependencies
* feat(record): refactor RecordDetail to use sessionId and improve session retrieval with error handling
* feat(progress-modal): adjust confetti display duration and update completion timing in ProgressModal
* chore: upgrade lock file
* refactor(snapshot): simplify element description rules for clarity and brevity
* feat(record): initialize optimizedEvents state as empty array and refactor logging methods for improved clarity
* refactor(export-controls): streamline event description generation and update progress step order for clarity
* feat(progress-modal): add ShinyText component for enhanced step title display during loading
* refactor(export-controls): consolidate test generation logic into a common function for improved maintainability
* refactor(export-controls, eventOptimizer): improve code readability and consistency by formatting and simplifying logic
* chore: delete unless config
* chore: add ignore file
* chore: add ignore file
* refactor: migrate from 'record' to 'recorder' package and consolidate related files
* refactor: remove playwright example file and integrate example code into prompts and YAML generation
* chore: hidden recorder
* chore: delete unless file
* chore: delete unless file
* refactor: migrate from 'record-form' to 'recorder-form' and update related configurations
* refactor: remove shiny-text component and update imports in related files
* refactor: enhance ProgressModal styling for better alignment and readability
* refactor: update finalEvents assignment in ExportControls for session title generation
* refactor: comment out recorder section in PlaygroundPopup for future review
* chore(ci): fix lint error
---------
Co-authored-by: Claude <noreply@anthropic.com>
* fix(core): fix planning test
* fix(report): report insight view
* chore(core): rename workflow
* fix(core): report replay scripts
* fix(core): test case
* docs(core): update api doc for ui-tars
* chore(core): fix test case
* fix(core): update error log of dump file
* chore(workflow): update ci config
* fix(core): update evaluation
* chore(core): add test case
* chore(core): update ci config
* chore(core): update ci config
* fix(core): ci
* fix(core): update inspect test case
* fix(core): e2e
* fix(puppeteer): show model info in report
* fix(core): update evaluation config
* chore(core): fix lint
* fix(web-integration): sanitize content for reportHtmlContent
* fix(workflow): version
* fix(web-integration): recover after retrieve
* fix(web-integration): test
* chore(core): fix lint
* fix(report): remove ! for content
* chore(core): add test case
* chore(core): add test case
---------
Co-authored-by: yutao <yutao.tao@bytedance.com>
* feat(web-integration): use xpath for cache instead of id
* feat(web-integration): enhance TaskCache to support xpaths for cache matching and add new test cases
* feat(web-integration): add debug log for unknown page types in TaskCache
* feat(web-integration): update caching logic and cache hit conditions for Plan and Locate tasks
* chore(core): update debug log
* feat(web-integration): update rspress.config and enhance TaskCache structure with new properties
* feat(web-integration): recalculate id when hit cache
* fix(web-integration): update mock implementation in task-cache test to use evaluate method
* feat(web-integration): enhance element caching by adding XPath support and improving cache hit logic
* chore(core): lint
* feat(web-integration): improve XPath handling in web-extractor
* test(web-integration): fix tests
* feat(core, web-integration): add attributes to LocateResultElement and enhance element handling
* fix(core): lint
* feat(web-integration): add midsceneVersion to TaskCache and update cache validation logic
* fix(core): test
* fix(web-integration): update cache validation logic to prevent reading outdated midscene cache files
* feat(web-integration): enhance TaskCache to track used cache items and improve cache retrieval logic
* fix(core): xpath logic (#710)
* feat(core): resue context for locate
* feat(core): build yamlFlow from aiAction
* feat(core): refine task-cache
* feat(core): update cache
* feat(core): refine task-cache
* feat(core): refine task-cache
* feat(core): remove unused checkElementExistsByXPath
* feat(core): use yaml file as cache
* chore(core): fix lint
* chore(core): print warning for previous cache
* refactor(core): remove quickAnswer references and improve element matching logic
* fix(core): update import path for buildYamlFlowFromPlans
* chore(web-integration): update output image and skip task error test
* fix(web-integration): update test snapshots to handle beta versions
* fix(web-integration): adjust test snapshots for version consistency
* fix(web-integration): track original cache length and adjust matching logic in tests
* fix(web-integration): update test URLs to reflect new target site and enable previously skipped test
* chore(core): update cache docs
* fix(core): test
* feat(core): try to match element from plan
* fix(web-integration): cache id stable when retry in palywright
* fix(web-integration): typo
* style(web-integration): lint
* fix(web-integration): stable cacheid in tests
* fix(web-integration): cache id
---------
Co-authored-by: quanruzhuoxiu <quanruzhuoxiu@gmail.com>
* feat(web-integration): add support for new agent method(aiBoolean, aiString, aiNumber, aiLocate)
* fix(core, web-integration): update data extraction logic and refine return types for agent methods
* feat(site): add detailed descriptions and examples
* fix(web-integration): remove unused properties from aiLocate return type and update UI titles
* fix(core): enhance agent methods for better extraction and querying
* fix(core): update test scripts and refine data extraction prompts
* fix(core): refine return type in ExecutionTaskApply and remove unused import in inspect
* fix(core): update data extraction logic and enhance type handling in Insight class
* fix(web-integration): update import path for puppeteerAgentForTarget in player test file
* fix(core): tests fix
* fix(core): fix tests
* fix(shared): enhance imageInfo tests by validating individual properties and updating snapshots
* fix(site): remove waitForNavigationTimeout from Playwright integration examples
* fix(site): update timeout configuration examples in FAQ for clarity
* test(web-integration): remove unused client-extractor test and related snapshots
* chore(core): fix query implementation
* fix(report): improve data handling in DetailSide component and enhance output display
* fix(core): refine data extraction prompts and improve error handling in PageTaskExecutor
* fix(core): update data extraction prompt for clarity and type validation
---------
Co-authored-by: yutao <yutao.tao@bytedance.com>
* feat(utils): optimize reportHTMLContent to handle large data with Buffer and newlines
* feat(report): enhance test data handling and filtering in PlaywrightCaseSelector
* feat(report): enhance reportHTMLContent to support file writing and improved dump data handling
* fix(core): correct file writing logic and improve dump data handling in reportHTMLContent
* feat(core): add tests for reportHTMLContent
* feat(core): add performance test for handling multiple large reports in utils
* feat(core): support custom midscene_run dir
* feat(report): add search functionality to PlaywrightCaseSelector component
* refactor(shared): simplify base directory resolution and remove unused environment variable
* feat(shared): integrate shared environment variables across multiple packages
* refactor(shared): update base directory resolution to use dynamic midscene_run directory
* fix(puppeteer): increase screenshot timeout from 3s to 10s for improved reliability
* refactor: android api
* refactor: enhance Android agent to accept options for device connection
* fix: type error
* fix: click after clearInput
* fix: click before clearInput
* feat: android playground
* feat: support npx package name
* feat: android playground joint
* fix: git ignore conflicts
* feat: ensure adb server is running before initializing adb client
* fix: deps consistency
* ci: add android playground
* feat: integrate shared constants and improve server configuration in android playground
* feat: android playground style
* feat: style opt
* feat: add @rsbuild/plugin-svgr dependency and improve URI handling in adb
* feat: remove unused water flow scripts and update comments to English
* feat: download report file
* feat: standalone android playground
* feat: use dynamic import
* feat: migrate CSS to LESS and remove unused styles in chrome extension and report
* feat: enhance Android playground with ScrcpyPlayer ref integration and device management improvements
* feat: optimize styles and layout in Android playground and visualizer components
* chore: add bin back
* chore: update build script to exclude documentation generation
* feat: add not ready message to PlaygroundResult for improved user guidance
* feat: add error handling for screenshot capture in Android page
* docs: update readme
* feat: add PNG validation for screenshot buffer in Android page
* feat: enhance UI components with improved styling and tooltips in ScrcpyPlayer and PromptInput
* docs: update uri parameter description in integrate-with-android documentation and improve uri handling in launch function
* style: update primary color to #2B83FF across multiple components and adjust margin in App.less
* refactor: replace userConfig with globalConfig for environment configuration management and update related functions
* feat: integrate server validation logic in App, AdbDevice, and ScrcpyPlayer components for improved connection handling
* style: enhance player component layout with overflow handling and margin adjustments
* style: refine player component layout with flex adjustments and improved spacing
* feat: add midscene model name display and improve layout in EnvConfig component
* feat: integrate ShinyText component for enhanced loading progress display in PlaygroundResult
* test: add test for isValidPNGImageBuffer
* style: remove background color from App.less and adjust AI config override behavior in env.ts
---------
Co-authored-by: yutao <yutao.tao@bytedance.com>
* feat: enable search area for locate
* fix: update evaluation
* fix: locator
* feat: show searchArea in report
* chore: add yaml support for aiTap
* feat: use adb instead appium
* feat: Adds debugging information and reconstructs input text capabilities
* feat: refactoring Android related functions and adding android modules
* feat: update the image scaling algorithm, adjust the Android page class to support device scaling, and remove test files that are no longer needed
* feat: adjust the Android page class to support device scaling, and remove test files that are no longer needed
* feat: use appium-adb instead of bare command
* fix: update entry for @midscene/android
* feat: optimize the screenshot processing logic, add a backup mechanism when screenshots fail, and update test cases to accommodate new features
* fix: rethrow error
* feat: add Android debug configuration options and update documentation
* chore: fix code style in #483 (#492)
* fix: remove try for error handle by outside
---------
Co-authored-by: yutao <yutao.tao@bytedance.com>
Co-authored-by: linyibing <linyibing@bytedance.com>
Co-authored-by: yuyutaotao <167746126+yuyutaotao@users.noreply.github.com>
* feat: enable search area for locate
* fix: update evaluation
* fix: build error
* fix: ci
* fix: locator
* feat: show searchArea in report
* chore: add yaml support for aiTap
* feat: update status tip
* fix: #473 (#484)
* chore: optimize unit test list
---------
Co-authored-by: zhouxiao.shaw <zhouxiao.shaw@bytedance.com>
* fix: add an field for CoT in qwen
* chore: update prompt
* fix: scroll in puppeteer
* fix: multiple action in qwen
* fix: multiple action in qwen
* fix: multiple action in qwen
* chore: update prompt
* chore: change fields order to enhance the CoT
