midscene

AI-powered, vision-driven UI automation for every platform.

What is midscene?

AI-powered, vision-driven UI automation for every platform.

Overview of midscene

What it is

Midscene.js is an AI-powered, vision-driven UI automation tool for E2E testing. It uses a visual GUI Agent to observe screens, act on elements, and verify results. It supports web, mobile, and desktop platforms.

Key features

  • Write test cases in natural language using YAML or JavaScript.
  • Locate and interact with UI elements by their appearance and position from screenshots, without maintaining selectors.
  • Verify visible UI state such as colors, highlights, layout, and content on canvas or native apps.
  • Cross-platform support: Web (Playwright, Puppeteer, desktop Chrome), Android (ADB), iOS (WebDriverAgent), HarmonyOS (HDC), Desktop (macOS, Windows, Linux, RDP), and custom interfaces.
  • Supports multiple AI models: GPT, Qwen, Doubao Seed, DeepSeek, Gemini, Kimi, and open-source options.
  • Midscene Test framework with YAML test cases, TypeScript Nodes for reusable business logic, lifecycle hooks, retries, concurrency, and step-level reports.

Use cases

  • E2E testing for web, mobile, and desktop applications.
  • Testing visual appearance and behavior that traditional selectors cannot capture.
  • Automating complex workflows that require visual verification.
  • Integrating AI-driven UI automation into existing test suites (Playwright, Vitest).

Getting started

See the repository README for installation and quick-start instructions.

License and pricing

Licensed under MIT. Open source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/web-infra-devmidscene.svg)](https://yunodb.com/tools/web-infra-devmidscene?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/web-infra-devmidscene?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/web-infra-devmidscene.svg" alt="Featured on YunoDB" height="20"></a>