Skip to content

ai-dashboad/flutter-skill

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

495 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

flutter-skill CLI demo β€” nav, tap, type, screenshot across websites

flutter-skill

Give any AI agent eyes and hands inside any running app.
10 platforms. Zero test code. One MCP server.

Stars pub.dev npm CI License

Demo β€’ Quick Start β€’ AI Platforms β€’ Platforms β€’ vs Others β€’ Docs

πŸš€ Zero config. Zero test code. Just talk to your AI.

If this saves you time, please consider starring the repo ⭐ β€” it helps others find it!


30-Second Demo

FINAL.mp4

One prompt. 28 AI-driven actions. Zero test code. The AI explores a TikTok clone, navigates tabs, scrolls feeds, tests search, fills forms β€” all autonomously.


Why This Exists

Writing E2E tests is painful. Maintaining them is worse. flutter-skill takes a different approach:

  • πŸ”Œ Connects any AI agent (Claude, Cursor, Windsurf, Copilot, OpenClaw) directly to your running app via MCP
  • πŸ‘€ The agent sees your screen β€” taps buttons, types text, scrolls, navigates β€” like a human tester who never sleeps
  • βœ… Zero test code β€” no Page Objects, no XPath, no brittle selectors. Just plain English
  • ⚑ Zero config β€” 2 lines of code, works on all 10 platforms
You: "Test the checkout flow with an empty cart, then add 3 items and complete purchase"

Your AI agent handles the rest β€” screenshots, taps, text entry, assertions, navigation.
No Page Objects. No XPath. No brittle selectors. Just plain English.

Quick Start

1. Install (30 seconds)

npm install -g flutter-skill

2. Add to your AI (copy-paste into MCP config)

{
  "mcpServers": {
    "flutter-skill": {
      "command": "flutter-skill",
      "args": ["server"]
    }
  }
}

Works with Claude Desktop, Cursor, Windsurf, Copilot, Cline, OpenClaw β€” any MCP-compatible agent.

3. Add to your app (2 lines for Flutter)

import 'package:flutter_skill/flutter_skill.dart';

void main() {
  if (kDebugMode) FlutterSkillBinding.ensureInitialized();
  runApp(MyApp());
}

4. Test β€” just talk to your AI:

"Launch my app, explore every screen, and report any bugs"

That's it. Zero configuration. Zero test code. Works in under 60 seconds.

πŸ“¦ More install methods (Homebrew, Scoop, Docker, IDE, Agent Skill)
Method Command
npm npm install -g flutter-skill
Homebrew brew install ai-dashboad/flutter-skill/flutter-skill
Scoop scoop install flutter-skill
Docker docker pull ghcr.io/ai-dashboad/flutter-skill
pub.dev dart pub global activate flutter_skill
VSCode Extensions β†’ "Flutter Skill"
JetBrains Plugins β†’ "Flutter Skill"
Agent Skill npx skills add ai-dashboad/flutter-skill
Zero-config flutter-skill init (auto-detects & patches your app)

Use with AI Platforms

MCP Server Mode (IDE Integration)

Works with any MCP-compatible AI tool. One config line:

{
  "mcpServers": {
    "flutter-skill": {
      "command": "flutter-skill",
      "args": ["server"]
    }
  }
}
Platform Config File Status
Cursor .cursor/mcp.json βœ…
Claude Desktop claude_desktop_config.json βœ…
Windsurf ~/.codeium/windsurf/mcp_config.json βœ…
VSCode Copilot .vscode/mcp.json βœ…
Cline VSCode Settings β†’ Cline β†’ MCP βœ…
OpenClaw Skill or MCP config βœ…
Continue.dev .continue/config.json βœ…

HTTP Serve Mode (CLI & Automation)

For standalone browser automation, CI/CD pipelines, or remote access:

# Start server
flutter-skill serve https://your-app.com

# Use CLI client commands
flutter-skill nav https://google.com
flutter-skill snap                    # Accessibility tree (99% fewer tokens)
flutter-skill screenshot /tmp/ss.jpg
flutter-skill tap "Login"
flutter-skill type "hello@example.com"
flutter-skill eval "document.title"
flutter-skill tools                   # List all available tools
Command Description
nav <url> Navigate to URL
snap Accessibility tree snapshot
screenshot [path] Take screenshot
tap <text|ref|x y> Tap element
type <text> Type via keyboard
key <key> [mod] Press key
eval <js> Execute JavaScript
title Get page title
text Get visible text
hover <text> Hover element
upload <sel> <file> Upload file
tools List tools
call <tool> [json] Call any tool

Supports --port=N, --host=H flags and FS_PORT/FS_HOST env vars.

Two Modes Compared

server (MCP stdio) serve (HTTP)
Use case IDE / AI agent integration CLI / automation / CI/CD
Protocol MCP (JSON-RPC over stdio) HTTP REST
Tools 253 (dynamic per page) 246 (generic)
Browser Auto-launches Chrome Connects to existing Chrome
Best for Cursor, Claude, VSCode OpenClaw, scripts, pipelines

Full CLI client reference: docs/CLI_CLIENT.md


10 Platforms, One Tool

Most testing tools work on 1-2 platforms. flutter-skill works on 10.

Platform SDK Test Score
Flutter (iOS/Android/Web) flutter_skill βœ… 188/195
React Native sdks/react-native βœ… 75/75
Electron sdks/electron βœ… 75/75
Tauri (Rust) sdks/tauri βœ… 75/75
Android (Kotlin) sdks/android βœ… 74/75
KMP Desktop sdks/kmp βœ… 75/75
.NET MAUI sdks/dotnet-maui βœ… 75/75
iOS (Swift/UIKit) sdks/ios βœ… 19/19
Web (any website) sdks/web βœ…
Web CDP (zero-config) No SDK needed βœ… 141/156

Total: 656/664 tests passing (98.8%) β€” each platform tested against a complex social media app with 50+ elements.


⚑ Performance

Real benchmarks from automated test runs against a complex social media app:

Operation Web (CDP) Electron Android
connect 93 ms 55 ms 103 ms
tap 1 ms 1 ms 2 ms
enter_text 1 ms 1 ms 2 ms
inspect 3 ms 12 ms 10 ms
snapshot 2 ms 8 ms 29 ms
screenshot 31 ms 80 ms 88 ms
eval 1 ms β€” β€”

Token efficiency: snapshot() returns a structured element tree instead of an image β€” 87–99% fewer tokens than sending screenshots to your AI agent.

How fast is that? A tap takes 1–2 ms end-to-end. Browser automation tools like Playwright and Selenium typically take 50–100 ms for the same operation. That's 50–100Γ— faster, because flutter-skill talks directly to the app runtime instead of going through WebDriver or CDP indirection.

Heavy DOM Sites (Real-World)

Tested 15 MCP tools against production websites β€” 75/75 passed, zero timeouts:

Site Tools Total Time snapshot screenshot count_elements
YouTube 15/15 βœ… 6.9s 43 ms 30 ms 4 ms
Amazon 15/15 βœ… 14.2s 1 ms 5 ms 2 ms
Reddit 15/15 βœ… 17.9s 6 ms 32 ms 51 ms
Hacker News 15/15 βœ… 4.8s 53 ms 188 ms 1 ms
Wikipedia 15/15 βœ… 7.8s 15 ms 336 ms 1 ms

Total time includes page load. Tool execution is consistently sub-100ms even on heavy DOM sites.


Why Not Playwright / Appium / Detox?

flutter-skill Playwright MCP Appium Detox
MCP tools 253 ~33 ❌ ❌
Platforms 10 1 (web) Mobile React Native
Setup time 30 sec Minutes Hours Hours
Test code needed ❌ None βœ… Yes βœ… Yes βœ… Yes
AI-native (MCP) βœ… βœ… ❌ ❌
Self-healing tests βœ… ❌ ❌ ❌
Monkey/fuzz testing βœ… ❌ ❌ ❌
Visual regression βœ… ❌ ❌ ❌
Network mock/replay βœ… ❌ ❌ ❌
API + UI testing βœ… ❌ ❌ ❌
Multi-device sync βœ… ❌ Partial ❌
Accessibility audit βœ… ❌ ❌ ❌
i18n testing βœ… ❌ ❌ ❌
Performance monitoring βœ… ❌ ❌ ❌
Natural language βœ… ❌ ❌ ❌
Flutter support βœ… Native Partial Partial ❌
Desktop apps βœ… βœ… ❌ ❌

| AI page understanding | βœ… AX Tree | ❌ Screenshots | ❌ | ❌ | | Boundary/security test | βœ… 13 payloads | ❌ | ❌ | ❌ | | Batch actions | βœ… 5+/call | 1/call | 1/call | 1/call |

flutter-skill is the only AI-native E2E testing tool that works across mobile, web, and desktop β€” with 7Γ— more tools than the nearest competitor.


CLI Commands

# πŸ€– AI autonomous exploration β€” finds bugs automatically
flutter-skill explore https://my-app.com --depth=3

# πŸ’ Monkey/fuzz testing β€” random actions, crash detection
flutter-skill monkey https://my-app.com --actions=100 --seed=42

# πŸš€ Parallel multi-platform testing
flutter-skill test --url https://my-app.com --platforms web,electron,android

# 🌐 Zero-config WebMCP server β€” any website becomes testable
flutter-skill serve https://my-app.com

🧠 AI-Native: 95% Fewer Tokens

Most AI testing tools send screenshots to the LLM β€” each one costs ~4,000 tokens.

flutter-skill uses Chrome's Accessibility Tree to give your AI a compact semantic summary of any page:

// page_summary β†’ ~200 tokens (vs ~4,000 for a screenshot)
{
  "title": "Shopping Cart",
  "nav": ["Home", "Products", "Cart", "Account"],
  "forms": [{"input:Coupon Code": "text"}],
  "buttons": ["Apply", "Checkout", "Continue Shopping"],
  "features": {"search": true, "pagination": true},
  "links": 47, "inputs": 3
}

Then batch multiple actions in one call:

// explore_actions β†’ 5 actions per call (vs 5 separate tool calls)
{"actions": [
  {"type": "fill", "target": "input:Coupon Code", "value": "SAVE20"},
  {"type": "tap", "target": "button:Apply"},
  {"type": "tap", "target": "button:Checkout"},
  {"type": "fill", "target": "input:Email", "value": "test@example.com"},
  {"type": "tap", "target": "button:Continue"}
]}

Result: Your AI agent tests faster, costs less, and understands pages better than screenshot-based tools.

flutter-skill Screenshot-based tools
Tokens per page ~200 ~4,000
Actions per call 5+ 1
Understands semantics βœ… roles, names, state ❌ pixels only
Works with Shadow DOM βœ… ❌

What It Can Do

πŸ‘€ See

  • screenshot β€” capture the screen
  • inspect_interactive β€” all tappable/typeable elements with semantic refs
  • find_element / wait_for_element
  • get_elements β€” full element tree

πŸ‘† Interact

  • tap / long_press / swipe / drag
  • enter_text / set_text / clear_text
  • scroll β€” all directions
  • go_back / press_key

πŸ” Inspect (v0.8.0)

  • Semantic refs: button:Login, input:Email
  • Stable across UI changes
  • tap(ref: "button:Submit")
  • 7 roles: button, input, toggle, slider, select, link, item

πŸš€ Control

  • launch_app β€” launch with flavors
  • hot_reload / hot_restart
  • get_logs / get_errors
  • scan_and_connect β€” auto-find apps
253 tools β€” full reference

AI Explore: page_summary, explore_actions, boundary_test, explore_report

Launch & Connect: launch_app, scan_and_connect, connect_cdp, hot_reload, hot_restart, list_sessions, switch_session, close_session, disconnect, stop_app

Screen: screenshot, screenshot_region, screenshot_element, native_screenshot, inspect, inspect_interactive, snapshot, get_widget_tree, find_by_type, get_text_content, get_visible_text

Interaction: tap, double_tap, long_press, enter_text, set_text, clear_text, swipe, scroll_to, drag, go_back, press_key, type_text, hover, fill, select_option, set_checkbox, focus, blur, native_tap, native_input_text, native_swipe

Smart Testing: smart_tap, smart_enter_text, smart_assert (self-healing with fuzzy match)

Assertions: assert_text, assert_visible, assert_not_visible, assert_element_count, assert_batch, wait_for_element, wait_for_gone, wait_for_idle, wait_for_stable, wait_for_url, wait_for_text, wait_for_element_count

Visual Regression: visual_baseline_save, visual_baseline_compare, visual_baseline_update, visual_regression_report, visual_verify, visual_diff, compare_screenshot

Network Mock: mock_api, mock_clear, record_network, replay_network, intercept_requests, clear_interceptions, block_urls, http_request

API Testing: api_request, api_assert

Coverage & Reliability: coverage_start, coverage_stop, coverage_report, coverage_gaps, retry_on_fail, stability_check

Data-Driven: test_with_data, generate_test_data

Multi-Device: multi_connect, multi_action, multi_compare, multi_disconnect, parallel_snapshot, parallel_tap

Accessibility: accessibility_audit, a11y_full_audit, a11y_tab_order, a11y_color_contrast, a11y_screen_reader

i18n: set_locale, verify_translations, i18n_snapshot

Performance: perf_start, perf_stop, perf_report, get_performance, get_frame_stats, get_memory_stats

Session: save_session, restore_session, session_diff

Recording & Export: record_start, record_stop, record_export (Playwright, Cypress, XCUITest, Espresso, Detox, Maestro, +5 more), video_start, video_stop

Auth: auth_inject_session, auth_biometric, auth_otp, auth_deeplink

CDP Browser: navigate, reload, go_forward, get_title, get_page_source, eval, get_tabs, new_tab, switch_tab, close_tab, get_cookies, set_cookie, clear_cookies, get_local_storage, set_local_storage, clear_local_storage, generate_pdf, set_viewport, emulate_device, throttle_network, go_offline, set_geolocation, set_timezone, set_color_scheme

Debug: get_logs, get_errors, get_console_messages, get_network_requests, diagnose, diagnose_project, reset_app


Platform Setup

Flutter (iOS / Android / Web)
dependencies:
  flutter_skill: ^0.9.36
import 'package:flutter_skill/flutter_skill.dart';

void main() {
  if (kDebugMode) FlutterSkillBinding.ensureInitialized();
  runApp(MyApp());
}
React Native
npm install flutter-skill-react-native
import FlutterSkill from 'flutter-skill-react-native';
FlutterSkill.start();
Electron
npm install flutter-skill-electron
const { FlutterSkillBridge } = require('flutter-skill-electron');
FlutterSkillBridge.start(mainWindow);
iOS (Swift)
// Swift Package Manager: FlutterSkillSDK
import FlutterSkill
FlutterSkillBridge.shared.start()

Text("Hello").flutterSkillId("greeting")
Android (Kotlin)
implementation("com.flutterskill:flutter-skill:0.8.0")

FlutterSkillBridge.start(this)
Tauri (Rust)
[dependencies]
flutter-skill-tauri = "0.8.0"
KMP Desktop

Add Gradle dependency β€” see sdks/kmp for details.

.NET MAUI

Add NuGet package β€” see sdks/dotnet-maui for details.


Example Prompts

Just tell your AI what to test:

Prompt What happens
"Test login with wrong password" Screenshots β†’ enters creds β†’ taps login β†’ verifies error
"Explore every screen and report bugs" Systematically navigates all screens, tests all elements
"Fill registration with edge cases" Tests emoji 🌍, long strings, empty fields, special chars
"Compare checkout flow on iOS and Android" Runs same test on both platforms, compares screenshots
"Take screenshots of all 5 tabs" Taps each tab, captures state

Contributing

See CONTRIBUTING.md for guidelines.

git clone https://github.com/ai-dashboad/flutter-skill
cd flutter-skill
dart pub get
dart run bin/flutter_skill.dart server  # Start MCP server

Links

πŸ“¦ pub.dev 🧩 VSCode
πŸ“¦ npm 🧩 JetBrains
🍺 Homebrew πŸ“– Docs
πŸ€– Agent Skill πŸ“‹ Changelog

⭐ If flutter-skill saves you time, star it so others can find it too!

MIT License Β© 2025

About

AI-powered E2E testing for 10 platforms. 253 MCP tools. Zero config. Works with Claude, Cursor, Windsurf, Copilot. Test Flutter, React Native, iOS, Android, Web, Electron, Tauri, KMP, .NET MAUI β€” all from natural language.

Topics

Resources

License

Contributing

Stars

Watchers

Forks

Sponsor this project

Packages

 
 
 

Contributors