Every AI agent that touches an Android device hits the same wall. Not permissions. Not compatibility. Latency. Latency. A single tap takes 300–1000ms through conventional tools. A UI tree dump? 1–5 seconds through ADB. Fill a 40-field form and you're staring at your screen for 17 minutes while your "intelligent" agent crawls through a protocol stack designed for human debuggers, not AI agents. I've spent years building Android tools — from SmartScreen (which anticipated Google's Adaptive Timeout by nine years) to production apps serving thousands of users. When I started building AI agents that needed to control Android devices, I expected the tooling to be ready. It wasn't even close. SmartScreen So I built NeuralBridge — an open-source Android companion app that gives AI agents sub-10ms control over any Android device. No ADB process spawning. No five-hop protocol chains. No middleware servers. NeuralBridge Just one HTTP call from agent to device. ⚡ The Problem Nobody's Talking About Here's what happens when an AI agent taps a button through a conventional tool: Agent → HTTP Request → Middleware Server → ADB Bridge → UIAutomator2 → Device Agent → HTTP Request → Middleware Server → ADB Bridge → UIAutomator2 → Device Five hops. Each one adds latency, error surface, and complexity. The "fast" path through other MCP-based tools isn't much better — every action spawns a new ADB process (100–200ms overhead) plus a UIAutomator dump (1–5 seconds). This isn't a minor inconvenience. It's an architectural bottleneck that makes real-time AI automation impossible. It's an architectural bottleneck that makes real-time AI automation impossible. Consider a realistic scenario: an AI agent filling out a health insurance form. 40 fields. Each tap-and-type cycle takes ~500ms through conventional tools. That's 20 seconds of pure waiting — and that's the optimistic case. With mandatory settle delays, you're looking at minutes. The agent isn't thinking. It isn't reasoning. It's waiting for infrastructure to catch up. 🏗️ One Hop Instead of Five NeuralBridge's key insight is deceptively simple: put the MCP server inside the Android app itself. put the MCP server inside the Android app itself. Agent → HTTP → NeuralBridge Companion App (in-process execution) Agent → HTTP → NeuralBridge Companion App (in-process execution) One hop. That's it. The companion app embeds a Ktor CIO HTTP server on port 7474, speaking native MCP protocol. When an agent sends a tap command, it doesn't cross process boundaries, spawn ADB shells, or traverse middleware. The app's AccessibilityService executes the gesture in-process, directly through Android's dispatchGesture() API. The result: a tap completes in ~2ms — down from the 300–500ms that conventional tools require. The result: a tap completes in ~2ms — down from the 300–500ms that conventional tools require. My rule for this project was simple: if you're crossing a process boundary to do something the OS can do in-process, you've already lost. Every millisecond of IPC overhead is a tax on intelligence. I'd rather spend six months putting the server inside the app than accept a 300ms floor on every action. Why AccessibilityService? Most Android automation tools treat ADB as the primary interface. ADB is powerful, but it was designed for debugging — not for real-time control by AI agents. Every ADB command: Spawns a new OS process
Establishes a USB/TCP connection
Serializes the command
Waits for the Android runtime to process it
Deserializes the response Spawns a new OS process Establishes a USB/TCP connection Serializes the command Waits for the Android runtime to process it Deserializes the response AccessibilityService bypasses all of this. It runs inside the Android runtime with direct access to the view hierarchy, gesture dispatch, and screen content. No IPC. No serialization. No process spawning. NeuralBridge uses a two-tier strategy that routes ~95% of operations through this fast path: Path

Operations

Latency

When Used



Fast (AccessibilityService)

Taps, swipes, text input, UI tree, screenshots

<10ms

95% of the time



Slow (ADB fallback)

Force-stop apps, clipboard on Android 10+, package management

~200ms

5% — only when Android restricts direct access Path

Operations

Latency

When Used



Fast (AccessibilityService)

Taps, swipes, text input, UI tree, screenshots

<10ms

95% of the time



Slow (ADB fallback)

Force-stop apps, clipboard on Android 10+, package management

~200ms

5% — only when Android restricts direct access Path

Operations

Latency

When Used Path Path Operations Operations Latency Latency When Used When Used Fast (AccessibilityService)

Taps, swipes, text input, UI tree, screenshots

<10ms

95% of the time Fast (AccessibilityService) Fast (AccessibilityService) Fast Taps, swipes, text input, UI tree, screenshots Taps, swipes, text input, UI tree, screenshots <10ms <10ms 95% of the time 95% of the time Slow (ADB fallback)

Force-stop apps, clipboard on Android 10+, package management

~200ms

5% — only when Android restricts direct access Slow (ADB fallback) Slow (ADB fallback) Slow Force-stop apps, clipboard on Android 10+, package management Force-stop apps, clipboard on Android 10+, package management ~200ms ~200ms 5% — only when Android restricts direct access 5% — only when Android restricts direct access The slow path exists because Android intentionally restricts certain operations from non-system apps. But even at 200ms, it's still faster than most tools' fast path. 📊 The Numbers That Matter I ran benchmarks against every major Android automation tool. Here's what I found: Operation

NeuralBridge

Tool A

Tool B

Tool C

Tool D



Tap

~2ms

300–1000ms

750ms–2s

500–2000ms

200–1000ms



Swipe

~2ms

300–1000ms

750ms–2s

500–2000ms

200–1000ms



Text Input

~1.4ms

500–3000ms

750ms–2s

500–2000ms

200–1000ms



UI Tree

18–33ms

500–2000ms

750ms–2s

1–5s

200–500ms



Screenshot

~60ms

300–500ms

~1s

300–500ms

~250ms



Average

~6.4ms

~800ms

~1.2s

~1.5s

~500ms Operation

NeuralBridge

Tool A

Tool B

Tool C

Tool D



Tap

~2ms

300–1000ms

750ms–2s

500–2000ms

200–1000ms



Swipe

~2ms

300–1000ms

750ms–2s

500–2000ms

200–1000ms



Text Input

~1.4ms

500–3000ms

750ms–2s

500–2000ms

200–1000ms



UI Tree

18–33ms

500–2000ms

750ms–2s

1–5s

200–500ms



Screenshot

~60ms

300–500ms

~1s

300–500ms

~250ms



Average

~6.4ms

~800ms

~1.2s

~1.5s

~500ms Operation

NeuralBridge

Tool A

Tool B

Tool C

Tool D Operation Operation NeuralBridge NeuralBridge Tool A Tool A Tool B Tool B Tool C Tool C Tool D Tool D Tap

~2ms

300–1000ms

750ms–2s

500–2000ms

200–1000ms Tap Tap ~2ms ~2ms ~2ms 300–1000ms 300–1000ms 750ms–2s 750ms–2s 500–2000ms 500–2000ms 200–1000ms 200–1000ms Swipe

~2ms

300–1000ms

750ms–2s

500–2000ms

200–1000ms Swipe Swipe ~2ms ~2ms ~2ms 300–1000ms 300–1000ms 750ms–2s 750ms–2s 500–2000ms 500–2000ms 200–1000ms 200–1000ms Text Input

~1.4ms

500–3000ms

750ms–2s

500–2000ms

200–1000ms Text Input Text Input ~1.4ms ~1.4ms ~1.4ms 500–3000ms 500–3000ms 750ms–2s 750ms–2s 500–2000ms 500–2000ms 200–1000ms 200–1000ms UI Tree

18–33ms

500–2000ms

750ms–2s

1–5s

200–500ms UI Tree UI Tree 18–33ms 18–33ms 18–33ms 500–2000ms 500–2000ms 750ms–2s 750ms–2s 1–5s 1–5s 200–500ms 200–500ms Screenshot

~60ms

300–500ms

~1s

300–500ms

~250ms Screenshot Screenshot ~60ms ~60ms ~60ms 300–500ms 300–500ms ~1s ~1s 300–500ms 300–500ms ~250ms ~250ms Average

~6.4ms

~800ms

~1.2s

~1.5s

~500ms Average Average Average ~6.4ms ~6.4ms ~6.4ms ~800ms ~800ms ~1.2s ~1.2s ~1.5s ~1.5s ~500ms ~500ms That's 100x faster on average. NeuralBridge numbers measured on a Pixel 7 running Android 14 over WiFi, averaged across 100 runs per operation. Competitor numbers sourced from their official documentation and community benchmarks (linked in the full benchmark doc). I'd welcome head-to-head benchmarks from the community — if you run your own, open an issue with results. That's 100x faster on average. full benchmark doc open an issue The form-filling scenario tells the story better than averages: NeuralBridge: 40 fields × ~100ms (tap + type + navigate) = ~4 seconds
Conventional tools: 40 fields × ~25 seconds (with settle delays) = ~17 minutes NeuralBridge: 40 fields × ~100ms (tap + type + navigate) = ~4 seconds NeuralBridge: ~4 seconds Conventional tools: 40 fields × ~25 seconds (with settle delays) = ~17 minutes Conventional tools: ~17 minutes Same task. Same device. Same AI agent. The gap widens further when you factor in mandatory settle delays between actions. To be fair about what these numbers don't show: Some existing tools have decade-old ecosystems with drivers for iOS, Windows, and web — NeuralBridge is Android-only. Some tools use intentional settle delays (750ms+) that buy near-zero flakiness at the cost of speed. And ADB-forwarded TCP approaches work over USB without needing WiFi connectivity. NeuralBridge wins on raw latency and token efficiency, but it's not a drop-in replacement for every use case. 🧠 Designed for AI, Not Adapted for AI Most automation tools were built for human-driven test scripts and later retrofitted with MCP adapters. NeuralBridge was built AI-native from day one. Three design decisions make this tangible: 1. Semantic Selectors (Not Pixel Coordinates) AI agents think in terms of "tap the Login button," not "tap pixel (540, 820)." NeuralBridge's selector system bridges this gap with a six-step resolution chain: Exact text match
Partial text match (contains)
Content description match
Resource ID match
Combined AND match (multiple criteria)
Fuzzy match (tolerance for typos and variations) Exact text match Partial text match (contains) Content description match Resource ID match Combined AND match (multiple criteria) Fuzzy match (tolerance for typos and variations) When multiple elements match, a tiebreaker picks the most likely target: visible elements beat hidden ones, interactive elements beat static ones, center-positioned elements beat edge ones. The agent says tap(text="Login"). NeuralBridge handles the rest. The agent says tap(text="Login"). NeuralBridge handles the rest. 2. Token-Optimized Responses Every token an AI agent processes costs money and time. A typical 200-node Android screen generates ~3,000 tokens of UI tree data in conventional tools. NeuralBridge compresses this to ~800 tokens — a 73% reduction — without losing actionable information. 73% reduction How: Compact format: Tabular representation instead of verbose JSON
Interactive-only filter: Strips ~80% of non-interactive decoration nodes
Smart omission: Empty fields, null values, and redundant metadata are dropped
Compressed bounds: [l,t,r,b] instead of {"left":0,"top":0,"right":1080,"bottom":2400} Compact format: Tabular representation instead of verbose JSON Compact format: Interactive-only filter: Strips ~80% of non-interactive decoration nodes Interactive-only filter: Smart omission: Empty fields, null values, and redundant metadata are dropped Smart omission: Compressed bounds: [l,t,r,b] instead of {"left":0,"top":0,"right":1080,"bottom":2400} Compressed bounds: This optimization is transparent — agents don't need to request it. It's the default. Your AI agent spends less on API calls and more on actual reasoning. 3. Context-Rich Observations The get_screen_context tool returns both a screenshot and the semantic UI tree in a single call (~70ms). This gives agents what human testers get intuitively: a visual snapshot paired with actionable element metadata. One call instead of two. Half the latency. Complete situational awareness. 🔧 32 Tools That Show AI-Native Design NeuralBridge ships with 32 tools organized across seven categories: observe, act, manage, wait, test, device, and meta. Rather than listing them all, here are three that show how the AI-native design plays out in practice: get_screen_context — Returns a screenshot and semantic UI tree in a single ~70ms call. Most tools force agents to make two round-trips (screenshot + tree dump), doubling latency and token cost. One call, complete situational awareness. get_screen_context scroll_to_element — The agent says "find the Submit button." NeuralBridge scrolls through the page, checking the accessibility tree after each scroll, until it finds the element or gives up after 30 seconds. No pixel-guessing. No "scroll down 3 times and hope." scroll_to_element accessibility_audit — Checks every element on screen for touch target size, missing content descriptions, and contrast issues. Returns results in under 50ms. What used to require a manual pass with Android's Accessibility Scanner now happens as a single MCP call. accessibility_audit The full tool reference covers all 32. full tool reference 🔬 The Technical Details That Took Months Building NeuralBridge wasn't just about the architecture insight. The hard part was making it production-grade across Android's fragmented ecosystem. Screenshot Pipeline Screenshots use MediaProjection with a JNI bridge to libjpeg-turbo for JPEG encoding in C++. The result: ~60ms for a full-resolution capture. When MediaProjection consent resets (Android 14+ does this on app restart), NeuralBridge automatically falls back to ADB screencap — slower, but reliable. Screenshots are only useful if the agent can reliably refer back to the elements it sees. Stable Element IDs Android's built-in view IDs are unstable — they change across app updates, language switches, and even screen rotations. NeuralBridge generates hash-based element IDs from the element's structural position and content. Same element, same ID, regardless of what changes around it. Cross-Version Compatibility The app runs on Android 7.0 through Android 15+. That's nine major versions with different permission models, accessibility APIs, and security restrictions. Each version required specific handling: Android 10+: Background clipboard access restricted → ADB fallback
Android 14+: MediaProjection consent lifecycle changes → automatic re-consent handling
Android 15+: Restricted settings for accessibility → manual enablement required Android 10+: Background clipboard access restricted → ADB fallback Android 10+: Android 14+: MediaProjection consent lifecycle changes → automatic re-consent handling Android 14+: Android 15+: Restricted settings for accessibility → manual enablement required Android 15+: Cold Start Reality There's one number I'm not proud of: the first request after app launch takes 30–120 seconds. Ktor's CIO engine JIT-compiles on the first incoming connection, and there's no way to avoid it short of switching to a pre-compiled server. Every subsequent request is sub-10ms, but that first one hurts. I'm exploring AOT compilation options to kill this, but for now, it's a known tradeoff — slow start, fast everything after. 🌍 Why This Matters Now Three things happened in the last 12 months: Claude gained computer-use capabilities, Google launched Project Mariner, and OpenAI shipped Operator. The industry has decided that AI agents will control real software on real devices. But the infrastructure is still stuck in the ADB era. The gap between "AI can reason about what to do on a phone" and "AI can actually do it fast enough to be useful" is enormous. Right now, an agent can plan a 10-step workflow in 200ms and then spend 30 seconds executing it. NeuralBridge closes that gap. 🚧 What NeuralBridge Can't Do Games that render via OpenGL, Unity, or Unreal have no accessibility tree — NeuralBridge can't see or interact with their UI elements. Banking apps with FLAG_SECURE block screenshots. Biometric prompts can't be automated. And Google Play won't distribute apps that use AccessibilityService for automation, so NeuralBridge is sideloaded. NeuralBridge is built for standard Android apps — which covers ~95% of what AI agents need to interact with. When an agent can execute 40 UI actions in 4 seconds instead of 17 minutes, new categories of automation become possible: When an agent can execute 40 UI actions in 4 seconds instead of 17 minutes, new categories of automation become possible: Real-time app testing where the AI adapts its strategy based on what it sees, without waiting seconds between observations
Multi-app workflows — Gmail to Calendar to Maps — completed in the time it takes a human to unlock their phone
Accessibility auditing at scale — automatically checking touch targets, content descriptions, and contrast ratios across every screen in an app
Continuous integration for mobile — running full UI test suites in seconds, not hours Real-time app testing where the AI adapts its strategy based on what it sees, without waiting seconds between observations Real-time app testing Multi-app workflows — Gmail to Calendar to Maps — completed in the time it takes a human to unlock their phone Multi-app workflows Accessibility auditing at scale — automatically checking touch targets, content descriptions, and contrast ratios across every screen in an app Accessibility auditing at scale Continuous integration for mobile — running full UI test suites in seconds, not hours Continuous integration for mobile 🚀 Getting Started NeuralBridge is open source (Apache 2.0) and requires no root access. Setup: Setup: Install the companion APK on your Android device
Enable the NeuralBridge AccessibilityService in Settings
Grant screen capture permission on first use
Point your AI agent to the device's IP: Install the companion APK on your Android device Enable the NeuralBridge AccessibilityService in Settings Grant screen capture permission on first use Point your AI agent to the device's IP: {
  "mcpServers": {
    "neuralbridge": {
      "type": "http",
      "url": "http://<device-wifi-ip>:7474/mcp"
    }
  }
} {
  "mcpServers": {
    "neuralbridge": {
      "type": "http",
      "url": "http://<device-wifi-ip>:7474/mcp"
    }
  }
} That's it. Your agent now has sub-10ms control over any Android device on your network. No middleware server. No ADB forwarding. No middleware. Security note: The MCP server currently runs without authentication — anyone on your WiFi network can send commands. This is fine for development and testing on trusted networks, but don't expose it on public WiFi. Token-based auth is on the roadmap. Security note: The Bigger Picture I didn't build NeuralBridge because the world needed another automation framework. I built it because I was tired of watching AI agents — systems capable of sophisticated reasoning — bottlenecked by infrastructure that treats latency as an afterthought. The first generation of mobile automation tools was built for human testers writing scripts. The second generation bolted on AI adapters. NeuralBridge is what happens when you start from zero with a single question: what would mobile automation look like if it were designed for AI agents from the ground up? what would mobile automation look like if it were designed for AI agents from the ground up? The answer is 100x faster, 73% cheaper on tokens, and fits in a single Android app. What's next: Multi-device orchestration, WebView inspection tools, and iOS support. If you want to see how fast your agents can actually move, grab the code and build APK. If you find a tool that's missing, open an issue — I read every one. What's next: grab the open an issue NeuralBridge is open source (Apache 2.0) on GitHub. NeuralBridge is open source (Apache 2.0) on GitHub. GitHub I build the tools I wish existed — and then open-source them. Follow me on HackerNoon for more on AI agents, Android internals, and the infrastructure nobody else wants to write. HackerNoon

The code in this story is for educational purposes. The readers are solely responsible for whatever they build with it.

Apache

I Built a 100x Faster Android Automation Tool Because AI Agents Deserve Better

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Building Multi-Agent Systems That Communicate Reliably with the A2A Protocol

20 Hours of Work, $258 in Sales — The First Week of Shade

4 Quick and Easy Tips to Make Your WordPress Website ADA Compliant

4 ways CSS :has() can make your HTML forms even better

5 Resources to Measure Website Accessibility

6 Tips to Host a Successful Virtual Networking Event

Building Multi-Agent Systems That Communicate Reliably with the A2A Protocol

20 Hours of Work, $258 in Sales — The First Week of Shade

4 Quick and Easy Tips to Make Your WordPress Website ADA Compliant

4 ways CSS :has() can make your HTML forms even better

5 Resources to Measure Website Accessibility

6 Tips to Host a Successful Virtual Networking Event

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps