Maestro Flakiness: Source Code Analysis

Source: Dev.to

Maestro markets itself as a test framework that “embraces the instability of mobile applications.”
But what does that actually mean in code? I dug into the source to find out.

The Marketing Promise

From Maestro’s documentation:

“UI elements will not always be where you expect them, screen tap will not always go through, etc. Maestro embraces the instability of mobile applications and devices and tries to counter it.”
“No need to pepper your tests with sleep() calls. Maestro knows that it might take time to load the content and automatically waits for it (but no longer than required).”

Sounds great. Let’s see what the code actually does.

1. Element Finding – The Hard‑coded 17‑Second Timeout

When you write tapOn: "Login", Maestro doesn’t look once and fail. It polls continuously. But for how long?

Source: Orchestra.kt

class Orchestra(
    private val maestro: Maestro,
    private val lookupTimeoutMs: Long = 17000L,        // Hardcoded: 17 seconds
    private val optionalLookupTimeoutMs: Long = 7000L // Hardcoded: 7 seconds for optional
)

What this means

• Every element lookup waits up to 17 seconds by default.
• Optional elements wait 7 seconds.
• You cannot change this per‑command.

Want to wait 30 seconds for a slow API response? Too bad. Want to fail‑fast in 3 seconds for performance testing? Also no.

2. The Polling Mechanism – Simple but Rigid

Source: MaestroTimer.kt

fun  withTimeout(timeoutMs: Long, block: () -> T?): T? {
    val endTime = System.currentTimeMillis() + timeoutMs

    do {
        val result = block()
        if (result != null) {
            return result
        }
    } while (System.currentTimeMillis()

Note: The maximum wait time is 10 seconds (10 polls × 1 second). This value is not configurable.

7. Platform Differences: Android vs iOS

Maestro handles “settling” differently per platform.

Android

Source: AndroidDriver.kt

// Checks if window is still updating
val windowUpdating = blockingStubWithTimeout.isWindowUpdating(...)

iOS

Source: IOSDriver.kt

// Uses screenshot comparison to detect animation end
val didFinishOnTime = waitUntilScreenIsStatic(SCREEN_SETTLE_TIMEOUT_MS)

Both approaches have different reliability characteristics, yet neither offers configurability.

Summary: The Hard‑Coded Reality

The Verdict

Maestro’s “built‑in flakiness handling” does more than raw XCUITest or Espresso, but it is a one‑size‑fits‑all solution with hard‑coded values.

• The code is clean and the approach is sound.
• The lack of escape hatches means that when defaults don’t work for your app, you’re stuck.

This isn’t necessarily bad—it’s a trade‑off for simplicity. However, the marketing suggests more intelligence and adaptability than the code actually provides.

Key Source Files

We’re Not Just Pointing Out Problems

We love Maestro’s YAML syntax—it’s arguably the best thing to happen to mobile test automation in years: simple, readable, and version‑control friendly.

But the execution engine has real limitations:

• Hard‑coded timeouts
• No configurability
• Platform inconsistencies

So we’re building something to fix it.

An open‑source engine that runs Maestro YAML tests on Appium’s battle‑tested infrastructure, offering:

• Configurable timeouts
• Real‑device support
• No magic numbers

Watch this space.

Learn More