We Ran 7,600+ Cloud Provisioning Tests Across AWS, Azure, and GCP — Here's What We Found
Source: Dev.to
Introduction
Nobody publishes this data, so we measured it ourselves. Cloud providers share uptime SLAs, pricing calculators, and feature comparison tables, but they don’t reveal how long it actually takes to provision infrastructure—or how often provisioning fails. To fill that gap we built ProvisioningIQ, which continuously runs real API calls (no simulations) to provision and then destroy resources across AWS, Azure, and GCP.
Methodology
- Scope: 7,600+ real provisioning tests (VMs or serverless containers).
- Frequency: 3 × day, across three regions per cloud, running continuously since January 2026.
- Process:
- Provision a real resource (VM or serverless container).
- Measure time at each phase: API accepted → allocating → ready → reachable.
- Record success/failure and failure category.
- Immediately destroy the resource.
Container Provisioning Times
| Cloud | Service | p50 Latency | p95 Latency | Success Rate |
|---|---|---|---|---|
| GCP | Cloud Run | 6–8 s | ~20 s | 100 % |
| AWS | ECS | ~20 s | ~40 s | 100 % |
| Azure | ACI | ~40 s | ~60 s | 100 % |
Observation: GCP Cloud Run provisions 10–20× faster than Azure ACI at the p50 level, and this advantage is consistent across all tested regions.
VM Provisioning Times
| Cloud | Service | p50 Latency | Success Rate |
|---|---|---|---|
| AWS | EC2 | ~34 s | 99.8 % |
| Azure | VM | 72–86 s | 99.7 % |
| GCP | GCE | ~100 s | 98.5 % |
Observation: AWS leads on VMs with the fastest p50 latency and the highest reliability. GCP’s VMs are noticeably slower than its containers, making Cloud Run the preferred GCP option for latency‑sensitive workloads.
Key Takeaways
-
On‑call impact: Engineers deal with the p95, not the average.
- AWS containers p95: ~40 s
- Azure containers p95: ~60 s
- GCP containers p95: ~20 s
-
Incident response: A 20‑second recovery (GCP) vs. a 60‑second recovery (Azure) can be the difference between users noticing an outage or not.
-
Regional variability: Provisioning times vary meaningfully between regions. Maintenance windows can temporarily double provisioning times in specific regions, and providers do not warn about these spikes.
-
Decision factors:
- Auto‑scaling under load
- Disaster recovery speed
- CI/CD pipeline velocity
Faster provisioning translates into tangible engineering time savings (e.g., ~144 hours recovered per team annually).
Additional Insights
-
Negotiation gap: Cloud contracts typically cover price, storage, network egress, and uptime SLA, but never provisioning latency. There is no industry‑wide commitment or benchmark for this metric.
-
Future benchmarking: We are extending our measurements to managed databases (RDS PostgreSQL, Cloud SQL, Azure Database for PostgreSQL) and to Terraform step‑level timings, which will pinpoint where each cloud spends time during provisioning.
ProvisioningIQ Offering
- Free tier: Daily benchmark snapshots at provisioningiq.appswireless.com.
- Pro tier: 90‑day history, p50/p95 trends, per‑region failure analysis, and a daily email digest.
Questions about methodology, failure categorization, or cleanup handling? Drop them in the comments.