[Paper] From Research to Practice: An Interactive Rapid Review of Autonomous Driving System Testing in Industry

Source: arXiv - 2605.00531v1

Overview

Autonomous Driving System (ADS) testing is one of the toughest hurdles before self‑driving cars can safely hit the road at scale. This paper bridges the long‑standing gap between academic research on ADS testing and the day‑to‑day realities of engineers working in a major automotive OEM. By running an interactive rapid review with 21 industry practitioners, the authors surface the most pressing testing challenges and evaluate how well existing research actually helps solve them.

Key Contributions

• Practitioner‑driven challenge map: Identification and prioritisation of 12 real‑world ADS testing challenges, with “testing completeness for End‑to‑End (E2E) ADS” and “effective test‑case generation” emerging as top concerns.
• First interactive rapid review: A novel review methodology that involves industry experts throughout the evidence‑gathering and analysis phases, ensuring the findings stay grounded in practice.
• Systematic assessment of 17 research studies: Mapping of current academic solutions to the two highest‑priority challenges, evaluating relevance, maturity, and ease of adoption.
• Actionable gap analysis: Clear illustration of where research falls short (e.g., lack of context‑aware scenario generation, limited integration with existing automotive toolchains).
• Roadmap for future work: Concrete recommendations for researchers aiming to produce industry‑ready ADS testing techniques.

Methodology

1. Stakeholder workshops: The authors hosted structured sessions with 21 engineers, test managers, and safety analysts from a leading car maker. Participants listed testing pain points and voted on the most critical ones.
2. Rapid literature scan: Using a focused set of keywords (e.g., “autonomous driving testing”, “scenario generation”, “end‑to‑end verification”), the team collected 17 recent peer‑reviewed studies that directly addressed the two top challenges.
3. Interactive relevance rating: Practitioners reviewed each paper’s abstract, methodology, and reported results, then scored them on practical relevance, readiness for integration, and coverage of real‑world constraints.
4. Synthesis & gap identification: Scores were aggregated, and the authors performed a thematic analysis to highlight common strengths, weaknesses, and missing pieces across the literature.

The approach is deliberately lightweight (hence “rapid”) yet still systematic, allowing the review to be completed within a few weeks while keeping industry voices at the core.

Results & Findings

• Challenge prioritisation: 70 % of participants flagged E2E testing completeness as the biggest obstacle, followed closely by generation of high‑impact test scenarios.
• Research focus mismatch: 12 of the 17 papers concentrate on scenario synthesis (e.g., adversarial or corner‑case generation) but rarely address full‑stack E2E validation that includes perception, planning, and control together.
• Readiness gap: Only 3 studies were rated “ready for adoption” (i.e., they provide tool support, clear APIs, and have been validated on production‑level vehicle models). The rest were deemed “conceptual” or “prototype‑level”.
• Context awareness lacking: Practitioners highlighted that most academic scenarios ignore domain‑specific constraints such as traffic regulations, vehicle dynamics limits, or sensor placement, limiting their usefulness.
• Toolchain integration: None of the evaluated works offered seamless hooks into the OEM’s existing simulation pipelines (e.g., CARLA, PreScan, or proprietary hardware‑in‑the‑loop setups).

Overall, the review reveals a significant disconnect: while academia excels at generating novel test cases, it often stops short of delivering end‑to‑end, industry‑compatible testing frameworks.

Practical Implications

• For developers: Expect to spend more time curating and adapting academic test‑case generators rather than plugging them straight into your CI pipeline. Look for open‑source tools that expose configurable scenario parameters and can export to your simulation environment.
• For test engineers: Prioritise solutions that provide traceability from generated scenarios back to safety requirements (e.g., ISO 26262, SOTIF). The paper’s gap analysis can serve as a checklist when evaluating new testing vendors.
• For tool vendors: There’s a clear market need for plug‑and‑play E2E testing suites that bundle scenario generation, sensor model fidelity, and automated result analysis. Investing in standards‑based APIs (e.g., OpenSCENARIO, OpenDRIVE) will lower adoption friction.
• For project managers: Allocate budget for co‑development projects with academia or research labs, focusing on proof‑of‑concept pilots that address the two high‑priority challenges identified.

In short, the study nudges the industry toward co‑design of testing solutions rather than a one‑sided adoption of academic prototypes.

Limitations & Future Work

• Single‑company focus: All practitioner input came from one automotive OEM, which may limit the generalisability of the challenge list to other manufacturers or emerging mobility players.
• Rapid review scope: By design, the literature search was intentionally narrow; relevant works outside the chosen keyword set or published after the cut‑off may have been missed.
• Depth of evaluation: Practitioners rated papers based on abstracts and limited demos; a deeper hands‑on trial could reveal additional integration hurdles.

Future research directions suggested by the authors include: building context‑aware scenario generators that embed regulatory and vehicle‑specific constraints; developing standardised evaluation metrics for E2E testing completeness; and conducting multi‑company longitudinal studies to validate the proposed industry‑research alignment framework.

Authors

• Qunying Song
• Ali Nouri
• Håkan Sivencrona
• Federica Sarro

Paper Information

• arXiv ID: 2605.00531v1
• Categories: cs.SE
• Published: May 1, 2026
• PDF: Download PDF