[Paper] Metamorphic Testing with the Rashomon Set: Explanation Faithfulness in Machine Learning
Source: arXiv - 2606.06056v1
Overview
Multiple machine learning models can achieve near-equivalent predictive performance on the same task, yet provide divergent feature-based explanations. This is called the Rashomon effect of (explainable) machine learning, and it raises the question of which explanations, if any, are trustworthy. We propose a framework based on metamorphic testing that assesses explanation faithfulness without requiring ground-truth labels by exploring attributed feature importance from post-hoc explanation methods. Five metamorphic relations formalize expected consistency properties between model behavior and feature attributions. We apply this general framework to two tabular regression datasets and two post-hoc explainers (SHAP and LIME) to demonstrate the approach. The framework offers a practical, model-agnostic tool for selecting accurate models with reliable and trustworthy explanations.
Key Contributions
This paper presents research in the following areas:
- cs.SE
- cs.AI
- cs.LG
Methodology
Please refer to the full paper for detailed methodology.
Practical Implications
This research contributes to the advancement of cs.SE.
Authors
- Helge Spieker
- Jørn Eirik Betten
- Arnaud Gotlieb
Paper Information
- arXiv ID: 2606.06056v1
- Categories: cs.SE, cs.AI, cs.LG
- Published: June 4, 2026
- PDF: Download PDF