[Paper] Measuring Agile Agreement: Development and Validation of the Manifesto and Principle Scales

Source: arXiv - 2512.08461v1

Overview

The paper introduces two rigorously tested questionnaires that let researchers and practitioners measure how much individuals “agree” with agile values versus agile practices. By separating agreement with the high‑level Agile Manifesto from agreement with the concrete 12 Principles, the authors fill a long‑standing gap in agile‑related measurement tools.

Key Contributions

• Manifesto Agreement Scale (MAS) – a brand‑new instrument that captures alignment with the four Agile Manifesto values.
• Principle Agreement Scale (PAS) – an improved, validated version of an existing scale that measures agreement with the 12 Agile Principles (the day‑to‑day practices).
• Systematic development process (item generation, expert review, pilot testing) that can be reused for building other psychometric tools.
• Robust validation using internal consistency (Cronbach’s α), construct validity, and convergent/divergent analyses (Proportional Odds Logistic Regression, Bland‑Altman plot, ICC).
• Open‑source release of both scales and the accompanying data set (Belgian IT professionals), enabling immediate adoption and further research.

Methodology

1. Item Creation – Drafted statements reflecting each of the four Manifesto values and each of the 12 Principles, phrased in plain language for software engineers, testers, and managers.
2. Expert Review – Agile coaches and scholars evaluated relevance, clarity, and redundancy; ambiguous items were revised or removed.
3. Pilot Survey – Sent a preliminary questionnaire to a small sample of Belgian IT workers to detect confusing wording and estimate response distributions.
4. Full‑scale Administration – Administered the refined MAS (4 items) and PAS (12 items) online to 352 professionals across various roles (developers, scrum masters, product owners, etc.).
5. Statistical Validation
   • Reliability: Cronbach’s α = 0.84 (MAS) and 0.88 (PAS).
   • Construct validity: Factor analysis confirmed a single underlying factor for each scale.
   • Convergent/Divergent analysis: Moderate correlation (r ≈ 0.45) indicated related but distinct constructs; ICC = 0.52 showed they are not interchangeable.
   • Additional checks: Proportional odds logistic regression tested predictive consistency across demographic groups; Bland‑Altman plots visualized agreement limits.

Results & Findings

• Both scales demonstrate high internal consistency, meaning respondents answer items in a coherent way.
• MAS and PAS are moderately correlated (they move together to some extent) but capture different dimensions: a developer can strongly endorse the Manifesto values yet disagree with specific principles (e.g., “continuous delivery” vs. “working software over comprehensive documentation”).
• The PAS proved slightly more discriminative across experience levels, suggesting that practice‑level agreement evolves with hands‑on exposure.
• Validation metrics (factor loadings > 0.6, ICC ≈ 0.5) confirm that the two instruments should be used together, not as substitutes.

Practical Implications

• Team health diagnostics: Scrum masters can quickly gauge whether a team’s cultural alignment (MAS) matches its operational habits (PAS), spotting mismatches before they become blockers.
• Hiring & onboarding: Recruiters can assess “person‑agile fit” by administering the scales, helping to match candidates whose values and practice preferences align with the target organization.
• Training impact measurement: Agile coaches can run pre‑ and post‑workshop surveys to see if a training shifts agreement on principles without necessarily altering core values.
• Process improvement: Organizations can identify which principles need reinforcement (e.g., “sustainable pace”) while preserving the overarching values that drive motivation.
• Research standardization: Future empirical studies on agile adoption can adopt MAS/PAS, enabling cross‑study comparisons and meta‑analyses.

Limitations & Future Work

• Sample bias: Validation was performed exclusively on Belgian IT professionals; cultural or regional differences may affect item interpretation.
• Cross‑sectional design: The study captures a snapshot in time; longitudinal data would reveal how agreement evolves with project cycles.
• Scope of constructs: The scales focus on agreement, not actual behavior; future work could link MAS/PAS scores to observable metrics (e.g., delivery frequency, defect rates).
• Extension to other agile frameworks: While rooted in Scrum‑centric environments, applying the scales to Kanban, XP, or SAFe contexts warrants further testing.

The authors have made the questionnaires and anonymized data publicly available, inviting the community to extend, translate, and refine these tools for broader, global use.

Authors

• Nicolas Matton
• Anthony Simonofski
• Marie-Ange Remiche
• Benoît Vanderose

Paper Information

• arXiv ID: 2512.08461v1
• Categories: cs.SE
• Published: December 9, 2025
• PDF: Download PDF