[Paper] The State of the SBOM Tool Ecosystems: A Comparative Analysis of SPDX and CycloneDX

Source: arXiv - 2512.21781v1

Overview

The paper The State of the SBOM Tool Ecosystems: A Comparative Analysis of SPDX and CycloneDX takes a deep dive into the two leading Software Bill of Materials (SBOM) standards—SPDX and CycloneDX—by measuring the health, maturity, and real‑world usage of their surrounding tool ecosystems. Understanding which ecosystem is stronger where helps developers and security teams choose the right format for their supply‑chain transparency needs.

Key Contributions

• Large‑scale empirical survey of 170 publicly advertised SBOM tools (171 CycloneDX‑focused vs. 470 SPDX‑focused).
• Health‑metric analysis (e.g., activity level, issue resolution speed, contributor count) for each ecosystem, revealing maturity gaps.
• Issue‑report mining of 36,990 tickets across open‑source SBOM projects to surface common pain points and development bottlenecks.
• Adoption study of the top 250 open‑source projects that rely on each ecosystem, comparing project health indicators (stars, forks, CI integration).
• Actionable recommendations for tooling vendors, standards bodies, and practitioners on how the two ecosystems can complement each other.

Methodology

1. Tool Identification – The authors scraped public repositories, package registries, and vendor listings to compile a catalog of SBOM tools that explicitly target SPDX or CycloneDX.
2. Metric Definition – For each tool they collected quantitative “health” signals: number of releases, commit frequency, open‑vs‑closed issue ratio, contributor count, and time‑to‑first‑response.
3. Issue‑Report Mining – Using GitHub’s REST API, they extracted 36,990 issue tickets, then applied natural‑language processing to classify them (bugs, feature requests, documentation, security).
4. Project‑Level Analysis – The top 250 open‑source projects using each format were identified via dependency graphs and build‑tool integrations; standard OSS health metrics (stars, forks, CI pass rate) were recorded.
5. Statistical Comparison – Non‑parametric tests (Mann‑Whitney U) were used to assess whether observed differences between the two ecosystems were statistically significant.

Results & Findings

• Tool Volume & Diversity – SPDX boasts roughly 2.8× more tools than CycloneDX, reflecting broader industry adoption and longer history.
• Developer Engagement – Projects that use CycloneDX tools show higher average contributor count (≈ 12 vs 7) and faster issue‑resolution times (median ≈ 2 days vs 5 days for SPDX).
• Issue Landscape – Both ecosystems suffer from documentation gaps, but CycloneDX tools have a higher proportion of feature‑request tickets (38 % vs 22 % for SPDX), indicating unmet user needs.
• Project Health – Open‑source projects built on CycloneDX tend to have higher CI pass rates and more recent releases, suggesting a more “living” ecosystem. SPDX‑based projects, however, enjoy greater tooling breadth (e.g., CI plugins, vulnerability scanners).
• Maturity Trade‑off – SPDX’s larger tool pool translates into better coverage of niche languages and build systems, while CycloneDX’s tighter community yields more rapid iteration and higher user satisfaction.

Practical Implications

• Tool Selection – Teams needing immediate, out‑of‑the‑box integration across many languages may gravitate toward SPDX, whereas organizations that value fast‑moving, community‑driven tooling (e.g., for container images) might pick CycloneDX.
• Supply‑Chain Automation – CI/CD pipelines can leverage the higher issue‑resolution speed of CycloneDX tools to reduce bottlenecks in SBOM generation and validation.
• Vendor Roadmaps – Tool vendors can prioritize the “feature‑request” clusters identified in the issue analysis (e.g., better support for monorepos, richer vulnerability linking) to close the gap between the ecosystems.
• Policy & Compliance – Regulators and auditors referencing SBOM standards can use these findings to justify allowing dual‑format compliance strategies, letting organizations benefit from the strengths of both ecosystems.
• Community Collaboration – The paper highlights concrete areas (e.g., shared parsers, cross‑format converters) where SPDX and CycloneDX communities could collaborate, reducing duplication of effort.

Limitations & Future Work

• Tool Discovery Bias – The catalog relies on publicly advertised tools; obscure or internal enterprise solutions may be under‑represented.
• Static Snapshot – Metrics were captured at a single point in time; rapid ecosystem changes (new releases, mergers) could shift the balance.
• Depth of Issue Classification – Automated NLP classification may mislabel nuanced tickets, affecting the accuracy of the “feature‑request” vs. “bug” split.
• Future Directions – The authors suggest longitudinal studies to track ecosystem evolution, deeper qualitative interviews with maintainers, and the development of a unified benchmark suite to evaluate SBOM tool performance across formats.

Authors

• Abdul Ali Bangash
• Tongxu Ge
• Zhimin Zhao
• Arshdeep Singh
• Zitao Wang
• Bram Adams

Paper Information

• arXiv ID: 2512.21781v1
• Categories: cs.SE
• Published: December 25, 2025
• PDF: Download PDF