[Paper] Bridging Forecast Accuracy and Inventory KPIs: A Simulation-Based Software Framework

Source: arXiv - 2601.21844v1

Overview

The paper presents a simulation‑based software framework that lets researchers and practitioners evaluate demand‑forecasting models through the lens of inventory‑management KPIs (total cost, service level, etc.) rather than just statistical error scores. By closing the loop between forecast generation and inventory decisions, the authors expose why a “more accurate” model on paper may not actually improve real‑world spare‑parts operations in the automotive aftermarket.

Key Contributions

• End‑to‑end simulation platform that integrates
  1. a synthetic spare‑parts demand generator,
  2. a plug‑and‑play forecasting engine, and
  3. an inventory control simulator.
• Decision‑centric evaluation metric: quantifies how forecast errors propagate to operational KPIs, shifting focus from MAE/RMSE to cost‑service trade‑offs.
• Empirical evidence that conventional accuracy improvements do not guarantee better inventory performance; models with similar error statistics can produce vastly different cost outcomes.
• Diagnostic analysis linking specific forecast error patterns (bias, variance, intermittent spikes) to inventory KPI deviations, offering actionable guidance for model selection.
• Open‑source‑ready design (modular code, configurable scenarios) that can be adapted to other intermittent‑demand domains beyond automotive spare parts.

Methodology

1. Synthetic Demand Generator – Uses statistical distributions (e.g., Poisson‑Gamma mixtures) calibrated to real spare‑parts data to mimic intermittent, bursty demand patterns.
2. Forecasting Module – A thin wrapper that can load any Python‑compatible model (ARIMA, Prophet, LSTM, Gradient Boosting, etc.). The framework records standard error metrics for each run.
3. Inventory Control Simulator – Implements a classic (s, S) policy with lead‑time, holding, shortage, and ordering costs. It consumes the forecasts as “demand signals” and outputs KPI values: total cost, fill‑rate, average inventory, etc.
4. Closed‑Loop Experiments – For each scenario (varying demand volatility, lead‑time, cost parameters), the authors run multiple forecast models, collect both statistical errors and KPI outcomes, and then analyze correlations and divergences.

The whole pipeline is automated, allowing batch runs and reproducible comparisons across dozens of model‑scenario combinations.

Results & Findings

• Weak Correlation: Pearson correlation between RMSE and total cost was often below 0.3, indicating that lower error does not reliably reduce cost.
• Bias Matters More Than Variance: Forecasts with a small systematic under‑forecast bias caused large stock‑out penalties, while higher variance but unbiased forecasts performed better on cost.
• Model‑Specific Trade‑offs: Gradient‑boosted trees achieved the lowest MAE but produced higher safety‑stock levels, inflating holding costs; a simple exponential smoothing model, though less accurate statistically, yielded a superior cost‑service balance in many scenarios.
• Scenario Sensitivity: In high‑lead‑time, high‑penalty environments, the choice of forecasting horizon (short vs. long) had a bigger impact on KPIs than raw accuracy.

These findings lead to a set of selection heuristics (e.g., prioritize unbiasedness for high service‑level contracts, tolerate higher MAE if it reduces safety stock) that can guide practitioners.

Practical Implications

• Tool for DevOps of Forecasting Pipelines: Teams can plug their production models into the framework to automatically assess downstream inventory impact before deployment.
• Cost‑Driven Model Tuning: Instead of optimizing solely for MAE, data scientists can incorporate KPI‑based loss functions or multi‑objective optimization that directly target total cost or fill‑rate.
• Risk Management: By simulating “what‑if” scenarios (e.g., sudden demand spikes, supplier delays), inventory planners can quantify the financial exposure of different forecasting strategies.
• Cross‑Domain Applicability: The modular design makes it easy to adapt the framework for other intermittent‑demand contexts such as aerospace parts, medical supplies, or e‑commerce “long‑tail” SKUs.
• Accelerated A/B Testing: Companies can run parallel simulations of legacy vs. new forecasting models, presenting decision makers with concrete KPI projections rather than abstract error numbers.

Limitations & Future Work

• Synthetic Demand Only: While the generator is calibrated to real data, the study does not validate the framework on live production streams; real‑world noise (e.g., promotions, warranty returns) may affect results.
• Single Inventory Policy: The simulator uses a classic (s, S) policy; more advanced policies (e.g., reinforcement‑learning‑based replenishment) could interact differently with forecast errors.
• Scalability: Running large‑scale Monte Carlo simulations can be computationally intensive; future work could explore cloud‑native parallelization or surrogate modeling.
• Extension to Multi‑Echelon Networks: The current focus is a single stocking location; extending the framework to multi‑tier supply chains would broaden its industrial relevance.

Overall, the paper delivers a practical bridge between AI‑driven forecasting and the hard‑nosed economics of inventory management, giving developers a concrete way to measure the true business value of their models.

Authors

• So Fukuhara
• Abdallah Alabdallah
• Nuwan Gunasekara
• Slawomir Nowaczyk

Paper Information

• arXiv ID: 2601.21844v1
• Categories: cs.AI, cs.SE
• Published: January 29, 2026
• PDF: Download PDF