[Paper] Benchmarking Stopping Criteria for Evolutionary Multi-objective Optimization

Source: arXiv - 2604.25458v1

Overview

The paper tackles a surprisingly overlooked piece of the evolutionary multi‑objective optimization (EMO) puzzle: when to stop the algorithm. By introducing a unified performance metric and a reproducible, file‑based benchmarking workflow, the authors make it far easier to compare and improve stopping criteria—an essential step for deploying EMO in real‑world systems where every function evaluation can be costly.

Key Contributions

• Scalar performance measure for stopping criteria – condenses the trade‑off between solution quality and computational effort into a single, easy‑to‑compare number.
• File‑based benchmarking framework – standardises data exchange, automates experiment orchestration, and enables anyone to reproduce results with a few command‑line steps.
• Compact text‑based population representation – stores entire EMO population snapshots efficiently, keeping benchmark files small without sacrificing fidelity.
• Empirical study of five popular stopping criteria – demonstrates how the proposed tools expose strengths and weaknesses that were previously hidden.

Methodology

1. Define the metric – For each run, the authors record the generation where the stopping criterion fires and the corresponding quality of the Pareto front (using a standard indicator like IGD⁺). The metric combines these two aspects into a single scalar:
   [ \text{Score} = \frac{\text{Quality}}{\text{Evaluations}} ]
   Higher scores mean “good solutions early”.

2. File‑based experiment pipeline –

   • Run phase: EMO algorithms write population snapshots (objective vectors) to a plain‑text log after every generation.
   • Post‑process phase: A lightweight parser reads the logs, applies each stopping rule offline, and computes the scalar scores.
     This decouples the algorithm from the stopping logic, allowing any EMO implementation (Python, Java, C++) to be benchmarked without code changes.

3. Data representation – The authors encode each individual as a space‑separated list of objective values, one line per individual, and compress the whole file with gzip. This reduces a typical 10 k‑generation run from dozens of megabytes to a few megabytes.

4. Benchmark suite – Five stopping criteria (e.g., fixed‑budget, convergence‑based, hypervolume stagnation) are evaluated on a set of standard multi‑objective test problems (ZDT, DTLZ) using two popular EMO algorithms (NSGA‑II, MOEA/D).

Results & Findings

• Hypervolume‑based stagnation consistently achieved the best trade‑off, stopping earlier while still delivering high‑quality Pareto fronts.
• Fixed‑budget approaches waste evaluations on already‑converged populations, confirming the need for smarter stopping.
• The scalar metric cleanly ranked the criteria, and the file‑based pipeline reproduced identical scores across multiple runs and hardware setups, proving its robustness.

Practical Implications

• Cost‑aware optimization – Developers can plug the hypervolume stagnation rule into existing EMO libraries (e.g., DEAP, jMetal) to automatically cut off expensive evaluations in engineering design, hyperparameter tuning, or resource allocation problems.
• Reproducible research & CI – The file‑based benchmark can be integrated into continuous‑integration pipelines: run an EMO job, dump logs, and let a post‑process script verify that a new stopping rule improves the scalar score before merging.
• Cross‑language interoperability – Because the data format is plain text, teams using mixed stacks (Python for prototyping, C++ for production) can share benchmark results without custom serializers.
• Scalable cloud deployments – The compact log format reduces storage and network transfer costs when running massive parallel EMO experiments on cloud clusters.

Limitations & Future Work

• The scalar metric collapses a multi‑dimensional trade‑off into one number, which may hide nuanced preferences (e.g., a developer might tolerate slightly worse quality for a big time saving).
• Experiments were limited to synthetic benchmark suites (ZDT/DTLZ); real‑world case studies (e.g., aerodynamic shape optimisation) are needed to validate the approach under noisy, expensive evaluations.
• The current hypervolume stagnation rule relies on a fixed threshold; future work could explore adaptive thresholds or learning‑based stopping signals that react to problem‑specific dynamics.

Bottom line: By giving the EMO community a simple, reproducible way to measure and compare stopping criteria, this work paves the way for smarter, cost‑effective multi‑objective optimisation in production software.

Authors

• Kenji Kitamura
• Ryoji Tanabe

Paper Information

• arXiv ID: 2604.25458v1
• Categories: cs.NE
• Published: April 28, 2026
• PDF: Download PDF