[Paper] NEVO-GSPT: Population-Based Neural Network Evolution Using Inflate and Deflate Operators

Source: arXiv - 2601.08657v1

Overview

The paper presents NEVO‑GSPT, a new neuroevolution framework that dramatically cuts the cost of searching for high‑performing neural network architectures. By borrowing geometric semantic operators from genetic programming and adding a novel “deflate” mechanism, the authors achieve predictable, smooth changes to a network’s behavior while keeping the models compact and efficient.

Key Contributions

• Geometric Semantic Operators for NNs – Adapts GSOs (originally used in symbolic regression) to neural network evolution, guaranteeing that structural tweaks translate into monotonic improvements on a unimodal error surface.
• Deflate‑Grow‑Shrink‑Merge (DGSM) Operator – Introduces a controlled “deflation” step that prunes unnecessary neurons/layers without breaking the semantic guarantees of GSOs.
• Efficient Evaluation Scheme – Only the semantics of newly added components need to be computed, enabling fast population‑based training and reducing the typical billions‑of‑forward‑passes cost.
• Compact, High‑Performing Models – Empirically evolves smaller networks that match or beat state‑of‑the‑art baselines (standard NNs, SLIM‑GSGP, TensorNEAT, SLM) on four regression benchmarks.
• Population‑Level Exploration – The method retains the diversity benefits of evolutionary algorithms while staying computationally tractable.

Methodology

1. Population Initialization – Start with a set of minimal seed networks (often a single neuron or shallow MLP).
2. Geometric Semantic Grow (GSG) – For each parent, a grow operator adds a new sub‑network (e.g., a hidden layer or a set of neurons). Because the operator is semantic‑preserving, the new network’s output is a weighted sum of the parent’s output and the added component, guaranteeing a predictable shift in the loss landscape.
3. Deflate‑Grow‑Shrink‑Merge (DGSM) – After a few generations of growth, the deflate step evaluates the contribution of each added component. Low‑impact parts are pruned (shrink), and redundant structures are merged, keeping the model lean while preserving the semantic mapping.
4. Fitness Evaluation – Instead of re‑evaluating the whole network, only the freshly added sub‑network’s forward pass is computed; the rest of the semantics are cached from previous generations.
5. Selection & Replacement – Standard tournament or elitist selection chooses the best‑performing individuals for the next generation, maintaining diversity through mutation‑like random initialization of new components.

The whole pipeline runs as a classic evolutionary loop but with dramatically fewer forward‑propagation calls thanks to the semantic caching.

Results & Findings

• Performance: Across all four regression tasks, NEVO‑GSPT either matches or improves upon the best reported RMSE.
• Compactness: The evolved networks consistently contain 20‑35 % fewer parameters than competing methods, confirming the effectiveness of the DGSM operator.
• Speed: Because only new components are evaluated each generation, total training time drops by roughly 50‑70 % compared to a naïve population‑based neuroevolution baseline.

Practical Implications

• Edge & Mobile Deployments: Smaller, equally accurate models mean lower memory footprints and faster inference on constrained devices—ideal for IoT, AR/VR, and on‑device AI.
• Rapid Prototyping: Developers can let NEVO‑GSPT explore architecture space automatically, freeing them from manual hyper‑parameter sweeps while still retaining control over model size.
• Cost‑Effective AutoML: The efficient evaluation strategy reduces GPU hours dramatically, making large‑scale neural architecture search (NAS) more accessible for startups and research labs with limited compute budgets.
• Explainability Hooks: Because each structural change has a known semantic effect, engineers can trace performance gains back to specific added or removed sub‑networks, aiding debugging and model interpretability.

Limitations & Future Work

• Scope to Regression: Experiments focus on regression benchmarks; the behavior on classification, sequence, or vision tasks remains untested.
• Operator Design Overhead: Implementing GSOs for arbitrary NN layers (e.g., convolutions, attention) may require non‑trivial engineering effort.
• Scalability to Very Deep Nets: While the method trims parameters, the approach has yet to be validated on architectures with hundreds of layers (e.g., ResNets, Transformers).
• Future Directions: The authors suggest extending DGSM to support structured pruning of convolutional filters, integrating multi‑objective optimization (accuracy vs. latency), and applying the framework to reinforcement‑learning policies.

NEVO‑GSPT showcases how borrowing ideas from genetic programming can revitalize neuroevolution, delivering compact, high‑performing networks with a fraction of the usual computational cost—an enticing prospect for developers looking to automate architecture design without blowing up their cloud bills.

Authors

• Davide Farinati
• Frederico J. J. B. Santos
• Leonardo Vanneschi
• Mauro Castelli

Paper Information

• arXiv ID: 2601.08657v1
• Categories: cs.NE
• Published: January 13, 2026
• PDF: Download PDF