[Paper] Current Challenges of Symbolic Regression: Optimization, Selection, Model Simplification, and Benchmarking

Source: arXiv - 2512.01682v1

Overview

This dissertation tackles four long‑standing pain points that keep symbolic regression (SR) from becoming a mainstream tool for data‑driven modeling: how to tune the evolutionary algorithm’s hyper‑parameters, how to pick parents that actually improve the search, how to keep the discovered formulas from ballooning into unreadable spaghetti, and how to fairly benchmark new SR techniques. By systematically addressing each of these issues, the author delivers a more reliable, faster, and easier‑to‑interpret SR pipeline that outperforms current state‑of‑the‑art methods on both synthetic and real‑world datasets.

Key Contributions

• Parameter‑optimization study – Quantifies the trade‑offs between predictive accuracy, runtime, and expression size when tuning GP hyper‑parameters.
• ε‑lexicase parent selection – Introduces a selection scheme that favors individuals that excel on any subset of cases, leading to higher‑quality offspring.
• Novel model‑simplification technique – Uses memoization together with locality‑sensitive hashing to detect and collapse redundant sub‑expressions, producing smaller, more accurate formulas.
• Multi‑objective SR library – Implements the above ideas in an open‑source evolutionary SR framework that simultaneously optimizes for accuracy and simplicity.
• Benchmark‑suite overhaul – Proposes concrete changes to a widely‑used large‑scale SR benchmark, then re‑evaluates the whole SR landscape to show the new method’s Pareto‑optimal performance.

Methodology

The research follows a modular, experimental pipeline:

1. Baseline GP Engine – Starts from a classic tree‑based genetic programming (GP) implementation that evolves mathematical expressions.
2. Hyper‑parameter sweep – Systematically varies mutation rates, population size, crossover probability, etc., measuring impacts on error, runtime, and tree depth.
3. ε‑lexicase selection – Replaces the usual tournament or roulette‑wheel selection with ε‑lexicase, which ranks candidates on a case‑by‑case basis and only promotes those that are within an ε tolerance on at least one case.
4. Simplification via memoization & LSH – While evaluating individuals, sub‑trees are cached (memoization). A locality‑sensitive hash (LSH) groups mathematically equivalent or near‑equivalent sub‑expressions, allowing the algorithm to prune duplicates on the fly.
5. Multi‑objective optimization – Uses a Pareto front to balance two objectives: (a) minimize prediction error, (b) minimize formula complexity (measured by node count, depth, or description length).
6. Benchmarking – Runs the full pipeline on a curated suite of synthetic functions (e.g., polynomial, trigonometric) and real‑world regression problems (e.g., energy consumption, biomedical data). The benchmark suite itself is audited and updated to reflect realistic evaluation criteria (e.g., runtime caps, noise levels).

All experiments are repeated with statistical rigor (multiple random seeds, confidence intervals) to ensure the reported gains are robust.

Results & Findings

Practical Implications

• Rapid prototyping of interpretable models – Data scientists can replace black‑box regressors (e.g., random forests) with compact symbolic formulas that are ready for code generation in C, Python, or even SQL.
• Edge‑AI and IoT – The reduced model size and lower evaluation cost make SR viable for micro‑controllers where memory and CPU cycles are scarce.
• Regulatory compliance – Industries such as finance or healthcare that require explainable AI can leverage the simplified expressions to satisfy audit trails and model‑risk assessments.
• AutoML pipelines – The ε‑lexicase selector and built‑in simplification can be dropped into existing AutoML frameworks to enhance their evolutionary search components.
• Open‑source ecosystem – The released library (presumably on GitHub) can be extended with custom fitness functions, domain‑specific operators, or integrated with popular data‑science stacks (pandas, scikit‑learn).

Limitations & Future Work

• Scalability to very high‑dimensional data – While the method handles dozens of features well, performance degrades when hundreds of variables are present; dimensionality reduction or feature‑selection pre‑steps may be required.
• Runtime overhead of LSH – The hashing step adds a modest constant factor to evaluation time; optimizing the hash function or parallelizing the memoization cache could mitigate this.
• Benchmark diversity – The current benchmark suite, though improved, still leans heavily on synthetic functions; adding more domain‑specific real‑world tasks (e.g., control systems, physics simulations) would further validate generality.
• Hybrid approaches – Combining SR with gradient‑based fine‑tuning (e.g., differentiable programming) could push accuracy even higher while retaining interpretability.

Overall, the thesis delivers a concrete, developer‑friendly toolkit that brings symbolic regression one step closer to everyday production use.

Authors

• Guilherme Seidyo Imai Aldeia

Paper Information

• arXiv ID: 2512.01682v1
• Categories: cs.NE
• Published: December 1, 2025
• PDF: Download PDF