[Paper] OR-Agent: Bridging Evolutionary Search and Structured Research for Automated Algorithm Discovery

Source: arXiv - 2602.13769v1

Overview

The paper introduces OR‑Agent, a modular multi‑agent framework that blends evolutionary search with a structured, tree‑based research workflow. By treating hypothesis generation, experiment execution, and reflective learning as coordinated “agents,” the system can automatically discover high‑performing algorithms for hard combinatorial problems and simulation‑based tasks—outperforming traditional evolutionary baselines while remaining transparent and extensible.

Key Contributions

• Hybrid Ideation Engine – Merges evolutionary selection of promising starting points with systematic generation of full research plans, enabling both exploration and exploitation.
• Tree‑Based Research Workflow – Represents hypothesis branching and back‑tracking as a structured tree, giving fine‑grained control over research trajectories beyond simple mutate‑crossover loops.
• Hierarchical Reflection Mechanism
  • Short‑term reflection: “verbal gradient” signals from immediate experimental feedback.
  • Long‑term reflection: “verbal momentum” aggregates cross‑experiment insights to guide future searches.
  • Memory compression: Regularizes the knowledge base (analogous to weight decay) to retain essential information while preventing drift.
• Open‑source, Extensible Platform – Full code and benchmark data are released, encouraging community reuse and adaptation to new domains.
• Empirical Validation – Demonstrates superior performance on a suite of classic combinatorial optimization problems (TSP, CVRP, Bin Packing, Orienteering, Multiple Knapsack) and a cooperative driving simulation.

Methodology

1. Agent Architecture – OR‑Agent consists of several cooperating agents:

   • Evolutionary Agent: samples and ranks initial algorithmic “genes” (e.g., heuristic components).
   • Systematic Ideation Agent: expands each gene into a full research plan, constructing a hypothesis tree where each node is a concrete algorithmic variant.
   • Execution Agent: runs the algorithm in the target environment, collecting performance metrics.
   • Reflection Agent: processes results at two timescales: immediate gradients (short‑term) and accumulated trends (long‑term), updating the search distribution.

2. Research Tree Construction – Starting from a selected root hypothesis, the system branches by applying predefined transformation operators (e.g., adding a local search, swapping selection criteria). Back‑tracking is explicit: if a branch underperforms, the tree prunes it and revisits alternative branches, preserving a clear audit trail.

3. Hierarchical Optimization‑Inspired Updates –

   • Verbal Gradient: a lightweight, differentiable proxy derived from experiment outcomes that nudges the search direction.
   • Verbal Momentum: a moving‑average of gradients across many experiments, smoothing noisy signals.
   • Memory Compression: periodic summarization of the knowledge base (e.g., via clustering or low‑rank approximation) to keep the search space tractable.

4. Evaluation Loop – The framework iterates over generations, each consisting of hypothesis expansion, execution, and reflection, until a stopping criterion (budget, convergence, or performance threshold) is met.

Results & Findings

• Consistent outperformance across all problem families, especially on larger instances where the search space explodes.
• Interpretability: The tree‑based workflow lets researchers inspect which hypothesis branches contributed most to final performance, a feature missing in black‑box EA runs.
• Scalability: Memory compression kept the knowledge base under control, allowing experiments with >10⁶ generated hypotheses without prohibitive memory growth.

Practical Implications

• Automated Heuristic Design – Developers can plug in domain‑specific operators (e.g., a new local search move for vehicle routing) and let OR‑Agent discover effective hybrid heuristics without hand‑tuning.
• Rapid Prototyping for Simulation Environments – In autonomous driving or robotics simulators, OR‑Agent can automatically evolve coordination strategies, reducing the time from concept to deployable policy.
• Explainable AI for Optimization – The explicit hypothesis tree provides a natural audit trail, useful for compliance, debugging, or knowledge transfer within engineering teams.
• Extensible Research Platform – Because the framework is open‑source and modular, organizations can integrate their own evaluation metrics, constraints, or domain‑specific knowledge bases, turning OR‑Agent into a “research assistant” for any experiment‑driven problem.

Limitations & Future Work

• Domain Dependency of Operators – The quality of generated algorithms hinges on the set of transformation operators supplied by the user; poorly chosen operators can limit exploration.
• Computational Overhead – Maintaining and reflecting over a large hypothesis tree adds runtime compared to lean pure‑EA loops, which may be a bottleneck for ultra‑large‑scale problems.
• Generalization Beyond Benchmarks – While the paper shows strong results on classic combinatorial tasks and a driving simulator, applying OR‑Agent to highly stochastic or non‑deterministic domains (e.g., reinforcement learning with sparse rewards) remains an open challenge.
• Future Directions suggested by the authors include: automated discovery of transformation operators via meta‑learning, tighter integration with differentiable simulators for richer gradient signals, and scaling the reflection system with distributed memory‑compression techniques.

Authors

• Qi Liu
• Wanjing Ma

Paper Information

• arXiv ID: 2602.13769v1
• Categories: cs.AI, cs.CE, cs.NE
• Published: February 14, 2026
• PDF: Download PDF