[Paper] MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery
Source: arXiv - 2606.06473v1
Overview
Large language model (LLM) agents are increasingly applied to long-horizon tasks such as scientific discovery and machine learning engineering (MLE), where sustained self‑evolution becomes a key capability. However, existing MLE agents suffer from inter‑branch information isolation, memoryless search, and lack of hierarchical control, which together hinder long‑horizon optimization. We present MLEvolve, an LLM‑based self‑evolving multi‑agent framework for end‑to‑end machine learning algorithm discovery.
By extending tree search to Progressive MCGS, MLEvolve enables cross‑branch information flow through graph‑based reference edges and gradually shifts the search from broad exploration to focused exploitation with an entropy‑inspired progressive schedule. To allow the agent to evolve with accumulated experience, we introduce Retrospective Memory, which combines a cold‑start domain knowledge base with a dynamic global memory for task‑specific experience retrieval and reuse. For stable long‑horizon iteration, we further decouple strategic planning from code generation with adaptive coding modes.
Evaluation on MLE‑Bench shows that MLEvolve achieves state‑of‑the‑art performance across multiple dimensions, including average medal rate and valid submission rate under a 12‑hour budget (half the standard runtime). Moreover, MLEvolve outperforms specialized algorithm discovery methods such as AlphaEvolve on mathematical algorithm optimization tasks, demonstrating strong cross‑domain generalization. The code is available at https://github.com/InternScience/MLEvolve.
Key Contributions
- Research Areas: cs.AI, cs.CL
Methodology
Please refer to the full paper for detailed methodology.
Practical Implications
This research contributes to the advancement of cs.AI.
Authors
- Shangheng Du
- Xiangchao Yan
- Jinxin Shi
- Zongsheng Cao
- Shiyang Feng
- Zichen Liang
- Boyuan Sun
- Tianshuo Peng
- Yifan Zhou
- Xin Li
- Jie Zhou
- Liang He
- Bo Zhang
- Lei Bai
Paper Information
- arXiv ID: 2606.06473v1
- Categories: cs.AI, cs.CL
- Published: June 4, 2026
- PDF: Download PDF