New AI framework autonomously optimizes training data, architectures and algorithms — outperforming human baselines
Source: VentureBeat
AI‑for‑AI Research: Automating the R&D Loop
AI R&D runs on a cycle of hypothesis, experiment, and analysis — each step demanding substantial manual engineering effort. A new framework from researchers at SII‑GAIR aims to close that bottleneck by automating the full optimization loop for training data, model architectures, and learning algorithms.
ASI‑EVOLVE Overview
A new framework called ASI‑EVOLVE, developed by the Generative Artificial Intelligence Research Lab (SII‑GAIR), is designed as an agentic system for AI‑for‑AI research. It uses a continuous “learn‑design‑experiment‑analyze” cycle to automate the optimization of the foundational AI stack.
- In experiments, this self‑improvement loop autonomously discovered novel designs that significantly outperformed state‑of‑the‑art human baselines.
- The system generated novel language‑model architectures, improved pre‑training data pipelines to boost benchmark scores by over 18 points, and designed highly efficient reinforcement‑learning algorithms.
For enterprise teams running repeated optimization cycles on their AI systems, the framework offers a path to reducing manual engineering overhead while matching or exceeding the performance of human‑designed baselines.
The Data and Design Bottleneck
- Engineering teams can only explore a tiny fraction of the vast possible design space for AI models at any given time.
- Executing experimental workflows requires costly manual effort and frequent human intervention.
- Insights from these expensive cycles are often siloed as individual intuition or experience, making systematic preservation and transfer across projects or teams difficult.
These constraints fundamentally limit the pace and scale of AI innovation.
Why Current Frameworks Fall Short
AI has made incredible strides in scientific discovery—from specialized tools like AlphaFold to agentic systems answering basic scientific questions. However, existing frameworks still struggle with open‑ended AI innovation and are mostly limited to narrow optimization within very specific constraints.
Advancing core AI capabilities is far more complex. It requires:
- Modifying large, interdependent codebases.
- Running compute‑heavy experiments that consume tens to hundreds of GPU hours.
- Analyzing multi‑dimensional feedback from training dynamics.
“Existing frameworks have not yet demonstrated that AI can operate effectively in this regime in a unified way, nor that it can generate meaningful advances across the three foundational pillars of AI development rather than within a single narrowly scoped setting,” the researchers write.
How ASI‑EVOLVE Learns to Research
ASI‑EVOLVE operates on a continuous loop between prior knowledge, hypothesis generation, experimentation, and refinement:
- Learn relevant knowledge and historical experience from existing databases.
- Design a candidate program representing its next hypothesis.
- Experiment by running the program and obtaining evaluation signals.
- Analyze outcomes into reusable, human‑readable lessons that are fed back into its knowledge base.
Core Components
| Component | Role |
|---|---|
| Cognition Base | Foundational domain expertise. Pre‑loaded with human knowledge, task‑relevant heuristics, and known pitfalls extracted from literature. Guides exploration toward promising directions from the first iteration. |
| Analyzer | Handles complex, multi‑dimensional feedback from experiments. Processes raw training logs, benchmark results, and efficiency traces, distilling them into compact, actionable insights and causal analyses. |
| Researcher | Reviews prior knowledge and past experimental results to generate new hypotheses, either proposing localized code modifications or writing entirely new programs. |
| Engineer | Executes the actual experiments. Equipped with efficiency measures (wall‑clock limits, early‑rejection quick tests) to filter out flawed candidates before they consume excessive GPU hours. |
| Database | Persistent memory that stores code, research motivations, raw results, and the Analyzer’s final reports for every iteration, ensuring insights compound systematically over time. |
By unifying these components, ASI‑EVOLVE ensures that an AI agent systematically learns from complex, real‑world experimental feedback without requiring constant human intervention.
While previous frameworks are designed to evolve candidate solutions, “ASI‑EVOLVE evolves cognition itself,” the researchers write. “Accumulated experience and distilled insights are continuously stored and retrieved to inform future exploration, ensuring that the system grows not only in the quality of its solutions but in its capacity to reason about where to search next.”
ASI‑EVOLVE in Action
Data Curation
- High‑quality data remains a persistent bottleneck for enterprise applications.
- When tasked with designing category‑specific cleaning strategies for massive pre‑training corpora, ASI‑EVOLVE inspected data samples and diagnosed quality issues such as HTML artifacts and formatting inconsistencies.
- The system autonomously formulated custom curation rules, discovering that systematic cleaning combined with domain‑aware preservation rules is far more effective than aggressive filtering.
Model Architecture & Learning Algorithms
- In benchmark tests, 3B‑parameter models trained on the AI‑curated data saw an average score boost of nearly 4 points over models trained on raw data.
- The gains were highest in knowledge‑intensive tasks (the text cuts off here, preserving the original content).
Takeaway
ASI‑EVOLVE demonstrates that automating the full AI‑for‑AI research loop—from data curation to model design and algorithmic innovation—can substantially reduce manual engineering effort while surpassing human‑crafted baselines. For organizations seeking to scale AI innovation, the framework offers a concrete path toward continuous, self‑improving R&D.
Performance Gains
- Massive Multitask Language Understanding (MMLU) – ASI‑EVOLVE achieved an increase of over 18 points, boosting performance across STEM, humanities, and social‑science tasks.
Neural Architecture Design
- Conducted 1,773 autonomous exploration rounds.
- Generated 105 novel linear‑attention architectures that outperformed DeltaNet, a highly efficient human‑designed baseline.
- Introduced multi‑scale routing mechanisms that dynamically adjust the model’s computational budget based on the specific content of the input.
Reinforcement‑Learning Algorithm Design
- Discovered new optimization mechanisms that surpassed the competitive GRPO baseline on complex mathematical‑reasoning benchmarks such as AMC32 and AIME24.
- One successful variant introduced a “Budget‑Constrained Dynamic Radius” that keeps model updates within a defined budget, effectively stabilizing training on noisy data.
What This Means for Enterprise AI
- Enterprise AI workflows constantly require optimizations to existing systems, from fine‑tuning open‑source models on proprietary data to making small changes to architectures and algorithms.
- The computational resources and engineering hours needed for such efforts are often immense and beyond the capabilities of most organizations, leaving many to run unoptimized versions of standard AI models.
- The research team states that the framework is designed so enterprises can integrate proprietary domain knowledge into the cognition repository and allow the autonomous loop to iterate on internal AI systems.
Open‑Source Release
- The research team has open‑sourced the ASI‑EVOLVE code, making the foundational framework available for developers and product builders.