[Paper] Recursive Concept Evolution for Compositional Reasoning in Large Language Models

Source: arXiv - 2602.15725v1

Overview

Large language models (LLMs) have become remarkably good at many reasoning tasks, but they still stumble when asked to compose multiple concepts—think of solving multi‑step math problems or answering nuanced scientific questions. The paper Recursive Concept Evolution for Compositional Reasoning in Large Language Models introduces a new inference‑time technique, Recursive Concept Evolution (RCE), that lets a frozen LLM reshape its own internal representation space on the fly, creating fresh “concept subspaces” whenever the existing ones prove insufficient.

Key Contributions

• Dynamic representation adaptation: RCE detects when a model’s latent space lacks the abstraction needed for a problem and spawns low‑rank concept subspaces during inference.
• Minimum Description Length (MDL) selection: New subspaces are kept only if they provide a more compact explanation of the data, preventing runaway growth.
• Synergistic merging & consolidation: Compatible subspaces are merged, and all active subspaces are jointly optimized under stability constraints, preserving the original knowledge of the base model.
• Plug‑and‑play integration: The authors demonstrate a drop‑in wrapper around the open‑source Mistral‑7B model, requiring no retraining of the underlying weights.
• Empirical gains on hard compositional benchmarks: RCE delivers 12‑18 % absolute improvements on ARC‑AGI‑2, 8‑14 % on GPQA and BBH, and reduces depth‑related errors on MATH and HLE.

Methodology

1. Detecting inadequacy – While the model processes a prompt, a lightweight monitor watches the activation patterns. If the variance or reconstruction error in the current representation exceeds a threshold, the system flags a “concept gap.”
2. Spawning a subspace – A small, trainable matrix (low‑rank) is initialized to capture the missing abstraction. This matrix lives alongside the frozen transformer layers and is updated only for the current inference episode.
3. MDL‑based pruning – Each candidate subspace is evaluated by how much it compresses the representation (i.e., reduces description length). Subspaces that do not improve the MDL score are discarded.
4. Merging & consolidation – When two active subspaces explain overlapping aspects of the problem, they are merged into a single subspace. All active subspaces are then jointly optimized with a constrained loss that penalizes drift from the original hidden states, ensuring the model remains stable.
5. Recursive application – The process repeats at each reasoning step (e.g., each chain‑of‑thought token), allowing the model to iteratively refine its internal concepts as the problem deepens.

All of this happens without fine‑tuning the base model weights, making RCE an inference‑time augmentation rather than a new training regime.

Results & Findings

• Consistent gains across diverse domains (science, math, logic) indicate that RCE is not just overfitting to a single dataset.
• Depth‑induced error (mistakes that accumulate as reasoning steps increase) drops noticeably, confirming the benefit of evolving concepts recursively.
• Computation overhead stays modest: the low‑rank subspaces add ~10‑15 % extra FLOPs, far cheaper than full‑model fine‑tuning or reinforcement‑learning loops.

Practical Implications

• Plug‑in inference optimizer: Developers can wrap existing LLM APIs (e.g., Mistral, Llama, Claude) with an RCE layer to boost performance on tasks that require multi‑step abstraction—think automated theorem proving, complex code synthesis, or AI‑assisted scientific research.
• Reduced need for massive fine‑tuning: Since RCE works at inference time, teams can avoid costly retraining pipelines while still extracting more reasoning power from a given model checkpoint.
• Better compositional AI assistants: Chatbots that need to combine disparate concepts (e.g., “explain the thermodynamic implications of a quantum algorithm”) can benefit from on‑the‑fly concept creation, leading to more accurate and coherent responses.
• Resource‑efficient scaling: The low‑rank nature of the added subspaces means the technique scales well to larger models; the same framework could be applied to 30‑B or 70‑B models with only linear overhead.

Limitations & Future Work

• Detection heuristics are handcrafted: The current variance‑based trigger may miss subtler representation gaps; learning a more nuanced adequacy predictor could improve subspace spawning.
• Stability‑constraint tuning: Balancing flexibility vs. drift requires careful hyper‑parameter selection; automated tuning methods are still an open question.
• Benchmark scope: While the paper covers several compositional suites, real‑world industrial workloads (e.g., large‑scale codebases, multimodal reasoning) remain to be tested.
• Extension to multimodal models: Future research could explore whether RCE‑style subspaces can be generated for vision‑language or audio‑language models, enabling compositional reasoning across modalities.

Bottom line: Recursive Concept Evolution offers a practical, inference‑time pathway for developers to unlock deeper compositional reasoning in existing LLMs without the heavy cost of full model retraining. As AI systems become more integrated into complex decision‑making pipelines, tools like RCE could become a standard part of the production stack.

Authors

• Sarim Chaudhry

Paper Information

• arXiv ID: 2602.15725v1
• Categories: cs.AI, cs.CL, cs.LG
• Published: February 17, 2026
• PDF: Download PDF