[Paper] Inside Out: Evolving User-Centric Core Memory Trees for Long-Term Personalized Dialogue Systems

Source: arXiv - 2601.05171v1

Overview

Long‑term personalized dialogue agents need to remember who a user is across countless conversations, but the finite context windows of modern LLMs cause older information to be lost or corrupted. The paper Inside Out: Evolving User‑Centric Core Memory Trees for Long‑Term Personalized Dialogue Systems introduces PersonaTree, a structured, tree‑based memory that grows in a controlled way while keeping a compact “core” representation of a user’s persona. A lightweight reinforcement‑learning component, MemListener, learns to issue explicit memory operations (ADD, UPDATE, DELETE, NO_OP), enabling the tree to evolve without blowing up the context size.

Key Contributions

• PersonaTree data structure: a globally maintained tree that separates a fixed “trunk” schema from mutable branches/leaves, providing deterministic growth and memory compression.
• MemListener agent: a small RL‑trained model that decides structured memory operations, achieving decision quality comparable to large reasoning models.
• Dual‑mode generation:
  • Latency‑sensitive mode reads directly from PersonaTree for fast responses.
  • Agentic mode expands on‑demand, pulling additional details while staying bounded by the tree.
• Comprehensive evaluation: shows PersonaTree reduces contextual noise and improves persona consistency versus naïve full‑text concatenation and existing personalized memory baselines.
• Open‑source friendly: the framework is built to work with off‑the‑shelf LLM APIs, making it easy to plug into existing chatbot pipelines.

Methodology

1. Tree Construction

   • The trunk encodes a predefined schema (e.g., user name, interests, preferences).
   • Branches represent topics or interaction episodes, and leaves store fine‑grained facts (e.g., “likes spicy ramen”).

2. Memory Operations

   • MemListener receives the current dialogue turn and the existing PersonaTree snapshot.
   • It outputs one of four symbolic actions:
     • ADD – insert a new leaf under a relevant branch.
     • UPDATE – modify an existing leaf’s value.
     • DELETE – prune outdated or contradictory information.
     • NO_OP – keep the tree unchanged.
   • Rewards are shaped around process metrics: consistency with the user’s stated facts, minimal tree size growth, and alignment with downstream response quality.

3. Response Generation

   • In fast mode, the LLM is prompted with a concise serialization of the trunk + selected branches, keeping token count low.
   • In agentic mode, the system first generates a high‑level answer, then queries the tree for supporting details, appending them only if they improve relevance.

4. Training & Evaluation

   • MemListener is trained via Proximal Policy Optimization (PPO) on simulated dialogues.
   • Benchmarks compare against full‑text memory (concatenating all past utterances) and other memory‑augmented chatbots (e.g., Retrieval‑Augmented Generation, Knowledge‑Graph‑based methods).

Results & Findings

• Memory compression: PersonaTree stores the same amount of user knowledge using ~30 % of the tokens required by full‑text concatenation.
• Consistency boost: The structured schema prevents contradictory statements (e.g., “I’m vegan” vs. “I love steak”).
• Speed advantage: Fast mode meets sub‑250 ms response times even with a 10‑turn dialogue history, making it viable for real‑time chat services.
• MemListener efficiency: A ~30 M‑parameter model matches or exceeds the operation decisions of 100 B‑parameter reasoning models, demonstrating that explicit symbolic actions can replace heavyweight inference.

Practical Implications

• Scalable personalization: SaaS chatbot platforms can maintain per‑user persona trees in a database, updating them on the fly without re‑indexing large text corpora.
• Cost reduction: By feeding only a compact tree to the LLM, token usage—and thus API spend—drops dramatically, especially for high‑volume services.
• Regulatory compliance: Structured memory makes it easier to audit, edit, or delete specific user facts (e.g., GDPR “right to be forgotten”) compared to opaque concatenated logs.
• Developer ergonomics: MemListener’s operation set is human‑readable, enabling developers to debug or manually intervene in the memory evolution process.
• Extensibility: The tree schema can be enriched with domain‑specific branches (e.g., medical history, financial preferences), allowing the same framework to power personalized assistants across industries.

Limitations & Future Work

• Schema rigidity: The initial trunk schema must be designed ahead of time; adapting to completely new user attributes may require schema revisions.
• Simulated training data: MemListener is trained on synthetic dialogues; real‑world user data could expose edge cases not covered in simulation.
• Scalability of tree traversal: While token‑efficient, retrieving the optimal subset of branches for a given turn still incurs a modest computational overhead that could grow with millions of users.
• Future directions suggested by the authors include:
  • Learning schema extensions automatically via meta‑learning.
  • Incorporating multimodal facts (images, voice snippets) as leaf nodes.
  • Exploring hierarchical RL where higher‑level policies decide which branches to surface before MemListener issues leaf‑level operations.

Inside Out demonstrates that a well‑designed, structured memory can give long‑term personalized chatbots the best of both worlds: consistency and low latency without the heavy token burden of naïve context concatenation. For developers building next‑generation conversational agents, adopting a tree‑based persona store and a lightweight operation controller could be a game‑changer both technically and economically.

Authors

• Jihao Zhao
• Ding Chen
• Zhaoxin Fan
• Kerun Xu
• Mengting Hu
• Bo Tang
• Feiyu Xiong
• Zhiyu Li

Paper Information

• arXiv ID: 2601.05171v1
• Categories: cs.CL
• Published: January 8, 2026
• PDF: Download PDF