The trick to AI coding memory isn't a bigger instruction file — it's smaller, layered knoledge
Source: Dev.to
The problem with large instruction files
People often try to fix the “my AI forgets everything” issue by making their instruction file huge—500, 1,000, 2,000 lines of CLAUDE.md (or .cursorrules, etc.).
Research shows that AI accuracy drops when the context gets too long, and instructions buried in large files are ignored. The result is a bloated file that consumes the context window before you even ask a question.
What actually works
The opposite approach works better: small, targeted files loaded only when relevant.
Tiered architecture
| Tier | Description | Typical size | Loading strategy |
|---|---|---|---|
| Tier 1 – Constitution | Core principles and routing table | ~200 lines | Always loaded |
| Tier 2 – Living Memory | Short‑term, frequently‑used rules | ~50 lines | Always loaded |
| Tier 3 – Project Brains | Project‑specific knowledge | Varies per project | Loaded per‑project |
| Tier 4 – Knowledge Store | Large reference material | Unlimited | Queried on demand |
Tier 1 – Constitution
Contains a routing table that tells the AI where to find each piece of information. Instead of stuffing everything into one file, Tier 1 simply points to the appropriate tier.
Tier 2 – Living Memory
Holds the most commonly needed instructions that should be instantly available.
Tier 3 – Project Brains
Separate files for each project, keeping project‑specific context isolated.
Tier 4 – Knowledge Store
A searchable repository that the AI can query when it needs deeper information.
Session Memory (continuity layer)
A lightweight layer that preserves context across a conversation, linking the appropriate tiers as the session progresses.
Lessons learned
Budget every tier strictly.
- 200 lines for Tier 1, 50 for Tier 2.
- Limits force you to prioritize quality and move excess content to the right tier instead of dumping it into the always‑loaded file.
Don’t store what the AI can derive.
- File structure, visible code patterns, and git history are already accessible to the model.
- Only store information that the AI would get wrong without explicit instruction.
Summarizer safeguards.
- An unsupervised summarizer that tried to process 50 sessions at once hit an API error, retried the full batch, and burned a third of a week’s token budget.
- Fixes:
- Batch cap of 5 sessions.
- Processed‑flag to avoid re‑summarizing completed sessions.
- Lock file to prevent concurrent runs.
Compatibility
The architecture works with Claude Code, Cursor, Copilot, Codex, Aider—any tool that reads instruction files. Filenames differ, but the layered approach remains the same.
Repository
The full system, including templates and an automated setup script, is available on GitHub:
https://github.com/sms021/SuperContext
Invitation for deeper discussion
Feel free to ask for more details on any part of the architecture, session memory handling, or how to migrate an existing giant instruction file to this tiered structure.