What if LLMs needed a spine, not a bigger brain?

Source: Dev.to

Problem Statement

I’ve been building something for the past few months, and I’m still trying to figure out whether I’m hitting a real problem or just over‑structuring something that better prompting would already solve.

My starting intuition is simple: LLMs are very good at generating, but much less reliable when you expect continuity from them. As soon as you want an agent that can hold a line, remember things cleanly, recover after tension, and stay coherent over time, you start seeing the limits of the model on its own. Not necessarily because it lacks intelligence, but because it lacks a kind of skeleton.

In many systems, the LLM does everything at once: it speaks, it decides, it improvises its own memory and its own frame. And that works, until it starts to drift. Prompting can take you pretty far, but it still feels fragile.

Proposed Approach

The idea is to move governance outside the model:

• The LLM generates, but it does not decide on its own.
• An explicit policy layer handles decisions, state, and memory, providing continuity.
• A timeline keeps an inspectable trace of what happened.

Observations

So far the approach yields:

• More stability and less invention around internal state.
• Better constraint‑following and firmer boundaries under prompt injection.
• Reduced drift over long sequences.

Open Questions

• I haven’t formally proven state causality, the actual impact of governed memory, or deterministic replay yet.
• Are there existing projects tackling the same problem?
• Could better prompting eventually absorb the need for this extra structure?

Core Question

If the LLM is the muscle, what does the skeleton look like?