Will AI Ever Be Good Enough to Not Need Spending Limits?
Source: Dev.to
Short answer
No. Understanding why reveals something fundamental about how we should think about AI safety.
Steel‑man the argument
AI capabilities are improving rapidly:
- Better alignment: Constitutional AI, RLHF, and new training techniques make models more reliable at following instructions.
- Longer context: Models can now hold millions of tokens, reducing “forgotten” instructions.
- Formal reasoning: Chain‑of‑thought and tool use make agents more predictable.
- Agent frameworks: LangChain, CrewAI, and others add structure around LLM decision‑making.
Given this trajectory, why wouldn’t AI agents eventually become trustworthy enough to handle financial transactions without external policy enforcement?
The core issue
LLMs are probabilistic by design. They predict the next token based on statistical patterns. Even a 99.99 % reliable model fails 1 in 10 000 times.
- For most applications, 99.99 % is excellent.
- For financial transactions, it’s not good enough.
Example: a trading agent
| Metric | Value |
|---|---|
| Transactions per day | 1 000 |
| Reliability (99.99 %) | 0.1 expected failures per day |
| Expected failures per month | ≈ 3 |
| Expected failures per year | ≈ 36 |
If one of those failures is “send the entire balance to the wrong address” instead of a minor formatting error, the tail risk is catastrophic.
A deterministic policy (e.g., if amount > dailyLimit then reject()) has a 0 % failure rate. The transaction either passes or it doesn’t—there is no statistical distribution of outcomes.
This isn’t about AI being “bad.” It’s about the mathematical difference between probabilistic and deterministic systems.
Analogy: automotive safety
Cars have become dramatically safer thanks to layered safety systems:
| Layer | Example |
|---|---|
| Crumple zones | – |
| Anti‑lock brakes | – |
| Electronic stability control | – |
| Autonomous emergency braking | – |
| Lane‑departure warnings | – |
Yet we still have seatbelts, airbags, and speed limits. Each layer handles different failure modes; the rarity of crashes doesn’t let us remove the protections for when they do occur.
Layered architecture for AI agents
| Layer | Purpose | Type |
|---|---|---|
| Training / RLHF | Make the model generally safe | Probabilistic |
| System prompts | Guide behaviour for this use case | Probabilistic |
| Agent framework | Add structure and validation | Mixed |
| Policy layer | Hard limits that cannot be exceeded | Deterministic |
Improving Layer 1 doesn’t eliminate the need for Layer 4—they serve different purposes.
Architectural principle
The system making decisions shouldn’t also control the guardrails.
If an AI agent both decides what to spend and enforces spending limits, those limits exist only as long as the agent respects them. A convincing prompt injection, hallucination, or edge‑case in the training data could bypass them.
Separation of concerns
┌─────────────────┐ ┌─────────────────┐ ┌─────────────┐
│ AI Agent │────▶│ Policy Layer │────▶│ Execution │
│ (decides) │ │ (enforces) │ │ (acts) │
└─────────────────┘ └─────────────────┘ └─────────────┘
│ │
Probabilistic Deterministic
Can be influenced Cannot be influenced
by inputs by the agentThe policy layer simply checks: Does this transaction comply with the rules? Yes → proceed. No → reject. It doesn’t care how good the AI is, whether it was jail‑broken, or whether it made a brilliant insight.
Human analogy
Even trusted humans have spending limits. A senior employee may be brilliant and reliable, yet they still can’t wire $1 M without approval. The limits aren’t about trust; they’re about:
- Risk management: Limiting the blast radius of any single decision.
- Compliance: Demonstrating controls to auditors and regulators.
- Process: Creating checkpoints for high‑stakes actions.
- Recovery: Ensuring mistakes can be caught before they become irreversible.
AI agents need the same constraints—not because they’re bad at their jobs, but because that’s how financial risk is managed in any system.
Regulatory reality
Regulators don’t accept “the AI is really good now” as a control. Standards such as SOC 2, PCI‑DSS, and various financial regulations require demonstrable, auditable controls:
- What limits exist.
- How they’re enforced.
- That they cannot be bypassed.
- An audit trail of decisions.
A policy engine provides all of this. An AI agent’s internal reasoning—no matter how sophisticated—doesn’t satisfy these requirements.
Evolving policies
There’s an implicit assumption that “policy layers are static while AI advances.” In reality, as agents become more capable, policies become more sophisticated.
Today’s policies
- Daily spending limits.
- Per‑transaction caps.
- Recipient whitelists.
Future policies (as agents take on more complex tasks)
- Cross‑agent coordination limits.
- Portfolio allocation constraints.
- Velocity detection across multiple assets.
- Conditional approvals based on market conditions.
Better AI means agents can do more, which in turn requires more sophisticated guardrails, not fewer.
Bottom line
The question isn’t “Will AI get good enough?” It’s “Good enough for what?”
- For making decisions? AI is already good and getting better.
- For eliminating the need for independent safety controls? Never. That’s not how safety engineering works.
Probabilistic systems require deterministic guardrails, whether the system is 90 % reliable or 99.99 % reliable. Guardrails aren’t a commentary on the AI’s capability—they’re a recognition that financial systems demand mathematical certainty, not just statistical confidence.
It can be brilliant. It should still have spending limits.
Ready to add deterministic guardrails to your AI agents?
- Quick Start Guide – Get running in 5 minutes
- GitHub – Open source SDK