[Paper] Policy Compiler for Secure Agentic Systems

Source: arXiv - 2602.16708v1

Overview

LLM‑driven agents are being handed increasingly sensitive responsibilities—think customer‑service bots that must follow strict escalation rules, or automated compliance checkers that need to respect data‑access policies. Embedding such rules directly in prompts is fragile; the model can simply ignore or bypass them. The paper introduces PCAS (Policy Compiler for Agentic Systems), a framework that automatically transforms any existing LLM‑based agent into a policy‑enforced system, guaranteeing that every action respects formally specified security and workflow rules.

Key Contributions

• Deterministic policy enforcement for LLM agents via a compile‑time instrumentation step, eliminating reliance on the model’s internal reasoning.
• Dependency‑graph state model that captures causal links between tool calls, tool outputs, and messages, enabling precise tracking of information flow across multiple agents.
• Datalog‑derived policy language allowing expressive, declarative specifications of transitive data‑flow and provenance constraints.
• Reference monitor that intercepts all agent actions, checks them against the compiled policy, and blocks violations before they execute.
• Empirical validation on three realistic scenarios (prompt‑injection defense, multi‑agent pharmacovigilance approvals, and enterprise customer‑service workflows), showing compliance jumps from ~48 % to >93 % across state‑of‑the‑art LLMs with zero violations in the instrumented runs.

Methodology

1. Policy Specification – Security engineers write rules in a lightweight Datalog‑style language. Rules can express “if data X originates from source Y, it must never be sent to Z” or “a drug‑adverse‑event report must be approved by a senior analyst before being logged.”
2. System Instrumentation – PCAS takes the original agent code (or a wrapper around any LLM API) and automatically injects a reference monitor that logs every observable event (messages, tool invocations, tool results).
3. Dependency Graph Construction – As the agent runs, the monitor builds a directed graph where nodes are events and edges denote causal influence (e.g., a tool’s output depends on a prior user message). This graph is the authoritative “state” for policy evaluation.
4. Policy Evaluation – Whenever the agent attempts an action (sending a message, calling an external API, writing to storage), the monitor queries the graph using the compiled Datalog rules. If any rule is violated, the action is blocked and an error is returned to the agent.
5. Compilation Guarantees – Because the monitor enforces policies before any side‑effect occurs, the system is provably compliant regardless of how the underlying LLM interprets prompts.

Results & Findings

• Policy compliance improves dramatically across all tested models, confirming that the reference monitor reliably blocks disallowed actions.
• Zero false positives in the instrumented runs: legitimate actions that satisfy the policy are never incorrectly blocked.
• Performance overhead stays modest (average latency increase of 12–18 ms per interaction), making PCAS suitable for real‑time services.

Practical Implications

• Secure Deployments – Companies can now ship LLM agents without fearing that a clever prompt will sidestep internal controls, because enforcement is external and deterministic.
• Regulatory Alignment – Industries with strict data‑handling rules (healthcare, finance, pharma) can encode GDPR, HIPAA, or FDA‑style constraints directly in the policy language and be confident of compliance.
• Multi‑Agent Coordination – In complex pipelines where one agent’s output feeds another, PCAS’s dependency graph ensures provenance is tracked end‑to‑end, preventing accidental leakage or unauthorized escalation.
• Rapid Prototyping – Developers can keep their existing LLM code unchanged; PCAS acts as a drop‑in wrapper, dramatically reducing the engineering effort required to harden agents.
• Auditability – The graph and rule evaluations provide a clear, queryable audit trail, simplifying post‑mortem investigations and satisfying compliance auditors.

Limitations & Future Work

• Policy Language Expressiveness – While Datalog covers many provenance and flow constraints, more nuanced temporal or probabilistic policies may require extensions.
• Scalability of the Dependency Graph – Extremely long conversations or high‑throughput multi‑agent systems could cause the graph to grow large; incremental pruning strategies are an open research direction.
• Integration with Closed‑Source LLM APIs – The current prototype assumes the ability to intercept all tool calls; tighter integration with commercial APIs (e.g., OpenAI, Anthropic) may need cooperation from providers.
• User‑Facing Error Handling – When a policy blocks an action, the agent currently returns a generic error; richer feedback mechanisms could improve user experience.

PCAS demonstrates that rigorous, compile‑time security can be brought to the fast‑moving world of LLM agents, bridging the gap between powerful generative models and the strict governance demands of production environments.

Authors

• Nils Palumbo
• Sarthak Choudhary
• Jihye Choi
• Prasad Chalasani
• Mihai Christodorescu
• Somesh Jha

Paper Information

• arXiv ID: 2602.16708v1
• Categories: cs.CR, cs.AI, cs.MA
• Published: February 18, 2026
• PDF: Download PDF