LLM Guardrails

LLM guardrails are checks that run outside the model: on the request before the model sees it, and on the response before the user does. They block, redact or reroute anything the policy forbids, such as personal data in a prompt, a jailbreak attempt, a toxic reply or an answer in a topic the product must not cover. Because they run outside the model, a clever prompt cannot argue its way past them the way it can past an instruction in the system prompt.

Most writing on guardrails stops at text filters. For a chatbot that is most of the job. For a system that calls tools, writes records or moves money, the filter on the text is the weaker layer, and the stronger guardrail is a limit on what the system can execute at all.

Input rails and output rails#

The two positions defend against different failures. Input rails inspect the request: they detect jailbreak and prompt injection attempts, enforce topic limits, and mask sensitive data so it never enters the prompt, the KV cache or your logs. Output rails inspect the response: they catch toxic or unsafe content, redact personal data the model produced itself, and validate the answer against policy or a schema.

Mask on the way in, not on the way out. If a customer pastes a card number and the input rail replaces it with a placeholder before the prompt is built, the raw number never reaches the model or the trace store. Redacting it from the reply afterwards still leaves it in every log the request touched.

What runs inside a rail#

A rail is a pipeline of checks at three costs. Pattern checks (regular expressions, deny-lists, checksums) are close to free and catch card numbers and known exploit strings. Small classifiers label text against a harm taxonomy and are fast enough to run on every request. An LLM judge can rule on nuanced policy, such as whether a reply amounts to regulated financial advice, at the price of a model call per check.

Input rails sit in series before generation, so their time adds directly to time to first token. Run pattern checks in sequence, run the heavier classifier alongside generation and cancel the stream if it flags, and buffer streamed output in sentence-sized pieces so the output rail can check each piece without holding back the whole reply.

The open-source pieces split along those layers. Meta's Llama Guard 4, released in April 2025, is a 12-billion-parameter safety classifier pruned from Llama 4 Scout that labels text and images against 14 hazard categories and fits on a single 24 GB GPU. Its companion Prompt Guard 2 comes in 86M and 22M sizes for fast injection screening. NVIDIA's NeMo Guardrails is an orchestration toolkit that wires checks like these into an ordered flow with topic and dialogue rules. Guardrails AI composes validators for toxicity, personal data, grounding and structured output. Microsoft Presidio detects and redacts personal data with named-entity recognition, patterns and checksums. Most stacks combine several of them rather than picking one.

Fail closed#

The decision that matters most never shows on a good day: what happens when a guard errors or times out. Many implementations pass the request through, choosing availability over safety, which turns the guard into decoration. If an attacker learns that a 300 ms classifier timeout means the check is skipped, they will send inputs built to be slow. At 99.9 percent guard availability and a million requests a day, failing open lets about a thousand unchecked requests through daily.

Bigcircle recommends failing closed, because the failure that a fail-open guard lets through is the one an attacker can trigger on purpose. On a guard error, block the request or route it to a safe fallback, such as a canned reply or a person. The cost is false positives, so measure them: track the block rate, and sample blocked traffic for review the way you sample answers for evals.

Put the hard rules in the action layer#

A classifier on text has its own error rate, and the obfuscation tricks that fool a model can fool it too. For anything the system does, as opposed to says, we put the rule where it cannot be talked around.

On our insurance voice agent, built for an insurer, compliance lives in the tool contract, not the prompt. Every policy, claim, payment and endorsement action goes through a typed tool that checks permissions and required disclosures before it executes. "Will the model always make the disclosure?" cannot be answered. "Can this operation run without the disclosure?" can, and the answer is no. On our DevOps agent, the agent acts only through purpose-built tools that wrap the cloud provider's APIs, with no general shell and no way for a plan step to become an arbitrary API call. On the desktop chief-of-staff agent, secrets are injected into the tool adapter at execution time and never enter the prompt, so there is nothing for an injected instruction to leak.

None of those is a filter. Each one removes an action or a secret from the model's reach, which a filter on text can never fully do. The same holds when a team runs coding agents in an AI software factory: keep the tests, the eval suite and the CI configuration outside every agent's write access. Keep the text rails for what the system says, and test the whole assembly with red-teaming, logging every block and every bypass to your observability traces. The OWASP GenAI LLM Top 10, whose 2026 edition was published on 3 August 2026, is the usual checklist for which risks the layers need to cover.