Lesson 5 · 6 min

Guardrails

Input/output filters that catch policy and PII issues.

Wrap LLM calls with two filters: pre-call (refuse out-of-scope inputs) and post-call (redact PII, block disallowed content). Make filters fast and explicit — slow guardrails murder p99.

Production scenario

Real-world example: Healthcare scheduling assistant

A scheduling assistant for a multi-clinic chain handles patient calls. Two guardrails wrap every model call:

[ pre-filter ]   blocks out-of-scope: medical advice → "I can only schedule appointments."
[ model ]        produces a draft response
[ post-filter ]  redacts PII from outbound text (SSN, DOB digits, etc.) before display

Both filters are deterministic regex + rule lists, no LLM in the hot path. They add ~3ms each. On day one, the post-filter caught a model output that included a digit-string the model thought was a phone number but was actually a partial SSN. Redacted before display.

Why this matters: guardrails must be fast and explicit. Save LLM-based judgments for async review queues, not the request path.

Knowledge points in this lesson

Pre-call filter blocks out-of-scope input
Post-call filter redacts PII
Guardrails must stay fast
Avoid LLM-based filters on hot path
Make filter decisions explicit and logged

Quick check

Context & ReliabilitySelect all that apply

Which metrics belong on an LLM-app observability dashboard? (Select all that apply)