Where it sits
- At the bottom of the architecture: a conditional router sends output through validation nodes before anything reaches the user (
12:49).
The three nodes
| Node | What it checks |
|---|---|
| Gatekeeper | Does the response actually answer the question? (12:49) |
| Auditor | Is the information grounded in the retrieved context and not hallucinated? (12:49) |
| Strategist | Does the response make sense given the broader context? (12:49) |
- The design goal is to mimic how a human would think before sending an answer: Does this address what was asked? Is this accurate? Does this make sense? (
12:49,13:15).
What these nodes catch in production
Stated as things they catch "a lot of" before reaching users (13:15):
- A response that sounds confident but contradicts the source documents.
- An answer that addresses a different question than the one asked.
- A recommendation that doesn't account for important constraints.