Video cover

Evidently AI · 5:11 · Watch on YouTube

Summary

A short educational video on LLM safety for application builders. It walks through four things that can go wrong with LLM apps — off-topic use, hallucinations, jailbreaks, and data leaks — each illustrated with a real-world incident (Klarna, Air Canada, Chevrolet, DPD). It then presents a defense strategy: start with risk assessment, then apply three lines of defense — product design choices, real-time guardrails, and safety testing through red-teaming — treating safety as an ongoing process, not a one-time gate.

Topics

  1. Introduction: why LLM safety matters
  2. Risk 1: Off-topic use
  3. Risk 2: LLM hallucinations
  4. Risk 3: LLM jailbreaks
  5. Risk 4: Data leaks
  6. Risk assessment
  7. First line of defense: product design choices
  8. Second line of defense: guardrails
  9. Third line of defense: safety testing and red-teaming

Overview

flowchart LR
    subgraph Risks
        A[Off-topic use]
        B[Hallucinations]
        C[Jailbreaks]
        D[Data leaks]
    end
    Risks --> RA[Risk assessment]
    RA --> D1[1. Product design<br/>RAG, classifiers,<br/>access control, HITL]
    D1 --> D2[2. Guardrails<br/>input/output screening]
    D2 --> D3[3. Safety testing<br/>red-teaming, adversarial<br/>prompt datasets]
    D3 -->|new attacks appear| D3

Key takeaways

  • LLM failures are a liability, not just a UX problem: a court ruled Air Canada had to compensate a customer after its chatbot gave refund advice contradicting company policy. 1:18
  • Hallucination is inherent to LLMs — "not a bug but a feature" — so never rely on them for factual information without grounding (e.g., RAG with quoted sources). 1:01
  • Jailbreaks have real business consequences: a Chevrolet chatbot was manipulated into "selling" a car for $1, and a DPD bot was prompted into swearing and mocking its own company. 1:48
  • Defense is layered: risk assessment first, then product design (classifiers, constrained data access, human-in-the-loop), then guardrails on inputs and outputs — accepting that guardrails add latency and can't cover everything. 2:49, 4:05
  • Test the whole system, not the model alone: red-team with a dataset of adversarial prompts and verify the app consistently denies them. 4:21, 4:38
  • Safety is ongoing — new attack types keep emerging, so apps in production must be re-tested continuously. 4:55