Video cover

Evidently AI · 1:31 · Watch on YouTube

Ninety-second welcome video for Evidently AI's course on LLM evaluations, presented by instructor Elena. It positions the course as a high-level, no-coding introduction for anyone building (or considering building) an LLM-powered product, and walks through the full curriculum: from LLM fundamentals and benchmarks, through product-level evaluation (datasets, test cases, methods and metrics, LLM-as-a-judge, safety, AI agents), to production observability and the business case for evaluations.

Topics

  1. Introduction and target audience
  2. Course curriculum
  3. Format and extras

Key takeaways

  • The course is a high-level introduction to LLM evaluations: no expertise and no coding required, only a general idea of building products with LLMs inside. 0:01
  • The curriculum spans the full lifecycle: LLM basics and benchmarks, then product evaluation — datasets, test cases, methods and metrics. 0:30
  • Dedicated coverage of LLM-as-a-judge (called "fascinating but slightly controversial"), LLM safety (hallucinations, jailbreaks), evaluating AI agents, and post-launch observability, monitoring, and guardrails. 0:45
  • The course closes with the business reasons to invest in AI evaluations, and includes a bonus video demonstrating how to run LLM evaluations with no code. 1:09