LLM testing usually comes down to two things 1:16:
- Capabilities — the app does what it's supposed to; responses are accurate, useful, helpful. The definition of "good" depends on the use case
1:16. - Risks — the outputs are safe and the system doesn't generate anything harmful
1:16.
Previous: Why evaluating LLMs is hard · Next: From vibe checks to manual labeling