Uploading the example dataset

  • A ready-made CSV of health-and-fitness questions and answers is provided for viewers who did not generate data. 4:36
  • Steps: Datasets → create from CSV, name it (e.g. "Health chatbot", description optional), drop the CSV, and map the columns — i.e. point to the columns containing text data. 4:36 4:56
  • The dataset holds questions plus answers ("something that you could have generated with your LLM chatbot"); evaluations start from the evaluation tab by choosing the dataset. 4:56

Selecting metrics (Text Evals on the answer column)

Choose Text Evals, point at the column to evaluate (the answers), and pick evaluation methods. 5:37

Four descriptors configured in the demo:

MetricTypePurpose
DenialsLLM-based classifier with a pre-built promptDetect when the LLM declines to respond 5:57
Semantic similarityDeterministic (embedding-based)Compare answers against questions; flags completely irrelevant answers 5:57 6:21
Text lengthDeterministicSpot unusually long/short answers 6:21
"Denials, regular expression"Contains descriptor (regex/word list)No-API-key alternative for detecting denials 6:39
  • The regex denial check uses a word list such as "sorry", "cannot", "apologies" (not case sensitive, extensible). 7:02
  • After defining the report, choose the API key (for the LLM-judge metrics) and run. 7:02