RAG

  • A RAG system first queries a source you gave it to find relevant data, then generates the response based on it. 0:34
  • It can be a component of a chatbot or an agent, or a standalone tool such as a smart search bar. 0:34

Chatbots

  • The evaluation complexity comes from holding complete conversations: you must evaluate the chat, not a single input/output pair. 0:51

AI agents

  • Definitions vary, but agents usually have three traits: 1:01
    1. Planning — deciding how to solve a task and choosing the right next step.
    2. Tool use — interacting with external APIs, internet search, or databases.
    3. Memory — keeping track of the complex task or conversation context.
  • Example: an e-commerce support agent handling a product return — it collects order details, verifies eligibility against company policies, checks the return window, processes the refund payment, all while maintaining the conversation. 1:18