• Common objection: "we don't need evaluations yet, it's just a proof of concept". 3:11
  • Counter-argument: without evaluations, a proof of concept is "just a glorified demo" — and building demos is already very easy. 3:11
  • To actually prove something with a concept you must test the AI product in limited but realistic conditions. 3:11
  • A real PoC needs to: handle at least some edge cases, and have a system for knowing how you're measurably improving it. 3:25
  • Conclusion: even a proof of concept needs evaluations — it is one of the most important parts of it. 3:25