Video cover

An educational lecture on turning a model-and-tools loop into a persistent, proactive agent. Lee explains the server and cloud-computer architecture, crash recovery, tool selection, context and memory management, and the evaluation loops used to improve the product. 2:08

Topics

  1. Introduction to always-on agents
  2. The evolution of AI development
  3. Model capabilities, files, and skills
  4. Architecture and agent lifecycle
  5. Server mechanics and durable execution
  6. Building the agent harness
  7. Context engineering and prompt caching
  8. Managing agent memory
  9. Training and product iteration
  10. Future outlook and open problems

Key takeaways

  • Always-on agents can sleep between events. Persistence comes from saving state and restoring it when a message or background event wakes the agent. 10:05 10:58
  • Keep the agent loop on a server separate from its Linux VM so the computer can shut down or be rebuilt independently. 14:30 15:04
  • The implementation uses Temporal durable workflows to resume after infrastructure failures; helpers also run as durable workflows. 16:42 17:06 24:34
  • Load tool details on demand, delegate verbose work to helpers, and return results to the main agent. This conserves its finite context window rather than creating literal infinite context. 20:48 23:30 24:07
  • Stable prompt prefixes preserve cached tokens. Compacting a long conversation while its cache is still warm reduces the cost of summarization and later turns. 28:13 32:03 32:30
  • Memory has different lifetimes: persistent user facts, dated activity logs, temporary scratchpad notes, and files retrieved when needed. 32:51 33:39 34:05
  • Product improvement has two loops: turn failures into evaluations and ship verified fixes frequently, then use those failures and evaluations to improve later models. 36:13 36:38