Future outlook and open problems

Product strategy

“Delete the product” means being willing to rebuild interfaces and harness assumptions as models improve. Lee says the architecture in the lecture already differs substantially from what he would have presented six months or a year earlier. 37:45
Build a small, simple interface that can benefit from the next generation of model capabilities. 38:17

Over the next 6–12 months relative to the talk, Lee expects agents to progress from hours or days of work toward weeks or months. 38:35
He expects better recall, broader access to the tools humans use, simpler or voice-based interfaces, and more learning of reusable skills on the job. 38:58 39:21 39:43
Companies may document more of their bespoke processes so agents can follow them. 40:09
Lee predicts products will either integrate agent capabilities or expose data for other agents to use, creating demand for workflows, sandboxes, memory, identity, payments, evaluations, and observability. 42:36

How do you fully evaluate a month-long agent task when new models ship every month? Lee presents better evaluations or slower releases as possible responses. 40:26
Memory algorithms and secure execution environments still need work, including protections against hostile takeover. 40:53
Notification policies must balance useful initiative against annoying interruptions. Lee says the current heuristics are imperfect. 41:10
Rising total token volume makes efficient inference, models, and harnesses an ongoing engineering priority. 41:38

Understand and review the system architecture. Its decisions compound even as agents write more code. 43:25
Build a small version of the harness and connect it to a sandbox or VM to learn how the components work together. 43:48 44:19
Learn how LLMs work, including their training, because model behavior influences product and harness decisions. Lee compares this to understanding databases when building web applications. 44:19 44:37