Video cover

Theo reviews Gemini 4 Argan through pricing, front-end and 3D demos, specialized visual tasks, private code audits, and a failed SQLite implementation. His central argument is that strong benchmark or demo results do not reliably predict usefulness in an existing codebase. 7:43 37:06

Watch the video · Theo - t3․gg · 38:30

Classification: Opinion/news, specifically a hands-on model review with benchmark evidence. These notes report the video's claims, measurements, and judgments, rather than independently verifying them. Model names follow the supplied transcript where identifiable; unclear names are flagged in the relevant notes.

Topic breakdown

  1. Release context, disclosures, and Google's claims
  2. Pricing, token use, and caching
  3. Skatebench, vision, and evaluation scope
  4. Front-end design and iteration speed
  5. YouTube access and specialized workflows
  6. Fishlop: visuals versus game-system reasoning
  7. Hallucination metrics versus code-audit results
  8. SQLite case study: diagnosis, implementation, and review
  9. Antigravity reliability and integration costs
  10. Final model choices and limits of the verdict

The original transcript and cover image are included unchanged.

Key takeaways

  • Low token rates did not guarantee low task cost. Theo reports heavy token use and 70–80% cache hits versus 99%+ with other models, while leaving the cause of the cache behavior unresolved. 4:31 4:53 5:24
  • Gemini achieved 100% on Theo's private skateboard-trick naming benchmark, compared with 79% for Opus 5.5 Max. This demonstrated a specific capability, not general coding competence. 7:43 8:14 8:53
  • Front-end designs improved with Anthropic's official design skill; Theo still preferred Fable 5.1 for distinctive marketing pages and emphasized fast iteration for product UX. 12:03 12:26 12:53 13:45
  • In the TS Rust audit, Gemini scored 4.38/10 with 24 false claims out of 36; 6.1 Soul scored 8.84 with 38 true claims and none wrong. 22:58 23:23 23:48 24:16
  • The pattern persisted across projects: Gemini scored 2.25/10 on Lakebed with 56 wrong findings out of 78, and 3.95/10 on T3 code architecture with 47 false claims. 27:13 27:33 28:07 28:34
  • Gemini's roughly 1,200-line SQLite PR failed checks and left main-thread blocking unresolved. Theo reports that the replacement change was 21 lines and merged. 30:47 31:37 32:22 33:33
  • Theo's final choice was to stop using Gemini for code, retain it for YouTube data retrieval, and primarily continue with Opus 5.5. He says he merged none of Gemini's work from this testing. 37:06 37:30 38:14