How the comparison works

  • Theo's “council” skill asks three to five model families to review the same proposed change or plan. This generates comparable outputs from the same task. 11:08 11:31
  • He has Opus and Soul judge their reviews, describing the judging as fully blind. The reported comparison contains 83 review pairs; it measures preference on his review workload. 11:31 12:00

Reported outcomes

  • The Opus judge preferred Opus in 66 of 83 comparisons, about 80%. The Soul judge preferred Opus in 54 of 83, about 65%. Both judges favored Opus overall, although some reviews favored Soul. 12:00
  • Theo gives the median Opus review cost as $1.65. The Soul figure is transcribed only as “36,” without a clear unit; $0.36 is a plausible reading but is not explicit in the supplied transcript. 12:00
  • His reported subscription estimates are about 205 Opus reviews versus 290 Soul reviews. He later rounds Opus to about 200 and describes Soul as giving 30–50% more work. 12:26 12:58
  • Theo says Soul finishes reviews slightly faster, but he favors Opus because of the review-quality difference. 12:58

Numerical caveats

  • This section quotes expected API-equivalent allowances of roughly $3,300 for Claude/Opus and $1,000 for ChatGPT Pro/Soul. Earlier, the video quoted $12,000 and $4,300. The supplied transcript does not explain how these sets of allowances relate. 5:03 12:26
  • The quoted figures do not reproduce the reported review counts. As an arithmetic check, $3,300 ÷ $1.65 equals 2,000 reviews, not 205; if “36” means $0.36, $1,000 ÷ $0.36 is about 2,778, not 290. Treat 205/290 as Theo's reported estimates, not a calculation these notes can validate from the transcript. 12:00 12:26
  • The usable finding is that both judges preferred Opus on this workload. The exact monthly throughput estimate needs clarification of the costs and effective allowances. 12:00 12:26