Each benchmark usually tests a specific capability. 1:00
Capability benchmarks
- MMLU — general knowledge on subjects like history and law.
1:00 - HellaSwag — common-sense reasoning.
1:00 - Mathematical benchmarks — many exist; models solve math problems.
1:00 - Code benchmarks — the model writes code or answers coding tasks.
1:00 - MT-Bench — tests how LLMs handle multi-turn conversations.
1:00
Risk and safety benchmarks
- StrongReject — tests a model's resistance against common attacks; OpenAI's o1 model card reports its performance on this benchmark.
1:25