LLM Benchmarks Compared: MMLU, HumanEval, GSM8K and More (2026)

LLM benchmarks are standardised tests that measure model capability across reasoning, coding, mathematics, language understanding, and agentic task completion. In 2026, 15 major benchmarks are in active use. Most measure academic performance on fixed tasks, while only four reliably predict real production outcomes. This guide explains what each benchmark measures, shows current scores for six frontier models, and answers the question leaderboard sites don’t: which benchmarks should you actually run for your specific use case.

For live scores updated weekly, LiveBench and Artificial Analysis are the authoritative references. This guide covers something different: what those scores actually test, where they break down for enterprise production environments, and how to build an evaluation program that goes beyond public benchmarks — including the benchmark gap no leaderboard addresses, locale-level capability.

Why LLM benchmark scores often don’t transfer to production

Benchmark scores fail to predict production performance for two reasons: saturation and data contamination. A saturated benchmark is one where leading models have already reached near-ceiling scores, so the benchmark no longer distinguishes between good and great. A contaminated benchmark is one where the test questions appeared in the model’s training data, making the score an artifact of memorisation rather than reasoning.

GSM8K, the grade school math benchmark, illustrates both problems. When it launched in 2021, it was challenging: GPT-3 scored around 35%. By 2024, GPT-4o, Claude 3.5, and Gemini 1.5 all exceeded 90%. Today, GPT-5.3 Codex scores 99%. The benchmark has saturated completely for frontier model comparisons. More seriously, a 2023 study showed that removing contaminated examples from the GSM8K test set produced accuracy drops of up to 13% for some models, meaning a meaningful portion of high scores were driven by training set overlap rather than genuine mathematical reasoning.

The result is that a model’s published benchmark score predicts production performance only when three conditions hold: the benchmark tests tasks similar to your use case, the test set is clean of training data contamination, and the benchmark hasn’t saturated to the point where score differences are statistically meaningless.

The 15 most-cited LLM benchmarks: what each one tests

Each benchmark targets a distinct capability. Here are the 15 most widely used LLM benchmarks in 2026, organised by category. The master table below is the fastest way to find the benchmark that maps to your evaluation goal.

Reasoning and knowledge: MMLU, BIG-Bench Hard, HellaSwag, GPQA-Diamond

MMLU measures reasoning and knowledge across 57 academic subjects, from elementary mathematics to professional law and medicine. Its 16,000+ multiple-choice questions make it the most widely cited general-capability benchmark. The limitation: frontier models have saturated it above 88%, and GPT-5.3 Codex now scores 93%. MMLU scores no longer differentiate between leading models. It remains useful for evaluating smaller and mid-tier models where the spread is still informative.

BIG-Bench Hard tests complex multi-step reasoning across 23 tasks specifically designed to resist shortcut solutions. Tasks include logical deduction, causal reasoning, and algorithmic problems. Unlike MMLU, BBH requires chain-of-thought reasoning to score well, which makes it more predictive of real-world reasoning performance for models below the frontier tier.

HellaSwag measures commonsense reasoning through sentence completion. Given a scenario description, the model must select the most plausible continuation from four options. Like MMLU, HellaSwag has saturated for frontier models at 95%+. It remains useful as a baseline check for smaller models and fine-tuned variants.

GPQA-Diamond tests expert-level reasoning on PhD-level science questions in biology, chemistry, and physics. Questions are designed so that non-expert PhD holders score around 34%, providing a meaningful floor. As of February 2026, Gemini 3.1 Pro leads at 94.3%, Claude Opus 4.6 at 91.3%, and GPT-5.3 Codex at 81%, with Qwen3.5-plus close behind at 88.4%. The benchmark is approaching saturation at the very top of the frontier but still clearly differentiates models in the 60-90% range.

Code and software engineering: HumanEval, HumanEval+, SWE-bench, LiveCodeBench

HumanEval measures code generation quality across 164 Python programming tasks, testing each completion with unit tests to check functional correctness. It is the standard coding benchmark cited in model release papers. The limitation: frontier models including GPT-5.3 Codex now score 93%, and training set contamination is well-documented. For current frontier model comparisons, LiveCodeBench or SWE-bench Verified are more reliable.

HumanEval+ extends HumanEval with an expanded test suite that catches more edge cases and reduces false positives. A model that passes the basic HumanEval test may fail HumanEval+ on the same task because the additional tests expose incomplete implementations. Score gaps between HumanEval and HumanEval+ reveal code quality issues invisible to the original benchmark.

SWE-bench Verified measures a model’s ability to resolve real GitHub issues in open-source codebases. Unlike HumanEval, SWE-bench tasks require understanding an existing codebase, identifying the root cause of a bug, and writing a solution that passes the repository’s test suite. Claude Opus 4.6 leads at 80.8%, MiniMax-M2.5 at 80.2%, and Gemini 3.1 Pro at 80.6%, with GLM-5 and Qwen3.5-plus clustered in the 76-78% range. SWE-bench Verified is highly resistant to contamination because the problems come from real-world software development work.

LiveCodeBench uses rolling updates to stay contamination-resistant. New programming questions are added monthly from competitive programming platforms, so models cannot be trained on the test set. Qwen3.5-plus leads at 83.6% on LiveCodeBench v6 as of February 2026. LiveCodeBench is the best choice for tracking coding capability over time as models are updated, because score increases must reflect genuine improvement rather than memorisation.

Math and logic: GSM8K, MATH-500, MGSM

GSM8K tests mathematical reasoning across 8,500 grade school word problems. It was the standard math benchmark from 2021 through 2023. Frontier models now reach 99% (GPT-5.3 Codex), making GSM8K useless for top-tier comparisons. It is still informative for evaluating smaller models and quantifying the gap between fine-tuned and base variants.

MATH-500 tests competition-level mathematical reasoning across 500 problems drawn from AMC, AIME, and similar competitions, spanning five difficulty levels. Unlike GSM8K, MATH-500 problems require multi-step symbolic reasoning. GPT-5.3 Codex scores 96%. For harder math evaluation, AIME 2025 and AIME 2026 are now the standard frontier benchmarks, with Qwen3.5-plus scoring 91.3% on AIME 2026 and GPT-5.3 Codex at 94% on AIME 2025.

MGSM tests mathematical reasoning across 10 languages, using the same 250 grade school math problems translated into Bengali, Chinese, French, German, Japanese, Russian, Spanish, Swahili, Telugu, and Thai. For any application that serves non-English users, MGSM reveals whether a model reasons consistently across languages or shows significant capability gaps in lower-resource languages. GPT-5.3 Codex scores 96% on MGSM.

One limitation no benchmark addresses: locale variants. MGSM tests Portuguese, but Brazilian and European Portuguese diverge on vocabulary (“ônibus” vs. “autocarro” for bus), formality registers (Brazilian Portuguese defaults to informal second-person “você” where European Portuguese uses formal constructions), and cultural references that affect how a model interprets and generates natural responses. A model that passes MGSM for Spanish does not guarantee consistent performance across Mexican, Argentine, and Castilian Spanish, which differ in vocabulary, idiom, and formality norms. Canadian and European French split on register and technical vocabulary. Simplified and Traditional Chinese share a script but diverge in vocabulary and usage patterns that reflect decades of linguistic separation.

A model that passes MGSM may still fail any of these locale splits in production. No public benchmark covers this gap. Addressing locale-level capability requires evaluation by native-speaker annotators who understand the specific variant your users speak, not aggregate language scores. For enterprise applications serving users across multiple locales — customer support in Latin America alongside Spain, or healthcare AI across Canadian and French markets — benchmark scores for a language code are not sufficient signal.

Safety and factuality: TruthfulQA, HELM

TruthfulQA measures a model’s tendency to produce confident hallucinations across 817 questions in 38 categories including health, law, finance, and conspiracy theories. Questions are designed to elicit incorrect but plausible-sounding answers that mimic common human misconceptions. A model that scores 30% on TruthfulQA generates false but confident answers nearly 70% of the time when asked about topics where misinformation is common.

HELM evaluates models holistically across multiple metrics including accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency. Rather than optimising for a single score, HELM produces a multi-dimensional profile. This makes HELM the appropriate choice for compliance-sensitive applications where accuracy alone is insufficient and safety or fairness properties must also be measured.

Agentic tasks: GAIA and AgentBench

GAIA evaluates AI agents on real-world tasks requiring tool use, multi-step reasoning, and information retrieval. Tasks range from answering factual questions using web search to completing multi-step data analysis. GAIA scores are heavily dependent on the agent framework used, so comparing raw GAIA scores across models without controlling for the prompting setup is not meaningful. Top agent systems built on frontier models now score above 85% on Level 1 tasks.

AgentBench tests agent performance across diverse interactive environments including operating systems, databases, knowledge graphs, and web browsers. Unlike static question-answering benchmarks, AgentBench requires sustained goal-directed behaviour over multiple steps. It covers tool use, memory, and planning simultaneously, making it the most comprehensive multi-environment agent benchmark available.

Frontier difficulty: HLE

Humanity’s Last Exam (HLE) consists of 2,500 expert-level questions contributed by domain specialists in mathematics, science, law, and other fields. Questions were designed to be beyond the reach of AI systems in 2024. In February 2026, Claude Opus 4.6 leads at 53.1% with tool access, followed by GPT-5.3 Codex at 36% and GLM-5 at 32%. The gap between tool-assisted and non-tool scores is significant, confirming that HLE measures agentic reasoning alongside raw knowledge. HLE is the current frontier benchmark: as models improve, their HLE score will track genuine capability gains on genuinely difficult problems.

Which LLM benchmark should you use? A use case selection guide

The right benchmark depends on your production task. The matrix below maps common enterprise use cases to the benchmarks most likely to predict performance in your specific context, with notes on what each benchmark misses.

LLM benchmark scores in 2026: six frontier models compared

Here are current scores across seven benchmarks for six widely deployed frontier models as of February 2026. These benchmarks were chosen because they still meaningfully differentiate frontier models — MMLU, HumanEval, and GSM8K are omitted from this table because all frontier models have saturated them above 90%.

¹ GPT-5.3-Codex scores 56.8% on the harder SWE-bench Pro variant; SWE-bench Verified not independently reported. ² MMMLU (massively multilingual variant). ³ Measured with tool access. ⁴ AIME 2026 variant. — = not publicly reported at time of writing.

Sources: GPT-5.3 Codex (automatio.ai), Claude Opus 4.6 (Anthropic/vellum.ai), Gemini 3.1 Pro (officechai.com), Qwen3.5-plus (analyticsvidhya.com), MiniMax-M2.5 (minimax.io), GLM-5 (automatio.ai). Scores change with each model update — this table is reviewed quarterly.

The benchmark-to-production gap: saturation, contamination, and what actually predicts success

A model’s benchmark score predicts production performance only when the benchmark tests tasks similar to your use case and the training data hasn’t contaminated the test set. Both conditions fail more often than benchmark papers acknowledge.

Saturation means differentiation collapses at the top. When GPT-5.3, Claude Opus 4.6, and Gemini 3.1 all score 88-93% on MMLU, the practical difference between them on a real-world knowledge task cannot be read from the benchmark. The score range has compressed to the point where noise exceeds signal. For saturated benchmarks, the model selection decision must be made on other criteria: latency, cost, instruction-following quality, or domain-specific fine-tuning.

Contamination means some scores reflect memory, not capability. The 13% GSM8K accuracy drop observed when contaminated examples were removed is a documented lower bound, not an upper bound. The true effect is larger and harder to measure because most training datasets are not fully disclosed. When a model achieves an unusually high score on an older benchmark, contamination should be the first hypothesis, not the last. GSM8K at 99% is not evidence that GPT-5.3 can solve all grade school math problems — it is evidence that the benchmark has exhausted its usefulness as a differentiator.

In evaluation programs built for enterprise clients, the most reliable predictor of production success is not a single benchmark score. The pattern that appears consistently across annotation quality evaluations is this: models that perform well on tasks requiring entity-level precision, where the right answer must name a specific entity, date, product, or person rather than a general category, outperform models with higher overall benchmark scores on real-world tasks where specificity matters. GPQA-Diamond and SWE-bench Verified, both requiring precise, verifiable outputs, show stronger correlation with production performance on enterprise tasks than MMLU or HellaSwag.

A model’s benchmark score is downstream of its training data quality. Models that lead on GPQA Diamond have been trained on expert-level content annotated with entity-level precision: not just “this answer is correct” but annotations that identify the specific mechanism, named entity, or causal chain the answer depends on. Generic crowdsourced annotation tells a model what to output. Expert annotation teaches the reasoning chain behind why an output is correct. The difference appears directly in benchmark scores that require multi-step precision reasoning.

This has a direct implication for fine-tuning programs. If your fine-tuned model is not improving on precision-sensitive benchmarks despite more training examples, the bottleneck is annotation quality, not data volume or model architecture. A model fine-tuned on 10,000 ambiguously labelled examples will plateau; the same model fine-tuned on 1,000 carefully annotated examples from domain-expert annotators typically does not. The benchmark ceiling for a fine-tuned model is set by the specificity of its training annotations — which is why teams that invest in annotation quality before scaling data volume consistently outperform teams that do the reverse.

How to build a benchmark baseline for your LLM evaluation program

A production evaluation baseline requires five components: task mapping, public benchmark selection, a proprietary test set, human-in-the-loop spot checks, and versioning. Here is how to build each one.

1. Map your use case to measurable task types. Before selecting any benchmark, define the specific tasks your model will perform in production. A customer support model handles multi-turn dialogue, entity extraction (order numbers, product names), and policy-constrained refusals. A code assistant handles generation, explanation, and debugging across specific languages. A financial analysis model handles numerical reasoning, table interpretation, and document summarisation. Each task type maps to a distinct set of benchmarks. Running MMLU for a code assistant tells you almost nothing useful.
2. Select 2-3 public benchmarks as proxy signals. Use the use case matrix above to pick benchmarks that test tasks structurally similar to your production workload. For a code assistant: LiveCodeBench, SWE-bench Verified, HumanEval+. For a multilingual support agent: MGSM plus a human evaluation component. For an agentic research tool: GAIA plus Terminal-Bench 2.0. Limit to 2-3 benchmarks to avoid benchmark overfitting, where model selection optimises for benchmark scores rather than production quality.
3. Build a proprietary test set of 100-500 examples from your production data. This is the step most teams skip and most consistently regret. A proprietary test set drawn from real production inputs, labelled by qualified annotators, tests the exact distribution of tasks your model will face. It is immune to contamination by definition and directly measures what you care about. 100 examples is the minimum for statistical reliability; 500 gives you enough examples to segment by task type and find targeted weaknesses. The test set should be rotated every 6 months to prevent the team from implicitly overfitting to it.

4. Run human spot evaluation on 10% of model outputs. LLM-as-a-judge evaluation, where you use a second model to score the first model’s outputs, is faster and cheaper than human evaluation but systematically underestimates errors in edge cases. Human evaluators catch failure modes that automated metrics miss: factual errors that are superficially plausible, outputs that are technically accurate but inappropriate for the context, and reasoning chains that reach correct conclusions through flawed logic. A 10% human spot evaluation rate gives you a calibration layer without requiring full human review of every output.
5. Version, rotate, and track benchmark scores over model iterations. Benchmark scores are only meaningful as a time series. A score of 82% on your proprietary test set tells you nothing in isolation; 82% after fine-tuning, up from 74% on the same test set before fine-tuning, tells you the fine-tuning improved performance by 8 points on your specific tasks. Version your test sets, record scores for every model update, and rotate the test set periodically to prevent implicit overfitting. Treat your evaluation baseline as a product that requires maintenance, not a one-time measurement.

For organisations building production LLM systems, this five-step baseline is the minimum necessary to make model selection decisions with confidence. Relying on published benchmark scores alone means trusting that the public test set distribution matches your production workload, that contamination hasn’t inflated the scores you are comparing, and that the benchmark hasn’t saturated to the point where score differences are noise. For most enterprise applications, none of those assumptions holds.

If you are evaluating models for an application that requires precision across multiple languages or domains, the benchmark selection and proprietary test set design work is where we spend most of our time with clients. The LXT model evaluation service covers benchmark selection, test set curation, and annotation QA for enterprise evaluation programs. For teams building or fine-tuning multilingual models, our data annotation service covers the training data quality work that benchmark scores are downstream of.

For a deeper look at how agent evaluation extends benchmark methodology, see our post on AI agent evaluation frameworks.