{"categories":["LLM"],"content":{"markdown":"A benchmark score means little on its own. It starts to mean something when you see how fast it moved: which model set the record, when, and how long the test lasted before the next one had to be invented.\n\nThis page keeps that record for 16 benchmarks, from the GPT-2 era to today. For each one you see every model that beat all earlier models at the time of its release, and the leaders as of now. The numbers come from the open benchmark archive of Epoch AI — 2,621 results in the 16 benchmarks shown here. We did not test any model ourselves and we changed no value; our part is the selection, the record lines and the comparison over time.\n\n## How to read the tables\n\n**Record** means: best score among all models released up to that date. The date is the release date of the model as the archive records it, not the day of the test. A model that was tested later still stands at its release date.\n\nScores are not comparable across benchmarks, and not always within one: results that Epoch AI ran itself use one fixed setup, while collected results come from papers and leaderboards with their own setups. Each benchmark names where its results come from.\n\nWhen a benchmark shows “no new results since”, that is usually a sign of success, not of neglect: the best models are close to the maximum, and the field has moved on to harder tests.\n\n## The records\n\n{{\u003c llm-benchmarks \u003e}}\n\n## What the record lines show\n\n**Tests are used up faster and faster.** On MMLU the record stood at 86.4% when GPT-4 arrived in March 2023, and it has barely moved since — the test was full. ARC-AGI-2 was introduced in March 2025 as a puzzle set that models could not solve; the first model above ten percent followed in July 2025, and the record now stands at 95.0%.\n\n**The length of tasks grows, not just the score.** METR's time horizon is the one measure here that is not capped at 100. GPT-4 handled tasks of about 5 minutes in 2023; the current record is over 17 hours.\n\n**One number hides the setup.** The same model appears with different scores depending on its reasoning effort or the agent software around it — the names in the tables carry these settings in brackets. If you compare two models for your own project, compare them in the setting you will actually use.\n\nIf you build with these models, the benchmarks are the start of the choice, not the end. The [LLM ranking](/en/llm/ranking/) puts all models on one scale, and the [LLM database](/en/llm/) lets you filter them and compare any four side by side.","text":"A benchmark score means little on its own. It starts to mean something when you see how fast it moved: which model set the record, when, and how long the test lasted before the next one had to be invented.\nThis page keeps that record for 16 benchmarks, from the GPT-2 era to today. For each one you see every model that beat all earlier models at the time of its release, and the leaders as of now. The numbers come from the open benchmark archive of Epoch AI — 2,621 results in the 16 benchmarks shown here. We did not test any model ourselves and we changed no value; our part is the selection, the record lines and the comparison over time.\nHow to read the tables Record means: best score among all models released up to that date. The date is the release date of the model as the archive records it, not the day of the test. A model that was tested later still stands at its release date.\nScores are not comparable across benchmarks, and not always within one: results that Epoch AI ran itself use one fixed setup, while collected results come from papers and leaderboards with their own setups. Each benchmark names where its results come from.\nWhen a benchmark shows “no new results since”, that is usually a sign of success, not of neglect: the best models are close to the maximum, and the field has moved on to harder tests.\nThe records All benchmarks at a glance BenchmarkModels from–toRecord todayModel Epoch Capabilities Index2023–2026167.3Claude Opus 5.5 Anthropic METR Time Horizon2019–202617 h 25 minClaude Mythos Preview (Early) Anthropic MMLU2021–202488.1%GPT-4o OpenAI GPQA Diamond2023–202695.8%GPT-6 Astra (max) OpenAI Humanity\u0026#39;s Last Exam2024–202654.8%GPT 6 Astra OpenAI SimpleQA Verified2024–202675.6%GPT-6 Astra (max) OpenAI GSM8K2020–202494.5%DS-Coder-V2-Instruct DeepSeek MATH Level 52023–202598.1%GPT-5 (high) OpenAI FrontierMath (Tiers 1–3)2024–202693.7%GPT-6 Astra (max) OpenAI SWE-bench Verified2024–202683.5%Claude Opus 4.7 (max) Anthropic Aider Polyglot2024–202588.0%GPT-5 (high) OpenAI Terminal-Bench2025–202684.7%GPT-5.5 OpenAI ARC-AGI2024–202698.5%Claude Fable 5 (XHigh) Anthropic ARC-AGI-22024–202695.0%GPT-6 Astra (Max) OpenAI HellaSwag2019–202495.3%GPT-4 OpenAI WinoGrande2019–202489.2%LLaMA-3.1 405B Base Meta AI Overall capability Epoch Capabilities Index published 2025 · models from 2023 to 2026 · 274 models in the archive · active · results: Epoch AI\nCombines the results of more than 50 benchmarks into one number per model, so that models can be compared even if they were never tested on the same benchmarks. It is an index, not a percentage: Epoch AI scales it so that Claude 3.5 Sonnet stands at 130 and GPT-5 at 150.\nRecords by model release date DateModelScore Feb 24, 2023LLaMA-65B Meta AI110.2 Mar 14, 2023GPT-4 (Mar 2023) OpenAI125.9 Jan 25, 2024GPT-4 Turbo (Nov 2023) OpenAI126.5 Feb 29, 2024Claude 3 Opus Anthropic126.9 Apr 9, 2024GPT-4 Turbo (Apr 2024) OpenAI127.2 May 13, 2024GPT-4o (May 2024) OpenAI129.0 Jun 20, 2024Claude 3.5 Sonnet Anthropic130.0 Sep 12, 2024o1-mini OpenAI135.8 Dec 17, 2024o1 OpenAI141.9 Mar 31, 2025Gemini 2.5 Pro (Mar 2025) Google DeepMind144.2 Apr 16, 2025o3 OpenAI146.9 Jun 10, 2025o3-pro OpenAI147.4 Aug 7, 2025GPT-5 OpenAI150.0 Oct 7, 2025GPT-5 Pro OpenAI150.3 Nov 18, 2025Gemini 3 Pro Google DeepMind152.9 Dec 11, 2025GPT-5.2 Pro OpenAI155.4 Feb 5, 2026GPT-5.3 Codex OpenAI156.8 Mar 5, 2026GPT-5.4 Pro OpenAI158.9 Apr 23, 2026GPT-5.5 Pro OpenAI162.1 Jul 24, 2026Claude Opus 5 Anthropic162.8 Sep 1, 2026Claude Fable 5.1 Anthropic164.7 Sep 3, 2026GPT-6 Astra OpenAI166.4 Sep 22, 2026Claude Opus 5.5 Anthropic167.3 Leaders today: Claude Opus 5.5 (167.3) · GPT-6 Astra (166.4) · GPT-6.1 Sol (166.1) · Claude Sonnet 5.5 (165.0) · Claude Fable 5.1 (164.7)\nMETR Time Horizon published 2025 · models from 2019 to 2026 · 39 models in the archive · active · results: METR\nThe length of the tasks a model completes correctly about half of the time, measured by how long a human expert needs for them. The number is a duration, not a score: a longer horizon means the model can work on its own for longer.\nRecords by model release date DateModelScore Nov 5, 2019GPT-2 (1.5B) OpenAI3 sec Mar 14, 2023GPT-4 (Mar 2023) OpenAI5.36 min Jan 25, 2024GPT-4 Turbo Preview (January 2024) OpenAI5.37 min Apr 9, 2024GPT-4 Turbo (Apr 2024) OpenAI6.57 min May 13, 2024GPT-4o (May 2024) OpenAI9.17 min Jun 20, 2024Claude 3.5 Sonnet (Jun 2024) Anthropic11 min Sep 12, 2024o1-preview OpenAI20 min Oct 22, 2024Claude 3.5 Sonnet (Oct 2024) Anthropic21 min Dec 17, 2024o1 (medium) OpenAI39 min Feb 24, 2025Claude 3.7 Sonnet Anthropic1 h Apr 16, 2025o3 (unknown thinking) OpenAI2 h Aug 7, 2025GPT-5 (high) OpenAI3 h 23 min Nov 18, 2025Gemini 3 Pro Preview Google DeepMind3 h 44 min Nov 24, 2025Claude Opus 4.5 (no thinking) Anthropic4 h 53 min Dec 11, 2025GPT-5.2 (high) OpenAI5 h 52 min Feb 5, 2026Claude Opus 4.6 (unknown thinking) Anthropic11 h 59 min Apr 7, 2026Claude Mythos Preview (Early) Anthropic17 h 25 min The bars use a logarithmic scale: each step to the right is a multiplication.\nLeaders today: Claude Mythos Preview (Early) (17 h 25 min) · Claude Opus 4.6 (unknown thinking) (11 h 59 min) · Gemini 3.1 Pro Preview (6 h 24 min) · GPT-5.2 (high) (5 h 52 min) · GPT-5.3 Codex (5 h 50 min)\nKnowledge and reasoning MMLU published 2020 · models from 2021 to 2024 · 158 models in the archive · no new results since 2024 · results: papers and Stanford HELM, collected by Epoch AI\nMultiple-choice questions from 57 subjects, from high-school level to professional exams. For years the standard test of general knowledge; today the best models are so close to the ceiling that hardly any new results are published.\nGPT-3\u0026#39;s well-known 43.9% is filed in the archive under the API version of January 2022 and therefore does not appear as a record here.\nRecords by model release date DateModelScore Aug 5, 2021GPT-J-6B EleutherAI, LAION25.7% Dec 8, 2021Gopher DeepMind60.0% Mar 15, 2022GPT-3.5 OpenAI70.0% Mar 14, 2023GPT-4 (initial release) OpenAI86.4% Jun 20, 2024Claude 3.5 Sonnet (20240620) Anthropic86.5% Sep 24, 2024Gemini 1.5 Pro (002) Google DeepMind86.9% Oct 22, 2024Claude 3.5 Sonnet (20241022) Anthropic87.3% Nov 20, 2024GPT-4o OpenAI88.1% Leaders today: GPT-4o (88.1%) · Claude 3.5 Sonnet (20241022) (87.3%) · DeepSeek v3 (87.2%) · Gemini 1.5 Pro (002) (86.9%) · Claude 3.5 Sonnet (20240620) (86.5%)\nGPQA Diamond published 2023 · models from 2023 to 2026 · 214 models in the archive · active · results: Epoch AI (own runs)\n198 multiple-choice questions in biology, physics and chemistry at graduate level, written so that non-experts fail even with web access. Guessing gives 25%.\nRecords by model release date DateModelScore Mar 14, 2023GPT-4 (Mar 2023) OpenAI35.7% Nov 6, 2023GPT-4 Turbo Preview (Nov 2023) OpenAI42.4% Feb 29, 2024Claude 3 Opus Anthropic47.2% May 13, 2024GPT-4o (May 2024) OpenAI48.9% Jun 20, 2024Claude 3.5 Sonnet (Jun 2024) Anthropic54.0% Sep 12, 2024o1-mini (high) OpenAI62.4% Dec 17, 2024o1 (high) OpenAI76.8% Jan 31, 2025o3-mini (high) OpenAI77.0% Feb 24, 2025Claude 3.7 Sonnet (64k thinking) Anthropic79.7% Mar 25, 2025Gemini 2.5 Pro Exp (Mar 2025) Google DeepMind83.8% Jun 5, 2025Gemini 2.5 Pro Preview (Jun 2025) Google DeepMind84.8% Jul 9, 2025Grok 4 xAI87.0% Nov 13, 2025GPT-5.1 (high) OpenAI87.6% Nov 18, 2025Gemini 3 Pro Preview Google DeepMind92.6% Feb 19, 2026Gemini 3.1 Pro Preview Google DeepMind94.4% Mar 5, 2026GPT-5.4 Pro (xhigh) OpenAI94.6% Aug 13, 2026Gemini 3.7 Flash (high) Google DeepMind94.8% Sep 2, 2026Gemini 3.8 Flash (high) Google DeepMind95.4% Sep 3, 2026GPT-6 Astra (max) OpenAI95.8% Leaders today: GPT-6 Astra (max) (95.8%) · Claude Sonnet 5.5 (max) (95.6%) · Gemini 3.8 Flash (high) (95.4%) · GPT-6.1 Sol (max) (95.4%) · Gemini 3.7 Flash (high) (94.8%)\nHumanity\u0026#39;s Last Exam published 2025 · models from 2024 to 2026 · 44 models in the archive · active · results: Humanity\u0026#39;s Last Exam leaderboard, collected by Epoch AI\nExpert-written questions at the edge of human knowledge across many fields, built to stay hard after older tests were solved.\nRecords by model release date DateModelScore Sep 24, 2024Gemini-1.5-Pro-002 Google DeepMind4.6% Dec 17, 2024o1 (December 2024) OpenAI8.0% Mar 19, 2025o1 Pro OpenAI8.1% Mar 25, 2025Gemini 2.5 Pro Experimental (March 2025) Google DeepMind18.2% Apr 16, 2025o3 (high) (April 2025) OpenAI20.3% Jun 5, 2025gemini-2.5-pro-preview-06-05 Google DeepMind21.6% Aug 7, 2025gpt-5-2025-08-07 OpenAI25.3% Oct 7, 2025gpt-5-pro-2025-10-06 OpenAI31.6% Nov 18, 2025gemini-3-pro-preview Google DeepMind37.5% Feb 19, 2026Gemini 3.1 Pro Preview (thinking high) Google DeepMind46.4% Sep 1, 2026Fable 5.1 (xhigh) Anthropic46.5% Sep 3, 2026GPT 6 Astra OpenAI54.8% Leaders today: GPT 6 Astra (54.8%) · Fable 5.1 (xhigh) (46.5%) · Gemini 3.1 Pro Preview (thinking high) (46.4%) · Gemini 3.8 Flash (44.5%) · GPT-5.4 Pro (44.3%)\nSimpleQA Verified published 2025 · models from 2024 to 2026 · 79 models in the archive · active · results: Epoch AI (own runs)\nShort factual questions with exactly one correct answer, answered without tools. It shows how much a model really knows and how often it invents an answer instead of admitting that it does not know.\nRecords by model release date DateModelScore Feb 29, 2024Claude 3 Opus Anthropic12.6% Aug 6, 2024GPT-4o (Aug 2024) OpenAI26.0% Dec 17, 2024o1 (high) OpenAI41.1% Apr 16, 2025o3 (high) OpenAI49.4% Aug 7, 2025GPT-5 (high) OpenAI50.1% Dec 17, 2025Gemini 3 Flash Preview Google DeepMind66.8% Feb 19, 2026Gemini 3.1 Pro Preview Google DeepMind73.5% Sep 3, 2026GPT-6 Astra (max) OpenAI75.6% Leaders today: GPT-6 Astra (max) (75.6%) · GPT-6.1 Sol (max) (73.9%) · Gemini 3.1 Pro Preview (73.5%) · Claude Opus 5.5 (max) (72.2%) · Claude Fable 5.1 (max) (70.8%)\nMathematics GSM8K published 2021 · models from 2020 to 2024 · 146 models in the archive · no new results since 2024 · results: papers and Stanford HELM, collected by Epoch AI\nMath word problems at grade-school level that need several calculation steps. Once a hard test of reasoning, now practically solved.\nRecords by model release date DateModelScore Jun 22, 2020babbage (1.3B) OpenAI0.7% Mar 15, 2022code-davinci-002 OpenAI56.8% Nov 28, 2022GPT-3.5 (8-shot) OpenAI78.2% Mar 14, 2023GPT-4 OpenAI92.0% Jun 17, 2024DS-Coder-V2-Instruct DeepSeek94.5% Leaders today: DS-Coder-V2-Instruct (94.5%) · Qwen2.5-Coder-14B-Instruct (94.2%) · Qwen2.5-Coder-32B-Instruct (93.0%) · GPT-4 (92.0%) · GPT-4 (92.0%)\nMATH Level 5 published 2021 · models from 2023 to 2025 · 94 models in the archive · active · results: Epoch AI (own runs)\nThe hardest level of the MATH dataset: competition problems from algebra, geometry, number theory and probability.\nRecords by model release date DateModelScore Jun 13, 2023GPT-4 (Jun 2023) OpenAI23.0% Nov 6, 2023GPT-4 Turbo Preview (Nov 2023) OpenAI40.0% Apr 9, 2024GPT-4 Turbo (Apr 2024) OpenAI46.7% May 13, 2024GPT-4o (May 2024) OpenAI51.0% Jun 20, 2024Claude 3.5 Sonnet (Jun 2024) Anthropic51.7% Jul 18, 2024GPT-4o mini OpenAI52.6% Aug 6, 2024GPT-4o (Aug 2024) OpenAI53.3% Sep 12, 2024o1-mini (high) OpenAI89.2% Dec 17, 2024o1 (high) OpenAI94.7% Jan 31, 2025o3-mini (high) OpenAI96.5% Apr 16, 2025o4-mini (high) OpenAI97.8% Aug 7, 2025GPT-5 (high) OpenAI98.1% Leaders today: GPT-5 (high) (98.1%) · o4-mini (high) (97.8%) · o3 (high) (97.8%) · GPT-5 mini (high) (97.8%) · Claude Sonnet 4.5 (32k thinking) (97.7%)\nFrontierMath (Tiers 1–3) published 2026 · models from 2024 to 2026 · 83 models in the archive · active · results: Epoch AI (own runs)\nOriginal, unpublished mathematics problems written by mathematicians, from advanced university level up to research level. The problems are kept private so that models cannot have seen them in training.\nThe archive lists this second version of the problem set since June 2026, with results for older models as well.\nRecords by model release date DateModelScore Jan 25, 2024GPT-3.5 Turbo (Jan 2024) OpenAI0.0% Apr 9, 2024GPT-4 Turbo (Apr 2024) OpenAI0.7% Dec 17, 2024o1 (high) OpenAI14.7% Jan 31, 2025o3-mini (high) OpenAI18.6% Apr 16, 2025o4-mini (high) OpenAI36.1% Aug 7, 2025GPT-5 (high) OpenAI55.4% Oct 7, 2025GPT-5 Pro OpenAI55.8% Dec 11, 2025GPT-5.2 Pro OpenAI74.0% Mar 5, 2026GPT-5.4 Pro (xhigh) OpenAI82.5% Apr 23, 2026GPT-5.5 Pro (xhigh) OpenAI87.7% Jul 9, 2026GPT-5.6 Sol (max) OpenAI89.1% Sep 1, 2026Claude Fable 5.1 (max) Anthropic90.2% Sep 3, 2026GPT-6 Astra (max) OpenAI93.7% Leaders today: GPT-6 Astra (max) (93.7%) · GPT-6.1 Sol (max) (93.7%) · Claude Opus 5.5 (max) (91.2%) · Claude Fable 5.1 (max) (90.2%) · GPT-6 Sol (max) (89.8%)\nCoding and agents SWE-bench Verified published 2024 · models from 2024 to 2026 · 32 models in the archive · active · results: Epoch AI (own runs)\n500 real issues from open-source Python projects, checked by humans. The model has to change the code so that the project\u0026#39;s tests pass.\nThese are Epoch AI\u0026#39;s own runs with one fixed setup; vendors often report higher numbers with their own tooling.\nRecords by model release date DateModelScore Nov 20, 2024GPT-4o (Nov 2024) OpenAI31.0% Feb 24, 2025Claude 3.7 Sonnet Anthropic61.0% Apr 16, 2025o3 (medium) OpenAI62.3% May 22, 2025Claude Opus 4 Anthropic70.7% Aug 5, 2025Claude Opus 4.1 Anthropic73.3% Aug 7, 2025GPT-5 (high) OpenAI73.6% Nov 24, 2025Claude Opus 4.5 (no thinking) Anthropic76.7% Feb 5, 2026Claude Opus 4.6 (no thinking) Anthropic78.7% Apr 16, 2026Claude Opus 4.7 (max) Anthropic83.5% Leaders today: Claude Opus 4.7 (max) (83.5%) · GPT-5.5 (xhigh) (80.6%) · Gemini 3.5 Flash (79.3%) · Claude Opus 4.6 (no thinking) (78.7%) · GLM-5.2 (78.7%)\nAider Polyglot published 2024 · models from 2024 to 2025 · 54 models in the archive · active · results: Aider leaderboard, collected by Epoch AI\n225 difficult programming exercises in C\u0026#43;\u0026#43;, Go, Java, JavaScript, Python and Rust. It measures whether a model can edit existing code correctly, with a second attempt after seeing the test output.\nRecords by model release date DateModelScore Jul 18, 2024GPT-4o mini OpenAI3.6% Aug 6, 2024GPT-4o (Aug 2024) OpenAI23.1% Sep 12, 2024o1-mini (unknown thinking) OpenAI32.9% Oct 22, 2024Claude 3.5 Sonnet (Oct 2024) Anthropic51.6% Dec 17, 2024o1 (high) OpenAI61.7% Feb 24, 2025Claude 3.7 Sonnet (32k thinking) Anthropic64.9% Mar 25, 2025Gemini 2.5 Pro Exp (Mar 2025) Google DeepMind72.9% Apr 16, 2025o3 (high) OpenAI81.3% Jun 5, 2025Gemini 2.5 Pro Preview (Jun 2025) Google DeepMind83.1% Jun 10, 2025o3-pro OpenAI84.9% Aug 7, 2025GPT-5 (high) OpenAI88.0% Leaders today: GPT-5 (high) (88.0%) · o3-pro (84.9%) · Gemini 2.5 Pro Preview (Jun 2025) (83.1%) · o3 (high) (81.3%) · Grok 4 (79.6%)\nTerminal-Bench published 2025 · models from 2025 to 2026 · 45 models in the archive · active · results: Terminal-Bench leaderboard, collected by Epoch AI\nTasks that an agent has to solve in a real command line: set up software, fix a system, process data. The score belongs to a model together with its agent software; we show the best result per model.\nRecords by model release date DateModelScore Jun 17, 2025Gemini 2.5 Flash Google DeepMind17.1% Jul 9, 2025Grok 4 xAI27.2% Jul 12, 2025Kimi K2 Instruct Moonshot27.8% Aug 5, 2025Claude Opus 4.1 Anthropic38.0% Aug 7, 2025GPT-5 OpenAI49.6% Nov 12, 2025GPT-5.1-Codex-Mini OpenAI61.6% Nov 18, 2025Gemini 3 Pro Google DeepMind69.4% Feb 5, 2026Claude Opus 4.6 Anthropic79.8% Feb 19, 2026Gemini 3.1 Pro Google DeepMind80.2% Mar 5, 2026GPT-5.4 OpenAI81.8% Apr 23, 2026GPT-5.5 OpenAI84.7% Leaders today: GPT-5.5 (84.7%) · GPT-5.4 (81.8%) · Gemini 3.1 Pro (80.2%) · Claude Opus 4.7 (80.2%) · Claude Opus 4.6 (79.8%)\nAbstraction ARC-AGI published 2019 · models from 2024 to 2026 · 91 models in the archive · active · results: ARC Prize leaderboard, collected by Epoch AI\nGrid puzzles in which a rule has to be worked out from a few examples and applied to a new case. Easy for people, long out of reach for language models — the test of learning something new instead of recalling it.\nRecords by model release date DateModelScore Sep 12, 2024o1-preview OpenAI18.0% Dec 17, 2024o1 (Medium) OpenAI30.7% Jan 31, 2025o3-mini (High) OpenAI34.5% Apr 16, 2025o3 (High) OpenAI60.8% Jul 9, 2025Grok 4 (Thinking) xAI66.7% Oct 7, 2025GPT-5 Pro OpenAI70.2% Nov 13, 2025GPT-5.1 (Thinking, High) OpenAI72.8% Nov 18, 2025Gemini 3 Pro Google DeepMind75.0% Nov 24, 2025Claude Opus 4.5 (64k thinking) Anthropic80.0% Dec 11, 2025GPT-5.2 Pro (X-High) OpenAI90.5% Feb 5, 2026Claude Opus 4.6 (120K, High) Anthropic94.0% Feb 12, 2026Gemini 3 Deep Think (2/26) Google DeepMind96.0% Feb 19, 2026Gemini 3.1 Pro (Preview) Google DeepMind98.0% Jun 9, 2026Claude Fable 5 (XHigh) Anthropic98.5% Leaders today: Claude Fable 5 (XHigh) (98.5%) · Gemini 3.8 Flash (High) (98.5%) · GPT-6 Astra (XHigh) (98.5%) · Claude Opus 5.5 (High) (98.5%) · GPT-6.1 Sol (XHigh) (98.5%)\nARC-AGI-2 published 2025 · models from 2024 to 2026 · 86 models in the archive · active · results: ARC Prize leaderboard, collected by Epoch AI\nThe successor to ARC-AGI from 2025, with puzzles chosen to resist the methods that solved the first version.\nRecords by model release date DateModelScore Jul 18, 2024GPT-4o-mini OpenAI0.0% Sep 12, 2024o1-mini OpenAI0.8% Jan 20, 2025Deepseek R1 DeepSeek1.3% Jan 31, 2025o3-mini (High) OpenAI3.0% Apr 16, 2025o3 (High) OpenAI6.5% May 22, 2025Claude Opus 4 (Thinking 16K) Anthropic8.6% Jul 9, 2025Grok 4 (Thinking) xAI16.0% Oct 7, 2025GPT-5 Pro OpenAI18.3% Nov 18, 2025Gemini 3 Pro Google DeepMind31.1% Nov 24, 2025Opus 4.5 (Thinking, 64K) Anthropic37.6% Dec 11, 2025GPT-5.2 Pro (High) OpenAI54.2% Feb 5, 2026Claude Opus 4.6 (120K, High) Anthropic69.2% Feb 12, 2026Gemini 3 Deep Think (2/26) Google DeepMind84.6% Apr 23, 2026GPT-5.5 (xHigh) OpenAI85.0% Jun 9, 2026Claude Fable 5 (Max) Anthropic89.2% Jul 9, 2026GPT-5.6 Sol (Max) OpenAI92.5% Sep 3, 2026GPT-6 Astra (Max) OpenAI95.0% Leaders today: GPT-6 Astra (Max) (95.0%) · GPT-6.1 Sol (Max) (94.2%) · Claude Opus 5.5 (High) (93.3%) · GPT-5.6 Sol (Max) (92.5%) · Claude Opus 5 (Max) (90.4%)\nLanguage understanding (classic) HellaSwag published 2019 · models from 2019 to 2024 · 97 models in the archive · no new results since 2024 · results: papers and Stanford HELM, collected by Epoch AI\nPick the most plausible continuation of an everyday scene out of four. A test of common sense from 2019 that people find trivial; guessing gives 25%.\nRecords by model release date DateModelScore Nov 5, 2019GPT2-XL-1.5B OpenAI40.0% Jun 22, 2020babbage (1.3B) OpenAI55.5% Aug 5, 2021GPT-J-6B EleutherAI, LAION66.2% Oct 11, 2021Megatron-NLG (530B) Microsoft, Nvidia82.4% Apr 4, 2022PaLM-540B Google Research83.8% Nov 28, 2022GPT-3.5 OpenAI85.5% Mar 14, 2023GPT-4 OpenAI95.3% Leaders today: GPT-4 (95.3%) · LLaMA-3.1 405B Base (89.2%) · Falcon-180B (89.0%) · DeepSeek-V3 Base (88.9%) · DeepSeek-V2 Base (87.1%)\nWinoGrande published 2019 · models from 2019 to 2024 · 84 models in the archive · no new results since 2024 · results: papers, collected by Epoch AI\nDecide what an ambiguous word in a sentence refers to — possible only with an understanding of the situation. Guessing gives 50%.\nRecords by model release date DateModelScore Nov 5, 2019GPT-2-XL 1.5B OpenAI58.3% Aug 5, 2021GPT-J-6B EleutherAI, LAION64.5% Oct 11, 2021Megatron-NLG (530B) Microsoft, Nvidia78.9% Dec 13, 2021GLaM Google79.2% Mar 15, 2022GPT-3.5 OpenAI81.6% Apr 4, 20225-shot PaLM Google Research85.1% Mar 14, 2023GPT-4 OpenAI87.5% Feb 29, 2024Claude 3 Opus Anthropic88.5% Jul 23, 2024LLaMA-3.1 405B Base Meta AI89.2% Leaders today: LLaMA-3.1 405B Base (89.2%) · Claude 3 Opus (88.5%) · GPT-4 (87.5%) · Falcon-180B (87.1%) · UNICORN (86.6%)\nSource and license Data: Epoch AI, “Capabilities \u0026amp; benchmarking”, retrieved on October 12, 2026, licensed CC BY 4.0. Results that Epoch AI collected from others keep their original source, which is named with each benchmark. ABXK computed the records and the leader groups from this data and changed no value.\nWhat the record lines show Tests are used up faster and faster. On MMLU the record stood at 86.4% when GPT-4 arrived in March 2023, and it has barely moved since — the test was full. ARC-AGI-2 was introduced in March 2025 as a puzzle set that models could not solve; the first model above ten percent followed in July 2025, and the record now stands at 95.0%.\nThe length of tasks grows, not just the score. METR\u0026rsquo;s time horizon is the one measure here that is not capped at 100. GPT-4 handled tasks of about 5 minutes in 2023; the current record is over 17 hours.\nOne number hides the setup. The same model appears with different scores depending on its reasoning effort or the agent software around it — the names in the tables carry these settings in brackets. If you compare two models for your own project, compare them in the setting you will actually use.\nIf you build with these models, the benchmarks are the start of the choice, not the end. The LLM ranking puts all models on one scale, and the LLM database lets you filter them and compare any four side by side.\n","words":3274},"date":"2026-10-12","description":"Which model held the record when? 16 LLM benchmarks from GPT-2 to today, with every record by release date and today's leaders — built from Epoch AI's open benchmark archive.","image":"https://abxk.ai/images/cover/llm-benchmarks.webp","json":"https://abxk.ai/en/llm/benchmarks/index.json","kind":"page","language":"en","llm_benchmarks":{"benchmarks":[{"active":true,"best":{"date":"2026-09-22","model":"Claude Opus 5.5","org":"Anthropic","src":"","value":167.3},"first_model":"2023-02-24","frontier":[{"date":"2023-02-24","model":"LLaMA-65B","org":"Meta AI","src":"","value":110.2},{"date":"2023-03-14","model":"GPT-4 (Mar 2023)","org":"OpenAI","src":"","value":125.9},{"date":"2024-01-25","model":"GPT-4 Turbo (Nov 2023)","org":"OpenAI","src":"","value":126.5},{"date":"2024-02-29","model":"Claude 3 Opus","org":"Anthropic","src":"","value":126.9},{"date":"2024-04-09","model":"GPT-4 Turbo (Apr 2024)","org":"OpenAI","src":"","value":127.2},{"date":"2024-05-13","model":"GPT-4o (May 2024)","org":"OpenAI","src":"","value":129},{"date":"2024-06-20","model":"Claude 3.5 Sonnet","org":"Anthropic","src":"","value":130},{"date":"2024-09-12","model":"o1-mini","org":"OpenAI","src":"","value":135.8},{"date":"2024-12-17","model":"o1","org":"OpenAI","src":"","value":141.9},{"date":"2025-03-31","model":"Gemini 2.5 Pro (Mar 2025)","org":"Google DeepMind","src":"","value":144.2},{"date":"2025-04-16","model":"o3","org":"OpenAI","src":"","value":146.9},{"date":"2025-06-10","model":"o3-pro","org":"OpenAI","src":"","value":147.4},{"date":"2025-08-07","model":"GPT-5","org":"OpenAI","src":"","value":150},{"date":"2025-10-07","model":"GPT-5 Pro","org":"OpenAI","src":"","value":150.3},{"date":"2025-11-18","model":"Gemini 3 Pro","org":"Google DeepMind","src":"","value":152.9},{"date":"2025-12-11","model":"GPT-5.2 Pro","org":"OpenAI","src":"","value":155.4},{"date":"2026-02-05","model":"GPT-5.3 Codex","org":"OpenAI","src":"","value":156.8},{"date":"2026-03-05","model":"GPT-5.4 Pro","org":"OpenAI","src":"","value":158.9},{"date":"2026-04-23","model":"GPT-5.5 Pro","org":"OpenAI","src":"","value":162.1},{"date":"2026-07-24","model":"Claude Opus 5","org":"Anthropic","src":"","value":162.8},{"date":"2026-09-01","model":"Claude Fable 5.1","org":"Anthropic","src":"","value":164.7},{"date":"2026-09-03","model":"GPT-6 Astra","org":"OpenAI","src":"","value":166.4},{"date":"2026-09-22","model":"Claude Opus 5.5","org":"Anthropic","src":"","value":167.3}],"group":"overall","introduced":"2025-10-01","key":"eci","last_model":"2026-09-29","measured":"epoch","models":274,"random":null,"results":274,"source":"Epoch AI","superseded":"","top":[{"date":"2026-09-22","model":"Claude Opus 5.5","org":"Anthropic","value":167.3},{"date":"2026-09-03","model":"GPT-6 Astra","org":"OpenAI","value":166.4},{"date":"2026-09-29","model":"GPT-6.1 Sol","org":"OpenAI","value":166.1},{"date":"2026-09-28","model":"Claude Sonnet 5.5","org":"Anthropic","value":165},{"date":"2026-09-01","model":"Claude Fable 5.1","org":"Anthropic","value":164.7},{"date":"2026-07-24","model":"Claude Opus 5","org":"Anthropic","value":162.8},{"date":"2026-09-22","model":"GPT-6 Sol","org":"OpenAI","value":162.7},{"date":"2026-04-23","model":"GPT-5.5 Pro","org":"OpenAI","value":162.1},{"date":"2026-06-09","model":"Claude Fable 5","org":"Anthropic","value":162.1},{"date":"2026-07-09","model":"GPT-5.6 Sol","org":"OpenAI","value":161.7}],"unit":"index"},{"active":true,"best":{"date":"2026-04-07","model":"Claude Mythos Preview (Early)","org":"Anthropic","src":"https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/","value":1044.78},"first_model":"2019-11-05","frontier":[{"date":"2019-11-05","model":"GPT-2 (1.5B)","org":"OpenAI","src":"https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/","value":0.05},{"date":"2023-03-14","model":"GPT-4 (Mar 2023)","org":"OpenAI","src":"METR - Measuring AI Ability to Complete Long Tasks","value":5.36},{"date":"2024-01-25","model":"GPT-4 Turbo Preview (January 2024)","org":"OpenAI","src":"METR - Measuring AI Ability to Complete Long Tasks","value":5.37},{"date":"2024-04-09","model":"GPT-4 Turbo (Apr 2024)","org":"OpenAI","src":"METR - Measuring AI Ability to Complete Long Tasks","value":6.57},{"date":"2024-05-13","model":"GPT-4o (May 2024)","org":"OpenAI","src":"METR - Measuring AI Ability to Complete Long Tasks","value":9.17},{"date":"2024-06-20","model":"Claude 3.5 Sonnet (Jun 2024)","org":"Anthropic","src":"https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/","value":11.4},{"date":"2024-09-12","model":"o1-preview","org":"OpenAI","src":"https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/","value":20.33},{"date":"2024-10-22","model":"Claude 3.5 Sonnet (Oct 2024)","org":"Anthropic","src":"https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/","value":20.52},{"date":"2024-12-17","model":"o1 (medium)","org":"OpenAI","src":"https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/","value":38.83},{"date":"2025-02-24","model":"Claude 3.7 Sonnet","org":"Anthropic","src":"","value":60.39},{"date":"2025-04-16","model":"o3 (unknown thinking)","org":"OpenAI","src":"","value":119.73},{"date":"2025-08-07","model":"GPT-5 (high)","org":"OpenAI","src":"","value":203.01},{"date":"2025-11-18","model":"Gemini 3 Pro Preview","org":"Google DeepMind","src":"","value":224.33},{"date":"2025-11-24","model":"Claude Opus 4.5 (no thinking)","org":"Anthropic","src":"","value":292.99},{"date":"2025-12-11","model":"GPT-5.2 (high)","org":"OpenAI","src":"","value":352.25},{"date":"2026-02-05","model":"Claude Opus 4.6 (unknown thinking)","org":"Anthropic","src":"","value":718.81},{"date":"2026-04-07","model":"Claude Mythos Preview (Early)","org":"Anthropic","src":"https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/","value":1044.78}],"group":"overall","introduced":"2025-03-19","key":"metr-time-horizon","last_model":"2026-04-07","measured":"external","models":39,"random":null,"results":49,"source":"METR","superseded":"","top":[{"date":"2026-04-07","model":"Claude Mythos Preview (Early)","org":"Anthropic","value":1044.78},{"date":"2026-02-05","model":"Claude Opus 4.6 (unknown thinking)","org":"Anthropic","value":718.81},{"date":"2026-02-19","model":"Gemini 3.1 Pro Preview","org":"Google DeepMind","value":384.15},{"date":"2025-12-11","model":"GPT-5.2 (high)","org":"OpenAI","value":352.25},{"date":"2026-02-05","model":"GPT-5.3 Codex","org":"OpenAI","value":349.53},{"date":"2026-03-05","model":"GPT-5.4 (xhigh)","org":"OpenAI","value":341.74},{"date":"2025-11-24","model":"Claude Opus 4.5 (no thinking)","org":"Anthropic","value":292.99},{"date":"2025-11-18","model":"Gemini 3 Pro Preview","org":"Google DeepMind","value":224.33},{"date":"2025-11-19","model":"GPT-5.1-Codex-Max","org":"OpenAI","value":223.71},{"date":"2025-08-07","model":"GPT-5 (high)","org":"OpenAI","value":203.01}],"unit":"minutes"},{"active":false,"best":{"date":"2024-11-20","model":"GPT-4o","org":"OpenAI","src":"Phi-4 Technical Report","value":88.1},"first_model":"2021-08-05","frontier":[{"date":"2021-08-05","model":"GPT-J-6B","org":"EleutherAI, LAION","src":"XGen-7B Technical Report","value":25.7},{"date":"2021-12-08","model":"Gopher","org":"DeepMind","src":"Scaling Language Models: Methods, Analysis \u0026 Insights from Training Gopher","value":60},{"date":"2022-03-15","model":"GPT-3.5","org":"OpenAI","src":"GPT-4 Technical Report","value":70},{"date":"2023-03-14","model":"GPT-4 (initial release)","org":"OpenAI","src":"GPT-4 Technical Report","value":86.4},{"date":"2024-06-20","model":"Claude 3.5 Sonnet (20240620)","org":"Anthropic","src":"Stanford CRFM Leaderboard","value":86.5},{"date":"2024-09-24","model":"Gemini 1.5 Pro (002)","org":"Google DeepMind","src":"Stanford CRFM Leaderboard","value":86.9},{"date":"2024-10-22","model":"Claude 3.5 Sonnet (20241022)","org":"Anthropic","src":"Stanford CRFM Leaderboard","value":87.3},{"date":"2024-11-20","model":"GPT-4o","org":"OpenAI","src":"Phi-4 Technical Report","value":88.1}],"group":"knowledge","introduced":"2020-09-07","key":"mmlu","last_model":"2024-12-26","measured":"external","models":158,"random":25,"results":249,"source":"papers and Stanford HELM, collected by Epoch AI","superseded":"","top":[{"date":"2024-11-20","model":"GPT-4o","org":"OpenAI","value":88.1},{"date":"2024-10-22","model":"Claude 3.5 Sonnet (20241022)","org":"Anthropic","value":87.3},{"date":"2024-12-26","model":"DeepSeek v3","org":"DeepSeek","value":87.2},{"date":"2024-09-24","model":"Gemini 1.5 Pro (002)","org":"Google DeepMind","value":86.9},{"date":"2024-06-20","model":"Claude 3.5 Sonnet (20240620)","org":"Anthropic","value":86.5},{"date":"2023-03-14","model":"GPT-4 (initial release)","org":"OpenAI","value":86.4},{"date":"2024-12-06","model":"Llama3.3 70b instruct","org":"Meta AI","value":86.3},{"date":"2024-05-14","model":"Gemini 1.5 Pro (May 2024)","org":"Google DeepMind","value":85.9},{"date":"2024-09-19","model":"Qwen2.5 72b instruct","org":"Alibaba","value":85.3},{"date":"2024-12-12","model":"Phi-4 14b","org":"Microsoft Research","value":84.8}],"unit":"percent"},{"active":true,"best":{"date":"2026-09-03","model":"GPT-6 Astra (max)","org":"OpenAI","src":"","value":95.8},"first_model":"2023-03-14","frontier":[{"date":"2023-03-14","model":"GPT-4 (Mar 2023)","org":"OpenAI","src":"","value":35.7},{"date":"2023-11-06","model":"GPT-4 Turbo Preview (Nov 2023)","org":"OpenAI","src":"","value":42.4},{"date":"2024-02-29","model":"Claude 3 Opus","org":"Anthropic","src":"","value":47.2},{"date":"2024-05-13","model":"GPT-4o (May 2024)","org":"OpenAI","src":"","value":48.9},{"date":"2024-06-20","model":"Claude 3.5 Sonnet (Jun 2024)","org":"Anthropic","src":"","value":54},{"date":"2024-09-12","model":"o1-mini (high)","org":"OpenAI","src":"","value":62.4},{"date":"2024-12-17","model":"o1 (high)","org":"OpenAI","src":"","value":76.8},{"date":"2025-01-31","model":"o3-mini (high)","org":"OpenAI","src":"","value":77},{"date":"2025-02-24","model":"Claude 3.7 Sonnet (64k thinking)","org":"Anthropic","src":"","value":79.7},{"date":"2025-03-25","model":"Gemini 2.5 Pro Exp (Mar 2025)","org":"Google DeepMind","src":"","value":83.8},{"date":"2025-06-05","model":"Gemini 2.5 Pro Preview (Jun 2025)","org":"Google DeepMind","src":"","value":84.8},{"date":"2025-07-09","model":"Grok 4","org":"xAI","src":"","value":87},{"date":"2025-11-13","model":"GPT-5.1 (high)","org":"OpenAI","src":"","value":87.6},{"date":"2025-11-18","model":"Gemini 3 Pro Preview","org":"Google DeepMind","src":"","value":92.6},{"date":"2026-02-19","model":"Gemini 3.1 Pro Preview","org":"Google DeepMind","src":"","value":94.4},{"date":"2026-03-05","model":"GPT-5.4 Pro (xhigh)","org":"OpenAI","src":"","value":94.6},{"date":"2026-08-13","model":"Gemini 3.7 Flash (high)","org":"Google DeepMind","src":"","value":94.8},{"date":"2026-09-02","model":"Gemini 3.8 Flash (high)","org":"Google DeepMind","src":"","value":95.4},{"date":"2026-09-03","model":"GPT-6 Astra (max)","org":"OpenAI","src":"","value":95.8}],"group":"knowledge","introduced":"2023-11-20","key":"gpqa-diamond","last_model":"2026-10-07","measured":"epoch","models":214,"random":25,"results":321,"source":"Epoch AI (own runs)","superseded":"","top":[{"date":"2026-09-03","model":"GPT-6 Astra (max)","org":"OpenAI","value":95.8},{"date":"2026-09-28","model":"Claude Sonnet 5.5 (max)","org":"Anthropic","value":95.6},{"date":"2026-09-02","model":"Gemini 3.8 Flash (high)","org":"Google DeepMind","value":95.4},{"date":"2026-09-29","model":"GPT-6.1 Sol (max)","org":"OpenAI","value":95.4},{"date":"2026-08-13","model":"Gemini 3.7 Flash (high)","org":"Google DeepMind","value":94.8},{"date":"2026-03-05","model":"GPT-5.4 Pro (xhigh)","org":"OpenAI","value":94.6},{"date":"2026-02-19","model":"Gemini 3.1 Pro Preview","org":"Google DeepMind","value":94.4},{"date":"2026-09-22","model":"GPT-6 Sol (max)","org":"OpenAI","value":94.3},{"date":"2026-07-21","model":"Gemini 3.6 Flash","org":"Google DeepMind","value":94.1},{"date":"2026-04-23","model":"GPT-5.5 (xhigh)","org":"OpenAI","value":94}],"unit":"percent"},{"active":true,"best":{"date":"2026-09-03","model":"GPT 6 Astra","org":"OpenAI","src":"","value":54.8},"first_model":"2024-09-24","frontier":[{"date":"2024-09-24","model":"Gemini-1.5-Pro-002","org":"Google DeepMind","src":"","value":4.6},{"date":"2024-12-17","model":"o1 (December 2024)","org":"OpenAI","src":"","value":8},{"date":"2025-03-19","model":"o1 Pro","org":"OpenAI","src":"","value":8.1},{"date":"2025-03-25","model":"Gemini 2.5 Pro Experimental (March 2025)","org":"Google DeepMind","src":"","value":18.2},{"date":"2025-04-16","model":"o3 (high) (April 2025)","org":"OpenAI","src":"","value":20.3},{"date":"2025-06-05","model":"gemini-2.5-pro-preview-06-05","org":"Google DeepMind","src":"","value":21.6},{"date":"2025-08-07","model":"gpt-5-2025-08-07","org":"OpenAI","src":"","value":25.3},{"date":"2025-10-07","model":"gpt-5-pro-2025-10-06","org":"OpenAI","src":"","value":31.6},{"date":"2025-11-18","model":"gemini-3-pro-preview","org":"Google DeepMind","src":"","value":37.5},{"date":"2026-02-19","model":"Gemini 3.1 Pro Preview (thinking high)","org":"Google DeepMind","src":"","value":46.4},{"date":"2026-09-01","model":"Fable 5.1 (xhigh)","org":"Anthropic","src":"","value":46.5},{"date":"2026-09-03","model":"GPT 6 Astra","org":"OpenAI","src":"","value":54.8}],"group":"knowledge","introduced":"2025-01-23","key":"hle","last_model":"2026-09-03","measured":"external","models":44,"random":4.8,"results":54,"source":"Humanity's Last Exam leaderboard, collected by Epoch AI","superseded":"","top":[{"date":"2026-09-03","model":"GPT 6 Astra","org":"OpenAI","value":54.8},{"date":"2026-09-01","model":"Fable 5.1 (xhigh)","org":"Anthropic","value":46.5},{"date":"2026-02-19","model":"Gemini 3.1 Pro Preview (thinking high)","org":"Google DeepMind","value":46.4},{"date":"2026-09-02","model":"Gemini 3.8 Flash","org":"Google DeepMind","value":44.5},{"date":"2026-03-05","model":"GPT-5.4 Pro","org":"OpenAI","value":44.3},{"date":"2026-04-08","model":"Muse Spark","org":"Meta AI","value":40.6},{"date":"2025-11-18","model":"gemini-3-pro-preview","org":"Google DeepMind","value":37.5},{"date":"2026-03-05","model":"gpt-5.4-2026-03-05 (xhigh thinking)","org":"OpenAI","value":36.2},{"date":"2026-04-16","model":"Claude Opus 4.7","org":"Anthropic","value":36.2},{"date":"2026-02-05","model":"claude-opus-4-6-thinking-max","org":"Anthropic","value":34.4}],"unit":"percent"},{"active":true,"best":{"date":"2026-09-03","model":"GPT-6 Astra (max)","org":"OpenAI","src":"","value":75.6},"first_model":"2024-02-29","frontier":[{"date":"2024-02-29","model":"Claude 3 Opus","org":"Anthropic","src":"","value":12.6},{"date":"2024-08-06","model":"GPT-4o (Aug 2024)","org":"OpenAI","src":"","value":26},{"date":"2024-12-17","model":"o1 (high)","org":"OpenAI","src":"","value":41.1},{"date":"2025-04-16","model":"o3 (high)","org":"OpenAI","src":"","value":49.4},{"date":"2025-08-07","model":"GPT-5 (high)","org":"OpenAI","src":"","value":50.1},{"date":"2025-12-17","model":"Gemini 3 Flash Preview","org":"Google DeepMind","src":"","value":66.8},{"date":"2026-02-19","model":"Gemini 3.1 Pro Preview","org":"Google DeepMind","src":"","value":73.5},{"date":"2026-09-03","model":"GPT-6 Astra (max)","org":"OpenAI","src":"","value":75.6}],"group":"knowledge","introduced":"2025-09-09","key":"simpleqa-verified","last_model":"2026-10-07","measured":"epoch","models":79,"random":null,"results":88,"source":"Epoch AI (own runs)","superseded":"","top":[{"date":"2026-09-03","model":"GPT-6 Astra (max)","org":"OpenAI","value":75.6},{"date":"2026-09-29","model":"GPT-6.1 Sol (max)","org":"OpenAI","value":73.9},{"date":"2026-02-19","model":"Gemini 3.1 Pro Preview","org":"Google DeepMind","value":73.5},{"date":"2026-09-22","model":"Claude Opus 5.5 (max)","org":"Anthropic","value":72.2},{"date":"2026-09-01","model":"Claude Fable 5.1 (max)","org":"Anthropic","value":70.8},{"date":"2026-06-09","model":"Claude Fable 5 (xhigh)","org":"Anthropic","value":70.7},{"date":"2026-07-09","model":"GPT-5.6 Sol (max)","org":"OpenAI","value":69.7},{"date":"2026-09-02","model":"Gemini 3.8 Flash (high)","org":"Google DeepMind","value":69.7},{"date":"2026-08-13","model":"Gemini 3.7 Flash (high)","org":"Google DeepMind","value":69.2},{"date":"2025-12-17","model":"Gemini 3 Flash Preview","org":"Google DeepMind","value":66.8}],"unit":"percent"},{"active":false,"best":{"date":"2024-06-17","model":"DS-Coder-V2-Instruct","org":"DeepSeek","src":"Qwen2.5-Coder Technical Report","value":94.5},"first_model":"2020-06-22","frontier":[{"date":"2020-06-22","model":"babbage (1.3B)","org":"OpenAI","src":"Stanford HELM","value":0.7},{"date":"2022-03-15","model":"code-davinci-002","org":"OpenAI","src":"Stanford HELM","value":56.8},{"date":"2022-11-28","model":"GPT-3.5 (8-shot)","org":"OpenAI","src":"Qwen Technical Report","value":78.2},{"date":"2023-03-14","model":"GPT-4","org":"OpenAI","src":"GPT-4 Technical Report","value":92},{"date":"2024-06-17","model":"DS-Coder-V2-Instruct","org":"DeepSeek","src":"Qwen2.5-Coder Technical Report","value":94.5}],"group":"math","introduced":"2021-10-27","key":"gsm8k","last_model":"2024-11-29","measured":"external","models":146,"random":null,"results":235,"source":"papers and Stanford HELM, collected by Epoch AI","superseded":"","top":[{"date":"2024-06-17","model":"DS-Coder-V2-Instruct","org":"DeepSeek","value":94.5},{"date":"","model":"Qwen2.5-Coder-14B-Instruct","org":"","value":94.2},{"date":"2024-11-21","model":"Qwen2.5-Coder-32B-Instruct","org":"Alibaba","value":93},{"date":"2023-03-14","model":"GPT-4","org":"OpenAI","value":92},{"date":"","model":"GPT-4","org":"","value":92},{"date":"","model":"GPT-4 (8-shot)","org":"","value":91.4},{"date":"2024-07-18","model":"GPT-4o-mini","org":"OpenAI","value":91.3},{"date":"2024-09-18","model":"Qwen2.5-Coder-32B","org":"Alibaba","value":91.1},{"date":"2023-06-13","model":"GPT-4","org":"OpenAI","value":90},{"date":"2024-08-17","model":"Phi-3.5-MoE","org":"Microsoft","value":88.7}],"unit":"percent"},{"active":true,"best":{"date":"2025-08-07","model":"GPT-5 (high)","org":"OpenAI","src":"","value":98.1},"first_model":"2023-06-13","frontier":[{"date":"2023-06-13","model":"GPT-4 (Jun 2023)","org":"OpenAI","src":"","value":23},{"date":"2023-11-06","model":"GPT-4 Turbo Preview (Nov 2023)","org":"OpenAI","src":"","value":40},{"date":"2024-04-09","model":"GPT-4 Turbo (Apr 2024)","org":"OpenAI","src":"","value":46.7},{"date":"2024-05-13","model":"GPT-4o (May 2024)","org":"OpenAI","src":"","value":51},{"date":"2024-06-20","model":"Claude 3.5 Sonnet (Jun 2024)","org":"Anthropic","src":"","value":51.7},{"date":"2024-07-18","model":"GPT-4o mini","org":"OpenAI","src":"","value":52.6},{"date":"2024-08-06","model":"GPT-4o (Aug 2024)","org":"OpenAI","src":"","value":53.3},{"date":"2024-09-12","model":"o1-mini (high)","org":"OpenAI","src":"","value":89.2},{"date":"2024-12-17","model":"o1 (high)","org":"OpenAI","src":"","value":94.7},{"date":"2025-01-31","model":"o3-mini (high)","org":"OpenAI","src":"","value":96.5},{"date":"2025-04-16","model":"o4-mini (high)","org":"OpenAI","src":"","value":97.8},{"date":"2025-08-07","model":"GPT-5 (high)","org":"OpenAI","src":"","value":98.1}],"group":"math","introduced":"2021-03-05","key":"math-level-5","last_model":"2025-10-15","measured":"epoch","models":94,"random":null,"results":108,"source":"Epoch AI (own runs)","superseded":"","top":[{"date":"2025-08-07","model":"GPT-5 (high)","org":"OpenAI","value":98.1},{"date":"2025-04-16","model":"o4-mini (high)","org":"OpenAI","value":97.8},{"date":"2025-04-16","model":"o3 (high)","org":"OpenAI","value":97.8},{"date":"2025-08-07","model":"GPT-5 mini (high)","org":"OpenAI","value":97.8},{"date":"2025-09-29","model":"Claude Sonnet 4.5 (32k thinking)","org":"Anthropic","value":97.7},{"date":"2025-09-24","model":"Qwen3-Max-Instruct","org":"Alibaba","value":97.1},{"date":"2025-05-28","model":"DeepSeek-R1 (May 2025)","org":"DeepSeek","value":96.6},{"date":"2025-01-31","model":"o3-mini (high)","org":"OpenAI","value":96.5},{"date":"2025-10-15","model":"Claude Haiku 4.5","org":"Anthropic","value":96.4},{"date":"2025-05-06","model":"Gemini 2.5 Pro Preview (Jun 2025)","org":"Google DeepMind","value":95.9}],"unit":"percent"},{"active":true,"best":{"date":"2026-09-03","model":"GPT-6 Astra (max)","org":"OpenAI","src":"","value":93.7},"first_model":"2024-01-25","frontier":[{"date":"2024-01-25","model":"GPT-3.5 Turbo (Jan 2024)","org":"OpenAI","src":"","value":0},{"date":"2024-04-09","model":"GPT-4 Turbo (Apr 2024)","org":"OpenAI","src":"","value":0.7},{"date":"2024-12-17","model":"o1 (high)","org":"OpenAI","src":"","value":14.7},{"date":"2025-01-31","model":"o3-mini (high)","org":"OpenAI","src":"","value":18.6},{"date":"2025-04-16","model":"o4-mini (high)","org":"OpenAI","src":"","value":36.1},{"date":"2025-08-07","model":"GPT-5 (high)","org":"OpenAI","src":"","value":55.4},{"date":"2025-10-07","model":"GPT-5 Pro","org":"OpenAI","src":"","value":55.8},{"date":"2025-12-11","model":"GPT-5.2 Pro","org":"OpenAI","src":"","value":74},{"date":"2026-03-05","model":"GPT-5.4 Pro (xhigh)","org":"OpenAI","src":"","value":82.5},{"date":"2026-04-23","model":"GPT-5.5 Pro (xhigh)","org":"OpenAI","src":"","value":87.7},{"date":"2026-07-09","model":"GPT-5.6 Sol (max)","org":"OpenAI","src":"","value":89.1},{"date":"2026-09-01","model":"Claude Fable 5.1 (max)","org":"Anthropic","src":"","value":90.2},{"date":"2026-09-03","model":"GPT-6 Astra (max)","org":"OpenAI","src":"","value":93.7}],"group":"math","introduced":"2026-06-12","key":"frontiermath","last_model":"2026-10-07","measured":"epoch","models":83,"random":null,"results":116,"source":"Epoch AI (own runs)","superseded":"","top":[{"date":"2026-09-03","model":"GPT-6 Astra (max)","org":"OpenAI","value":93.7},{"date":"2026-09-29","model":"GPT-6.1 Sol (max)","org":"OpenAI","value":93.7},{"date":"2026-09-22","model":"Claude Opus 5.5 (max)","org":"Anthropic","value":91.2},{"date":"2026-09-01","model":"Claude Fable 5.1 (max)","org":"Anthropic","value":90.2},{"date":"2026-09-22","model":"GPT-6 Sol (max)","org":"OpenAI","value":89.8},{"date":"2026-07-09","model":"GPT-5.6 Sol (max)","org":"OpenAI","value":89.1},{"date":"2026-09-28","model":"Claude Sonnet 5.5 (max)","org":"Anthropic","value":88.8},{"date":"2026-04-23","model":"GPT-5.5 Pro (xhigh)","org":"OpenAI","value":87.7},{"date":"2026-06-09","model":"Claude Fable 5 (max)","org":"Anthropic","value":87},{"date":"2026-07-09","model":"GPT-5.6 Terra (max)","org":"OpenAI","value":86}],"unit":"percent"},{"active":true,"best":{"date":"2026-04-16","model":"Claude Opus 4.7 (max)","org":"Anthropic","src":"","value":83.5},"first_model":"2024-11-20","frontier":[{"date":"2024-11-20","model":"GPT-4o (Nov 2024)","org":"OpenAI","src":"","value":31},{"date":"2025-02-24","model":"Claude 3.7 Sonnet","org":"Anthropic","src":"","value":61},{"date":"2025-04-16","model":"o3 (medium)","org":"OpenAI","src":"","value":62.3},{"date":"2025-05-22","model":"Claude Opus 4","org":"Anthropic","src":"","value":70.7},{"date":"2025-08-05","model":"Claude Opus 4.1","org":"Anthropic","src":"","value":73.3},{"date":"2025-08-07","model":"GPT-5 (high)","org":"OpenAI","src":"","value":73.6},{"date":"2025-11-24","model":"Claude Opus 4.5 (no thinking)","org":"Anthropic","src":"","value":76.7},{"date":"2026-02-05","model":"Claude Opus 4.6 (no thinking)","org":"Anthropic","src":"","value":78.7},{"date":"2026-04-16","model":"Claude Opus 4.7 (max)","org":"Anthropic","src":"","value":83.5}],"group":"coding","introduced":"2024-08-13","key":"swe-bench-verified","last_model":"2026-06-16","measured":"epoch","models":32,"random":null,"results":35,"source":"Epoch AI (own runs)","superseded":"","top":[{"date":"2026-04-16","model":"Claude Opus 4.7 (max)","org":"Anthropic","value":83.5},{"date":"2026-04-23","model":"GPT-5.5 (xhigh)","org":"OpenAI","value":80.6},{"date":"2026-05-19","model":"Gemini 3.5 Flash","org":"Google DeepMind","value":79.3},{"date":"2026-02-05","model":"Claude Opus 4.6 (no thinking)","org":"Anthropic","value":78.7},{"date":"2026-06-16","model":"GLM-5.2","org":"Z.ai (Zhipu AI)","value":78.7},{"date":"2026-04-24","model":"DeepSeek v4 Pro (max)","org":"DeepSeek","value":77.6},{"date":"2026-05-19","model":"Qwen3.7 Max","org":"Alibaba","value":77.3},{"date":"2026-03-05","model":"GPT-5.4 (high)","org":"OpenAI","value":76.9},{"date":"2025-11-24","model":"Claude Opus 4.5 (no thinking)","org":"Anthropic","value":76.7},{"date":"2026-04-20","model":"Qwen 3.6 Max (Preview)","org":"Alibaba","value":76.7}],"unit":"percent"},{"active":true,"best":{"date":"2025-08-07","model":"GPT-5 (high)","org":"OpenAI","src":"https://aider.chat/docs/leaderboards/","value":88},"first_model":"2024-07-18","frontier":[{"date":"2024-07-18","model":"GPT-4o mini","org":"OpenAI","src":"https://aider.chat/docs/leaderboards/","value":3.6},{"date":"2024-08-06","model":"GPT-4o (Aug 2024)","org":"OpenAI","src":"https://aider.chat/docs/leaderboards/","value":23.1},{"date":"2024-09-12","model":"o1-mini (unknown thinking)","org":"OpenAI","src":"https://aider.chat/docs/leaderboards/","value":32.9},{"date":"2024-10-22","model":"Claude 3.5 Sonnet (Oct 2024)","org":"Anthropic","src":"https://aider.chat/docs/leaderboards/","value":51.6},{"date":"2024-12-17","model":"o1 (high)","org":"OpenAI","src":"https://aider.chat/docs/leaderboards/","value":61.7},{"date":"2025-02-24","model":"Claude 3.7 Sonnet (32k thinking)","org":"Anthropic","src":"https://aider.chat/docs/leaderboards/","value":64.9},{"date":"2025-03-25","model":"Gemini 2.5 Pro Exp (Mar 2025)","org":"Google DeepMind","src":"Aider LLM Leaderboards","value":72.9},{"date":"2025-04-16","model":"o3 (high)","org":"OpenAI","src":"https://aider.chat/docs/leaderboards/","value":81.3},{"date":"2025-06-05","model":"Gemini 2.5 Pro Preview (Jun 2025)","org":"Google DeepMind","src":"https://aider.chat/docs/leaderboards/","value":83.1},{"date":"2025-06-10","model":"o3-pro","org":"OpenAI","src":"https://aider.chat/docs/leaderboards/","value":84.9},{"date":"2025-08-07","model":"GPT-5 (high)","org":"OpenAI","src":"https://aider.chat/docs/leaderboards/","value":88}],"group":"coding","introduced":"2024-12-21","key":"aider-polyglot","last_model":"2025-12-01","measured":"external","models":54,"random":null,"results":72,"source":"Aider leaderboard, collected by Epoch AI","superseded":"","top":[{"date":"2025-08-07","model":"GPT-5 (high)","org":"OpenAI","value":88},{"date":"2025-06-10","model":"o3-pro","org":"OpenAI","value":84.9},{"date":"2025-06-05","model":"Gemini 2.5 Pro Preview (Jun 2025)","org":"Google DeepMind","value":83.1},{"date":"2025-04-16","model":"o3 (high)","org":"OpenAI","value":81.3},{"date":"2025-07-09","model":"Grok 4","org":"xAI","value":79.6},{"date":"2025-05-06","model":"Gemini 2.5 Pro Preview (Jun 2025)","org":"Google DeepMind","value":76.9},{"date":"2025-09-29","model":"DeepSeek-V3.2-Exp","org":"DeepSeek","value":74.2},{"date":"2025-12-01","model":"DeepSeek-V3.2 (Thinking)","org":"DeepSeek","value":74.2},{"date":"2025-03-25","model":"Gemini 2.5 Pro Exp (Mar 2025)","org":"Google DeepMind","value":72.9},{"date":"2025-04-16","model":"o4-mini (high)","org":"OpenAI","value":72}],"unit":"percent"},{"active":true,"best":{"date":"2026-04-23","model":"GPT-5.5","org":"OpenAI","src":"https://www.tbench.ai/leaderboard/terminal-bench/2.0","value":84.7},"first_model":"2025-06-17","frontier":[{"date":"2025-06-17","model":"Gemini 2.5 Flash","org":"Google DeepMind","src":"https://www.tbench.ai/leaderboard/terminal-bench/2.0","value":17.1},{"date":"2025-07-09","model":"Grok 4","org":"xAI","src":"Terminal-Bench v2 Leaderboard","value":27.2},{"date":"2025-07-12","model":"Kimi K2 Instruct","org":"Moonshot","src":"Terminal-Bench v2 Leaderboard","value":27.8},{"date":"2025-08-05","model":"Claude Opus 4.1","org":"Anthropic","src":"Terminal-Bench v2 Leaderboard","value":38},{"date":"2025-08-07","model":"GPT-5","org":"OpenAI","src":"https://www.tbench.ai/leaderboard/terminal-bench/2.0","value":49.6},{"date":"2025-11-12","model":"GPT-5.1-Codex-Mini","org":"OpenAI","src":"https://www.tbench.ai/leaderboard/terminal-bench/2.0","value":61.6},{"date":"2025-11-18","model":"Gemini 3 Pro","org":"Google DeepMind","src":"https://www.tbench.ai/leaderboard/terminal-bench/2.0","value":69.4},{"date":"2026-02-05","model":"Claude Opus 4.6","org":"Anthropic","src":"https://www.tbench.ai/leaderboard/terminal-bench/2.0","value":79.8},{"date":"2026-02-19","model":"Gemini 3.1 Pro","org":"Google DeepMind","src":"https://www.tbench.ai/leaderboard/terminal-bench/2.0","value":80.2},{"date":"2026-03-05","model":"GPT-5.4","org":"OpenAI","src":"https://www.tbench.ai/leaderboard/terminal-bench/2.0","value":81.8},{"date":"2026-04-23","model":"GPT-5.5","org":"OpenAI","src":"https://www.tbench.ai/leaderboard/terminal-bench/2.0","value":84.7}],"group":"coding","introduced":"2025-05-19","key":"terminal-bench","last_model":"2026-04-23","measured":"external","models":45,"random":null,"results":202,"source":"Terminal-Bench leaderboard, collected by Epoch AI","superseded":"","top":[{"date":"2026-04-23","model":"GPT-5.5","org":"OpenAI","value":84.7},{"date":"2026-03-05","model":"GPT-5.4","org":"OpenAI","value":81.8},{"date":"2026-02-19","model":"Gemini 3.1 Pro","org":"Google DeepMind","value":80.2},{"date":"2026-04-16","model":"Claude Opus 4.7","org":"Anthropic","value":80.2},{"date":"2026-02-05","model":"Claude Opus 4.6","org":"Anthropic","value":79.8},{"date":"2026-02-05","model":"GPT-5.3-Codex","org":"OpenAI","value":78.4},{"date":"2025-11-18","model":"Gemini 3 Pro","org":"Google DeepMind","value":69.4},{"date":"2025-12-18","model":"GPT-5.2-Codex","org":"OpenAI","value":66.5},{"date":"2025-12-11","model":"GPT-5.2","org":"OpenAI","value":64.9},{"date":"2025-12-17","model":"Gemini 3 Flash","org":"Google DeepMind","value":64.3}],"unit":"percent"},{"active":true,"best":{"date":"2026-06-09","model":"Claude Fable 5 (XHigh)","org":"Anthropic","src":"https://arcprize.org/leaderboard","value":98.5},"first_model":"2024-09-12","frontier":[{"date":"2024-09-12","model":"o1-preview","org":"OpenAI","src":"ARC Prize Leaderboard","value":18},{"date":"2024-12-17","model":"o1 (Medium)","org":"OpenAI","src":"ARC Prize Leaderboard","value":30.7},{"date":"2025-01-31","model":"o3-mini (High)","org":"OpenAI","src":"ARC Prize Leaderboard","value":34.5},{"date":"2025-04-16","model":"o3 (High)","org":"OpenAI","src":"https://arcprize.org/leaderboard","value":60.8},{"date":"2025-07-09","model":"Grok 4 (Thinking)","org":"xAI","src":"","value":66.7},{"date":"2025-10-07","model":"GPT-5 Pro","org":"OpenAI","src":"ARC Prize Leaderboard","value":70.2},{"date":"2025-11-13","model":"GPT-5.1 (Thinking, High)","org":"OpenAI","src":"https://arcprize.org/leaderboard","value":72.8},{"date":"2025-11-18","model":"Gemini 3 Pro","org":"Google DeepMind","src":"ARC Prize Leaderboard","value":75},{"date":"2025-11-24","model":"Claude Opus 4.5 (64k thinking)","org":"Anthropic","src":"ARC Prize Leaderboard","value":80},{"date":"2025-12-11","model":"GPT-5.2 Pro (X-High)","org":"OpenAI","src":"","value":90.5},{"date":"2026-02-05","model":"Claude Opus 4.6 (120K, High)","org":"Anthropic","src":"https://arcprize.org/leaderboard","value":94},{"date":"2026-02-12","model":"Gemini 3 Deep Think (2/26)","org":"Google DeepMind","src":"https://arcprize.org/leaderboard","value":96},{"date":"2026-02-19","model":"Gemini 3.1 Pro (Preview)","org":"Google DeepMind","src":"ARC Prize Leaderboard","value":98},{"date":"2026-06-09","model":"Claude Fable 5 (XHigh)","org":"Anthropic","src":"https://arcprize.org/leaderboard","value":98.5}],"group":"abstraction","introduced":"2019-11-05","key":"arc-agi","last_model":"2026-09-29","measured":"external","models":91,"random":null,"results":279,"source":"ARC Prize leaderboard, collected by Epoch AI","superseded":"","top":[{"date":"2026-06-09","model":"Claude Fable 5 (XHigh)","org":"Anthropic","value":98.5},{"date":"2026-09-02","model":"Gemini 3.8 Flash (High)","org":"Google DeepMind","value":98.5},{"date":"2026-09-03","model":"GPT-6 Astra (XHigh)","org":"OpenAI","value":98.5},{"date":"2026-09-22","model":"Claude Opus 5.5 (High)","org":"Anthropic","value":98.5},{"date":"2026-09-29","model":"GPT-6.1 Sol (XHigh)","org":"OpenAI","value":98.5},{"date":"2026-02-19","model":"Gemini 3.1 Pro (Preview)","org":"Google DeepMind","value":98},{"date":"2026-07-09","model":"GPT-5.6 Sol (xHigh)","org":"OpenAI","value":97.5},{"date":"2026-07-24","model":"Claude Opus 5 (High)","org":"Anthropic","value":97.5},{"date":"2026-09-01","model":"Claude Fable 5.1 (Max)","org":"Anthropic","value":97.5},{"date":"2026-04-23","model":"GPT-5.5 Pro (High)","org":"OpenAI","value":96.5}],"unit":"percent"},{"active":true,"best":{"date":"2026-09-03","model":"GPT-6 Astra (Max)","org":"OpenAI","src":"","value":95},"first_model":"2024-07-18","frontier":[{"date":"2024-07-18","model":"GPT-4o-mini","org":"OpenAI","src":"","value":0},{"date":"2024-09-12","model":"o1-mini","org":"OpenAI","src":"","value":0.8},{"date":"2025-01-20","model":"Deepseek R1","org":"DeepSeek","src":"","value":1.3},{"date":"2025-01-31","model":"o3-mini (High)","org":"OpenAI","src":"","value":3},{"date":"2025-04-16","model":"o3 (High)","org":"OpenAI","src":"","value":6.5},{"date":"2025-05-22","model":"Claude Opus 4 (Thinking 16K)","org":"Anthropic","src":"","value":8.6},{"date":"2025-07-09","model":"Grok 4 (Thinking)","org":"xAI","src":"","value":16},{"date":"2025-10-07","model":"GPT-5 Pro","org":"OpenAI","src":"","value":18.3},{"date":"2025-11-18","model":"Gemini 3 Pro","org":"Google DeepMind","src":"","value":31.1},{"date":"2025-11-24","model":"Opus 4.5 (Thinking, 64K)","org":"Anthropic","src":"","value":37.6},{"date":"2025-12-11","model":"GPT-5.2 Pro (High)","org":"OpenAI","src":"","value":54.2},{"date":"2026-02-05","model":"Claude Opus 4.6 (120K, High)","org":"Anthropic","src":"","value":69.2},{"date":"2026-02-12","model":"Gemini 3 Deep Think (2/26)","org":"Google DeepMind","src":"","value":84.6},{"date":"2026-04-23","model":"GPT-5.5 (xHigh)","org":"OpenAI","src":"","value":85},{"date":"2026-06-09","model":"Claude Fable 5 (Max)","org":"Anthropic","src":"","value":89.2},{"date":"2026-07-09","model":"GPT-5.6 Sol (Max)","org":"OpenAI","src":"","value":92.5},{"date":"2026-09-03","model":"GPT-6 Astra (Max)","org":"OpenAI","src":"","value":95}],"group":"abstraction","introduced":"2025-03-24","key":"arc-agi-2","last_model":"2026-09-29","measured":"external","models":86,"random":null,"results":259,"source":"ARC Prize leaderboard, collected by Epoch AI","superseded":"","top":[{"date":"2026-09-03","model":"GPT-6 Astra (Max)","org":"OpenAI","value":95},{"date":"2026-09-29","model":"GPT-6.1 Sol (Max)","org":"OpenAI","value":94.2},{"date":"2026-09-22","model":"Claude Opus 5.5 (High)","org":"Anthropic","value":93.3},{"date":"2026-07-09","model":"GPT-5.6 Sol (Max)","org":"OpenAI","value":92.5},{"date":"2026-07-24","model":"Claude Opus 5 (Max)","org":"Anthropic","value":90.4},{"date":"2026-09-01","model":"Claude Fable 5.1 (Max)","org":"Anthropic","value":90},{"date":"2026-09-22","model":"GPT-6 Sol (Max)","org":"OpenAI","value":89.6},{"date":"2026-06-09","model":"Claude Fable 5 (Max)","org":"Anthropic","value":89.2},{"date":"2026-09-02","model":"Gemini 3.8 Flash (High)","org":"Google DeepMind","value":89.2},{"date":"2026-04-23","model":"GPT-5.5 (xHigh)","org":"OpenAI","value":85}],"unit":"percent"},{"active":false,"best":{"date":"2023-03-14","model":"GPT-4","org":"OpenAI","src":"The Falcon Series of Open Language Models","value":95.3},"first_model":"2019-11-05","frontier":[{"date":"2019-11-05","model":"GPT2-XL-1.5B","org":"OpenAI","src":"Textbooks Are All You Need II: phi-1.5 technical report","value":40},{"date":"2020-06-22","model":"babbage (1.3B)","org":"OpenAI","src":"Stanford HELM","value":55.5},{"date":"2021-08-05","model":"GPT-J-6B","org":"EleutherAI, LAION","src":"XGen-7B Technical Report","value":66.2},{"date":"2021-10-11","model":"Megatron-NLG (530B)","org":"Microsoft, Nvidia","src":"GLaM: Efficient Scaling of Language Models with Mixture-of-Experts","value":82.4},{"date":"2022-04-04","model":"PaLM-540B","org":"Google Research","src":"PaLM: Scaling Language Modeling with Pathways","value":83.8},{"date":"2022-11-28","model":"GPT-3.5","org":"OpenAI","src":"The Falcon Series of Open Language Models","value":85.5},{"date":"2023-03-14","model":"GPT-4","org":"OpenAI","src":"The Falcon Series of Open Language Models","value":95.3}],"group":"language","introduced":"2019-05-19","key":"hellaswag","last_model":"2024-12-26","measured":"external","models":97,"random":25,"results":135,"source":"papers and Stanford HELM, collected by Epoch AI","superseded":"","top":[{"date":"2023-03-14","model":"GPT-4","org":"OpenAI","value":95.3},{"date":"2024-07-23","model":"LLaMA-3.1 405B Base","org":"Meta AI","value":89.2},{"date":"2023-09-06","model":"Falcon-180B","org":"Technology Innovation Institute","value":89},{"date":"2024-12-26","model":"DeepSeek-V3 Base","org":"DeepSeek","value":88.9},{"date":"2024-05-07","model":"DeepSeek-V2 Base","org":"DeepSeek","value":87.1},{"date":"","model":"PaLM 2-L","org":"","value":86.8},{"date":"2023-12-11","model":"Mixtral 8x7B","org":"Mistral AI","value":86.7},{"date":"","model":"UNICORN","org":"","value":86.6},{"date":"2022-11-28","model":"GPT-3.5","org":"OpenAI","value":85.5},{"date":"2023-03-15","model":"Falcon 40B","org":"Technology Innovation Institute","value":85.3}],"unit":"percent"},{"active":false,"best":{"date":"2024-07-23","model":"LLaMA-3.1 405B Base","org":"Meta AI","src":"DeepSeek-V3 Technical Report","value":89.2},"first_model":"2019-11-05","frontier":[{"date":"2019-11-05","model":"GPT-2-XL 1.5B","org":"OpenAI","src":"","value":58.3},{"date":"2021-08-05","model":"GPT-J-6B","org":"EleutherAI, LAION","src":"XGen-7B Technical Report","value":64.5},{"date":"2021-10-11","model":"Megatron-NLG (530B)","org":"Microsoft, Nvidia","src":"GLaM: Efficient Scaling of Language Models with Mixture-of-Experts","value":78.9},{"date":"2021-12-13","model":"GLaM","org":"Google","src":"GLaM: Efficient Scaling of Language Models with Mixture-of-Experts","value":79.2},{"date":"2022-03-15","model":"GPT-3.5","org":"OpenAI","src":"GPT-4 technical report","value":81.6},{"date":"2022-04-04","model":"5-shot PaLM","org":"Google Research","src":"GPT-4 Technical report","value":85.1},{"date":"2023-03-14","model":"GPT-4","org":"OpenAI","src":"GPT-4 technical report","value":87.5},{"date":"2024-02-29","model":"Claude 3 Opus","org":"Anthropic","src":"The Claude 3 Model Family: Opus, Sonnet, Haiku","value":88.5},{"date":"2024-07-23","model":"LLaMA-3.1 405B Base","org":"Meta AI","src":"DeepSeek-V3 Technical Report","value":89.2}],"group":"language","introduced":"2019-07-24","key":"winogrande","last_model":"2024-12-26","measured":"external","models":84,"random":50,"results":145,"source":"papers, collected by Epoch AI","superseded":"","top":[{"date":"2024-07-23","model":"LLaMA-3.1 405B Base","org":"Meta AI","value":89.2},{"date":"2024-02-29","model":"Claude 3 Opus","org":"Anthropic","value":88.5},{"date":"2023-03-14","model":"GPT-4","org":"OpenAI","value":87.5},{"date":"2023-09-06","model":"Falcon-180B","org":"Technology Innovation Institute","value":87.1},{"date":"","model":"UNICORN","org":"","value":86.6},{"date":"2024-05-07","model":"DeepSeek-V2 Base","org":"DeepSeek","value":86.3},{"date":"2024-12-26","model":"DeepSeek-V3 Base","org":"DeepSeek","value":85.2},{"date":"2022-04-04","model":"5-shot PaLM","org":"Google Research","value":85.1},{"date":"2024-06-17","model":"DS-Coder-V2-Base","org":"DeepSeek","value":83.7},{"date":"2024-04-18","model":"LLaMA 3 70B","org":"Meta AI","value":83.5}],"unit":"percent"}],"license":"CC BY 4.0","retrieved":"2026-10-12","source":"Epoch AI, 'Capabilities \u0026 benchmarking', epoch.ai/benchmarks"},"modified":"2026-10-12","section":"llm","tags":["llm-benchmarks","leaderboard","model-comparison","ai-progress"],"title":"LLM Benchmarks 2019–2026: The Records on 16 Leaderboards","translations":[{"json":"https://abxk.ai/de/llm/benchmarks/index.json","language":"de","url":"https://abxk.ai/de/llm/benchmarks/"}],"url":"https://abxk.ai/en/llm/benchmarks/"}