AI comparison: models, releases and scores since 2022

On 14 March 2023, three publishers announced a model on the same day: OpenAI presented GPT-4, Anthropic opened Claude and Zhipu published the weights of ChatGLM-6B. ChatGPT, released in November 2022, had started the race a few months earlier. This comparison puts the major releases of ten publishers in one place: what each one changed, and the score the publisher published that day.

Last updated: 18 October 2026. The log at the end of the article says what changed at each pass.

How to read the announced scores

Publishers do not always cite the same benchmarks, because a test that every model passes no longer says anything. Here are the tests that come up in the tables below.

Three precautions before comparing two figures

Each publisher measures in-house, in its own way. In December 2023, Google presented Gemini Ultra as the first model to beat human experts on MMLU, with 90.04%. The table in the Gemini technical report shows how: this figure comes from a method that generates 32 lines of reasoning before choosing an answer (CoT@32), while GPT-4’s 86.4% comes from a prompt with five examples (5-shot). With five examples, Gemini Ultra scored 83.7%. Number of attempts, tools allowed, thinking time granted: two scores can only be compared under equal conditions.

A public test ends up leaking. The AIME 2024 problems have been public since the competition, and the MathArena team (ETH Zurich and INSAIT) reports strong signs of contamination on them.

MathArena’s counter is to evaluate models on a competition’s problems as soon as they are published.

An excellent score does not tell the whole story. MathArena and Stanford’s AI Index 2026 report each show it with a contrast:

That is why a published score serves to spot candidates, and why you then test a model on your own tasks before putting it into production.

The timeline at a glance

The releases that mattered at the ten publishers, in the order they arrived. The details of each, and the score announced at release, are in the next part, publisher by publisher.

The star ★ marks around ten releases that made their mark: those that opened a path other publishers followed, served as a reference for their competitors or took the lead on release, according to the pages cited in this comparison.

2022 and 2023

2024

2025

2026

Publisher by publisher

For each publisher, a paragraph on how it works (open weights or not, licences, what it emphasises), then its major milestones. The right-hand column gives the score the publisher published at release, with a link to the page that publishes it; when a score was only published later, the date is given.

OpenAI: GPT and o

OpenAI started the race with ChatGPT, in November 2022. From September 2024 to summer 2025, the company ran two ranges side by side: the GPT models, and a series of reasoning models, o1 then o3 and o4-mini, trained with reinforcement learning for complex reasoning tasks. GPT-5 brought them together in August 2025 in a single system: a fast model for most questions, a model that thinks for the hardest ones, and a router that chooses in real time. The same week, OpenAI published open weights, gpt-oss, under Apache 2.0. GPT-5.6 comes in three tiers: Sol, the flagship model; Terra, cheaper; Luna, the fastest and most economical; GPT-6 keeps Sol and Luna, and adds Astra above them.

On safety, GPT-5.4 Thinking is OpenAI’s first general-purpose model treated as having “High” capability in cybersecurity, and GPT-6 Astra the first to reach the “Critical” level of its preparedness framework. From GPT-4 Turbo to o3, the scores come from OpenAI’s public evaluation repository, simple-evals, which has not tracked new models since July 2025; since then, the system cards publish few scores in text, apart from HealthBench. Between two milestones, intermediate versions (GPT-4o mini, o3-mini, GPT-5.1 to 5.5, GPT-6 Sol and Luna, GPT-6.1 Sol); the table keeps only the milestones.

Anthropic: Claude

Anthropic does not publish the weights of its models: Claude is used through its app, through the API and through the major cloud platforms. The company has bet on code and agents: computer use from October 2024, then a model presented as the first hybrid reasoning model on the market, delivered with Claude Code, in February 2025; SWE-bench Verified comes back in almost every announcement. Since 2026, safety has divided the range: above Opus, the Mythos class was first reserved for cybersecurity partners, then opened to everyone under the name Fable, with safeguards that route some questions to a less powerful model. Between two milestones, intermediate versions come out every two or three months (Opus 4.1, Haiku 4.5, Sonnet 4.6, Opus 4.7 and 4.8, Sonnet 5, Sonnet 5.5, Haiku 5.5); the table keeps only the milestones.

Google: Gemini and Gemma

Google runs two families side by side. Gemini is closed, usable through the app and the API; Gemma publishes its weights, drawn from the same research as Gemini, under the Apache 2.0 licence since Gemma 4. The company bet early on multimodality (Gemini 1.0 is trained from the start on several types of data) and on very long contexts (one million tokens with Gemini 1.5). Its models first come out in an experimental version or in preview: Gemini 3.1 Pro still was in October 2026. In 2026, a new Gemini Flash came out almost every month (3.6 in July, 3.7 in August, 3.8 in September); the table keeps only the families.

Meta: Llama and Muse

Meta published the weights of the whole Llama line, under in-house licences that changed with each version: research only for the first LLaMA, commercial use from Llama 2, the right to train other models with Llama’s outputs from Llama 3.1. With Llama 3.2, the licence for the models that read images excluded individuals and companies established in the European Union. In 2026, the new models come from Meta Superintelligence Labs, under the name Muse: Muse Spark comes out closed, served in the Meta AI assistant and then through an API, while Muse Glimmer publishes its weights under Apache 2.0. Muse Spark has had three updates (1.1 in July, 1.2 in August with the coding agent Muse Code, 1.3 in September), whose scores are only published as images or in PDF. In October came Muse Wonder, the series Meta presents as its most capable, served through its API and in Muse Code, then Muse Spark Open, which publishes the weights of a version of Muse Spark under an in-house licence with limited commercial terms.

xAI: Grok

xAI launched Grok in November 2023 with an argument of its own: real-time knowledge of the world through X. The company published the weights of Grok-1 under Apache 2.0 in March 2024, then those of Grok 2 in August 2025, under an in-house licence that allows commercial use in line with its usage policy, but forbids using it, or its outputs, to train other general-purpose models. Since Grok 3, the reasoning models have been trained with reinforcement learning, at pre-training scale with Grok 4; in 2026, the publisher, which now signs as SpaceXAI, is focusing on code and agents, with Grok 4.5 trained alongside Cursor and its own coding agent, Grok Build, in beta since May. Between two milestones, intermediate versions (Grok 4.1 Fast, Grok 4.20 and its multi-agent variant in March 2026, Grok Build 0.1 in May); the table keeps only the milestones.

Mistral AI

France’s Mistral AI alternates between three regimes. Open weights under Apache 2.0, usable without restriction (Mistral 7B, Mixtral, Mistral 3, Small 4); weights published under a licence that limits use (research only for Large 2, non-production for Codestral, modified MIT for Medium 3.5); and models reserved for its API (the 2024 Large, Medium 3). It emphasises the ratio between performance and cost, and raised €3 billion in September 2026 to bring a “sovereign”, open-weight AI up to the level of the best. The first milestone funded by this round, Mistral Large 4, was trained in its own data centres in Europe; in preview since 6 October 2026, its weights are announced for the end of the month.

DeepSeek

DeepSeek has published the weights of its general-purpose models since the first, DeepSeek LLM 67B: first under an in-house licence that allows commercial use, then under the MIT licence from R1 onwards (“Distill & commercialize freely!”), and up to V4. The company bets on efficiency: a mixture of experts from V2, which saves 42.5% of the training cost compared with the previous model; V3 trained in 2.788 million H800 GPU hours; sparse attention with V3.2-Exp, together with a cut of more than half in its API prices; then one million tokens of context by default with V4. Between two milestones, dated updates (V3-0324, R1-0528, V3.1-Terminus); the table keeps only the milestones.

Alibaba: Qwen

Alibaba publishes most of its Qwen models with open weights: first under its own licence, then mostly under Apache 2.0 from Qwen2 onwards. Its “Max” models long remained reserved for the API, until Qwen3.8, whose flagship model published its weights under an in-house licence that allows commercial use under conditions. The company emphasises multilingualism (201 languages and dialects with Qwen3.5, against 119 for Qwen3), native multimodality since Qwen3.5, and agents in 2026. Between two milestones, other releases slot in (Qwen1.5, Qwen3-Coder, Qwen3.6, Qwen3.7-Plus); the table keeps only the milestones. At the end of September 2026, Qwen 4 was being trained.

Zhipu: GLM

Zhipu AI, which publishes as Z.ai internationally, has opened its weights since its first chat model: ChatGLM-6B was released as open source on the very day ChatGLM launched, 14 March 2023. First under in-house licences that allowed commercial use after registration, then under the MIT licence, from the GLM-Z1 reasoning model in April 2025 up to GLM-5.2. GLM-5.3 moves to a licence of its own, which requires a security review from the largest operators of hosted models, above $10 billion in revenue over twelve months. The company bets on agents and code: since GLM-5.1, its model cards no longer publish SWE-bench Verified but SWE-bench Pro, Terminal-Bench and agent tests. Between two milestones come vision models (GLM-4.5V, GLM-4.6V), Flash and Turbo variants, and GLM-5.3-Flash, the first natively multimodal model of the GLM-5 series; the table keeps only the milestones.

Moonshot AI: Kimi

Since Kimi K2, in July 2025, Moonshot AI has published the weights of its flagship models, after smaller open models (Moonlight, Kimi-VL). The licence is a modified MIT, with a single condition: display “Kimi K2” in the interface of a product that exceeds 100 million monthly active users or $20 million in monthly revenue. Kimi K3 moves to a licence of its own, which also requires a separate agreement from operators of hosted models above $20 million in revenue over twelve months. The company bets on agents: K2 Thinking chains 200 to 300 tool calls in a row, K2.5 directs up to 100 sub-agents in parallel; context goes from 128,000 to 256,000 tokens with the September 2025 update, then to one million with K3. Between two milestones, dated updates (K2-Instruct-0905) and specialised models (Kimi-Dev, Kimi-Linear); the table keeps only the milestones.

What four years of releases have changed

Prices have collapsed. Stanford’s AI Index 2025 report measured the fall at equal performance. A task that was not cost-effective two years ago can be today, provided you choose the right model for each task.

Open models have caught up. The same report measures the gap between open-weight and closed models. A model you can run on your own server is no longer a second-rate choice.

China is at the same level. According to the AI Index 2026, American and Chinese models have traded the lead several times since early 2025: DeepSeek-R1 briefly matched the best American model in February 2025, and in March 2026 Anthropic’s best model led by only 2.7%.

Models have learned to think before answering. In September 2024, OpenAI presented o1-preview, trained with reinforcement learning for complex reasoning tasks. The others followed within a few months: QwQ-32B-Preview at Alibaba at the end of November, with open weights, then on the same day, 20 January 2025, DeepSeek-R1, with open weights under the MIT licence, and Kimi k1.5 at Moonshot AI, both announced at the level of o1. The idea then merged into general-purpose models: Claude 3.7 Sonnet, Qwen3 and DeepSeek-V3.1 answer straight away or think for longer, as you choose. Today, the thinking effort, and so the bill, is often set task by task.

Code and agents made the biggest leap. In 2023, the best model tested fixed 1.96% of SWE-bench bugs. The AI Index 2026 records what followed on two tests: the Verified version of SWE-bench, and OSWorld, which has agents carry out real tasks on a computer, under several operating systems.

Agents still fail roughly one time in three on these structured tests.

Industry has taken over. Again according to the AI Index 2026, more than 90% of the notable frontier models of 2025 came from industry, not from public research.

Which model should you choose, then?

No table names the best model for you, because it does not exist in the abstract. An expensive frontier model is justified for writing a sales page or running a complex agent; for sorting thousands of messages, a small model, far cheaper, is enough, and it answers faster. Choosing the model is in fact only the third of the five questions in our guide to integrating AI in your business: the target task and where the data goes are decided first.

To get a first idea, Arena (formerly Chatbot Arena, created by researchers at the University of California, Berkeley) ranks models based on public votes: two anonymous models answer the same question, the person chooses the better answer, and the names are only revealed after the vote. It is a measure of preference, useful, but it knows neither your documents nor your constraints.

Three questions do the rest:

1. The task. One model per use, tried on about twenty of your real examples: that is the method in our article on evaluating a model before production.
2. Volume and price. What a task repeated ten thousand times a month costs, and the discounts available: see how an AI bill is calculated.
3. Where your data goes. A publisher’s API, a European host or an open model on your server: the four levels of private AI help you decide.

If you have to make this choice, or bring down an AI bill that keeps climbing, that is what our AI at the right cost offer is for: a test bench on your tasks, then a dated routing table that you keep.

How we keep this comparison up to date

Monitoring is automatic, publishing goes through a person:

This tracking feeds our own way of working. The table that assigns a model to each task in our automations is dated and revalidated every quarter: we rerun the test bench on the current tasks, note the suppliers’ prices and new releases, then update the table.

Update log

• 18 October 2026: on 17 October, DeepSeek published a dated update of DeepSeek-V4.1-Flash, with its weights; it is in line with the model. No new milestone at OpenAI, Anthropic, Google, Mistral AI, xAI, Alibaba and Moonshot AI; Meta and Zhipu were not reread that day.
• 16 October 2026: added Muse Wonder, announced on 12 October, and Muse Spark Open, published on 15 October, at Meta; Gemini 4 Argon has been open in the Gemini API since 13 October. No new milestone at OpenAI, Anthropic, Zhipu, DeepSeek, Alibaba and Moonshot AI; Mistral AI and xAI were not reread that day.
• 8 October 2026: Claude Haiku 5.5, released on 7 October, joins Anthropic’s intermediate versions; no new milestone at the eight other publishers reread that day.
• 7 October 2026: added Mistral Large 4, in public preview since 6 October.
• 5 October 2026: first version. Ten publishers, releases from November 2022 to the end of September 2026.

Frequently asked questions

Sources

1. GPT-4 (new tab)community.openai.com
2. Claude (new tab)anthropic.com
3. ChatGLM-6B (new tab)arxiv.org
4. In November 2022 (new tab)edunewsletter.openai.com
5. MMLU (new tab)arxiv.org
6. GPQA Diamond (new tab)arxiv.org
7. SWE-bench Verified (new tab)swebench.com
8. The first version (new tab)arxiv.org
9. Humanity’s Last Exam (new tab)arxiv.org
10. ARC-AGI-2 (new tab)arcprize.org
11. Gemini technical report (new tab)arxiv.org
12. MathArena (new tab)arxiv.org
13. AI Index 2026 report (new tab)hai.stanford.edu
14. MathArena (new tab)arxiv.org
15. Trained with reinforcement learning (new tab)developers.openai.com
16. A single system (new tab)deploymentsafety.openai.com
17. Treated as having “High” capability in cybersecurity (new tab)deploymentsafety.openai.com
18. The first to reach the “Critical” level (new tab)deploymentsafety.openai.com
19. Simple-evals (new tab)github.com
20. MMLU: 86.4%; HumanEval: 67.0%; simulated bar exam: around the top 10% of candidates (new tab)arxiv.org
21. MMLU: 86.5%; GPQA: 49.1%; HumanEval: 87.6% (new tab)github.com
22. MMLU: 88.7%; GPQA: 53.6%; HumanEval: 90.2% (new tab)github.com
23. MMLU: 90.8%; GPQA: 73.3%; HumanEval: 92.4% (new tab)github.com
24. MMLU: 90.8%; GPQA: 69.5%; SimpleQA: 62.5% (new tab)github.com
25. MMLU: 90.2%; GPQA: 66.3%; HumanEval: 94.5% (new tab)github.com
26. MMLU: 92.9%; GPQA: 82.8%; MATH-500: 97.8% for o3; GPQA: 77.6% for o4-mini (new tab)github.com
27. AIME 2025: 92.5%; GPQA Diamond: 80.1%; SWE-bench Verified: 62.4% (new tab)arxiv.org
28. HealthBench Professional, adjusted for answer length: 60.5 for Sol, 8.7 points more than GPT-5.5 (new tab)deploymentsafety.openai.com
29. HumanEval: 71.2%, against 56.0%; GSM8k: 88.0% (new tab)anthropic.com
30. The Claude 2.1 announcement (new tab)anthropic.com
31. The Claude 3 announcement (new tab)anthropic.com
32. Internal agentic coding evaluation: 64% of problems solved, against 38% for Claude 3 Opus (new tab)anthropic.com
33. SWE-bench Verified: 49.0%, against 33.4%; OSWorld, screenshot only: 14.9% (new tab)anthropic.com
34. SWE-bench Verified: 63.7%, and 70.3% with a custom scaffold (new tab)anthropic.com
35. SWE-bench: 72.5% for Opus 4, 72.7% for Sonnet 4; Terminal-bench: 43.2% for Opus 4 (new tab)anthropic.com
36. SWE-bench Verified: 77.2%, averaged over 10 attempts; OSWorld: 61.4% (new tab)anthropic.com
37. At the medium setting, Sonnet 4.5’s best score on SWE-bench Verified with 76% fewer tokens (new tab)anthropic.com
38. Search in a very long text (MRCR v2, one million tokens): 76%, against 18.5% for Sonnet 4.5 (new tab)anthropic.com
39. SWE-bench Verified: 93.9%; GPQA Diamond: 94.6% (new tab)anthropic.com
40. The announcement (new tab)anthropic.com
41. HLE: 57.8% without tools, 63.8% with (new tab)anthropic.com
42. On ARC-AGI 3, a score three times higher than the next model (new tab)anthropic.com
43. HLE with tools: 67.7%; Terminal-Bench 4.0: 66.4% (new tab)anthropic.com
44. The Bard announcement (new tab)blog.google
45. The PaLM 2 announcement (new tab)blog.google
46. MMLU: 90.0% (new tab)blog.google
47. Finds a passage hidden in a very long text 99% of the time (new tab)blog.google
48. MMLU: 64.3% for Gemma 7B (new tab)huggingface.co
49. SWE-bench Verified: 51.8% (new tab)developers.googleblog.com
50. The release notes (new tab)ai.google.dev
51. HLE: 18.8% without tools, SWE-bench Verified: 63.8% with a custom agent (new tab)blog.google
52. HLE: 37.5% without tools, SWE-bench Verified: 76.2% (new tab)blog.google
53. ARC-AGI-2: 77.1% (new tab)blog.google
54. 3rd open model in the Arena text ranking (new tab)blog.google
55. GPQA Diamond: 84.3% for the 31B (new tab)huggingface.co
56. Terminal-Bench 2.1: 76.2% (new tab)blog.google
57. DeepSWE v1.1: 77.9%; AutomationBench: 51.3%, in first place (new tab)blog.google
58. LLaMA-13B beats GPT-3 on most benchmarks, LLaMA-65B rivals Chinchilla-70B and PaLM-540B (new tab)arxiv.org
59. MMLU: 68.9% for the 70B (new tab)huggingface.co
60. MMLU: 82.0% for the 70B Instruct (new tab)huggingface.co
61. MMLU: 87.3%; HumanEval: 89.0% for the 405B Instruct (new tab)huggingface.co
62. MMMU: 60.3% for the 90B Vision (new tab)huggingface.co
63. GPQA Diamond: 50.5%, against 49.0% for the 405B (new tab)huggingface.co
64. GPQA Diamond: 69.8%; MMLU Pro: 80.5% for Maverick (new tab)huggingface.co
65. HLE: 58% in Contemplating mode (new tab)ai.meta.com
66. SWE-bench Verified: 76.0%; AIME 2026: 94.7% (new tab)huggingface.co
67. The announcement (new tab)research.meta.ai
68. The announcement (new tab)research.meta.ai
69. Real-time knowledge of the world through X (new tab)x.ai
70. An in-house licence (new tab)huggingface.co
71. The weights announcement (new tab)x.ai
72. MMLU: 81.3%; MATH: 50.6%; HumanEval: 74.1% (new tab)x.ai
73. GPQA: 56.0%; MMLU: 87.5%; HumanEval: 88.4% (new tab)x.ai
74. AIME 2025: 93.3% at the highest compute setting; GPQA: 84.6%; LiveCodeBench: 79.4% (new tab)x.ai
75. ARC-AGI-2: 15.9%, a record for a closed model; HLE, text-only questions: 50.7% for Grok 4 Heavy (new tab)x.ai
76. GPQA Diamond: 85.7%; AIME 2025 without tools: 92.0%; HLE without tools: 20.0% (new tab)x.ai
77. LMArena ranking (text): 1st with 1,483 Elo points for Grok 4.1 Thinking; preferred 64.78% of the time over the previous model (new tab)x.ai
78. DeepSWE 1.0: 62.0%; Terminal-Bench 2.1: 83.3%; SWE-bench Pro: 64.7% (new tab)x.ai
79. Artificial Analysis Intelligence Index: 61, at the level of GPT-5.6 Sol; CursorBench 3.2: 69.9%; DeepSWE 1.1: 65.9% (new tab)x.ai
80. CursorBench 4.0: 46.3%; DeepSWE 1.1: 71.0% at high effort; Terminal-Bench 4.0: 37.6% (new tab)x.ai
81. Raised €3 billion in September 2026 (new tab)mistral.ai
82. Mistral Large 4 (new tab)mistral.ai
83. Better than Llama 2 13B on all benchmarks tested (new tab)mistral.ai
84. MT-Bench: 8.30 (new tab)mistral.ai
85. Second model available through an API, after GPT-4 (new tab)mistral.ai
86. The Codestral announcement (new tab)mistral.ai
87. MMLU: 84.0% (new tab)mistral.ai
88. MMMU: 52.5% (new tab)mistral.ai
89. MMLU: over 81% (new tab)mistral.ai
90. At least 90% of the level of Claude Sonnet 3.7 (new tab)mistral.ai
91. AIME 2024: 73.6% for Medium (new tab)mistral.ai
92. 2nd among open non-reasoning models on LMArena (new tab)mistral.ai
93. AA LCR: 0.72 (new tab)mistral.ai
94. SWE-bench Verified: 77.6% (new tab)mistral.ai
95. “Distill & commercialize freely!” (new tab)api-docs.deepseek.com
96. Saves 42.5% of the training cost (new tab)arxiv.org
97. 2.788 million H800 GPU hours (new tab)huggingface.co
98. One million tokens of context by default (new tab)api-docs.deepseek.com
99. MMLU: 71.3% for the base version; HumanEval: 73.78% for the Chat version (new tab)github.com
100. MMLU: 78.5% for the base version; HumanEval: 81.1% for the Chat version (new tab)huggingface.co
101. The announcement (new tab)api-docs.deepseek.com
102. GPQA Diamond: 71.5%; AIME 2024: 79.8%; SWE-bench Verified: 49.2% (new tab)huggingface.co
103. GPQA Diamond: 80.1% with thinking; SWE-bench Verified: 66.0% in agent mode, without thinking (new tab)huggingface.co
104. GPQA Diamond: 79.9%; HLE: 19.8%; SWE-bench Verified: 67.8% (new tab)huggingface.co
105. The model card (new tab)huggingface.co
106. GPQA Diamond: 90.1%; HLE: 37.7%; SWE-bench Verified: 80.6% (new tab)huggingface.co
107. Dated update on 17 October (new tab)api-docs.deepseek.com
108. GPQA Diamond: 90.9%; Terminal-Bench 2.1: 90.6% (new tab)huggingface.co
109. 201 languages and dialects with Qwen3.5 (new tab)alibabagroup.com
110. Qwen 4 was being trained (new tab)alibabacloud.com
111. The Tongyi Qianwen announcement (new tab)alibabagroup.com
112. MMLU: 56.7% (new tab)huggingface.co
113. MMLU: 77.4%; GSM8K: 78.9% (new tab)github.com
114. MMLU: 84.2% for the 72B; HumanEval: 86.0% for the 72B Instruct (new tab)qwenlm.github.io
115. MMLU: 86.1% for the 72B; GPQA: 49.0% for the 72B Instruct (new tab)qwenlm.github.io
116. GPQA: 65.2%; AIME: 50.0% (new tab)qwenlm.github.io
117. The announcement (new tab)qwenlm.github.io
118. The Qwen3 announcement (new tab)qwenlm.github.io
119. SWE-bench Verified: 69.6% (new tab)alibabacloud.com
120. GPQA: 88.4%; SWE-bench Verified: 76.4% (new tab)huggingface.co
121. SWE-bench Verified: 80.4%; GPQA Diamond: 92.4%; HLE: 41.4% (new tab)alibabacloud.com
122. The announcement (new tab)alibabacloud.com
123. GPQA Diamond: 92.6%; HLE: 43.6% (new tab)huggingface.co
124. A security review (new tab)huggingface.co
125. The announcement (new tab)github.com
126. MMLU: 40.63% (new tab)github.com
127. MMLU: 61.4%; GSM8K: 72.3% (new tab)github.com
128. The model card (new tab)huggingface.co
129. 3rd of all the models compared, closed and open, with an average of 63.2 over 12 benchmarks (new tab)huggingface.co
130. SWE-bench Verified: 64.2% (new tab)arxiv.org
131. The model card (new tab)huggingface.co
132. SWE-bench Verified: 68.0% (new tab)huggingface.co
133. SWE-bench Verified: 77.8%; HLE: 30.5% without tools, 50.4% with (new tab)huggingface.co
134. SWE-bench Pro: 58.4%; HLE: 31.0% without tools, 52.3% with (new tab)huggingface.co
135. SWE-bench Pro: 62.1%; GPQA Diamond: 91.2% (new tab)huggingface.co
136. Terminal-Bench 2.1: 88.2%; HLE with tools: 62.5% (new tab)huggingface.co
137. Display “Kimi K2” in the interface (new tab)huggingface.co
138. A separate agreement (new tab)huggingface.co
139. AIME: 77.5; MATH 500: 96.2; MathVista: 74.9 (new tab)arxiv.org
140. SWE-bench Verified: 65.8%; GPQA Diamond: 75.1%; MMLU: 89.5% (new tab)kimi.ai
141. HLE with tools: 44.9%; SWE-bench Verified: 71.3% (new tab)kimi.ai
142. HLE: 30.1% without tools, 50.2% with; SWE-bench Verified: 76.8% (new tab)huggingface.co
143. SWE-bench Verified: 80.2%; GPQA Diamond: 90.5% (new tab)huggingface.co
144. Terminal-Bench 2.1: 88.3%; GPQA Diamond: 93.5% (new tab)huggingface.co
145. HLE: 43.5% without tools, 56.0% with (new tab)github.com
146. AI Index 2025 report (new tab)hai.stanford.edu
147. Arena (new tab)arena.ai