Contents7
In short
Independent evaluations of machine translation crown a different system from one year to the next, and the ranking changes from one language pair to another.
Translation services and general-purpose AI models are converging: DeepL and Google now offer their own LLM-type models.
For a specialised text, the answer comes from a trial on your texts, judged by a person who reads the language.
“DeepL or ChatGPT?” “Is Google Translate enough?” The question comes up in every translation project, and every provider has its answer, drawn from its own tests. This article does not name a winner: it says what independent evaluations show, what really sets the tools apart, and how to decide between them on your texts. It extends our articles on website translation and opening new markets.
What independent evaluations show
The reference here is WMT, the annual conference on machine translation, which every year has professional annotators judge the translations of dozens of systems: those of research teams, general-purpose AI models and online translation services.
- 2024
According to the WMT24 findings, the best system overall is Claude 3.5 Sonnet, winning 9 of the 11 language pairs evaluated.
- 2025
According to the WMT25 findings, it is Gemini 2.5 Pro, in the top cluster for 14 of the 16 pairs evaluated by humans.
Three limits prevent drawing a choice for your company from this.
The winner changes. Two editions, two different providers on top. Models evolve every few months: a ranking holds for the versions tested, on the date of the test.
Online services are anonymised. Translations from public machine translation services appear in the evaluation under code names (ONLINE-B, ONLINE-G, ONLINE-W): WMT does not say which one is DeepL or Google Translate.
The texts are not yours. The domains tested are news, social media, speech, literary and educational texts, chosen so that they can be judged without specialist expertise. A contract, a technical sheet or an offer page are not among them. The same findings also note persistent weaknesses with a field’s terminology.
The ranking also changes from one language to another. On English to Egyptian Arabic, for example, the order is not that of the overall ranking:
WMT25, English to Egyptian Arabic: human score out of 100
The system at the top of the overall ranking comes only fourth on this pair, and the human reference translation stays ahead of every system. That is the strongest argument for testing language by language.
What providers say
Every provider publishes its own comparisons, and each gives itself the advantage. In July 2024, DeepL announced a new language model built for translation, citing blind tests with language professionals in which its translations were preferred to those of Google Translate and ChatGPT-4. These are the provider’s own tests, on the competing versions of the time.
These announcements say one useful thing: the line between families of tools is fading. DeepL relies on an LLM-type model, Google offers a “Translation LLM” in Cloud Translation alongside its classic translation model, and general-purpose models translate. Opposing “machine translator” and “generative AI” makes less and less sense.
What really sets the tools apart
Rather than an overall quality score, compare what changes your day-to-day work.
Translation services: DeepL, Google Cloud Translation
- A text goes in, its translation comes out. Few instructions to write: the tool translates, with a few settings depending on the plan, such as the level of formality or style rules at DeepL.
- The glossary is built in: DeepL and Google accept a glossary loaded once and applied to every translation. We detail its use in brand glossary: keeping translations consistent.
- Billing per character or by subscription, easy to forecast.
General-purpose AI models: Claude, Gemini, GPT, Mistral, DeepSeek and others
- They follow instructions: tone, formal or informal address, maximum length, output format, glossary supplied with the text.
- They translate a whole document with its context, which helps with references from one paragraph to another.
- They need more checking: a generative model can add, omit or rephrase. Numbers, required terms and length are verified afterwards.
- Billing per token, which depends on the model chosen and the length of the instructions.
Translation service
Built-in glossary, no instructions to write
Output close to the source structure
More limited tone settings
General-purpose model
Instructions on tone, format and length
Context of the whole document
Checks needed on what it adds or omits
Data usage terms
One criterion weighs as much as quality: what happens to the text you send. The terms change with the plan, not only with the provider. DeepL states “no data training” on its paid plans. For the Gemini API, Google’s price list states that content on the paid tier is not used to improve its products, and that content on the unpaid tier is. Read the terms of the plan you actually use, not those of the provider in general.
How to test on your texts
The only ranking that counts is the one on your texts. The trial fits in six steps:
- Gather twenty to thirty real extracts.
An offer page, terms of sale, a product page, a customer service reply, a technical text: what you will really translate, difficult cases included.
- Set the same rules for all.
The same glossary and, for general-purpose models, the same instructions.
- Compare three to five tools
Including at least one translation service and one small model, often much cheaper.
- Have them judged blind
By a person who reads the language, without knowing which tool produced which translation. Record errors by severity: wrong meaning, wrong term, tone, form.
- Calculate the cost of a usable translation
Correction included, not just the price per character.
- Start again at every change
Of a tool’s version, or when opening a new language.
The general method is detailed in evaluating an AI model before putting it into production. A second AI model can help the judge sort a large volume, but the decision rests with someone who reads the language: the WMT organisers themselves conclude that human evaluation should remain the final arbiter of translation quality.
From the field
Our routing table does not use the same tool for German as for Chinese, and Thai is settled on the test bench, on the client’s own texts. On Palmora Property, a real-estate listings site in six languages that we operate, the translation tool was chosen with this method, and changing provider cut the cost of the pipeline by a factor of 60. That is a cost result for that site, not a quality ranking that holds elsewhere.
What the tool does not solve
The best tool replaces neither the review of the pages that commit the company, nor the glossary, nor the translation memory, nor the localization of prices and formats, which we detail in localize, don’t just translate. On a catalogue, it does not remove the need for the checks on figures and units described in translating product pages for export. Nor does it replace the translator: it shifts their work towards review, terminology and the texts that need a real rewrite. We detail that split in translating your website with AI, automatic or reviewed, and the full budget of a language in website translation cost: per word, per page, per market.
Where to start
Choose the language of your first market and ten extracts from your most important texts, have them translated by three tools, and give the lot, without the names, to someone who reads the language. You will quickly know which one needs the least correction.
For a full test bench on your content, language by language, describe your texts and your languages: we start from your documents, never from a public ranking.
Frequently asked questions
Is DeepL better than Google Translate?
It depends on the language and the text. Independent evaluations anonymise online services, and the comparisons published by providers favour whoever publishes them. Test both, with your glossary, on twenty or so of your own extracts, judged by someone who reads the language.
Can ChatGPT replace a translator?
No. An AI model produces a fast first draft, sometimes excellent, but it can get a term, a figure or a tone wrong without flagging it. A translator or reviewer remains necessary for the pages that sell or commit the company, and to judge quality: the reference evaluations themselves rely on professional annotators.
Which tool should you choose for Chinese, Russian or Thai?
The one that wins your trial in that language. The ranking changes from one language pair to another, and a tool that is excellent into German can be average into Thai. That is why a routing table can use a different tool per language.
Are consumer versions suitable for professional use?
Check the terms before pasting a contract or customer data into them. Data usage terms differ by plan: what is true of a paid plan or an API is not necessarily true of the same provider’s consumer version. For sensitive texts, use a plan whose terms you have read.
Sources
- Findings of the WMT24 General Machine Translation Shared Task (new tab)aclanthology.org
- Findings of the WMT25 General Machine Translation Shared Task (new tab)aclanthology.org
- DeepL, announcement of its next-gen language model (new tab)deepl.com
- DeepL, Translator plans (new tab)deepl.com
- Google, Gemini API pricing (new tab)ai.google.dev
Findings of the WMT24 General Machine Translation Shared Task
Read on aclanthology.org(new tab)Findings of the WMT25 General Machine Translation Shared Task
Read on aclanthology.org(new tab)Jim · founder of Brain Plus AI
Also in translation and new markets
Choose and price7 min
Translating your website with AI: automatic or reviewed?
Having everything reviewed is expensive, and reviewing nothing lets errors through in languages that nobody at your company reads. The split is decided page by page, depending on what each page commits you to.
Choose and price8 min
Website translation cost: per word, per page, per market
A translation quote is read per word or per page, an API is billed per character, and neither tells you what an open language really costs. Here are the three ways of counting, with public prices, and what to add to know whether a market is worth its translation.
Practical guide8 min
Brand glossary: keeping translations consistent
The same product called three different things on three pages, and readers wonder whether there are three products. A brand glossary fixes the terms that are not up for debate, before the first translation, whatever the tool or the translator.