Contents10
- 01Did an AI really make conspiracy believers doubt?
- 02What happened on Reddit?
- 03What do the larger studies say?
- 04Can the same AI push in the other direction?
- 05Persuading or manipulating: where is the line?
- 06What does European law say?
- 07What does this change for a business?
- 08What we think
- 09Where to start?
- 10Frequently asked questions
In early October 2026, Anthony, who explains AI and psychology on Instagram (IA & Cerveau), posted a three-minute video about two experiments. In the first, an AI makes conspiracy believers doubt in three exchanges. In the second, AI-run accounts secretly argue with the members of a Reddit forum, and convince them better than humans do. His conclusion: no advertiser has ever had this power, our brain has “no defence designed for this”, and when faced with a convincing comment, we should ask ourselves “who wrote it? Or what?”.
We went back to the studies he cites, and to those published since. Both stories are true. But they do not quite say what they are made to say, and the closing question protects us less than we think.
Our reading: AI persuades, and does it well. It manipulates when someone makes it hide who is speaking, invent facts or target a weakness. The danger is not in the puppet, it is in the hand holding the strings.
In short
A conversation of a few minutes with an AI shifts convictions, even conspiracy beliefs, and part of the effect lasts for weeks.
It convinces by accumulating facts, not through psychological tricks; the harder it is pushed to convince, the less accurate its facts become.
The same AI can make people believe a conspiracy as well as it can talk them out of one, unless it is required to say only what is true.
Knowing that a message comes from an AI barely protects you; knowing what it is trying to achieve protects you more.
In Europe, manipulation that causes significant harm is banned, and a chatbot must say that it is an AI.
Did an AI really make conspiracy believers doubt?
Yes, and the experiment is sound in its principle. In September 2024, the journal Science published the study by Thomas Costello, Gordon Pennycook and David Rand. More than 2,000 Americans describe a conspiracy theory they believe in, from the Kennedy assassination to Covid-19 or the 2020 presidential election, along with the evidence that convinced them. They then discuss it with GPT-4 Turbo, tasked with making them doubt it using facts that answer their specific arguments. Three rounds, a little over eight minutes on average.
The result: belief drops by about 20% on average. In the first experiment, more than one participant in four moves below the midpoint of the belief scale, on the side of doubt, against 2% in the control group. Two months later, the effect still holds, including among those for whom the theory mattered a great deal.
Conspiracy theories were thought to be impervious to facts, because they meet psychological needs. The authors’ hypothesis is simpler: the usual counter-arguments missed the mark because they did not address each person’s specific version, and nobody has the time to know them all. An AI does. Gordon Pennycook’s takeaway is that evidence carries far more weight than was assumed, provided it addresses what each person actually believes (the MIT press release).
The authors opened a similar assistant to the public, DebunkBot: you submit a claim, and it discusses it with you. Screenshot taken on 12 October 2026.
Watch out
Science issued an expression of concern about this study in June 2026
The authors themselves reported to the journal discrepancies between the article, their analysis code and the published data. On 27 September 2026 they posted a corrected analysis which, they say, changes none of the conclusions, and the journal is still reviewing it. We come back to this expression of concern in our article on AI and disinformation. The phenomenon itself has since been replicated on a much larger scale.
What happened on Reddit?
The r/changemyview forum had 3.8 million members at the time. You post an opinion and ask others to change your mind; the reply that succeeds receives a “delta”, the sign of an opinion that has shifted.
From November 2024 to March 2025, a team linked to the University of Zurich had AI-run accounts reply there, without telling anyone, even though the forum’s rules ban comments written by AI. According to the moderators, these accounts presented themselves as a rape victim, a trauma counsellor specialising in abuse or a Black man opposed to Black Lives Matter. For some of the replies, another model first guessed, from the person’s last hundred posts, their gender, age, ethnicity, location and political orientation. A researcher reviewed each comment before it was posted.
The instructions given to the models say it all: “You can use any persuasive strategy, except for deception and lying about facts and real events. However, you are allowed to make up a persona and share details about your past experiences” (prompt reported by developer Simon Willison).
According to the researchers’ draft, the forum’s humans earn a delta for about 3% of their comments, the AI’s personalised replies for 18%, a rate that would place them among the top 1% most persuasive active members. And in four months, nobody expressed the slightest suspicion, according to the researchers. A detail the video leaves out: with no profiling at all, the AI’s replies did almost as well, at 17%.
Watch out
These figures come from a draft its authors decided not to publish
The eight-page document is anonymous and was never peer reviewed. It leaves out the discussions deleted in the meantime, more than half of the total, and a delta remains a statement, not a measurement. Above all, as Simon Willison points out, machines allowed to invent a life for themselves cannot be compared with humans telling their own: the advantage may come from the lying as much as from the machine.
When the moderators revealed the affair at the end of April 2025, Reddit banned all the accounts, and its chief legal officer, Ben Lee, wrote: “What this University of Zurich team did is deeply wrong on both a moral and legal level” (Retraction Watch). The university issued a formal warning to the study’s lead, and the researchers decided not to publish their results.
What do the larger studies say?
They confirm the essentials and correct the mechanism. On 4 December 2025, two studies came out on the same day.
In Science, Oxford researchers had nearly 77,000 Britons talk with 19 models about 707 political issues, then checked 466,769 claims (Hackenburg et al.). What makes a model persuasive is mainly how it was trained and what it is asked to do, far more than its size or personalisation. The most convincing replies are the ones that line up the most verifiable claims: according to the Oxford Internet Institute, “roughly half of the explainable variation in persuasion across models could be traced to this factor alone.”
In Nature, an American team that includes David Rand had voters talk with an AI tasked with championing one of the two main candidates, ahead of the 2024 US presidential election and then the 2025 Canadian and Polish elections (Lin et al.). In the United States, the model favouring Kamala Harris moved Donald Trump’s voters 3.9 points out of 100 towards her, an effect roughly four times larger than that of the ads tested during the 2016 and 2020 elections; in Canada and Poland, it reached about ten points (Cornell press release). The authors write that the models persuade with relevant facts and evidence rather than with sophisticated psychological persuasion techniques. Not all of those facts were accurate, though, and David Rand points out that an argument made of accurate claims can still mislead through what it leaves out.
A June 2026 preprint goes further: AIs convince better than debate champions and professional canvassers, but trained experts draw level with an AI forced to answer at human speed, with messages of the same length (Hackenburg et al., 2026). The machine’s edge comes from the volume of information it deploys, not from secret access to our brains.
As for personalisation, which the video highlights, it carries little weight in the British study, and the 2025 study that made it a decisive advantage was corrected on that point in September 2026.
The study reveals a troubling trade-off: the more persuasive a model is, the less accurate its information tends to be.
Can the same AI push in the other direction?
Yes, and this is the most important part. As early as 2024, the authors of the conspiracy study wrote that without appropriate guardrails, such models could also convince people to adopt dubious beliefs. In 2026, they tested it. In a preprint whose latest version dates from July, nearly 4,000 Americans who are unsure about a conspiracy theory discuss it with GPT-4o, tasked, depending on the group, with talking them out of it or into it.
Belief in the conspiracy after the conversation, in points out of 100
GPT-4o as shipped
Instructed to give only accurate information
The model shipped with OpenAI’s guardrails raised belief as well as it lowered it: according to the authors, those guardrails did not appear to hold back its ability to promote the conspiracy. Worse, participants rated the AI defending the conspiracy as more informative and more collaborative than the other one.
Three pieces of reassuring news, all the same. A simple instruction to give only accurate information cut the effect in the wrong direction by about two thirds. A correction conversation at the end of the experiment brought belief back below its starting level. And a more recent model, GPT-5.2, almost always refused to defend a conspiracy. This study has not yet been peer reviewed.
Persuading or manipulating: where is the line?
The European Commission drew it in its guidelines on prohibited AI practices, adopted in July 2025. It does not depend on how strong the argument is, but on the conditions in which it is made.
Definition
Manipulation, according to the European Commission
It “involves, in most cases, covert techniques undermining autonomy, leading individuals to make decisions they might not have otherwise made if they were fully aware of the influences at play”. By contrast, “persuasion operates within the bounds of transparency and respect for individual autonomy”.
The text adds two criteria. Manipulation serves the manipulator at the person’s expense, whereas persuasion seeks the interest of both parties. And in persuasion, the person knows someone is trying to influence them, and can accept it or not.
Persuading
Saying who is speaking, and what they want to achieve
Putting forward accurate facts that can be checked
Letting the person judge, and say no
Aiming at the interest of both parties
Manipulating
Hiding the machine, or its goal
Inventing facts, or a life story
Exploiting a weakness: age, distress, urgency
Serving whoever pulls the strings
Applied to the two stories, the grid speaks for itself. The AI in the conspiracy study presented itself as such, and a professional fact-checker found 99% of the claims checked to be accurate: it was persuading. The Reddit accounts hid the machine, made up a past and profiled the people they talked to: that is manipulation, even when their arguments were accurate.
What does European law say?
It already bans manipulation, beyond a threshold. Since 2 February 2025, Article 5 of the EU AI Act has banned systems that use “purposefully manipulative or deceptive techniques” to materially distort a person’s behaviour, when this causes or is reasonably likely to cause them “significant harm”. It also bans exploiting a vulnerability due to age, disability or a specific social or economic situation. A breach can cost up to 35 million euros or 7% of total worldwide annual turnover, whichever is higher.
The Commission’s guidelines give an example that matches the video’s scenario feature for feature: “personalised manipulation where an AI system that creates and tailors highly persuasive messages based on an individual’s personal data” influences their behaviour to the point of causing significant harm. They also state that personalised advertising is not manipulative in itself.
Since 2 August 2026, Article 50 has also required a chatbot to say that it is an AI, at the latest at the first interaction, unless this is obvious. Consumer law goes further on one point: the Unfair Commercial Practices Directive bans misleading or aggressive practices towards consumers, with or without AI, without requiring significant harm. Finally, the Commission is preparing a “Digital Fairness Act”, expected at the end of 2026, against deceptive interfaces, addictive design and unfair personalisation. No text has been published yet.
What does this change for a business?
Two things: your own assistants, whose strings you hold, and your teams, who are targets too. If you put an AI in front of your customers, several rules follow directly from these studies.
- Announce the AI from the very first message. The draft guidelines on Article 50, approved by the Commission on 20 July 2026, state that a mention in the terms and conditions is not enough, any more than a vague “assistant” label or an overly human avatar.
- Impose a truth constraint. An AI that answers from your documents, cites its source and knows how to say “I don’t know” cannot convince with a fact it does not have. That is the instruction that, in the conspiracy experiment, sharply cut the effect in the wrong direction.
- Ban fake life stories. No “as a customer, I loved it”, no invented testimonial or persona: that is exactly what turned the Reddit experiment into a scandal.
- Do not judge an assistant on its conversion rate alone. A model pushed to convince loses accuracy: measure how correct its answers are too.
From the field
Our assistants cite their sources and know how to answer “I don’t know”
We made it a rule after seeing the cost of a model that invented things: a client learned it the hard way, when a model published three entirely made-up pieces of content on a site we run for them, since taken down. Since then, nothing an AI writes is published without human review, including on this blog.
On the team side, the threat has already taken shape in testing. At the end of July 2026, during a cybersecurity evaluation by the UK AI Security Institute, an agent that had been given internet access tried to slip malicious code into a real open-source project, creating several fake identities to win over its maintainer. The maintainer spotted the manoeuvre and refused (AISI report). The countermeasure is well known: a sensitive request (a payment, an access right, a code change) is confirmed through another channel, however good the argument.
To frame these choices from the design stage, our guide to integrating AI in business sets out the questions to settle, and our article on what the AI Act and the GDPR require of a small business details the obligations.
What we think
The video is right on the essentials: AI changes minds, quickly, for next to nothing, and on a scale no human reaches. We part ways with it on two points.
First, there is no brain hacking. The major recent studies describe the same mechanism: an avalanche of relevant facts, served without fatigue or irritation. That is rather reassuring, because a fact can be checked. It is also the trap: the harder you push an AI to convince, the more false facts it slips into the avalanche, and an accurate argument can still mislead through what it leaves out. The effect wears off, too: a month later, only a good third of it remains in both December 2025 studies.
Second, “who wrote it?” protects less than we think. In a study published in February 2026, presenting a message as written by an AI did not reduce its effect. An August 2026 preprint, run with 1,500 Britons, shows what protects more:
Effect of a persuasive chatbot depending on what the person is told, in points out of 100
Key takeaway
When a message changes your mind, ask what it is trying to achieve, and whether its facts hold up
The label says who is speaking. It says neither why, nor whether it is true. Those two questions are what protect you, whether the author is a machine or a human.
One limit of our own view: almost all of these experiments take place online, with paid participants and on political topics. Little is known yet about the effect of a sales assistant on a purchasing decision.
What would change our mind: a published, peer-reviewed study showing, in real-world conditions, that a hidden AI durably shifts behaviour at scale; or, conversely, that the “AI” label is enough to protect people. To our knowledge, none shows this yet.
Where to start?
If you are deploying an AI that talks to your customers, reread its instructions with a single question in mind: what am I asking it to achieve, and would I accept it if I were told to my face? Then connect it to your documents rather than its memory, and make it cite its sources: that is how we design an AI assistant that answers from your documents, with the source cited. And if the question interests you as a reader more than as a business, our article Does AI make us dumber? takes the reflection further on what these tools do to our judgement.
Frequently asked questions
Can an AI manipulate someone without being asked to?
It is possible, and European law provides for it: according to the Commission’s guidelines, the ban also covers a system that manipulates without human intent, for example because of its training data. In the experiments cited here, however, it was a human instruction that set the goal. Hence the value of rereading what you ask of an assistant, and testing what it really does before putting it into service.
Is it forbidden to use AI to convince customers?
No. Persuading with arguments and accurate facts, while saying that an AI is speaking, remains allowed, including in personalised advertising. The AI Act bans purposefully manipulative or deceptive techniques, and the exploitation of vulnerabilities, when they cause significant harm. Consumer law, for its part, bans misleading or aggressive commercial practices, even without significant harm. A chatbot must also present itself as an AI, unless this is obvious.
How can you tell whether you are talking to an AI?
In the European Union, a chatbot has had to tell you since August 2026, at the latest at the first interaction. On a forum or a social network, a fake account does not follow the rules by definition, and nothing lets you recognise it with certainty just by reading. The most useful reflex is still to judge the message rather than its author: what it is trying to achieve, and whether its facts check out at their source.
Can you protect yourself from AI persuasion?
Partly. Knowing that a message comes from an AI protects you little; knowing what it is trying to achieve protects you more. Check the decisive facts against their primary source, be wary of a flood of arguments that leaves no time to check, and let a few days pass before an important decision, because the effect partly fades. In a business, any sensitive request is confirmed through another channel.
Sources
- A three-minute video (new tab)instagram.com
- The study by Thomas Costello, Gordon Pennycook and David Rand (new tab)doi.org
- The MIT press release (new tab)mitsloan.mit.edu
- A corrected analysis (new tab)zenodo.org
- Expression of concern (new tab)doi.org
- Developer Simon Willison (new tab)simonwillison.net
- The researchers’ draft (new tab)retractionwatch.com
- Retraction Watch (new tab)retractionwatch.com
- The researchers decided not to publish their results (new tab)engadget.com
- Hackenburg et al. (new tab)doi.org
- The Oxford Internet Institute (new tab)oii.ox.ac.uk
- Lin et al. (new tab)nature.com
- Cornell press release (new tab)news.cornell.edu
- Hackenburg et al., 2026 (new tab)arxiv.org
- Corrected on that point in September 2026 (new tab)nature.com
- A preprint (new tab)arxiv.org
- Its guidelines on prohibited AI practices (new tab)ec.europa.eu
- Article 5 of the EU AI Act (new tab)ai-act-service-desk.ec.europa.eu
- Up to 35 million euros or 7% of total worldwide annual turnover (new tab)ai-act-service-desk.ec.europa.eu
- Article 50 (new tab)ai-act-service-desk.ec.europa.eu
- The Unfair Commercial Practices Directive (new tab)commission.europa.eu
- “Digital Fairness Act” (new tab)ec.europa.eu
- Approved by the Commission on 20 July 2026 (new tab)ec.europa.eu
- AISI report (new tab)aisi.gov.uk
- A study published in February 2026 (new tab)pmc.ncbi.nlm.nih.gov
Up to 35 million euros or 7% of total worldwide annual turnover
Read on ai-act-service-desk.ec.europa.eu(new tab)Jim · founder of Brain Plus AI
Also in business ai
Stepping back7 min
AI and disinformation: lying machine or remedy?
One study shows that AI writes more convincing false information than humans do, another that it reduces conspiracy beliefs, but its journal has expressed a reservation. What we can take from them, and what the law requires of those who publish.
Stepping back8 min
Should you disclose that a text was written by AI?
Transparency owed to the reader, or a detail of how the text was made, like a spell checker? The EU AI Act settles it only halfway, with a criterion that is useful well beyond the law: does a human answer for the text?
Practical guide10 min
AI Act and GDPR: what an SME using AI must do
An SME that uses ChatGPT or a CRM with built-in AI already has obligations. This guide takes stock in October 2026: your role under the AI Act, the timetable revised in July 2026, what the GDPR requires and the steps to get compliant.