Automated decisions: can AI decide on its own?

Software can sort job applications, score a credit risk, adjust a price or flag an “underperforming” employee. The question is no longer whether it can do this, but whether an automated decision of this kind can dispense with a person to validate it.

The two opposing camps each have a solid argument, and the law has already begun to settle the question, in Europe as in California. For an SME that automates its HR or its pricing, the stakes are very concrete: knowing which decisions to leave to the software and which to have someone validate.

Position 1: the algorithm is fairer than a rushed human

Those who defend automated decision-making have good arguments. A tired or rushed person does not judge two identical cases the same way. A system applies the same criteria to everyone, at any hour, with no favouritism and no personal sympathies. It can process thousands of credit applications in a few seconds, where a team would take weeks, and it leaves a record of every criterion used.

They add that human biases are rarely documented, whereas an algorithm’s can be measured: you can test a model on thousands of made-up cases and correct what goes wrong, which is never done with a recruiter.

Position 2: a decision that affects someone’s life must be explainable

On the other side, the argument comes down to one sentence: an employee who is dismissed, or a customer who is refused credit, must be able to understand why and to contest the decision before someone who can change it. A language model cannot always explain what led it to an answer, and a system trained on past decisions reproduces the biases of those decisions, with the appearance of neutrality.

This camp also points to a risk of diluted responsibility: when “the system decided”, nobody feels responsible any more. The rushed manager of position 1 at least has a name.

What the law already says

In Europe, since 2018. Article 22 of the General Data Protection Regulation provides:

The text provides for three exceptions: the decision is necessary for entering into or performing a contract, it is authorised by Union or Member State law, or it is based on the person’s explicit consent. For the contract and consent cases, the company must guarantee the person at least the right to obtain human intervention, to express their point of view and to contest the decision; where it is the law that authorises the decision, that law must itself provide for suitable measures.

The EU AI Act. Annex III of the AI Act classifies as high-risk the systems used to recruit and select candidates, those that make decisions on promotion or dismissal, those that monitor and evaluate employees’ performance, and those that assess people’s creditworthiness or establish their credit score, except for systems used to detect financial fraud. The requirements and obligations specific to these systems will apply from 2 December 2027: Regulation (EU) 2026/1744 of 8 July 2026 postponed that deadline, originally set for 2 August 2026. An employer using one of these systems will then have to inform workers’ representatives and the employees concerned before putting it into service (Article 26). In Article 86, the text also gives a person affected by a decision the right to obtain “clear and meaningful explanations of the role of the AI system in the decision-making procedure and the main elements of the decision taken”.

In California, a law of September 2026. On 30 September 2026, the governor signed the “No Robo Bosses Act” (the name its author gave it), and the official text of SB 947 provides that it applies from 1 July 2027. It bans an employer from relying solely on an automated decision system to discipline or dismiss an employee. When the employer relies primarily on such a system, a person must corroborate the decision using the data the system used or other evidence, such as the employee’s evaluations or personnel file; if the person cannot do so, the system’s output cannot be used. The employee also receives, when the decision is announced to them, a written notice stating that the decision was based primarily on the system and that a person has verified it.

And what about prices? A price displayed to everyone is not, as a general rule, an individual decision within the meaning of the GDPR. A personalised price, calculated for one person from their profile, is closer to such a decision and also falls under consumer law. It is the least settled of the three areas.

What we think

We think each position is right on one point. An algorithm is often more consistent than a human, and that is a real argument for using one. But consistency is not justice: a system can be perfectly consistent in its errors, across thousands of cases at once. And a decision that can be neither explained nor contested before someone is not acceptable when it affects a job or credit.

The rule we defend fits in one line: a person validates anything that commits the company to someone or that involves money. The system prepares, sorts, flags, proposes; an identified person decides and can say why. Our own expense-entry agent, in production since July 2026, follows this pattern:

Internal tasks with nothing at stake for any individual can, for their part, run automatically, with a review of a sample.

What would change our mind: systems able to explain each decision in a verifiable way, audited by an independent third party, and an effective way to appeal to a human. We are not there yet with language models.

What this changes in practice for a company

The approach takes five steps:

1. 01Take an inventory of automated decisions.

   List what your tools already decide on their own: reminders, account blocks, prospect scoring in the CRM, sorting of job applications, discounts granted. The exercise can reveal automatic decisions that nobody had formalised.

2. 02Classify by what is at stake.

   A priority score on a prospect does not carry the same weight as a credit refusal or a performance alert on an employee. Whatever significantly affects a person goes through human validation.

3. 03Keep a record of the validation.

   Note who validated, on what evidence, on what date. In a well-kept CRM, that is a field and a history, not a major project. To choose the tool, our guide to choosing between Twenty, Dolibarr and Odoo compares the three, and our CRM and customer data section also covers migrating existing data.

4. 04Plan for explanation and appeal.

   For each sensitive decision, prepare the answer to the question “why?” and a simple way to contest it. That is what the GDPR already requires, and what the AI Act provides for high-risk uses. Our guide AI Act and GDPR for an SME details what these uses require of the company that deploys them.

5. 05Inform before deploying.

   If a tool touches on staff management, notify workers’ representatives and the employees concerned before it goes live, and train the managers who use it to spot its errors. Automations that remove tasks also raise the question of how juniors learn the job, which we cover in our analysis of AI and young people’s first jobs.

Where to start

The law does not forbid automating: it refuses to let a machine decide alone what weighs on a person’s life. The first step is therefore the inventory described above, then a validation field and a history in the tool where these decisions are made, often the CRM. If yours cannot say who validated what, our custom CRM set up in your own vocabulary covers that configuration and the migration of your existing data. To choose what to automate first, see our Automation and bots section.

Frequently asked questions

Sources

1. General Data Protection Regulation (new tab)eur-lex.europa.eu
2. AI Act (new tab)eur-lex.europa.eu
3. Regulation (EU) 2026/1744 of 8 July 2026 (new tab)eur-lex.europa.eu
4. Official text of SB 947 (new tab)leginfo.legislature.ca.gov