Book a call

Digital workers

A human in the loop is now a legal control with a price

Amol Patil, Founder · 26 August 2026 · 8 min read

Explore with AI
ChatGPTPerplexityClaudeGoogleGrok
  • The Dutch data protection authority fined Uber 824,990,000 euros because its software deactivated drivers with no human assessment. Uber has appealed.
  • GDPR Article 22 limits decisions about people based solely on automated processing. EU regulators say a token sign-off does not count as human involvement.
  • A real review gives a person the information, the time and the authority to change the outcome, and leaves a record that shows it.
  • Budget reviewers as a working step. A rubber stamp still costs a reviewer and leaves the decision solely automated.

We think human review of automated decisions about people is now a legal control, and a missing or token review carries a fine. Operations, HR and compliance leads should staff it like any other control, with a named step, time for each case and the authority to say no.

On 21 August 2026 the Dutch data protection authority fined Uber for deactivating drivers' accounts by algorithm. The CNIL, France's data protection regulator, received the original complaint, cooperated on the case and published its own notice on 24 August.

What the regulator found

Between 2018 and 2022, Uber's software deactivated drivers' accounts when it detected suspected fraud or customer ratings that were too low. Accounts were blocked temporarily, or permanently after persistently low ratings, and drivers lost their Uber income while blocked. The Dutch authority found no human assessment in these decisions and ruled that they broke the GDPR's ban on fully automated decision-making. It also found that Uber had not told drivers enough about the automation.

Uber has stopped the practice and has appealed the fine. The authority says it aligned the decision with other European regulators, who all calculate fines the same way.

The Uber decision in numbers

What the law asks for

Article 22 of the GDPR gives people the right not to be subject to a decision based solely on automated processing, including profiling, when it has legal effects on them or similarly significant effects. Recital 71 gives two examples: automatic refusal of an online credit application, and e-recruiting with no human intervention.

There are exceptions, for decisions necessary for a contract, authorised by law, or based on explicit consent. Under the contract and consent routes, the company must still offer at least the right to obtain human intervention, to express a point of view and to contest the decision.

Guidelines from the Article 29 Working Party, endorsed by the European Data Protection Board, close the obvious loophole. A person who routinely applies the system's output, with no real influence on the result, does not make the decision any less automated.

To count, oversight must be "meaningful, rather than just a token gesture". The reviewer needs the authority and competence to change the decision and should consider all the relevant data. The guidelines also ask companies to record, in their impact assessment, how much human involvement the process has and at which stage.

The EU Court of Justice made a related point in December 2023. In the SCHUFA case, it held that a credit agency's automated score can itself count as an automated decision when the lender that receives it draws strongly on it. In our view, when staff follow a score without examining the case, the score is making the decision, and a signature does not change that.

This is not legal advice. Ask counsel which of your decisions have legal or similarly significant effects.

Platform work spells out the staffing

The EU Platform Work Directive goes further for people who work through apps. Article 10 requires that any decision to restrict, suspend or terminate the account of a person doing platform work be taken by a human being. Platforms must provide "sufficient human resources" for oversight. The people doing it need the competence, training and authority to override automated decisions, and protection from dismissal or other adverse treatment when they use it.

Article 11 lets workers ask for a review of decisions like these and requires a written answer within two weeks. Member States must bring the directive into national law by 2 December 2026.

Read together, these rules describe the job. The person in the loop needs the facts, the time and the power to say no, and the company needs a record that proves all three.

A rubber stamp and a real review, side by side

What separates the two is whether the person could have changed the outcome, and whether the records show it.

Rubber stamp or real review

Rubber stampReal review
When the person actsAfter the decision takes effect, if someone complainsBefore an adverse decision takes effect
What the reviewer seesThe system's verdictThe inputs, the system's reasons and the affected person's side
Time for each caseWhatever the queue allowsA budget set in the staffing plan
AuthorityConfirm, or escalate and waitOverturn alone, with no penalty for doing so
RecordA click and a timestampThe decision, the reasons, the reviewer and the time spent
What gets measuredCases closed per hourHow often reviewers change outcomes, and how appeals end

  • Rubber stamp

    When the person acts
    After the decision takes effect, if someone complains
    What the reviewer sees
    The system's verdict
    Time for each case
    Whatever the queue allows
    Authority
    Confirm, or escalate and wait
    Record
    A click and a timestamp
    What gets measured
    Cases closed per hour

  • Real review

    When the person acts
    Before an adverse decision takes effect
    What the reviewer sees
    The inputs, the system's reasons and the affected person's side
    Time for each case
    A budget set in the staffing plan
    Authority
    Overturn alone, with no penalty for doing so
    Record
    The decision, the reasons, the reviewer and the time spent
    What gets measured
    How often reviewers change outcomes, and how appeals end

Our design view, drawn from the Article 29 Working Party guidelines and Article 10 of the Platform Work Directive.

Price the control, then staff it

A real review costs reviewer hours. We think that cost belongs in the business case next to the automation savings, because it is the price of using automation on decisions about people at all. A rubber stamp still pays for a reviewer and, on the regulators' reading, leaves the decision solely automated.

Start with an inventory. List every automated decision that blocks, refuses, filters out or ends something for a person, such as account suspensions, applicant rejections, shift allocation, fraud holds and credit refusals. For each one, write down whether a person decides before it takes effect, reviews it on appeal, or never sees it.

Then set reviewers up to disagree. Show them the inputs and the system's reasons along with its verdict. Give each case a time budget, and protect anyone who overturns the system. The same discipline applies to any AI system doing real work, and it starts with writing down what it may do alone and what needs sign-off.

Then watch the override rate. If reviewers almost never change an outcome, either the system is very good or the review is a stamp. Time spent per review and a sample of confirmed cases will tell you which. A sudden rise in overrides usually means the system has started getting cases wrong. Treat it as an incident, as you would for any AI worker in production.

Check your last month

Pull last month's adverse automated decisions about people. For each one, find who reviewed it, how long they spent, what they saw and whether they could have changed it. If your records cannot answer those four questions, a regulator's request will find the same gap.

Our human-in-the-loop method starts the same way, by measuring where the model fails, and then designs the routing, reviewer rubrics and decision log around that gap.

Sources

· views
Amol PatilFounderFounded extendfuture in 2019. Has shipped computer vision, voice and agentic systems into production across ten industries. Amol Patil on LinkedIn

Working on something in this territory?

Tell us what you are trying to win. We answer within one business day, from the people who build.

Start a projectBook a call