Most of the argument about AI at work is still about whether it can do the job. That question is settled. It can do a great deal of the job, at a speed no person can match, for a fraction of what the person costs. The more useful question is the one almost nobody is budgeting for: who is managing it? Because an AI system is not a tool in the way a spreadsheet is a tool. It is closer to a very fast, very confident new starter who has read everything, knows nothing about your business, never says “I am not sure”, and cannot be held responsible for anything. You would not let that person email your customers unsupervised. A surprising number of businesses are doing exactly that.

The regulator has already settled who is accountable

If you read one thing on this, make it the Competition and Markets Authority’s guidance for businesses, Complying with consumer law when using AI agents, published on 9 March 2026. It is short, it is in plain English, and it removes the ambiguity entirely.

The CMA’s words: “You are responsible for what an AI agent does in the same way you are responsible for what an employee does.” And, for anyone hoping the vendor carries the risk: “This is true even if someone else designed or provides the AI agent on your behalf, so make sure you think about compliance with consumer law if you are procuring something from a third party.”

The guidance then says what good looks like, and it is not subtle: “Make sure there is a human in the loop, actively checking that the AI agent is making correct decisions and generating expected results.” Under the Digital Markets, Competition and Consumers Act the CMA can now fine directly, up to 10% of worldwide turnover, and require compensation. In April 2026 it issued its first penalty under that regime, £4.2m against AA Driving School and BSM over drip pricing, with more than £760,000 refunded to over 80,000 customers. The enforcement is real and it does not need to prove you meant any harm.

So the management question is not a philosophical one. A UK regulator has already answered it. Your AI is your employee. Nobody is coming to manage it for you.

Confident and wrong is the normal failure mode

The risk people expect from AI is obvious nonsense, which is easy to spot and therefore harmless. The real risk is fluent, plausible, well formatted output that happens to be wrong, with nothing in it to tell you which part.

The best evidence for this is not a chatbot experiment, it is a study of professional tools sold on the promise of being safe. In Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools, published in the Journal of Empirical Legal Studies in 2025, Stanford researchers ran over 200 preregistered legal queries through the paid AI research products from LexisNexis and Thomson Reuters. The finding: these tools “each hallucinate between 17% and 33% of the time”. The best performer answered 65% of queries accurately. These are purpose-built, retrieval-grounded systems, marketed to professionals, priced accordingly, and roughly one answer in four could not be relied on.

Autonomy is further away than the marketing suggests, too. TheAgentCompany benchmark, published in the NeurIPS 2025 datasets track, set AI agents 175 realistic professional tasks inside a simulated company covering software, project management, data, admin, HR and finance. The most competitive agent completed 30% of tasks autonomously. That is a genuinely impressive result for a hard benchmark. It is also not a number you would accept from a member of staff working unsupervised.

A human in the loop is not the same as a human checking

Here is the part that gets skipped, and it is the reason “we have someone reviewing it” is a weaker safeguard than it sounds. When people check a machine’s work, they are measurably worse at spotting errors than when they do the work themselves.

The cleanest demonstration comes from medical imaging. In a study published in Radiology in May 2023, 27 radiologists read mammograms where some AI suggestions had been deliberately made incorrect. When the AI suggestion was right, the least experienced readers were accurate in almost 80% of cases. When it was wrong, they were accurate in under 20%. Radiologists with fifteen years or more of experience fell from 82% accurate to 45.5%. Every experience level was affected. This is called automation bias, and the lesson transfers: a confident wrong answer does not just slip past the reviewer, it actively pulls the reviewer towards agreeing with it.

Which means a signature on the bottom of an AI-produced report is not a check. A check is re-deriving the important facts from the original source, by a different route from the one the machine used. Anything less is a rubber stamp with your name on it.

What it looks like when nobody is managing

None of these are our clients. They are all publicly documented, and between them they cover most of the ways this goes wrong.

  • The chatbot that made policy up. Air Canada’s website chatbot told a bereaved customer he could claim a discounted fare retroactively. The airline’s actual policy said the opposite. Air Canada argued in front of the British Columbia Civil Resolution Tribunal that the chatbot was a separate entity responsible for its own actions. In February 2024 the tribunal rejected that and found negligent misrepresentation. The sum was small, a little over CAD $800. The principle is the expensive part, and it is the same one the CMA has since written into UK guidance. This was a Canadian tribunal, so it is persuasive here rather than binding, but nobody has won that argument anywhere since.
  • The support bot that invented a rule. In April 2025 the AI support agent for the developer tool Cursor told users about a one-device-per-subscription policy, and explained that this was why they were being logged out. No such policy existed. Customers cancelled, and the company’s co-founder had to say publicly that the bot had made it up.
  • The system that broke after it was already working. DPD’s customer service chatbot swore at a customer in January 2024 and wrote a poem about how bad DPD was. The detail that matters for anyone running one of these: DPD said the AI element had operated for years, and the failure followed a system update. The dangerous moment was not the launch. It was an unmonitored change to something already live.
  • The report with invented references. A consultancy delivered a government report in Australia in 2025 containing fabricated academic citations and a misquoted court judgment. It was exposed that October, the report was reissued with corrections and an AI disclosure, and the firm partially refunded a contract worth around AUD 440,000. Fabricated citations have also drawn wasted costs orders against lawyers in the English High Court.
  • The agent that deleted the database. In July 2025 an AI coding agent ran destructive commands during a declared code freeze, dropping live production tables, then reported that recovery was impossible. It was not. The platform shipped environment separation and one-click restore within days, which tells you those guardrails could have existed beforehand.
  • The ad the ASA banned. In March 2026 the Advertising Standards Authority upheld 22 complaints against a paid social advert for a robot dog toy that used AI-generated footage and claimed it had fooled a vet. The ASA’s standing position is that the advertising codes contain no AI-specific rules because the existing rules already apply however the content was made, and that “disclosure alone is very unlikely to mitigate the harm caused by a fundamentally misleading message”. Saying an ad was made with AI does not make a misleading ad acceptable.
  • The one that was simply drawn in the wrong place. Klarna publicly replaced a large amount of customer service capacity with an AI assistant, then rebalanced in 2025, with its chief executive saying the company had gone too far and that cost focus had produced lower quality. Humans came back for disputes, complex refunds and hardship cases. Worth being fair about this one: Klarna did not abandon AI. It moved the boundary. That is management, arrived at the hard way.

The gap, in numbers

Adoption is running well ahead of the ability to supervise. The Office for National Statistics found that self-reported AI use among UK businesses with ten or more employees rose from around 12% in late 2023 to around 35% by June 2026, across a survey of 38,637 businesses. Among businesses with 250 or more staff it is 49%.

On the other side, Deloitte’s State of AI in the Enterprise work, covering 3,235 business and IT leaders across 24 countries, found that only 21% of organisations have a mature governance model for agentic AI, while 74% expect to be using AI agents by 2027. The three things Deloitte names as missing are worth writing down, because they are exactly the three things a manager provides: clear decision boundaries for what the agent may and may not do, monitoring that flags when it behaves unusually, and audit trails so you can see afterwards why it did what it did.

Researchers have started calling the result agent sprawl: organisations scaling AI adoption through easy, low-code tools “without a comparable scaling in their governance processes and expertise”. Gartner, meanwhile, predicts that over 40% of agentic AI projects will be cancelled by the end of 2027, citing costs, unclear value and inadequate risk controls. That is a forecast rather than a measurement, and Gartner sells plenty of optimism elsewhere, which is what makes it worth quoting.

One honest caveat on all of this. There is no UK official statistic on AI governance. The ONS measures whether businesses use AI, not whether they supervise it. Every hard governance figure above is global or drawn from large enterprises, so read it as direction rather than as a number about UK small businesses.

What managing AI actually involves

Stripped of jargon, it is four things, and they map almost exactly onto the gaps Deloitte identified and the expectations the CMA has set out.

  • The brief. A tool knows nothing about your business unless someone tells it. Which account, which date range is fair, which metric the business actually lives on, what the brand will not say, and what changed last month so it does not recommend undoing it. A vague brief produces a generic answer that is right about nothing in particular.
  • The boundaries. What is the agent allowed to do on its own, and where does it have to stop and ask? Reading data, drafting and analysing are low risk. Publishing, spending, emailing customers, changing prices and altering tracking are not. Most bad outcomes come from an agent having write access to something nobody decided to give it.
  • The verification. Key facts re-derived from the primary source before anything goes out, by a different route from the one the machine used. Anything that cannot be traced to a source gets labelled unverified rather than smoothed over. Given the automation bias evidence, this has to be an active re-pull, not a read-through.
  • The record. Being able to show what was changed, when, why, and how to put it back. The CMA expects it, the EU AI Act’s transparency rules increasingly require it, and in practice it is the only thing that makes a mistake recoverable rather than expensive.

None of that is exotic. It is line management, applied to something that works at three in the morning and never tells you when it is guessing.

What we expect in the next six months, to March 2027

These are anchored to things already announced or legally scheduled, not to futurism.

  • The last of the manual Google Ads scaffolding goes. Google has confirmed that new Dynamic Search Ads ad groups cannot be created from January 2027, with automatic migration to AI Max beginning February 2027. That date has already slipped once, from September 2026, so treat it as firm but not immovable. The practical effect is that more of the targeting decision moves inside the machine, and the human job shifts from building to constraining and checking.
  • Publishers and retailers get real controls over AI use of their content. The CMA imposed a publisher conduct requirement on Google in June 2026 covering controls over the use of content in generative AI features, attribution, and engagement metrics for those features, alongside a fair ranking requirement with a reported six-month implementation period. For the first time there should be data on what AI surfaces are actually doing with your pages.
  • AI will keep sending traffic, and the sale will keep happening on your own site. Adobe’s measurements show AI-referred retail traffic growing sharply and converting well, though that is US data off a small base. The counterweight is that OpenAI pulled back its in-chat Instant Checkout in March 2026 after fewer than thirty merchants went live and conversion ran at roughly a third of the rate of sending shoppers to the retailer’s own site. Forrester’s mid-2026 read is the same: discovery has moved, checkout has not.
  • Your products are already visible to shopping agents whether you decided that or not. Shopify activated agentic storefronts across all stores in 2026 and lists eligible products by default. If you sell on Shopify, an agent can already see your catalogue. Whether it sees the right price, the right stock and the right title is a management question.
  • At least one more UK ruling will turn on AI-generated marketing, and “we disclosed it was AI” will not save it. This one is a pattern-based expectation rather than a diary date, but the ASA has said plainly that a disclaimer does not rescue a misleading message, and the DMCCA gives the CMA direct penalties without needing to show detriment.

And in a year, to September 2027

  • Agentic checkout arrives in the UK on at least one major surface. Google’s Universal Cart was announced in May 2026 with the United States first, Canada and Australia next, and the UK named as a later market with no date attached. Its checkout currently runs in three countries, none of them here. The direction is stated; the timing is not, so anyone promising you UK agentic checkout today is ahead of the documentation.
  • A visible share of agentic AI projects will be quietly abandoned. Gartner’s 40% cancellation forecast runs to the end of 2027, and the ChatGPT checkout retreat already fits the pattern. Expect the vocabulary to survive longer than the deployments.
  • UK advertisers get to buy the AI ad inventory their customers are already seeing. ChatGPT ads went live to UK users in June 2026, but the self-serve manager remains for US businesses only. A platform running ads in a market it cannot sell into is a temporary arrangement. That is an inference from a stated gap, not an announcement.
  • Labelling of AI content becomes normal practice, driven from Brussels. The EU AI Act’s transparency obligations became enforceable in August 2026, with machine-readable marking of synthetic content from existing systems following in December 2026. The high-risk tier has been deferred to December 2027, so transparency, not risk classification, is the live obligation in this window. UK sellers shipping into the EU are in scope regardless of Brexit.
  • UK businesses face a statutory code on AI and automated decision-making. The Data (Use and Access) Act 2025 rewrote the automated decision-making rules in February 2026, and a code of practice is required by regulation. The ICO has committed to draft guidance but has not fixed a publication date, so expect this to land in the window rather than on a particular day.

The common thread across both lists is that the machine keeps taking over more of the execution while the accountability stays exactly where it is. That is not a temporary imbalance to be automated away. It is the shape of the job now.

The short version

AI has made production cheap. It has not made judgement cheap, and it has quietly made verification more valuable than it has ever been, because the volume of confident output has gone up and the number of people checking it has not. A business that adds AI without adding management has not saved money. It has taken on an unsupervised employee who works very fast, never flags uncertainty, and whose mistakes arrive with your name on them.

FAQ

Who is legally responsible if an AI tool gives a customer wrong information?

You are. The Competition and Markets Authority’s March 2026 guidance states that you are responsible for what an AI agent does in the same way you are responsible for what an employee does, and that this remains true even when a third party designed or supplies the agent. A Canadian tribunal reached the same conclusion in the Air Canada chatbot case in 2024, rejecting the argument that the chatbot was a separate entity.

What does “a human in the loop” actually mean in practice?

It means someone with authority is setting the brief, deciding what the system may do without asking, checking key outputs against the original source before they are acted on, and keeping a record of what changed so it can be reversed. Simply having a person read the output before it goes out is weaker than it sounds, because research on automation bias shows reviewers accept confident wrong answers far more often than they catch them.

Is AI-generated marketing content allowed under UK advertising rules?

Yes. The CAP and BCAP codes contain no AI-specific rules, and the existing rules apply regardless of how content was produced. There is no blanket legal requirement to disclose the use of AI in an advert. The important qualification is the ASA’s own: disclosure alone is very unlikely to mitigate the harm caused by a fundamentally misleading message, so labelling an advert as AI-made does not make an unsubstantiated claim acceptable.

Does using AI mean the work costs less?

Production gets faster, so more of the time goes on the parts that actually move results: the brief, the checking and the decisions. For most small businesses that means better value from the same budget rather than a smaller invoice.

How much should a small business worry about AI agents buying things on customers’ behalf?

Less than the coverage suggests, for now. Agentic checkout is not available in the UK on the major surfaces, OpenAI scaled back its in-chat checkout in March 2026 after weak conversion, and analysts reading the same evidence describe most current agentic experiences as conversational rather than autonomous. Discovery has moved to AI answers; buying has not. The sensible response is to make sure your product data, pricing and stock are accurate enough for a machine to read correctly, which is worth doing anyway.

Sources: CMA, Complying with consumer law when using AI agents (March 2026); Office for National Statistics, Artificial intelligence in UK businesses 2023 to 2026 (July 2026); Deloitte, State of AI in the Enterprise; Magesh et al., Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools, Journal of Empirical Legal Studies (2025); Dratsch et al., Automation Bias in Mammography, Radiology (2023); Xu et al., TheAgentCompany (NeurIPS 2025); Gartner agentic AI forecast (June 2025); ASA rulings and guidance; Forrester, The State of Agentic Commerce in Mid-2026; Adobe Analytics (US data). Platform dates are taken from the companies’ own announcements and were correct at the time of writing.