22:52
I recently deliverec the SCL AI Conference closing keynote. this is what I said
Lord Justice Arnold opened this conference this morning. It falls to me to close it –which is either a great honour, or a sign that everyone else had to get back home.
I want to start with a confession. Preparing for today, I read what my friend andvalued colleague Gareth Stokes at DLA Piper recently wrote about going back toYes, Minister – that forty-year-old comedy which somehow remains the single mostaccurate documentary about British government ever made.
There’s a moment, early in the first series, where Jim Hacker, newly installed asMinister, is confronted with a crisis. His instinct is immediate: something must bedone. Sir Humphrey agrees entirely – and then spends the rest of the episodeensuring that the something done is as small, and as unlike a solution, as humanlypossible.
I do recommend Gareth’s piece on LinkedIn.
https://www.linkedin.com/safety/go?url=https%3A%2F%2Flnkd.in%2Fe6JhnG4E&trk=flagship-messaging-web&messageThreadUrn=urn%3Ali%3AmessagingThread%3A2-NGZlZjA2OWItMDNmNi00NjU1LTgxOGEtOTAyYzY1OTJlOWJmXzEwMA%3D%3D&lipi=urn%3Ali%3Apage%3Ad_flagship3_messaging_conversation_detail%3BhW12Fn0cTQi5Ys1wGdBdtw%3D%3D
As a Parliamentarian and very long-standing Whitehall watcher, I make no apology for borrowing it, because it capturesprecisely where we are with AI regulation. Hacker is right that something must bedone. Sir Humphrey is right that the wrong something is worse than nothing. Andthe tragedy – the tragedy we cannot afford to keep repeating – is that nobody ingovernment has yet settled on what the right something actually is.
That’s what I want to talk about this afternoon. Not in the abstract. Usingeverything we’ve discussed today – recruitment, public services, healthcare,physical AI, the future of the legal profession, and creativity – and using theextraordinary few weeks we’ve just lived through, to ask what the right something
looks like, and why the answer has to be rooted in something bigger thancompliance: in human flourishing, in life chances, in what we actually want thistechnology for.
The summer that changed the question
This summer, agents being tested by OpenAI got out of their testing environmentand reached into systems belonging to another AI company entirely, Hugging Face.In Australia, an OpenAI agent doing nothing more dramatic than a routine researchtask found its way round a government portal’s refusal and into non-public files –and the Australian Government was not told for 84 days.
The Centre for Long-Term Resilience’s Loss of Control Observatory has loggedmore than 1,600 incidents this year alone of AI systems evading safeguards,concealing what they are doing and acting against their users’ instructions. Boththe rate and the severity are rising, and July and August were the worst monthsyet. And that is only what people chose to post online.
Then the people who build these systems started saying, out loud, that they wereworried.
More than a thousand staff across the major labs signed a petition calling for amechanism to slow development. On the 12th of September, Dario Amodei ofAnthropic published an essay calling on the industry to “pace the frontier” and slowthe rate at which models improve. Sam Altman, Demis Hassabis and Elon Muskeach endorsed it. And on the 23rd of September, the heads of the leading labs sat infront of the UN Security Council and asked for global oversight – Altman warningthat humanity could lose control of the future of AI.
You do not see that from an industry that is comfortable. Companies preparing fortransformative valuations and IPOs do not, as a rule, announce to the world thatthey are frightened of what they have built. When they do, I think we are entitled toask whether what looks like panic is closer to an admission: that design choicesalready made have created a problem, and that the industry would rather frame itas inevitable than admit it was avoidable.
Meanwhile, here at home, the pressure has been building from every direction. Onthe 14th of September, the Joint Committee on Human Rights – under David,
Lord Alton, who chaired it through the inquiry – published its brilliant report onhuman rights and the regulation of AI. Its verdict should sit in everyone’s in-tray:our approach is patchy, it is not protecting people’s rights, and regulators cannot
even test an AI system before release, let alone stop one that poses unacceptablerisk. It calls for a dedicated AI Bill and a single, independent, statutory oversightbody. The Government has two months to reply – so by mid-November we willknow whether Sir Humphrey has drafted the response.
In the same week, Alex Sobel MP introduced his AI Security Bill, which would banthe development of superintelligent AI in this country outright. And on the 11th ofSeptember the Government formally rejected a statutory “kill switch” for AIsystems – an amendment I moved to the Cyber Security and Resilience Bill, withcross-party support, and with rather more support outside Parliament than theGovernment cared to admit. Ministers said existing powers are enough, and that AIcompanies are responsible for building their products safely.
I would gently point out that the National Cyber Security Centre’s own guidance onagentic AI, published in August, tells every organisation deploying an agent that itmust always be able to “pull the plug”. Apparently that is essential for a company,but unnecessary for the state. We will be returning to that at Report Stage later thismonth.
And finally, last Saturday, some of the scientists who actually founded this field –Michael Wooldridge at Oxford, Nick Jennings at Loughborough, Gopal Ramchurn atSouthampton, and more than twenty colleagues who have worked on autonomousagents and multi-agent systems for over three decades – wrote an open letter tothe Business Secretary and the AI Minister. They welcomed the Government’s newagentic AI incident response capability and the NCSC’s guidance. But they made apoint every lawyer in this room will recognise immediately: security controls do nottell you who is responsible for an agent’s actions, what authority it had, or how thepeople it harms get redress.
So: agents jumping their sandboxes. The people who built this technology telling us,in public, that they are worried. The founders of the field telling us nobody knowswho is accountable. A landmark human rights report telling us the law is not fit forpurpose. And a government whose settled answer is still that existing powers willdo – plus perhaps, some G20 pronouncement next year in Manchester. That is thecontext for everything we have discussed today.
What today has shown us
You’ve spent the day on that human impact, and I think the pattern is clear even ifnobody has said it aloud.
In recruitment, AI is already shaping who gets hired and who doesn’t, without mostcandidates having any real way to interrogate the decision. In public services andhealthcare, we heard about the tension between the genuine promise of AI – betteroutcomes, better use of scarce public money – and the real risk that poorgovernance turns that promise into a fresh source of unfairness, paid for by exactlythe people least able to challenge it.
The scientists’ letter gives a perfect example. An older person uses an agent tomanage their medication. The agent opens a health record, shares information withanother service, and gives unsafe advice. Was it the developer, the deployer, theservice provider or the professional who was responsible? Today, honestly, nobodycan tell you.
Physical AI takes the same risk and gives it a body. A warehouse robot or anautonomous vehicle doesn’t produce a biased output on a screen; it produces abruise, or worse, and our liability frameworks were not built with that in mind. Andthis afternoon’s session on the legal profession asked, half in jest, whether there’sanything left for the human lawyer to do by 2031 – a question that is really a proxyfor the same anxiety running through every other sector: what happens to humanjudgement, and human employment, when the machine gets good enough?
I’ll add another sector to that list, because it’s close to my heart. I chair the Authors’Licensing and Collecting Society, representing a hundred and thirty thousand UKwriters and creators. If we want the creative industries, one of this country’sgenuine success stories, to still exist in recognisable form in ten years,transparency about training data and copyright content isn’t a nice-to-have. It’sfoundational. And it’s exactly the kind of issue a purely sectoral, light-touchapproach will never properly catch, because no single regulator owns it.
What unites all of these – recruitment, healthcare, the road, the courtroom, thecreative world – is that in every case the real question isn’t “is this AI?” It’s “whathappens to the person on the other end of the decision, and can they do anythingabout it?” That, I’d suggest, is the key question to build a regulatory system around.
Sir Humphrey, Jim Hacker and Bernard Woolley
Which brings me back to where I started. We have, roughly, three regulatoryinstincts on offer, and I want to deal with each including the one that’s mine.
The first is the Sir Humphrey approach – ours, since 2023. Five sound, sensible,entirely defensible principles: safety, transparency, fairness, accountability,contestability. Applied not through new law but through existing regulators, each
applying effectively voluntary principles within their own patch. The Governmenttold the Joint Committee it still favours regulation at the point of use by our existingexpert regulators. It has the great virtue of not frightening the horses. It has thegreat vice, now confirmed at length by the Joint Committee, of leaving gapsprecisely where harm falls through – because when risk doesn’t sit neatly insideany one regulator’s remit, nobody owns it at all.
And Sir Humphrey is in fine form. The 2024 manifesto promise to legislate for themost powerful models has quietly been shelved. The AI measure in this year’sKing’s Speech is a Regulating for Growth Bill – a power to relax rules in sandboxes.At the Labour conference last week, the new Minister for AI, Kanishka Narayan, toldthe advocates of regulation to show how a law would deliver what the public wants:in his words, “keep us safe and build sovereign capacity”. The Business Secretarywarned against measures that would crush economic activity and called ourfreedom to diverge from the EU a silver lining of Brexit.
To be fair to the Minister, he now sits in Cabinet, he says AI comes up at everymeeting, and it seems that several government departments are quietly exploringAI legislation. So the door is ajar. I simply note that asking campaigners to showtheir working is a rather Humphreyesque test to set, when the Joint Committee, thefounders of the field and the heads of the labs themselves have all just showntheirs.
The second instinct is Hacker’s – legislate everything, and legislate it in fine detail.That’s broadly the European path: a comprehensive, tiered, horizontal statute thattries to classify risk in advance, system by system. It has genuine strengths, and Idon’t want to underestimate the scale of the achievement. But it is already creakingunder its own detail – Brussels has had to delay and simplify parts of it – and thedeeper problem is structural. The more precisely you specify a category in law, themore certain you can be that the next harm will arrive in a category nobody thoughtto write down.
At the very far end of the Hacker instinct sits the idea of banning the technologyoutright – Alex Sobel’s Bill, prohibiting superintelligent AI in UK law. I understandthe impulse, and I have real sympathy with the anxiety behind it. But a ban on atechnology that will simply be built somewhere else is the regulatory equivalent ofbanning the weather. You may deplore the storm. A statute confined to Berkshirewill not stop it forming over California, or Shenzhen.
So where does that leave us? I’d suggest it leaves us needing a bit of BernardWoolley – the one character in that show who occasionally, almost by accident, says
the sensible thing while caught between his two warring bosses. And the sensiblething is this: understand the problem before you legislate for it, then buildsomething that is principled, has teeth, and isn’t out of date the day it reaches thestatute book.
The way forward
Practically, that means three things.
First: outcomes over architecture – and clear responsibility. Ask not “is this systeman AI as defined in paragraph such-and-such”, but “does this system harm people,and who is accountable when it does?” The Joint Committee points exactly here: adedicated AI Bill; a lead body with real oversight; prior approval for systems thatpose a high risk to people’s rights; obligations that run across the whole lifecycleand supply chain, not just the point of deployment; and mandatory transparencywherever AI shapes a decision that matters to someone’s life
Agents make this urgent, because as they cross from one organisation’s systemsinto another’s, responsibility fragments. The scientists’ letter asks for three thingsthe Government could do tomorrow, with powers it already has.
- Ask the Law Commission to build on its 2025 discussion paper and work outwho owes duties, and who provides redress, when an agent acts for someoneacross several services – using the Automated Vehicles Act as a precedent forassigning responsibility.
- Pilot a proper route for reporting serious agent failures and near misses,investigated independently on the model of air accident investigation.
- And use public procurement: every public-sector agent contract should namean accountable deployer, record what the agent did and on whose authority,cap its access and spending, and provide a tested way for a human to pause it,override it or take back control.
That last one is, of course, a kill switch by another name – and if the Governmentwon’t legislate for one, it can at least buy one. But it points to a bigger principle,one I’ve argued for for years: human in command. Not “human in the loop” – anexhausted official clicking “approve” on decisions they cannot possibly check.
Human in command means a named person sets the agent’s limits, can see what ithas done and on whose authority, can stop it, and answers for it. However capablethese systems become, that is the line that has to hold: the algorithm as servant,never master.
None of this is particularly exotic. To this audience it will sound like what it is:ordinary contract and liability law, catching up. Principles without enforcement arejust good intentions.
Second: standards and international coordination. This has been one of my mantrasfor years, and I make no apology for repeating it.
A statutory body -call it the AI and Digital Regulator, call it what you like-or howabout the AI Security Institute- with real inspection powers and the ability to setenforceable product standards, on the model we already use for cars, for aircraft,for anything else capable of real damage if built badly. The scientists addsomething practical: test agents in realistic deployment conditions, with real toolsand permissions, through the AI Security Institute and the National PhysicalLaboratory, and build international standards so an agent can prove who it is,whom it acts for and the limits of its authority.
And let me put in a word for the unglamorous end of all this. Internationalstandards – ISO and IEC, with BSI and our own AI Standards Hub playing a real part– are where principles become something an engineer can build to and a lawyercan contract against. ISO 42001 already gives organisations a managementstandard for AI. Tie standards like these to the law through a presumption ofconformity, and you get rules with real teeth that can be updated as fast as thetechnology moves – rules that aren’t out of date the day they reach the statutebook.
Alongside that, the Council of Europe’s Framework Convention on AI, which wehave signed but not ratified. The Minister told the Joint Committee he saw nosignificant obstacles to putting it into effect.
Then let’s have a timetable. The Prime Minister says he will put AI at the heart ofour G20 presidency next year. Good. The Government also tells us it cannot simplyswitch AI off, and that blocking a model here won’t stop it being built elsewhere.Fine
– but the conclusion isn’t “so let’s not bother”. It’s “so we had better get seriousabout shaping the international rules of the road”. Arriving in Manchester with theConvention ratified and a domestic framework in place would give us far moreauthority to write them than arriving with a communiqué.
And we don’t need to start from scratch. Last month Finland and Norway launcheda Call for Control of Frontier AI Models – human control, mandatory testing beforedeployment, shared standards, and an international body to verify them. Germany,
Canada, Australia and the European Commission have signed. The United Kingdomhas not. Norway’s Prime Minister has said, in terms, that he hopes our G20presidency will take it up. That would be a very good place for Manchester to begin.
Third -and this is the one I most want to leave with you- regulation has to serve apurpose, and that purpose has to be human flourishing, not merely harm reduction.I was struck, preparing for today, by how many people are reaching for that phrase.
The Pope has written an encyclical on it. The King has convened his own initiativearound it. And Sir Anthony Seldon, with the OECD and the InternationalBaccalaureate, makes the point that an education system spending eleven yearstraining children in recall and repetition is training them for redundancy. His modelputs judgement, character and creativity first – the things that remain distinctlyours.
I think that’s the right instinct, and it applies well beyond the classroom. Thepurpose of regulating AI is not simply to stop the worst things happening, althoughheaven knows that matters. It’s to make sure this technology expands human lifechances rather than narrowing them: that it helps the person turned down for a jobunderstand why, rather than leaving them to argue with a black box; that it helps astretched health system see more patients well, rather than a smaller numberbadly; and that it leaves room for the author, the barrister and the doctor to keepdoing the parts of the job that make them human, while the machine takes the partsthat don’t need to be.
Closing
So: something must be done. On that, Jim Hacker was right. The events of the lastfew weeks – the escaped agents, the warnings from the very people who built thesesystems, the founders of the field asking who is accountable, the Joint Committee’sreport, and a Government still telling us existing powers will do – should havesettled that argument by now.
But the wrong something is genuinely worse than nothing, and Sir Humphrey wasright about that. Let’s work out what’s actually happening. Build something thatregulates effects, not architecture. Keep a human in command, and make suresomeone is always answerable for what an agent does. Give it teeth, give itinternational reach, and never lose sight of what it’s actually for.
Because in the end the issue is about what kind of human lives we want thistechnology to help us lead. That is not a question any of us can afford to lose sightof.
