On the quiet end of the most reassuring phrase in modern enterprise.
The first essay in this series argued that the tense of defence is wrong. The second argued that the subject of defence is wrong. This third and final essay argues something that follows from the first two but that nobody, in polite institutional company, wishes to say aloud: the speed of defence is wrong, and the human-in-the-loop — that comforting phrase deployed at every board meeting, every regulatory submission, every vendor demo — is increasingly a polite fiction.
The phrase exists because it does useful work. It reassures the regulator, who would like to know that a person is responsible. It reassures the board, who would like to know that the machine has not been left in charge. It reassures the executive, who would like to know that, when things go wrong, there will be someone to call. It is a phrase of governance theatre, and the theatre is, in most enterprises, already over.
It is a phrase of governance theatre, and the theatre is, in most enterprises, already over.
This essay is, again, written from the vantage point of financial services. As before, the diagnosis travels. Insurance, logistics, healthcare, manufacturing, defence, and the public sector are all running on the same assumption — that there is, somewhere in the system, a human deciding — at speeds that no longer permit a human to decide.
This is the final essay. It is about what comes next, and who is going to be honest about it.
Nine seconds
It is 3:47am on a Sunday at a large bank. An autonomous adversary — a software agent, not a person — has been probing the bank’s third-party integrations for the past six minutes. It has identified a misconfiguration in a credential rotation policy for a vendor account that handles payment reconciliations. It has used the misconfiguration to obtain a temporary token. It has used the token to query the customer data store. It has issued seventeen API calls, each one shaped like a legitimate request, none of them, in aggregate, normal. By the time the bank’s autonomous defensive system has noticed, mapped the behaviour, decided it was malicious, and revoked the credentials, the entire interaction is over. The whole thing has taken nine seconds.
Somewhere in the bank’s Security Operations Centre, a human analyst is now receiving an alert. The alert says: autonomous response executed; incident contained; no human action required at this time. The analyst will, in due course, write a paragraph in a daily report. The daily report will, in due course, be summarised in a weekly report. The weekly report will be summarised in a monthly report, which will be presented, in due course, to a sub-committee of the board. The sub-committee will be told that the bank’s cybersecurity defences performed as expected. They will nod. They will move on. They will have been told, in the strictest sense, the truth.
They will have been told, in the strictest sense, the truth.
A polite fiction
The truth they will not have been told is this: nine seconds is not a window in which a human is in any loop. It is a window in which a human is in the after-action report. The human-in-the-loop has not disappeared. It has simply been moved to a different room. The room has comfortable chairs. The view is excellent. The events under discussion, however, finished some hours ago.
This is the part of the essay where someone, somewhere, is preparing to push back. We have humans in the loop, they will say. Every alert is reviewed. Every escalation has a human owner. We have not, in any sense, ceded control. They will say it with great conviction. They will be entirely sincere. They will also be wrong, in approximately the same way someone is wrong who says they are the one flying the aircraft because they were the one who, at some earlier moment, programmed the destination into the autopilot.
The honest position — the one most CISOs will admit to in private, usually after the second drink — is that the defensive systems in any large modern enterprise are now operating at speeds, scales, and concurrencies that no human can meaningfully supervise in real time. There is no command centre with screens where alert analysts sit, eyes darting, fingers hovering over the block button. There has not been, for some years. There are autonomous systems making thousands of decisions per second, escalating only the cases they cannot resolve, and producing reports — beautifully formatted reports — for humans to review at human speed, after the events they describe have completed.
This is not, in itself, a problem. It is, in fact, the only architecture that works. The adversary automated long ago. The first ransomware-as-a-service offerings appeared a decade back. The first fully autonomous reconnaissance agents were observed in the wild several years before the polite discourse caught up. Adversarial AI did not arrive on a Tuesday in 2026. It has been here. Defence has been catching up, and is now, in the better institutions, more or less caught up. Machine-speed offence is met with machine-speed defence. The actual machine-versus-machine warfare that everyone has been writing thinkpieces about for five years is happening, quietly, at three in the morning, in your enterprise, right now. The thinkpieces are running slightly behind.
The problem is not the architecture. The problem is the fiction we maintain around the architecture. We continue to say human-in-the-loop in the same tone we used in 2014, when the loop ran at human speed and a person could meaningfully decide. The phrase has not been updated to acknowledge that the human is now in a different loop — a slower, advisory, policy-setting, after-the-fact loop. This is fine. It is even appropriate. It is, however, not what the phrase used to mean, and nobody has bothered to tell the board.
Worse, the asymmetry inside this picture is widening. Attackers operate without a human-in-any-loop, because no regulator obliges them to. Defenders operate with a human in a loop for compliance reasons that are, increasingly, ceremonial. The result is that the attacker is unconstrained while the defender is constrained, and the constraint produces no actual safety — it produces only the appearance of accountability. We are not, in fact, slower because of human oversight. We are merely required to pretend we are slower, in a particular tone of voice, for the auditor. The auditor knows this. The auditor’s job is also, increasingly, ceremonial.
We are merely required to pretend we are slower, in a particular tone of voice, for the auditor. The auditor knows this. The auditor’s job is also, increasingly, ceremonial.
The judge and the operator
The implication, for any board that is not actively hiding from the question, is that the role of the human in security is undergoing a quiet but profound shift. It is no longer to decide, in real time, what should happen. The machines are doing that. It is to set the policies under which the machines decide, to audit the outcomes, to investigate the edge cases, and to take responsibility — actual responsibility, not pantomime — for what the system was instructed to do. This is closer to the role of a judge than the role of an operator. Most security teams are still organised, recruited, and rewarded as operators. The judges have not yet arrived. The advertisements for them have not yet been placed. The job descriptions have not been written, because the people who would write them are still, themselves, operators.
This has consequences. The skill profile required for security in a machine-speed world is no longer “able to read alerts fast.” It is able to reason about complex automated decisions, design policy at the right level of abstraction, and accept accountability for outcomes the system produced autonomously. There are, at present, approximately fourteen people in the world who can do this well. The other institutions are still hiring SOC analysts and hoping.
There are, at present, approximately fourteen people in the world who can do this well. The other institutions are still hiring SOC analysts and hoping.
The same pattern, everywhere
What is true of security is becoming true of risk, fraud, operations, compliance, and, increasingly, parts of credit and treasury. In each, the same pattern is forming. The decisions are made by software, at speeds that exclude human intervention, monitored by humans who are providing meaningful oversight at the policy level and ceremonial oversight at the transaction level. In each, the regulatory framework was written for a world in which the transaction-level oversight was real. In each, the framework has not been rewritten.
This is not, in the long run, sustainable. It is, in the short run, the situation. The institutions that will navigate the next decade well are not the ones with the most humans-in-the-loop. They are the ones that have stopped pretending the loop is what it used to be.
“The institutions that will navigate the next decade well are not the ones with the most humans-in-the-loop. They are the ones that have stopped pretending the loop is what it used to be.”
= Sumir Nagar
There is a version of this essay that ends in alarm, and a version that ends in resignation, and I am not interested in writing either. The version I want to end with is the one that says: the era of human-as-operator in enterprise defence is closing. This is neither tragic nor triumphal. It is simply true. The interesting question is what role the human plays next, and that question — unlike the previous two in this series — has not yet been answered, because most institutions have not yet asked it.
The answer will not come from buying another tool. The tools are already here. It will come from reorganising governance around the actual decision points — which are now policies, not transactions — and from accepting, at board level, that we have a human reviewing this is no longer the reassurance it once was. The reassurance that matters is we have set the policy correctly, and we can prove the system is operating within it. That is a different kind of governance. It requires different skills, different reporting, different audit processes, and a different relationship between the board and the technology it oversees. None of these are impossible. All of them require admitting that the old reassurance no longer reassures.
The first question to ask, the next time anyone says human-in-the-loop at a meeting you are chairing, is the most uncomfortable one available: which loop, exactly, and at what speed? If the answer is convincing, you have a defence. If the answer is the loop where we look at the report afterwards, you have a different kind of governance, and you should probably start describing it accurately.
This concludes the three-part series. The longer working paper I have been writing for boards — drawing the three essays together and providing a structured set of questions for risk, audit, and technology committees to put to their executives — will be published shortly. If you’d like to read it, leave a note in the comments or write to me directly. I read everything. I reply to most of it. I do not, ever, add anyone to a mailing list. That is a promise of a kind.
If you have read all three essays in this series, you already understand the argument. The remaining question is what you propose to do with it. That, as it happens, is the kind of conversation I am usually available for.
The series
Part 1. Cyber, InfoSec, Operational Risk: Defending in the Wrong Tense
Part 2. The Identities That Do Not Sleep
Part 3. The Last Human in the Loop (this essay)

Leave a Reply