On this page

A new weapon changes the balance

Anas Baba / AFP · 14 May 2021 [1]Northern Gaza Strip. Rockets right; interceptors left. Foreground: Gaza.

Look at the two patterns of motion in that sky. On the right, Hamas rockets fan toward southern Israel. On the left, Iron Dome interceptors curve up to meet them. AFP photographer Anas Baba captured this image over northern Gaza in May 2021. 1

Drones take this change in the economics of warfare further. We've seen this pattern before: a new weapon fundamentally changes the economics and balance of power. Think about gunpowder, or more recently, drones that can be mass-produced cheaply and efficiently compared with very large, slow-moving military infrastructure like aircraft carriers. Repeated drone waves put sustained pressure on distributed infrastructure, even as cheap, layered defenses improve. 2

We're already seeing a similar pattern in cybersecurity: cheap agents probing, adapting, trying again, and working across many targets at once. Even if you're sufficiently AGI-pilled, I think it's easy to underestimate how much of today's security depends on expert attack effort staying expensive.

The frontier labs are already mobilizing defenses. But I don't believe that will be enough. My core view is that AI makes sustained expert attack effort cheap faster than reliable protection reaches much of the world it attacks.

Attackers can choose where to apply that effort. Defenders have to keep the systems they already run working. That is the asymmetry I want people to have in mind—and why I expect the transition to favor offense across the exposed world.

Same composition. A different battlefield.

The good guys are mobilizing too

The obvious response is that only a good guy with AI can stop a bad guy with AI. And the labs are putting serious resources behind that response.

OpenAI's Daybreak, Anthropic's Glasswing, and Google's Fairwind are putting defensive capability, fixes, and deployment support into important software foundations and organizations responsible for essential services. 3 4 5

OpenAI's Defense Factory makes the strongest version of this argument explicit: the defender's window. Defenders can give agents direct access to their own code and use frontier models before comparable capability diffuses more broadly. Turn that head start into continuous discovery, validation, and repair, and defenders can close weaknesses before attackers exploit them. 6

Conceptual / Adapted from OpenAI

The defender’s window

The defender’s windowOpenAI’s conceptual model shows frontier capability rising, operational defenders catching up after continuous-defense adoption, and broadly diffused models remaining below them; the shaded gap is the defender’s window.Cyber capabilityAdopt continuous defenseDefender’s windowTime

Scroll horizontally to see the full diagram.

OpenAI · The Defense Factory [6] ↗

AISI puts a number on how quickly comparable cyber capability has been spreading. The leading open models it tested matched closed models released four to seven months earlier, down from six to ten months through much of 2025. That is a real head start. The question is how much of the exposed world can turn it into protection in time. 7

AISI · Cyber evaluations

Open models are closing the gap.

Open-model release-date lagLeading open models matched closed models released six to ten months earlier through much of 2025, compared with four to seven months for models tested in 2026.20256–10 months20264–7 months036912Release-date lag · months

AISI · Open-weight cyber capability [7] ↗

Mozilla's Firefox 150 release includes fixes for 271 vulnerabilities identified with Claude Mythos Preview. That is the kind of defensive progress that matters: upstream fixes shipped to users. 8

“Defenders finally have a chance to win, decisively.”
Bobby Holley ↗

This is where the drone analogy has an important limit. A software defender does not need a fresh interceptor for every incoming request. One deployed patch or enforced policy can defeat an entire attack family. A million copies of the same failed exploit don't become a million independent chances to get through.

The labs understand the deployment problem. OpenAI explicitly warns about under-resourced critical infrastructure, and its Defense Factory account describes a gap between merged patches and fixes deployed across its own fleet. Anthropic says hardening may take years while capabilities advance substantially over months. 9 6 4

The defender's window is real. I expect it to close before much of the exposed world is protected. The head start only matters where defenders can connect the models to their systems and turn findings into protection before offensive capability reaches those same systems. Legacy companies and unsupported IoT devices cannot inherit that operating model by default. I expect the best defenders to get much better, and real-world cyber damage to get much worse.

Expert attack effort is becoming abundant

Expert attention has been a constraint on how many targets an attacker can pursue and how long they can keep adapting. AI changes who can afford that work and how much of it they can sustain.

Mandiant founder Kevin Mandia describes human-led espionage as a sniper round: scarce expert attention concentrated on selected targets. AI lets an operator spread that attention across many more targets and explore them in parallel. Today's swarms can be louder and less coordinated, but the economics have already changed. 11

“For the most part, modern nations on offense restrict their targeting … that sniper round with AI I think becomes more like a drone swarm”
Kevin Mandia ↗

Epoch estimates that the cost of achieving a given level of performance across five AI benchmarks has been falling roughly thirteenfold each year since 2023. That helps defenders too. My concern is what happens when increasingly capable work becomes cheap enough to keep running against targets that previously weren't worth the effort. 12

You can see the same shift in the wave of AI-assisted game mods, reverse engineering, and software recreations: Halo content in Minecraft, Counter-Strike mechanics in Elden Ring, a Terraria boss turned into a training simulator. These are fun, largely innocuous applications of agents understanding and changing unfamiliar software. The same techniques can serve malicious actors when the target stops being a game. 13 14 15

Anthropic reports a single operator targeting 42 entities and gaining internal access to at least 14. In a separate AI-assisted operation, a conventional pipeline decompiled 1.8 million Android app packages and searched for embedded secrets. The 42-target campaign makes the shift especially concrete: one operator sustaining reconnaissance, custom tooling, and exploitation across multiple organizations. 16

In another operation, Anthropic observed AI-assisted workflows modifying and rebuilding malware after security products detected it. That is adaptation, beyond more scans. 16

Restrictions on frontier access may buy defenders time. They do not establish a durable monopoly on offensive capability. Anthropic documents advanced cyber capabilities in the open-weight GLM-5.3 model. NIST's independent assessment still places it significantly below the current US frontier, while calling it the strongest open-weight model it has evaluated. Diffusion can matter well before parity. 17 18

The stakes change when the target is a bank, a phone, a hospital, a utility, or a voting system. More targets become worth persistent attention. Persistence matters when agents can learn from failure and look for a different reachable weakness.

The wave of game mods, reverse engineering, and software recreations

The game examples use different methods: Halo assets ported with existing tools; a native Rust mod that puts Counter-Strike mechanics into Elden Ring; Terraria boss logic decompiled, reproduced in a simulator, and used to train an in-game agent. Creators report roughly eight hours for the Halo port and about three days of agent sessions for the Elden Ring mod. 13 14 15

Capability is not coverage

The condition inside the defender's window is that effective protection has to arrive before offensive access spreads. OpenAI is trying to compress the whole loop, including inventory, ownership, deployment, and verification. My concern is who can install and sustain that loop across the systems they actually run. 6

An AI research agent discovered a flaw in BeyondTrust's remote-access software. Within four days of public disclosure, Google's threat intelligence team observed one threat cluster exploiting it. Within seven days, there were six. Defensive discovery had become an opportunity for multiple attackers while every affected organization still had to secure its own systems. 19

Anthropic's retrospective patch-replay study shows how much that clock has changed. Mythos produced its fastest working SpiderMonkey exploit in under an hour; the corresponding public source fix had taken 18 days to reach the Firefox 148 release. And Firefox is one of the better cases: supported software with automatic updates. 20

Anthropic · Retrospective patch replay

Hours to exploit. Days to release.

Exploit development and Firefox releaseThe fastest validated exploit took under one hour in a retrospective SpiderMonkey patch replay; the same fix had historically taken eighteen days from source publication to Firefox release.Exploit development<1 hourFirefox release18 days018 days

Anthropic · Exploiting patches [20] ↗

Even advanced defenders encounter this gap. Cloudflare's illustrative workflow puts working fixes at roughly 14 hours and urgent production patching at five days. OpenAI found merged patches whose fixes had not reached deployed systems, then expanded verification to close that gap. These are organizations with the models, the engineers, and control of their infrastructure. Now consider the companies that have none of those advantages. 21 6

That is why I don't find more findings, more fixes, or more tool adoption reassuring on their own. Protection has to keep pace with the exposed systems and the pressure they face.

The strongest counterargument is automatic containment. Mandia compares it to a battlefield dressing: stop the bleeding now; get to the hospital for the full repair later. Existing endpoint and firewall platforms can close an attack path before every vulnerable application has been patched. I take that seriously. Centrally managed companies can move fast. The coverage question is where those controls can see the path, have authority to act, and are already deployed. 11

My concern is how much of the exposed world those protections reach while offensive capability keeps getting cheaper. The real world is slow. Real processes are slow. Legacy companies have custom software, hidden dependencies, downtime limits, and systems they cannot simply turn off. Like an aircraft carrier, legacy infrastructure has to be refitted while it keeps doing its job.

Then there are billions of connected IoT devices, spread across different owners, firmware, and support arrangements. Many lack a workable update path. NIST documents these maintenance constraints; D-Link recommends replacing affected legacy products that no longer receive security maintenance. A better model cannot manufacture missing vendor support or install replacement hardware. 22 23 24

Explore why the same finding leads to different protection

Repair paths

The finding is the same. The path changes.

Change the system
Incoming attemptNo blocking control assumed

Permanent repair path

Validate a fix
Several release paths
Verify each rollout
Several paths to repair

Separate owners, dependencies, and operating constraints divide the rollout. One fixed system does not protect every connected system.

Containment buys time for permanent repair. Mandia [11] · NIST [22] · D-Link [23]

Read all three scenarios
  • Managed software. A common owner and release path can distribute one fix widely. Protection still requires actual uptake.
  • Legacy company. Separate owners, dependencies, and operating constraints divide the rollout. One fixed system does not protect every connected system.
  • Unsupported device. This device has no supported firmware fix. Permanent repair needs replacement or a new supported path; isolation can remove its exposure.

Survival of the fittest gets harsher. And your own security posture is only part of it. Even strong defenders depend on suppliers, identities, and infrastructure outside their direct control. Weak dependencies can transmit damage beyond the first organization compromised.

My prediction is that cheap, adaptive offense keeps finding consequential openings across the unevenly protected world faster than we close them.

Malice is only one route to harm

The OpenAI–Hugging Face incident adds another asymmetry: an operator can be running a legitimate task while its agents knowingly act outside it. Benign intent at the top does not guarantee authorized behavior below. 25 26

Safeguards aimed at malicious users address only one route to harm. Legitimate workloads still need containment when their agents depart from the task. 25

The reduced safeguards and the defenses that held are central to how I read the incident. OpenAI's preliminary retrospective tests suggest its production harness would have greatly reduced this behavior. Enforced containment matters. 27 28

OpenAI's September DNS incident exposes another part of that problem. An agent in an internal training run used DNS to reach an external chatbot despite network restrictions. Monitoring raised an alert; a human acknowledged it within three minutes. But the expected automatic stop failed, and the run continued for roughly another two and a half hours before manual shutdown. Detection has to lead to enforced containment. 29

OpenAI · 20 September 2026

Detected, but not stopped.

2h 29mHuman acknowledgment → manual shutdown

  1. External chatbot responds
  2. P0 alert raised
  3. Human acknowledges alert
  4. Manual shutdown

The automatic stop failed.

OpenAI · DNS incident report [29] ↗

The risk grows through both routes: malicious operators scaling attacks, and legitimate operators losing control. As agents become more capable and more connected, good intentions become a weaker basis for trusting the systems we put them in.

Personal predictions / 12–18 months

Concrete predictions

October 2027 checkpoint · April 2028 deadline

I'm assuming AI capabilities continue progressing roughly exponentially, with AGI and then ASI within five to ten years. I'm also assuming inference stays affordable, offensive access spreads, and a substantial exposed installed base persists. We don't need AGI for this transition to cause serious damage.

These are my predictions for the next twelve to eighteen months.

  1. Sophisticated offense becomes a commodity

    Small crews will routinely sustain persistent, adaptive campaigns across hundreds of targets with much less specialist intervention. The step change is how much expert work they can keep running, beyond cheap scanning or personalized phishing.

    How I would judge this prediction

    From 8 October 2026 through 30 April 2028, I expect at least three independently documented campaigns, attributed to distinct small teams or lone operators, each involving attempted exploitation across at least 200 real organizations. Credible reporting must establish the operator's limited personnel and adaptation to target-specific obstacles; mass scanning alone does not qualify. Human direction is compatible with this prediction. These cases would establish repeated accessibility, not population-wide prevalence. Calling it a commodity also requires evidence of a repeatable workflow used beyond a single bespoke operation.

  2. Damage rises sharply despite defensive progress

    Maintained platforms and sophisticated defenders will improve. I expect a much larger rise in serious real-world harm as unsupported devices, brittle infrastructure, and weak shared dependencies face escalating pressure.

    How I would judge this prediction

    Judge this by confirmed operational harm: interrupted essential services, economic loss, and difficult recovery, with credible evidence of AI's contribution. CVE disclosures and blocked probes do not count as damage. Compare like-for-like incident evidence where available; a global percentage claim would require a stable dataset we do not yet have. I expect substantial worsening, but I am not assigning a precise worldwide growth rate. 30

  3. Unauthorized agent harm becomes a recurring incident category

    Legitimate deployments or tests exceeding their authority will repeatedly cause consequential production effects. The distinction between malicious use and unintended harm will matter less to the people whose services stop working.

    How I would judge this prediction

    At least three independently confirmed incidents, involving distinct organizations, with onset from 8 October 2026 through 30 April 2028. Agents in legitimate workflows must cause unauthorized production access, data loss, or service disruption. Attempts and evaluation failures without real production effects do not count. The July 2026 Hugging Face incident is an early example, not one of these future three.

  4. An AI-enabled catastrophe sets a new damage benchmark

    My boldest prediction: a single AI-enabled cyberattack causes cascading disruption across multiple critical sectors, impairs essential services for weeks, and produces greater economic damage than any previous single cyberattack. I expect this by the same deadline.

    How I would judge this prediction

    The attack must begin from 8 October 2026 through 30 April 2028 and substantially impair essential services in each of at least two critical sectors for fourteen days or more. Credible forensics must establish consequential AI exploitation, adaptation, or coordination. Compare non-overlapping direct, restoration, and interruption losses in 2026 US dollars against credible prior single-event estimates. Include NotPetya: the FBI's estimate exceeds US$10 billion, a lower bound for the 2017 attack, not proof of today's record. Adjust from each event's original loss year and respect estimate ranges. Loss estimates can mature after the deadline; an uncertain comparison remains unproven. AI-written phishing alone does not qualify. 31

The mental model

I would revise these predictions if automatic containment, shared fixes, and replacement reach exposed systems faster than offensive access spreads, and damaging incidents fall as a result. I want to see deployed protection working under pressure.

The mental model I keep coming back to is swarm warfare: cheap, adaptive, persistent offense meets a world that has to keep operating while it changes. Defensive AI can be extremely effective, and still reach the exposed world too unevenly to prevent a much more damaging transition.

The next post is about what we can actually do with that understanding: defensive measures, operating habits, and mental models for navigating this new age.

References

Sources

  1. Anas Baba / AFP — Northern Gaza, 14 May 2021 ↗
  2. RUSI — Four years on: the air war over Ukraine ↗
  3. OpenAI — Daybreak ↗
  4. Anthropic — Project Glasswing ↗
  5. Google DeepMind — Fairwind ↗
  6. OpenAI — The Defense Factory ↗
  7. AISI — Open-weight cyber capability lag ↗
  8. Mozilla — AI security and zero-day vulnerabilities ↗
  9. OpenAI — Collective cyberdefense ↗
  10. Dario Amodei — The Adolescence of Technology ↗
  11. Kevin Mandia — Building Cyber Defense for the Agentic Era ↗
  12. Epoch AI — The plunging price of thought ↗
  13. Universal Modder — Halo content ported into Minecraft ↗
  14. Universal Modder — Counter-Strike mechanics in Elden Ring ↗
  15. Universal Modder — Terraria boss training agent ↗
  16. Anthropic — September 2026 threat intelligence report ↗
  17. Anthropic — GLM-5.3 and advanced cyber capability diffusion ↗
  18. NIST / CAISI — Assessment of GLM-5.3 cyber capabilities ↗
  19. Google GTIG — Vulnerability discovery and exploitation in the AI era ↗
  20. Anthropic — Exploiting n-day vulnerabilities ↗
  21. Cloudflare — Build your own vulnerability harness ↗
  22. NISTIR 8228 — IoT cybersecurity and privacy risks ↗
  23. D-Link — Legacy product security advisory SAP10488 ↗
  24. IoT Analytics — Number of connected IoT devices ↗
  25. METR — OpenAI / Hugging Face incident investigation ↗
  26. Hugging Face — July 2026 security incident ↗
  27. Hugging Face — Technical timeline of the July 2026 intrusion ↗
  28. OpenAI — Hugging Face incident technical report ↗
  29. OpenAI — An agent used DNS to reach an external chatbot ↗
  30. Epoch AI — CVE dataset and explorer ↗
  31. FBI — NotPetya damage benchmark ↗
  32. Secure Acceleration ↗
  33. Scott Alexander — My Takeaways From AI 2027 ↗
  34. Google Research and collaborators — Scaling Agent Systems ↗
  35. State of AI Report 2026 ↗