Arvind Narayanan

Here are 14 points about AI safety I made on the Ezra Klein show, and some impressions about the experience. ​​https://www.nytimes.com/2026/10/09/opinion/ezra-klein-podcast-arvind-narayanan.html 1. Intelligence isn’t power — An A.I. system being powerful is not a property of the model itself. It’s a property of what powers we choose to give it in the real world. 2. The capabilities and behaviors we saw in the HuggingFace and other hacks weren’t entirely emergent. The models were trained for persistence, cooperation, and cyber tasks. The RL environments in which they were trained had various issues that led to the unintended behavior. 3. I agree that controlling highly capable AI agents is different in some ways from controlling an airplane or a nuclear reactor. The agent’s intentionality does make a difference to our approach compared to traditional engineering problems and domains. But it is still an engineering problem. It is not a solved engineering problem, but it is a solvable one. 4. A lot of effort and money has been put toward alignment. That’s important and has proved useful, but if the same amount of effort and money had additionally been put toward control (sandboxes, monitoring, and other guardrails outside the model itself), we’d be seeing very different results. 5. Models getting smarter doesn’t necessarily mean safety gets harder, because capability gains improve defensive abilities too. In many ways it makes safety easier rather than harder. What matters is how the attacker-defender balance shifts, and we have agency to shift it back to defenders. 6. In cybersecurity, long before AI we reached a point where automated systems were superhuman at finding certain vulnerabilities. It helped defenders over attackers because the same tools are used by defenders to find and fix vulnerabilities in software before even shipping. There is a constantly shifting equilibrium, but it is not a new problem. 7. I pushed back on Dario Amodei’s term “operational excellence” to describe recent organizational failures. Excellence is still far out on the horizon. Ezra offered “operational adequacy”. I think that would be a good start — behaving like any mature company instead of a startup-like culture. 8. Defensive AI agents will have to be part of the solution, as uncomfortable as that seems. It’s a new paradigm. We won’t be able to figure it out right away. But the fact that both offensive and defensive technologies escalate in capabilities and complexity is a fact of technological civilization, whether in the digital or physical realm. 9. Ezra expressed some incredulity about my being okay with the ratcheting of complexity. I think it’s fair to be concerned. But safety technologies will never be fully legible to the public. People will have the same reaction if they see the immense complexity of safety in any other high-stakes domain such as nuclear or aerospace. We govern them pretty technocratically. 10. The weirdness of the current moment in cybersecurity absolutely does have historical precedents. There was a prolonged period of profound destabilization when worms and viruses started a new offensive paradigm. It took us well over a decade to develop effective defenses against it. This time we can hopefully get there in less than a decade. 11. I continue to be skeptical about the possibility of superhuman persuasion abilities from superintelligence. (Claims to the contrary tend to confuse two very different senses of the word persuasion.) 12. What worries me is not the capabilities of the technology itself or the possibility that we will get no warnings before catastrophic risks materialize. We are getting plenty of warnings! But our policy and organizational responses to the warnings remain inadequate. …(truncated to API 4000-char source_post limit; full text at source URL)