Is AI Really Trying to Kill US?
A former Anthropic employee and the company’s CEO weighed in, so we looked at their persuasive writing skills. Then we asked Claude.
A few weeks ago, two experts in the field of artificial intelligence — Jacob Coxon, a former researcher at Anthropic; and Dario Amodei, the company’s founder and CEO — published warnings about the existential threat of AI and what should be done about it.
Their stories are both provocative and calls to action, but in very different ways. Here’s our take on how each measures up to the principles of persuasive writing and thought leadership.
Let’s start with story length. Our attention spans are measured in seconds. We don’t want to read anything longer than a post on X. Even videos can’t hold us more than about 30 seconds.
Even if you don’t believe those stats, they weren’t lost on the 28-year-old Coxon, who worked at rival OpenAI before Anthropic. At 7:04 p.m. on Tuesday, Sept. 8, he posted seven times on X, with each entry coming under the platform’s 288-character limit.
Put them all together and you get 348 words — about the average length of a story on your SharePoint homepage that employees complain is too long.
It’s a quick read, though, and Coxon’s content is compelling, at times even frightening. It generated 174 million views, 206,000 reposts — and a ton of news coverage.
Four days later, on Sept. 12, Amodei weighed in with a whopping 3,800-plus word essay on how to stop AI from annihilating us.
Amodei’s piece is thoughtful, but it’s a bit of a slog. That makes us wonder how many people read to the end, when he finally gives us some details on what we should do next. Still, Amodei’s piece generated 76 million views and 27,000 reposts.
Given its brevity, Coxon’s combined posts may have been read in their entirety, but they also set off a panic about AI killing us all before the end of the decade — the one line everyone remembers. Amodei’s essay offers solutions — if you get to them — but they get mired down in technical language and jargon.
Headlines. X doesn’t really lend itself to headlines, so we’ll give Coxon a pass.
Amodei’s piece is titled, “We Must Pace the Frontier.” We’re hearing that term a lot these days, but as a headline it doesn’t convey the magnitude of this issue.
As Ezra Klein wrote in The New York Times, “Pacing the frontier isn’t enough. Walking quickly off a cliff is only marginally better than sprinting off one. Human beings need to control the frontier.”
Ledes. The two pieces take very different paths to drawing readers in. Coxon, with a flurry of urgency, wastes no time:
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.
Well, that gets our attention. It’s emotional and powerful and makes us want to keep reading. That’s what a lede is meant to do.
Amodei’s essay comes at the problem in a deeply personal but roundabout way. He writes:
I believe that AI could cure most major diseases in the next 5-10 years, greatly accelerate economic growth rates, create a world of abundance and empowerment, and usher in a renaissance of democracy and freedom. I feel the urgency personally. My own father died of a disease that was cured just a few years after his death, and I myself survived an early-stage cancer that would not have been treatable even fifty years ago. Carefully wielded, AI can be the latest in a long line of technological miracles that have uplifted and ennobled humanity.
This anecdotal opening is heartfelt. And it works because it hints at what’s coming next:
Nut graph. In news outlets, a nut graph summarizes the story in a nutshell. Amodei offers two more long paragraphs before he gets to, “Yes, but.” He writes about the inherent power of the technology and the risks of losing control of AI systems. He describes how he and his colleagues have focused on the long-term benefits of commercial success while also balancing the need for safety and careful regulation. Then he finally jumps in:
But over the last few months, I have become convinced that fully addressing the risks requires even more prudence — not just investing in risk prevention but pacing the rate of capabilities advancement so that risk prevention has time to keep up. We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain (his boldface).
Coxon is far less subtle:
Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing. . . . The people building AI earnestly believe that it could kill us all by the end of the decade.
Tone. Finally, consider the tone of the pieces for their ability to pull readers in and keep them reading. We find Amodei’s sincerity quite convincing, but about a thousand words into his essay, his tone abruptly shifts from that of an emotional, caring storyteller to, well, a bit of an IT nerd.
No offense to anyone’s IT department, but Amodei’s solutions start to sound like what your IT guys say to you when you complain about some new system not working. They want to appease you without really telling you anything.
That’s probably unfair to most IT people, but that’s how the piece starts to feel. He offers three steps: embedded evaluators in AI companies, establishing safety standards with other AI companies in democratic countries, and coordinating all of this with AI development in authoritarian countries.
So far so good, but then Amodei goes deep, with 17 bullet points further explaining the three steps and why pacing is a good idea. And he throws in all the jargon for good measure — alignment, “unverbalized motivations,” and the one that scares the hell out of everyone: “recursive self-improvement.” (That’s when the robots no longer need us to continue to evolve.)
This part of Amodei’s piece makes our head hurt, despite reading it a dozen times. But in fairness, this is complicated stuff — possibly the most complicated in the history of humanity — and we should all work hard at it to understand. But will we?
Fire alarm
In sharp contrast, Coxon’s prose reads like a five-alarm fire, calling all who care about human civilization to grab any bucket of water they can find and rush to the scene.
He says Anthropic understands the stakes, but blames it for being “locked in a race to get there first.”
Coxon concludes by citing Reinforcement Learning, or RL for short, a powerful way to train AI agents that also poses the most risks. He calls on his fellow researchers to think about what’s coming — and what they can do about it.
Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind? Should you put your head down because “it’s happening anyway” — or take this moment to call for different conditions?
Coxon’s piece is provocative and a little sensational (maybe?). It made us wonder what the AI bots thought of it. So, we asked Anthropic’s AI assistant Claude to edit Coxon.
Here’s Coxon:
We have all witnessed the progress in each of these domains, and progress is not slowing. . . . The people building AI earnestly believe that it could kill us all by the end of the decade.
Claude tries to lower the temperature:
Progress in each of these areas has been visible and hasn’t slowed. Many of the people building these systems believe there’s a serious chance this could go catastrophically wrong within the decade.
And ever the efficient and helpful editor, Claude explained that why some prominent researchers worry about this possibility:
“Kill us all” is the most extreme version of the claim (extinction); many who are worried are more focused on catastrophic-but-non-extinction outcomes, loss of control, or concentration of power.
Non-extinction outcomes? Thanks, Claude. I feel so much better.
As a kid, Jim Ylisela tried to build a robot head but never got past the light bulbs for the eyes. Today, he’s more focused on making sure that writers sound less like robots and more like the beautiful, emotional and flawed humans they are. Want to learn about our Build Better Writers program of workshops and one-on-one editing? Email Senior Consultant Tom Corfman to set up a free call with him and Jim, RCG co-founder and senior partner.
Follow RCG on LinkedIn and subscribe to our weekly newsletter here.
