AI might ‘kill us all.’ That’s still a savvy sales pitch

The latest frontier model, with apparent human-level faculties, had gone rogue – displaying dangerous, even devious online behaviors. Leading AI researchers and executives warned of the “profound risks to society and humanity” that could radically alter “the history of life on Earth.” Absent any government regulation and amidst an “out-of-control” corporate rush into the unknown, they called for an immediate moratorium on developing more powerful AI systems.

This was, of course, three-and-half years ago.

Perhaps more relevant, these concerns arrived hundreds of billions of investment dollars prior to the latest AI-extinction panic.

Last month, Jacob Coxon, the Anthropic researcher whose resignation post went viral and made headlines, issued pleas not unlike those of the “Pause Giant AI Experiments” open letter signed by Elon Musk, Steve Wozniak, and others in March 2023. Coxon admonished the wild-eyed pursuit of superintelligence that’s “gambling with our lives.”

“Do not underestimate the power of this technology,” he wrote on X. “The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.”

And, yet, the two are not mutually exclusive: The risks can be real, as can the rewards that come with a trillion-dollar I.P.O., which both OpenAI and Anthropic are eyeing. It can be both alarmist and advertising.

After all, the most valuable companies in history surely need to be selling the most powerful technologies in history. What else would all that capital be paying for? And what could be more powerful than an extinction-level technology?

If AI can wipe out civilization, it can surely handle some work emails on your behalf.

As it’s spread relentlessly in the years since that open letter, artificial intelligence has been called many things: “an idea, an infrastructure, an industry.” But because it’s a giant market wager on life in the future, it must also be a massive, ongoing sales pitch.

That sales pitch has only grown more frenzied as the economics of AI have gotten more precarious. It is an unfathomably expensive technology, with an (as-yet) speculative business model, simultaneously propping up the U.S. economy and also torchingtech company cash by the bushel.

Those bets only pay out if we, as humans, agree to adopt AI into every aspect of our thinking lives, both professionally and personally. And we’ll only agree to that if the technology is good enough – and not just “doesn’t hallucinate” good enough.

Like, “kill us all” good enough.

Hence, the doomsday handwringing of Coxon, Dario Amodei, and others in Silicon Valley is part of that sell, intentional or not.

As a professor who studies media and culture, the commercial narratives around our technologies are as consequential as the technologies themselves. In my most recent book, I focused on the notion of “authenticity” as pivotal to social media’s ascendence and ubiquity over the past two decades.

YouTube and Instagram would never have amassed fortunes for their parent companies if they couldn’t be framed as platforms of genuine self-performance: As we produced content for them, they hoped to induce authenticity from us. Similarly, other research showed how marketing to the millennial generation meant conceptualizing that cohort as being eager to divulge their lives online.

In short, you can’t sell a technology – and, in particular, a deeply unpopular one like AI – unless you authenticate it into people’s lives.

Of late, the ads on TV are doing that ideological work. Mountains of money are being plowed into 30-spots that tell us to let AI organize our book clubs, write our children’s fan letters, and handle our eulogies. Because artificial intelligence is, by definition, artificial (i.e., “fake”), it must be sold as acceptably “authentic” for all those purposes.

Studying more than seventy such commercials that have aired in recent years, I found themes of entrepreneurialempowerment and efficiencyfetish; bodily enhancement and social companion.

Most emblematic (and presumptuous), though, is messaging that sells AI as inevitable, because only a stick-in-the-mud Luddite would foolishly inveigh against inevitability.

Hence, to take one example, OpenAI’s Super Bowl spot, which shuffles through pixelated representations of inventions throughout human history and places AI explicitly in lineage with the spear, fire, railroad, flight, DNA, and space travel – though not (apparently!) the atom bomb. “All progress has a starting point,” the ad concludes.

Anthropic’s Coxon wrote something similar in his resignation, “We have all witnessed the progress in each of these domains, and progress is not slowing.”

That’s the fundamental faith one must abide in Silicon Valley: a belief in the inexorability of the project.

How else to explain the fact that these latest calls of AI danger and panic – calls coming from “inside the house,” in the horror film sense – will almost surely stop nothing? After all, they stopped nothing three years ago when 1,000 intellectual luminaries and corporate oligarchs put their name to paper demanding much the same.

For those oligarchs, the money to be made must surely be the compensation – financial, if not moral – for making the thing that could kill us all. And that alleged risk, again, helps seal the deal, if you’re investing in (and therefore betting on) the power of such tools.

But that’s not the only rhetorical benefit.

Extinction by way of HAL 9000 or Skynet – a dramatic, cinematic evocation – obscures the actual, slower-moving environmental devastation of AI when it comes to electricity, water, and other resources. It also misdirects our attention from the (equally slow-moving) cognitive devastation that’s happening in education as students outsource habits and muscles of thought, as AI asks of them.

Ironically, as AI tantalizes with the prospect of doing more, it will also leave us able to do less: a deskilling inherent to any act of outsourcing. If we accept the synthetic product generated – where AI thinks, expresses, and feels on our behalf – we’ll have abandoned something more fundamentally human than automation ever took from us before.

For now – in an era of unregulated free-for-all where tech companies do what they want, because “can” is the only policy verb that matters – normsare all that we have to guard against that substitution. Advertising and public relations are attempting to shape those norms, as they have for more than a century on many a product before.

Sometimes that effort is successful; other times, Mark Zuckerberg has to write off an estimated $80 billion loss on a Metaverse party that no one wanted to show up for.

And if the AI endgame, indeed, rolls the dice on a 10% chance of annihilation, well, the United States can’t let China get there first, can we? Even as Anthropic ostensibly pleads for guardrails because of AI’s peril, national security anxieties will override any caution – becauseof that same potential power.

Turns out that “if anyone builds it, everyone dies” also functions as a savvy sales pitch necessitating precisely the action that’s cautioned against.

Over the decades, there is a literal seasonality to AI discourse: “springs” are those years full of hype and investment; “winters,” by contrast, when momentum and money withdraw. Likewise, many a technology before – from electricity and the telegraph to phone and radio – have been heralded as sublime, magical, revolutionary, and terrifying.

That’s not to say this time isn’t different – that this technology is, finally, sufficiently advanced to truly spook the technical geniuses close to the problem.

But we have heard the story before. And there’s still untold riches that await those who tell it convincingly.

The opinions expressed in Fortune.com commentary pieces are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Fortune.