If you’re looking to understand the philosophy that underpins Silicon Valley’s latest gold rush, look no further than OpenAI’s Scarlett Johansson debacle. The story, according to Johansson’s lawyers, goes like this: Nine months ago, OpenAI CEO Sam Altman approached the actor with a request to license her voice for a new digital assistant; Johansson declined. She alleges that just two days before the company’s keynote event last week, in which that assistant was revealed as part of a new system called GPT-4o, Altman reached out to Johansson’s team, urging the actor to reconsider. Johansson and Altman allegedly never spoke, and Johansson allegedly never granted OpenAI permission to use her voice. Nevertheless, the company debuted Sky two days later—a program with a voice many believed was alarmingly similar to Johansson’s.

Johansson told NPR that she was “shocked, angered and in disbelief that Mr. Altman would pursue a voice that sounded so eerily similar to mine.” In response, Altman issued a statement denying that the company had cloned her voice and saying that it had already cast a different voice actor before reaching out to Johansson. (I’d encourage you to listen for yourself.) Curiously, Altman said that OpenAI would take down Sky’s voice from its platform “out of respect” for Johansson. This is a messy situation for OpenAI, complicated by Altman’s own social-media posts. On the day that OpenAI released ChatGPT’s assistant, Altman posted a cheeky, one-word statement on X: “Her”—a reference to the 2013 film of the same name, in which Johansson is the voice of an AI assistant that a man falls in love with. Altman’s post is reasonably damning, implying that Altman was aware, even proud, of the similarities between Sky’s voice and Johansson’s.

On its own, this seems to be yet another example of a tech company blowing past ethical concerns and operating with impunity. But the situation is also a tidy microcosm of the raw deal at the center of generative AI, a technology that is built off data scraped from the internet, generally without the consent of creators or copyright owners. Multiple artists and publishers, including The New York Times, have sued AI companies for this reason, but the tech firms remain unchastened, prevaricating when asked point-blank about the provenance of their training data. At the core of these deflections is an implication: The hypothetical superintelligence they are building is too big, too world-changing, too important for prosaic concerns such as copyright and attribution. The Johansson scandal is merely a reminder of AI’s manifest-destiny philosophy: This is happening, whether you like it or not.

Altman and OpenAI have been candid on this front. The end goal of OpenAI has always been to build a so-called artificial general intelligence, or AGI, that would, in their imagining, alter the course of human history forever, ushering in an unthinkable revolution of productivity and prosperity—a utopian world where jobs disappear, replaced by some form of universal basic income, and humanity experiences quantum leaps in science and medicine. (Or, the machines cause life on Earth as we know it to end.) The stakes, in this hypothetical, are unimaginably high—all the more reason for OpenAI to accelerate progress by any means necessary. Last summer, my colleague Ross Andersen described Altman’s ambitions thusly:

As with other grand projects of the 20th century, the voting public had a voice in both the aims and the execution of the Apollo missions. Altman made it clear that we’re no longer in that world. Rather than waiting around for it to return, or devoting his energies to making sure that it does, he is going full throttle forward in our present reality.

Part of Altman’s reasoning, he told Andersen, is that AI development is a geopolitical race against autocracies like China. “If you are a person of a liberal-democratic country, it is better for you to cheer on the success of OpenAI” rather than that of “authoritarian governments,” he said. He noted that, in an ideal world, AI should be a product of nations. But in this world, Altman seems to view his company as akin to its own nation-state. Altman, of course, has testified before Congress, urging lawmakers to regulate the technology while also stressing that “the benefits of the tools we have deployed so far vastly outweigh the risks.” Still, the message is clear: The future is coming, and you ought to let us be the ones to build it.

Other OpenAI employees have offered a less gracious vision. In a video posted last fall on YouTube by a group of effective altruists in the Netherlands, three OpenAI employees answered questions about the future of the technology. In response to one question about AGI rendering jobs obsolete, Jeff Wu, an engineer for the company, confessed, “It’s kind of deeply unfair that, you know, a group of people can just build AI and take everyone’s jobs away, and in some sense, there’s nothing you can do to stop them right now.” He added, “I don’t know. Raise awareness, get governments to care, get other people to care. Yeah. Or join us and have one of the few remaining jobs. I don’t know; it’s rough.” Wu’s colleague Daniel Kokotajlo jumped in with the justification. “To add to that,” he said, “AGI is going to create tremendous wealth. And if that wealth is distributed—even if it’s not equitably distributed, but the closer it is to equitable distribution, it’s going to make everyone incredibly wealthy.” (There is no evidence to suggest that the wealth will be evenly distributed.)

This is the unvarnished logic of OpenAI. It is cold, rationalist, and paternalistic. That such a small group of people should be anointed to build a civilization-changing technology is inherently unfair, they note. And yet they will carry on because they have both a vision for the future and the means to try to bring it to fruition. Wu’s proposition, which he offers with a resigned shrug in the video, is telling: You can try to fight this, but you can’t stop it. Your best bet is to get on board.

You can see this dynamic playing out in OpenAI’s content-licensing agreements, which it has struck with platforms such as Reddit and news organizations such as Axel Springer and Dotdash Meredith. Recently, a tech executive I spoke with compared these types of agreements to a hostage situation, suggesting they believe that AI companies will find ways to scrape publishers’ websites anyhow, if they don’t comply. Best to get a paltry fee out of them while you can, the person argued.

The Johansson accusations only compound (and, if true, validate) these suspicions. Altman’s alleged reasoning for commissioning Johansson’s voice was that her familiar timbre might be “comforting to people” who find AI assistants off-putting. Her likeness would have been less about a particular voice-bot aesthetic and more of an adoption hack or a recruitment tool for a technology that many people didn’t ask for, and seem uneasy about. Here, again, is the logic of OpenAI at work. It follows that the company would plow ahead, consent be damned, simply because it might believe the stakes are too high to pivot or wait. When your technology aims to rewrite the rules of society, it stands that society’s current rules need not apply.

Hubris and entitlement are inherent in the development of any transformative technology. A small group of people needs to feel confident enough in its vision to bring it into the world and ask the rest of us to adapt. But generative AI stretches this dynamic to the point of absurdity. It is a technology that requires a mindset of manifest destiny, of dominion and conquest. It’s not stealing to build the future if you believe it has belonged to you all along.

“Our focus with AI is to help create more healthy and equitable relationships.” Whitney Wolfe Herd, the founder and executive chair of the dating app Bumble, leans in toward her Bloomberg Live interviewer. “How can we actually teach you how to date?”

When her interviewer, apparently bemused, asks for an example of what this means, Herd launches into a mind-bending disquisition on the future of AI-abetted dating: “Okay, so for example, you could in the near future be talking to your AI dating concierge, and you could share your insecurities. ‘I just came out of a breakup. I have commitment issues.’ And it could help you train yourself into a better way of thinking about yourself. And then it could give you productive tips for communicating with other people. If you want to get really out there, there is a world where your dating concierge could go and date for you with other dating concierges.” When her audience lets out a peal of uneasy laughter, the CEO continues undeterred, heart-shape earrings bouncing with each sweep of her hands. “No, no, truly. And then you don’t have to talk to 600 people. It will then scan all of San Francisco for you and say, These are the three people you really ought to meet.”

What Herd provides here is much more than a darkly whimsical peek into a dystopian future of online dating. It’s a window into a future in which people require layer upon layer of algorithmic mediation between them in order to carry out the most basic of human interactions: those involving romance, sex, friendship, comfort, food. Implicit in Herd’s proclamation—that her app will “teach you how to date”—is the assumption that AI will soon understand proper human behavior in ways that human beings do not. Despite Herd’s insistence that such a service would empower us, what she’s actually describing is the replacement of human courtship rituals: Your digital proxy will go on innumerable dates for you, so you don’t have to practice anything so pesky as flirting and socializing.

[Read: America is sick of swiping]

Hypothetical AI dating concierges sound silly, and they are not exactly humanity’s greatest threat. But we might do well to think of the Bumble founder’s bubbly sales pitch as a canary in the coal mine, a harbinger of a world of algorithms that leave people struggling to be people without assistance. The new AI products coming to market are gate-crashing spheres of activity that were previously the sole province of human beings. Responding to these often disturbing developments requires a principled way of disentangling uses of AI that are legitimately beneficial and prosocial from those that threaten to atrophy our life skills and independence. And that requires us to have a clear idea of what makes human beings human in the first place.

In 1977, Ivan Illich, an Austrian-born philosopher, vagabond priest, and ruthless critic of metastatic bureaucracies, declared that we had entered “the age of Disabling Professions.” Modernity was characterized, in Illich’s view, by the standardization and professionalization of everyday life. Activities that were once understood to be within the competencies of laypeople—say, raising children or bandaging the wounded—were suddenly brought under the purview of technical experts who claimed to possess “secret knowledge,” bestowed by training and elite education, that was beyond the ken of the untutored masses. The licensed physician displaced the local healer. Child psychologists and their “cutting edge” research superseded parents and their instincts. Data-grubbing nutritionists replaced the culinary wisdom of grandmothers.

Illich’s singular insight was that the march of professional reason—the transformation of Western civilization into a technocratic enterprise ruled by what we now call “best practices”—promised to empower us but actually made us incompetent, dependent on certified experts to make decisions that were once the jurisdiction of the common man. “In any area where a human need can be imagined,” Illich wrote, “these new professions, dominant, authoritative, monopolistic, legalized—and, at the same time, debilitating and effectively disabling the individual—have become exclusive experts of the public good.” Modern professions inculcate the belief not only that their credentialed representatives can solve your problems for you, but also that you are incapable of solving said problems for yourself. In the case of some industries, like medicine, this is plainly a positive development. Other examples, like the ballooning wellness industry, are far more dubious.

If the entrenchment of specialists in science, schooling, child-rearing, and so on is among the pivotal developments of the 20th century, the rise of online dating is among the most significant of the 21st. But one key difference between this more recent advancement and those of yesteryear is that websites such as Tinder and Hinge are defined not by disabling professionals with fancy degrees, but by disabling algorithms. The white-coated expert has been replaced by digital services that cut out the human middleman and replace him with an (allegedly) even smarter machine, one that promises to know you better than you know yourself.

[Faith Hill: ‘Nostalgia for a dating experience they’ve never had’]

And it’s not just dating apps. Supposed innovations including machine-learning-enhanced meal-kit companies such as HelloFresh, Spotify recommendations, and ChatGPT suggest that we have entered the Age of Disabling Algorithms as tech companies simultaneously sell us on our existing anxieties and help nurture new ones. At the heart of it all is the kind of AI bait-and-switch peddled by the Bumble CEO. Algorithms are now tooled to help you develop basic life skills that decades ago might have been taken as a given: How to date. How to cook a meal. How to appreciate new music. How to write and reflect. Like an episode out of Black Mirror, the machines have arrived to teach us how to be human even as they strip us of our humanity. We have reason to be worried.

As conversations over the dangers of artificial intelligence have heated up over the past 18 months—largely thanks to the meteoric rise of large language models like ChatGPT—the focus of both the media and Silicon Valley has been on Skynet scenarios. The primary fear is that chat models may experience an “intelligence explosion” as they are scaled up, meaning that LLMs might proceed rapidly from artificial intelligence to artificial general intelligence to artificial superintelligence (ASI) that is both smarter and more powerful than even the smartest human beings. This is often called the “fast takeoff” scenario, and the concern is that if ASI slips out of humanity’s control—and how could it not—it might choose to wipe out our species, or even enslave us.

These AI “existential risk” debates—at least the ones being waged in public—have taken on a zero-sum quality: They are almost exclusively between those who believe that the aforementioned Terminator-style dangers are real, and others who believe that these are Hollywood-esque fantasies that distract the public from more sublunar AI-related problems, like algorithmic discrimination, autonomous weapons systems, or ChatGPT-facilitated cheating. But this is a false binary, one that excludes another possibility: Artificial intelligence could significantly diminish humanity, even if machines never ascend to superintelligence, by sapping the ability of human beings to do human things.

The epochal impact of online dating is there for all to see in a simple line graph from a 2019 study. It shows the explosive growth of online dating since 1995, the year that Match.com, the world’s first online-dating site, was launched. That year, only 2 percent of heterosexual couples reported meeting online. By 2017, that figure had jumped to 39 percent as other ways of meeting—through friends or family, at work or in church—declined precipitously.

Besides online dating, the only way of meeting that increased during this period was meeting at a bar or restaurant. However, the authors of the study noted that this ostensible increase was a mirage: The “apparent post-2010 rise in meeting through bars and restaurants for heterosexual couples is due entirely to couples who met online and subsequently had a first in-person meeting at a bar or restaurant or other establishment where people gather and socialize. If we exclude the couples who first met online from the bar/restaurant category, the bar/restaurant category was significantly declining after 1995 as a venue for heterosexual couples to meet.” In other words, online dating has become hegemonic. The wingman is out. Digital matchmaking is in.

But even those selling online-dating services seem to know there’s something unsettling about the idea that algorithms, rather than human beings, are now spearheading human romance. A bizarre Tinder ad from last fall featured the rapper Coi Leray playing the role of Cupid, perched on an ominously pink stage, tasked with finding a date for a young woman. A coterie of associates, dressed in Hunger Games chic, grilled a series of potential suitors as Cupid swiped left until the perfect match was found. These characters put human faces on an inhuman process.

Leif Weatherby, an expert on the history of AI development and the author of a forthcoming book on large language models, told me that ads like this are a neat distillation of Silicon Valley’s marketing playbook. “We’re seeing a general trend of selling AI as ‘empowering,’ a way to extend your ability to do something, whether that’s writing, making investments, or dating,” Weatherby explained. “But what really happens is that we become so reliant on algorithmic decisions that we lose oversight over our own thought processes and even social relationships. The rhetoric of AI empowerment is sheep’s clothing for Silicon Valley wolves who are deliberately nurturing the public’s dependence on their platforms.” Curtailing human independence, then, is not a bug, but a feature of the AI gold rush.

Of course, there is an extent to which this nurtured dependence isn’t unique to AI, but is an inevitable by-product of innovation. The broad uptake of any new technology generally atrophies the human skills for the processes that said technology makes more efficient or replaces outright. The advent of the vacuum was no doubt accompanied by a corresponding decline in the average American’s deftness with a broom. The difference between technologies of convenience, like the vacuum or the washing machine, and platforms like Tinder or ChatGPT is that the latter are concerned with atrophying competencies, like romantic socializing or thinking and reflection, that are fundamental to what it is to be a human being.

[Read: AI has lost its magic]

The response to our algorithmically remade world can’t simply be that algorithms are bad, sensu stricto. Such a stance isn’t just untenable at a practical level—algorithms aren’t going anywhere—but it also undermines unimpeachably positive use cases, such as the employment of AI in cancer diagnosis. Instead, we need to adopt a more sophisticated approach to artificial intelligence, one that allows us to distinguish between uses of AI that legitimately empower human beings and those—like hypothetical AI dating concierges—that wrest core human activities from human control. But making these distinctions requires us to re-embrace an old idea that tends to leave those of us on the left rather squeamish: human nature.

Both Western intellectuals and the progressive public tend to be hostile to the idea that there is a universal “human nature,” a phrase that now has right-wing echoes. Instead, those on the left prefer to emphasize the diversity, and equality, of varying human cultural traditions. But this discomfort with adopting a strong definition of human nature compromises our ability to draw red lines in a world where AI encroaches on human territory. If human nature doesn’t exist, and if there is no core set of fundamental human activities, desires, or traits, on what basis can we argue against the outsourcing of those once-human endeavors to machines? We can’t take a stand against the infiltration of algorithms into the human estate if we don’t have a well-developed sense of which activities make humans human, and which activities—like sweeping the floor or detecting pancreatic cancer—can be outsourced to nonhuman surrogates without diminishing our agency.  

One potential way out of this impasse is offered by the so-called capability approach to human flourishing developed by the philosopher Martha Nussbaum and others. In rejection of the kind of knee-jerk cultural relativism that often prevails in progressive political thought, Nussbaum’s work insists that advocating for the poor or marginalized, at home or abroad, requires us to agree on universal “basic human capabilities” that citizens should be able to develop. Nussbaum includes among these basic capabilities “being able to imagine, to think, and to reason” and “to engage in various forms of familial and social interaction.” A good society, according to the capability approach, is one in which human beings are not just theoretically free to engage in these basic human endeavors, but are actually capable of doing so.

As AI is built into an ever-expanding roster of products and services, covering dating, essay writing, and music and recipe recommendations, we need to be able to make granular, rational decisions about which uses of artificial intelligence expand our basic human capabilities, and which cultivate incompetence and incapacity under the guise of empowerment. Disabling algorithms are disabling precisely because they leave us less capable of, and more anxious about, carrying out essential human behaviors.

Of course, some will object to the idea that there is any such thing as fundamental human activities. They may even argue that describing behaviors like dating and making friends, critical thinking, or cooking as central to the human condition is ableist or otherwise bigoted. After all, some people are asexual or introverted. Others with mental disabilities might not be adept at reflection, or written or oral communication. Some folks simply do not want to cook, an activity which is historically gendered besides. But this objection relies on a sleight of hand. Identifying certain activities as fundamental to the human enterprise does not require you to believe that those who don’t or can’t engage in them are inhuman, just as embracing the idea that the human species is bipedal does not require you to believe that people born without legs lack full personhood. It only asks that you acknowledge that there are some endeavors that are vital aspects of the human condition, taken in the aggregate, and that a society where people broadly lack these capacities is not a good one.

Without some minimal agreement as to what those basic human capabilities are—what activities belong to the jurisdiction of our species, not to be usurped by machines—it becomes difficult to pin down why some uses of artificial intelligence delight and excite, while others leave many of us feeling queasy.

What makes many applications of artificial intelligence so disturbing is that they don’t expand our mind’s capacity to think, but outsource it. AI dating concierges would not enhance our ability to make romantic connections with other humans, but obviate it. In this case, technology diminishes us, and that diminishment may well become permanent if left unchecked. Over the long term, human beings in a world suffused with AI-enablers will likely prove less capable of engaging in fundamental human activities: analyzing ideas and communicating them, forging spontaneous connections with others, and the like. While this may not be the terrifying, robot-warring future imagined by the Terminator movies, it would represent another kind of existential catastrophe for humanity.

Whether or not the Bumble founder’s dream of artificial-intelligence-induced dalliances ever comes to fruition is an open question, but it is also somewhat beside the point. What should give us real pause is the understanding of AI, now ubiquitous in Big Tech, that underlies her dystopian prognostications. Silicon Valley leaders have helped make a world in which people feel that everyday social interactions, whether dating or making simple phone calls, require expert advice and algorithmic assistance. AI threatens to turbocharge this process. Even if your personalized dating concierge is not here yet, the sales pitch for them has already arrived, and that sales pitch is almost as dangerous as the technology itself: AI will teach you how to be a human.