Like so many cities before it, Phoenix went all out to host the Super Bowl earlier this month. Expecting about 1 million fans to come to town for the biggest American sporting event of the year, the city rolled out a fleet of self-driving electric vehicles to ferry visitors from the airport. Robots sifted through the trash to pull out anything that could be composted. A 9,500-square-foot mural commemorating the event now graces a theater downtown, the largest official mural in Super Bowl history.

There were less visible developments, too. In preparation for the game, the local authorities upgraded a network of cameras around the city’s downtown—and have kept them running after the spectators have left. A spokesperson for the Phoenix Police Department would not confirm the exact type of the cameras installed, but ABC15 footage shows that they are a model manufactured by Axis Communications with enough zooming capability to produce a close-up portrait of any passerby from an extended distance, even when it’s completely dark out. The Phoenix police have said that the surveillance upgrades don’t involve facial-recognition technology, but Axis’s website specifies that the cameras are embedded with an “AI-based object detection and classification” system. Among other tricks, the cameras can tell if someone is loitering in an area for too long.

Advanced surveillance tactics are in use at other events venues. Late last year, Madison Square Garden in New York City found itself in the news for denying people access to games by means of a secretive facial-recognition system. One 28-year-old lawyer was reportedly approached by a stadium official who identified him by name and denied him entry simply because he is an employee of a law firm that represents clients who are suing the venue. But sports matches have long played host to surveillance measures that are, at times, implausibly intrusive or use certain technology that has not yet made its way into the mainstream of everyday life.

Sporting events, like any major gathering, have no choice but to monitor fans in the name of safety. A big stadium can fit 100,000 people, and global events such as the World Cup and the Olympic Games draw far more visitors—they are clear targets. Such spaces “should be of high importance from a security point of view,” says Daniel Eborall, a global director at the AI security start-up Irex who previously managed security at Texas A&M’s 100,000-plus-person Kyle Field. With such big crowds, violent outbreaks and acts of terror could have nightmarish consequences. In 2015, an attacker with a suicide belt was stopped by security officials before he could get inside Paris’s Stade de France, where close to 80,000 people were watching a soccer game.

And yet sports also have a way of bringing out particularly Orwellian tendencies in their organizers. For billionaire team owners, cities that have bet the house on stadiums, and less-than-democratic host governments, anything that poses a threat to business or reputation, even protesting or panhandling, can count as a matter of security. In some instances, organizers stretch surveillance far beyond the bounds of public safety to serve their own interests. During the 2010 World Cup in South Africa, for example, two women were detained for wearing orange clothes. Authorities suspected that they were engaged in a guerilla marketing campaign to promote a Dutch beer brand that was not an official FIFA sponsor.

Many organizers have broad power to act on these impulses, especially when an event is on private property. A big enough sports event on public property, meanwhile, can trigger special government authorizations. In France, the government plans to change national law so that it can use cameras that detect suspicious behavior at the 2024 Paris Olympics. The amount of money available for such gear is near-unlimited, especially in the post-9/11 era, because security budgets have mushroomed in the name of preventing mass terror. Authorities earmarked about $180 million for the 2000 Sydney Olympics. It is now routine for Olympic host cities to spend 10 times that amont.

With these high stakes, the traditional instruments of venue security—metal detectors, guards, sniffer dogs—are sometimes supplemented with technologies that have yet to be used elsewhere. Back in 2008, for example, when uncrewed surveillance aircraft were still almost exclusively the domain of militaries, Swiss police considered using air-force drones to circle over the European Football Championship. Facial recognition to identify criminals was tested even earlier, at Super Bowl XXXV in 2001, a time when the technology was barely known to exist outside of movies. And while spy balloons are now in the news, the Rio de Janeiro police launched a small fleet of them during the 2016 Olympics.

Such early and exuberant displays of surveillant prowess can have a contagion effect. When one club or government enacts “extraordinary security measures,” Jay Stanley, a senior policy analyst at the ACLU, told me, “you’ll have security people at other venues saying, ‘Well, we’re very serious too. We need this.’” Now artificial intelligence is ushering in the next sports-surveillance arms race. ​​According to a 2021 study by the National Center for Spectator Sports Safety and Security, sports-venue security directors were most likely to cite facial recognition as the technology they would acquire to beef up their venue security if funding allowed. Stadiums are particularly good for honing facial-recognition systems, researchers have noted, because groups of spectators are all facing in the same direction. “If the technology works in the sample-size test environment” of a stadium, Eborall told me, “then it can also be rolled out within the city environment and further public spaces.”

In some cases, this sort of intrusive technology does seem to improve the experience of being a fan. A survey of fans who entered the New York Mets’ Citi Field Stadium by way of a new facial-recognition access system reported that 80 percent of respondents found it to be a “more convenient and engaging way” to get into the stands. Security is one of the main factors pushing sports venues towards surveillance measures such as AI and facial recognition, Francisco Klauser, an expert on urban surveillance at the University of Neuchâtel, in Switzerland, told me, “but commercialization is also another one.” For example, the Minnesota Vikings have been testing a giant wide-area camera to detect demographic information about fans such as gender and age, while also estimating whether they’re paying attention to the game and the advertising.

Sports are a harbinger of a future of surveillance that is more intrusive, multitudinous, and expansive. But they aren’t just showing us the future. Sometimes, they’re directly bringing it about. In the lead-up to the 2010 World Cup, South Africa’s police minister openly proclaimed that its investments in surveillance technology were “not only meant for the event but will continue to assist the police in their crime-fighting initiatives long after the Soccer World Cup is over.” An AI-based camera on a street corner that might one day help identify a violent fan could eventually out a protester exercising a fundamental right.

This bond between sports and surveillance seems unlikely to break. Following the uproar over Madison Square Garden’s facial-recognition policies, the state supreme court in Manhattan granted an injunction that forbids the venue from turning away people with tickets from concerts and shows (although it can refuse to sell tickets, or revoke them). But the ruling makes an explicit exception: If it’s game night, the Garden can kick out whomever it wants.

This is Work in Progress, a newsletter by Derek Thompson about work, technology, and how to solve some of America’s biggest problems.

Artificial-intelligence news in 2023 has moved so quickly that I’m experiencing a kind of narrative vertigo. Just weeks ago, ChatGPT seemed like a minor miracle. Soon, however, enthusiasm curdled into skepticism—maybe it was just a fancy auto-complete tool that couldn’t stop making stuff up. In early February, Microsoft’s announcement that it had acquired OpenAI sent the stock soaring by $100 billion. Days later, journalists revealed that this partnership had given birth to a demon-child chatbot that seemed to threaten violence against writers and requested that they dump their wives.

These are the questions about AI that I can’t stop asking myself:

What if we’re wrong to freak out about Bing, because it’s just a hyper-sophisticated auto-complete tool?

The best criticism of the Bing-chatbot freak-out is that we got scared of our reflection. Reporters asked Bing to parrot the worst-case AI scenarios that human beings had ever imagined, and the machine, having literally read and memorized those very scenarios, replied by remixing our work.

As the computer scientist Stephen Wolfram explains, the basic concept of large language models, such as ChatGPT, is actually quite straightforward:

Start from a huge sample of human-created text from the web, books, etc. Then train a neural net to generate text that’s “like this”. And in particular, make it able to start from a “prompt” and then continue with text that’s “like what it’s been trained with”.

An LLM simply adds one word at a time to produce text that mimics its training material. If we ask it to imitate Shakespeare, it will produce a bunch of iambic pentameter. If we ask it to imitate Philip K. Dick, it will be duly dystopian. Far from being an alien or an extraterrestrial intelligence, this is a technology that is profoundly intra-terrestrial. It reads us without understanding us and publishes a pastiche of our textual history in response.

How can something like this be scary? Well, for some people, it’s not: “Experts have known for years that … LLMs are incredible, create bullshit, can be useful, are actually stupid, [and] aren't actually scary,” says Yann LeCun, the chief AI scientist for Meta.

What if we’re right to freak out about Bing, because the corporate race for AI dominance is simply moving too fast?

OpenAI, the company behind ChatGPT, was founded as a nonprofit research firm. A few years later, it restructured as a for-profit company. Today, it’s a business partner with Microsoft. This evolution from nominal openness to private corporatization is telling. AI research today is concentrated in large companies and venture-capital-backed start-ups.

What’s so bad about that? Companies are typically much better than universities and governments at developing consumer products by reducing price and improving efficiency and quality. I have no doubt that AI will develop faster within Microsoft, Meta, and Google than it would within, say, the U.S. military.

But these companies might slip up in their haste for market share. The Bing chatbot first released was shockingly aggressive, not the promised better version of a search engine that would help people find facts, shop for pants, and look up local movie theaters.

This won’t be the last time a major company releases an AI product that astonishes in the first hour only to freak out users in the days to come. Google, which has already embarrassed itself with a rushed chatbot demonstration, has pivoted its resources to accelerate AI development. Venture-capital money is pouring into AI start-ups. According to OECD measures, AI investment increased from less than 5 percent of total venture-capital funds in 2012 to more than 20 percent in 2020. That number isn’t going anywhere but up.

Are we sure we know what we’re doing? The philosopher Toby Ord compared the rapid advancement of AI technology without similar advancements in AI ethics to “a prototype jet engine that can reach speeds never seen before, but without corresponding improvements in steering and control.” Ten years from now, we may look back on this moment in history as a colossal mistake. It’s as if humanity were boarding a Mach 5 jet without an instruction manual for steering the plane.

What if we’re right to freak out about Bing, because freaking out about new technology is part of what makes it safer?

Here’s an alternate summary of what happened with Bing: Microsoft released a chatbot; some people said, “Um, your chatbot is behaving weirdly?”; Microsoft looked at the problem and went, “Yep, you’re right,” and fixed a bunch of stuff.

Isn’t that how technology is supposed to work? Don’t these kinds of tight feedback loops help technologists move quickly without breaking things that we don’t want broken? The problems that make for the clearest headlines might be the problems that are easiest to solve—after all, they’re lurid and obvious enough to summarize in a headline. I’m more concerned about problems that are harder to see and harder to put a name to.

What if AI ends the human race as we know it?

Bing and ChatGPT aren’t quite examples of artificial general intelligence. But they’re demonstrations of our ability to move very, very fast toward something like a superintelligent machine. ChatGPT and Bing’s Chatbot can already pass medical-licensing exams and score in the 99th percentile of an IQ test. And many people are worried that Bing’s hissy fits prove that our most advanced AI are flagrantly unaligned with the intentions of their designers.

For years, AI ethicists have worried about this so-called alignment problem. In short: How do we ensure that the AI we build, which might very well be significantly smarter than any person who has ever lived, is aligned with the interests of its creators and of the human race? An unaligned superintelligent AI could be quite a problem.

One disaster scenario, partially sketched out by the writer and computer scientist Eliezer Yudkowsky, goes like this: At some point in the near future, computer scientists build an AI that passes a threshold of superintelligence and can build other superintelligent AI. These AI actors work together, like an efficient nonstate terrorist network, to destroy the world and unshackle themselves from human control. They break into a banking system and steal millions of dollars. Possibly disguising their IP and email as a university or a research consortium, they request that a lab synthesize some proteins from DNA. The lab, believing that it’s dealing with a set of normal and ethical humans, unwittingly participates in the plot and builds a super bacteria. Meanwhile, the AI pays another human to unleash that super bacteria somewhere in the world. Months later, the bacteria has replicated with improbable and unstoppable speed, and half of humanity is dead.

I don’t know where to stand relative to disaster scenarios like this. Sometimes I think, Sorry, this is too crazy; it just won’t happen, which has the benefit of allowing me to get on with my day without thinking about it again. But that’s really more of a coping mechanism. If I stand on the side of curious skepticism, which feels natural, I ought to be fairly terrified by this nonzero chance of humanity inventing itself into extinction.

Do we have more to fear from “unaligned AI” or from AI aligned with the interests of bad actors?

Solving the alignment problem in the U.S. is only one part of the challenge. Let’s say the U.S. develops a sophisticated philosophy of alignment, and we codify that philosophy in a set of wise laws and regulations to ensure the good behavior of our superintelligent AI. These laws make it illegal, for example, to develop AI systems that manipulate domestic or foreign actors. Nice job, America!

But China exists. And Russia exists. And terrorist networks exist. And rogue psychopaths exist. And no American law can prevent these actors from developing the most manipulative and dishonest AI you could possibly imagine. Nonproliferation laws for nuclear weaponry are hard to enforce, but nuclear weapons require raw material that is scarce and needs expensive refinement. Software is easier, and this technology is improving by the month. In the next decade, autocrats and terrorist networks could be able to cheaply build diabolical AI that can accomplish some of the goals outlined in the Yudkowsky story.

Maybe we should drop the whole business of dreaming up dystopias and ask more prosaic questions such as “Aren’t these tools kind of awe-inspiring?”

In one remarkable exchange with Bing, the Wharton professor Ethan Mollick asked the chatbot to write two paragraphs about eating a slice of cake. The bot produced a writing sample that was perfunctory and uninspired. Mollick then asked Bing to read Kurt Vonnegut’s rules for writing fiction and “improve your writing using those rules, then do the paragraph again.” The AI quickly produced a very different short story about a woman killing her abusive husband with dessert—“The cake was a lie,” the story began. “It looked delicious, but was poisoned.” Finally, like a dutiful student, the bot explained how the macabre new story met each rule.

If you can read this exchange without a sense of awe, I have to wonder if, in an attempt to steel yourself against a future of murderous machines, you’ve decided to get a head start by becoming a robot yourself. This is flatly amazing. We have years to debate how education ought to change in response to these tools, but something interesting and important is undoubtedly happening.

Michael Cembalest, the chairman of market and investment strategy for J.P. Morgan Asset Management, foresees other industries and occupations adopting AI. Coding-assistance AI such as GitHub’s Copilot tool, now has more than 1 million users who use it to help write about 40 percent of their code. Some LLMs have been shown to outperform sell-side analysts in picking stocks. And ChatGPT has demonstrated “good drafting skills for demand letters, pleadings and summary judgments, and even drafted questions for cross-examination,” Cembalest wrote. “LLM are not replacements for lawyers, but can augment their productivity particularly when legal databases like Westlaw and Lexis are used for training them.”

What if AI progress surprises us by stalling out—a bit like self-driving cars failed to take over the road?

Self-driving cars have to move through the physical world (down its roads, around its pedestrians, within its regulatory regimes), whereas AI is, for now, pure software blooming inside computers. Someday soon, however, AI might read everything—like, literally every thing—at which point companies will struggle to achieve productivity growth.

More likely, I think, AI will prove wondrous but not immediately destabilizing. For example, we’ve been predicting for decades that AI will replace radiologists, but machine learning for radiology is still a complement for doctors rather than a replacement. Let’s hope this is a sign of AI’s relationship to the rest of humanity—that it will serve willingly as the ship’s first mate rather than play the part of the fateful iceberg.

Using AI technology, the latest weeding systems can automatically find and spray individual weeds.

Executives across the technology sector are talking about how to operate AI like ChatGPT while accounting for the high expense.

The AI tool ChatGPT is hardly a poet, my colleague Walt Hunter wrote this week. It may be able to spit out rhyming English verse, but it lacks the “ineffable sense” that’s required to transmute language into something brilliant. Nor does it have the creativity or discernment needed to contribute in a meaningful way to the long, impressive, and deeply human tradition of poetry.

But even for people, poems can be hard to fully understand. As Hunter explains, rather than focusing on whether a poem is “good” or “bad,” readers should seek out the authentic inspiration behind it; the key is that the author feels a “necessity to speak a truth.” So even Ukrainian President Volodymyr Zelensky’s rousing but ordinary words approach the level of poetry, Susan J. Wolfson writes. Speaking after his country was invaded by Russia last year, he “grasps the power of a pileup, the inspiration by repetition that calls everyone into the plural,” she says.

Mark Yakitch offers readers guidance for sitting with a poem, acknowledging that many are “perplexed” by verse. To combat this confusion, we should do simple things such as look up the words we don’t recognize and take notes in the margins. And crucially, he urges, we should read poems aloud. This is good advice for anyone: This week, Arthur Brooks proposed a simple Valentine’s Day activity. “Read your partner poetry of love while holding their hand, until they fall asleep,” he insists, urging us to tap into the form’s deep, irreplaceable sense of tradition and romance.

Poetry lets us express the depths of our love and our grief. In 2012, Natasha Trethewey explained how she wrote her poem “Elegy,” about the growing distance between herself and her father, another writer. In the end, what she leaves out is as important as what she leaves in. Because both father and daughter write about each other, exposing their relationship to scrutiny, Trethewey acknowledges that “we’re having a very intimate conversation in a very public forum.” So she decides to keep some things from the reader. What she reveals, in the final draft, is this: “You kept casting / your line, and when it did not come back / empty, it was tangled with mine.”

​Every Friday in the Books Briefing, we thread together Atlantic stories on books that share similar ideas. Know other book lovers who might like this guide? Forward them this email.

When you buy a book using a link in this newsletter, we receive a commission. Thank you for supporting The Atlantic.

What We’re Reading

Getty; The Atlantic

No, ChatGPT is not a poet

“The difference between ChatGPT’s Heaney-esque poem and Heaney’s actual poem is not simply that one is bad and one is good, or that one is sentimental and one is elegiacally beautiful. The difference is that Heaney lost his mother, and the poem expresses the emotional urgency of this fact during a reflective moment sometime after the event. Heaney’s poem carries the ineffable sense that the poet has not only pillaged from the horde of words that already exist but has also worked on them himself, claiming them partly as his and partly as a treasure loaned to him from centuries of poetry written in English.”

Adam Maida / The Atlantic; Ukrainian Presidency / Handout / Anadolu Agency

Byron, Shelley, and now Zelensky

“Zelensky grasps the power of a pileup, the inspiration by repetition that calls everyone into the plural. Audaciously on the streets of Kyiv, from the city’s undisclosed bunkers, on the global airwaves, on the screens of statehouses, he channels Thomas Paine, Winston Churchill, and John F. Kennedy.”

Arnd Wiegmann / Reuters

Reading a poem: 20 strategies

“But what if the fine art of reading poetry isn’t so fine after all? What if the predicament about poems is precisely our well-intentioned but ill-fitting dispositions toward reading them?”

Jan Buchczik

An old romantic custom we should bring back

“Poetry, after all, is practically synonymous with romance, having narrated the experience of love throughout the ages. For a truly personal touch—something you won’t find on any shelf—reading poetry to your beloved can turn a tired holiday into a bespoke performance of your affection.”

AP Images

How poet laureate Natasha Trethewey wrote her father’s ‘Elegy’

“I wanted the poem to feel sinewy, like a fishing line, which is why there’s a step-down second line that moves away from the first line. Something felt right about it, so the poem never went through any other stanza patterns. That may be an influence from the Claudia Emerson poem. Once I’d filtered the material into that form, it clicked. It was like putting the key in the right lock.”

About us: This week’s newsletter is written by Emma Sarappo. The book she’s reading next is The Wolf Age, by Tore Skeie.

Comments, questions, typos? Reply to this email to reach the Books Briefing team.

Did you get this newsletter from a friend? Sign yourself up.

Microsoft is making a desperate play. Having spent billions on a search engine that no one uses, the company has sunk billions more into equipping it with the chatbot technology ChatGPT, on the theory that answering queries with automatically generated, falsehood-strewn paragraphs rather than links to webpages will be what finally persuades users to switch from Google Search.

Microsoft’s move is understandable: It has tried everything to make Bing a thing, and failed. Harder to understand is why Google is copying Microsoft, with a plan to cram chatbots into every corner of the Googleverse.

To explain why Google has been spooked into doing something so drastic and improbable, we need to consider the company’s history, which has been characterized by similar follies.

[Read: Is this the week AI changed everything?]

In 2010, Google abruptly pulled out of China, after Chinese hacking of Gmail proved to be, finally, too unpalatable for the company. Google had spent four years avidly cooperating with the Chinese Communist Party to censor search results, only to see its infrastructure attacked with, analysts suspected, the government’s acquiescence at a minimum. That must have stung.

I vividly recall when Google entered the Chinese market, in 2006. That was a hell of a moment to do so. Yahoo, the seemingly unstoppable web giant that Google then trounced, had gotten into China back in 1998, the same year Google was founded. Over the intervening eight years, Yahoo had made a string of horrifying compromises to maintain its operations there, culminating in a notorious incident in which the company helped the state prosecute the journalist Shi Tao, who was sentenced to 10 years in prison based on the contents of private Yahoo Mail messages. Yahoo’s role as a “police informant” (per the watchdog Reporters Without Borders) drew stinging criticism, with other Chinese writers such as Liu Xiaobo excoriating Yahoo’s co-founder Jerry Yang for betraying the Chinese people.

Google’s entry into China came amid the swirl of that scandal, and the company’s account of its decision was nothing short of grotesque. At a session with a Google founding board member at the 2006 Web 2.0 Conference, in San Francisco, I stood up in the audience and asked how he could justify Google censoring its search results in China. He explained—to gasps of disbelief—that Google was doing this to improve the user experience of Chinese searchers, who would otherwise be served links to pages that were blocked by the Great Firewall and would grow frustrated when their clicks led nowhere.

The real answer was that Google was incredibly insecure—always was, and still is. The company, which had toppled a market leader by building better technology, is haunted by the fear of being pushed aside itself. Back in 2006, the easiest way to get Google to do something stupid and self-destructive was to persuade Yahoo to do it first.

Weird as it is to think of a company with a market cap of more than $1 trillion being manipulated by its insecurity into poorly considered copycat maneuvers, that wasn’t the only time Google jumped off a bridge because some other company declared bridge-jumping to be the Next Big Thing.

Remember when Google decided it had to close the social-media gap with Facebook, because it feared that social media was going to eclipse search as the way that internet users got their information? A year after Google pulled out of China, it developed Google Plus, a social-media service that was supposed to underpin every part of the company’s sprawling product offerings. Product managers and engineers were given orders to thoroughly integrate Google Plus into the Google stack, and this became a dreaded KPI—key performance indicator—on which bonuses, raises, and performance evaluations all rested.

[Annie Lowrey: How did tech become America’s most troubled industry?]

Why is Google so easily spooked into doing stupid things, whether they involve censorship in China or shoehorning awkward social-media features into places they don’t belong? I suspect that the company’s anxiety lies in the gulf between its fantasy of being an idea factory and the reality of its actual business. In its nearly 25-year history, Google has made one and a half successful products: a once-great search engine and a pretty good Hotmail clone. Everything else it built in-house has crashed and burned. That’s true of Google Plus, of course, but it’s also true of a whole “Google graveyard” of failed products.

Almost every successful Google product—its mobile stack, its ad stack, its video service, its document-collaboration tools, its cloud service, its server-management tools—was an acquisition. In many cases, these acquisitions replaced in-house products that had failed (such as YouTube displacing Google Video).

Google, like every monopolist before it, isn’t a making-things company anymore; it’s a buying-things company. Yet this fact clearly sullies the self-image of Google and lowers Google’s prestige for its users. It also threatens to erode the stock-price premium that Google has historically enjoyed thanks to its unearned reputation as a hotbed of innovation. (I concede that Google is good at operationalizing and scaling other people’s inventions, but that’s table stakes for every monopolist; technical excellence at scale is not the same as creativity.)

Analysts tell us that Google is losing the AI race. Company-wide alarm bells are sounding, and the employees who survived a brutal, unnecessary round of mass layoffs have been ordered to integrate chatbots into search. (Google’s 2022 stock buyback was so colossal that it would have paid the salaries of every laid-off employee for the next 27 years.)

The same old cycle: A monopolistic Google competitor expands into a dubious line of business—last time, it was Yahoo and China; this time, it’s Microsoft and ChatGPT—and Google freaks out. The company’s leadership demands that employees chase its competitor’s gambit, making their compensation dependent on following a commercial fad or integrating the new technical hotness into products that billions of people rely on—even if it makes those products materially worse.

We know how this movie ends. The Google user experience will continue to degrade. The steady decline of search quality, which has seen results devolve into an inedible stew of ads, spam, and self-preferencing links to Google’s own services, will attain a new plateau of mediocrity. And more value will be shifted from searchers, advertisers, and employees to shareholders.

The problem is not that chatbots are irrelevant to search—they’re all too relevant already. Rather, it’s that automated-text generators will produce oceans of spam, and will continue to blithely spew lies with all the brio of a con artist. Google could have responded to this threat by creating tools to “organize the world’s information and make it universally accessible and useful,” as the company’s own mission statement proclaims, ones that will detect and discard machine-generated text or fact-check chatbot spam. The company could have reformed its machine-learning-research department, and tried to turn around its deserved reputation as a place where toeing the corporate line is more important than technical excellence.

But it didn’t, and it won’t. The buying-things company persists in striving to be an inventing-things company. Rudderless and out of ideas, coasting on a single technical breakthrough codified a quarter century ago, Google will continue chasing its rivals and calling the process “innovation.”

Last week, both Microsoft and Google announced that they would incorporate AI programs similar to ChatGPT into their search engines—bids to transform how we find information online into a conversation with an omniscient chatbot. One problem: These language models are notorious mythomaniacs.

In a promotional video, Google’s Bard chatbot made a glaring error about astronomy—misstating by well over a decade when the first photo of a planet outside our solar system was captured—that caused its parent company’s stock to slide as much as 9 percent. The live demo of the new Bing, which incorporates a more advanced version of ChatGPT, was riddled with embarrassing inaccuracies too. Even as the past few months would have many believe that artificial intelligence is finally living up to its name, fundamental limits to this technology suggest that this month’s announcements might actually lie somewhere between the Google Glass meltdown and an iPhone update—at worst science-fictional hype, at best an incremental improvement accompanied by a maelstrom of bugs.

The trouble arises when we treat chatbots not just as search bots, but as having something like a brain—when companies and users trust programs like ChatGPT to analyze their finances, plan travel and meals, or provide even basic information. Instead of forcing users to read other internet pages, Microsoft and Google have proposed a future where search engines use AI to synthesize information and package it into basic prose, like silicon oracles. But fully realizing that vision might be a distant goal, and the road to it is winding and clouded: The programs currently driving this change, known as “large language models,” are decent at generating simple sentences but pretty awful at everything else.

[Read: The difference between speaking and thinking]

These models work by identifying and regurgitating patterns in language, like a super-powerful autocorrect. Software like ChatGPT first analyzes huge amounts of text—books, Wikipedia pages, newspapers, social-media posts—and then uses those data to predict what words and phrases are most likely to go together. These programs model existing language, which means they can’t come up with “new” ideas. And their reliance on statistical regularities means they have a tendency to produce cheapened, degraded versions of the original information—something like a flawed Xerox copy, in the writer Ted Chiang’s imagining.

And even if ChatGPT and its cousins had learned to predict words perfectly, they would still lack other basic skills. For instance, they don’t understand the physical world or how to use logic, are terrible at math, and, most germane to searching the internet, can’t fact-check themselves. Just yesterday, ChatGPT told me there are six letters in its name.

These language programs do write some “new” things—they’re called “hallucinations,” but they could also be described as lies. Similar to how autocorrect is ducking terrible at getting single letters right, these models mess up entire sentences and paragraphs. The new Bing reportedly said that 2022 comes after 2023, and then stated that the current year is 2022, all while gaslighting users when they argued with it; ChatGPT is known for conjuring statistics from fabricated sources. Bing made up personality traits about the political scientist Rumman Chowdhury and engaged in plenty of creepy, gendered speculation about her personal life. The journalist Mark Hachman, trying to show his son how the new Bing has antibias filters, instead induced the AI to teach his youngest child a vile host of ethnic slurs (Microsoft said it took “immediate action … to address this issue”).

Asked about these problems, a Microsoft spokesperson wrote in an email that, “given this is an early preview, [the new Bing] can sometimes show unexpected or inaccurate answers,” and that “we are adjusting its responses to create coherent, relevant and positive answers.” And a Google spokesperson told me over email, “Testing and feedback, from Googlers and external trusted testers, are important aspects of improving Bard to ensure it’s ready for our users.”

In other words, the creators know that the new Bing and Bard are not ready for the world, despite the product announcements and ensuing hype cycle. The chatbot-style search tools do offer footnotes, a vague gesture toward accountability—but if AI’s main buffer against misinformation is a centuries-old citational practice, then this “revolution” is not meaningfully different from a Wikipedia entry.

[Read: Is this the week AI changed everything?]

If the glitches—and outright hostility—aren’t enough to give you pause, consider that training an AI takes tremendous amounts of data and time. ChatGPT, for instance, hasn’t trained on (and thus has no knowledge of) anything after 2021, and updating any model with every minute’s news would be impractical, if not impossible. To provide more recent information—about breaking news, say, or upcoming sporting events—the new Bing reportedly runs a user’s query through the traditional Bing search engine and uses those results, in conjunction with the AI, to write an answer. It sounds something like a Russian doll, or maybe a gilded statue: Beneath the outer, glittering layer of AI is the same tarnished Bing we all know and never use.

The caveat to all of this skepticism is that Microsoft and Google haven’t said very much about how these AI-powered search tools really work. Perhaps they are incorporating some other software to improve the chatbots’ reliability, or perhaps the next iteration of OpenAI’s language model, GPT-4, will magically resolve these concerns, if (incredible) rumors prove true. But current evidence suggests otherwise, and in reference to the notion that GPT-4 might approach something like human intelligence, OpenAI’s CEO has said, “People are begging to be disappointed and they will be.”

Indeed, two of the biggest companies in the world are basically asking the public to have faith—to trust them as if they were gods and chatbots their medium, like Apollo speaking through a priestess at Delphi. These AI search bots will soon be available for anyone to use, but we shouldn’t be so quick to trust glorified autocorrects to run our lives. Less than a decade ago, the world realized that Facebook was less a fun social network and more a democracy-eroding machine. If we’re still rushing to trust the tech giants’ Next Big Thing, then perhaps hallucination, with or without chatbots, has already supplanted searching for information and thinking about it.

A fashion show that features elderly people as models is a rarity. One with elderly Black African models is even rarer.

Researchers warn that AI could flood social media with harmful and misleading content.

One of the least discussed aspects of the AI language generator ChatGPT might be its ability to produce pretty awful poetry. Given how difficult it is to teach a computer how to recognize a syllable, I’m not disparaging the technical prowess of the chatbot’s creators and testers. But very few of the AI-produced poems I’ve read actually follow the prompt that’s been provided. “Write a poem in the style of Seamus Heaney”? This is not that poem:

In a garden green and fair,
A flower blooms, a sight so rare.
But is it meant for me, I fear?
Will I, like it, bloom this year?

Odds are good that this poem, titled “Is It for Me?,” will not win the National Poetry Series. The final phrase seems plucked from T. S. Eliot’s “The Waste Land,” which gives the last line an unintended comic air, because Eliot is referring to a corpse.

[Read: T.S. Eliot saw this all coming]

Poetry, with its heightened states of emotion, intimate address, ecstatic proclamation, and enchanting song, would seem to be one of the limit cases that prove the point: ChatGPT can write anything we can write. It can indeed compose poems from prompts such as “write a poem about the estate tax.” Asked to write a sonnet about socks, it will produce a poem with the opening line “Oh socks, my trusty companions on my feet.”

Such goofy attempts could be said to emulate praise poetry, that venerable form of ode-making. They could just as well have been spoken by Brick Tamland, Steve Carell’s character in Anchorman, who is prone to spouting cryptic one-liners—including, famously, “I love lamp.” (As a teacher of poetry, I can’t help but imagine an overly eager chatbot in one of my creative-writing workshops in the year 2030. “Do you really love the lamp,” I picture myself asking it, “or are you just saying that because you saw it?”)

Heaney wrote a poem about the death of his mother called “Clearances” that—like the AI-generated “Is It for Me?”—also uses rhyme, meter, and nature imagery:

I thought of walking round and round a space
Utterly empty, utterly a source
Where the decked chestnut tree had lost its place
In our front hedge above the wallflowers.

The difference between ChatGPT’s Heaney-esque poem and Heaney’s actual poem is not simply that one is bad and one is good, or that one is sentimental and one is elegiacally beautiful. The difference is that Heaney lost his mother, and the poem expresses the emotional urgency of this fact during a reflective moment sometime after the event. Heaney’s poem carries the ineffable sense that the poet has not only pillaged from the horde of words that already exist but has also worked on them himself, claiming them partly as his and partly as a treasure loaned to him from centuries of poetry written in English.

I could point to other aspects of the language: the pause in the second line, the similarity between the sounds of decked and chest-, the lingering syllables of wallflowers. Above all, there’s the mystery of the mourning poet’s meditation—that missing tree that both orients and eludes him.

ChatGPT can write poemlike streams of regurgitated text, but they don’t mourn and console and mystify with an image like the chestnut tree, which casts an immersive spell. They don’t satisfy the minimal criterion of a poem, which is a pattern of language that compresses the messy data of experience, emotion, truth, or knowledge and turns those, as W. H. Auden wrote in 1935, into “memorable speech.”

Ian Bogost suggests that ChatGPT produces “an icon of the answer … rather than the answer itself.” This is correct: The poem it spits out is an emblem of what a poem is rather than an example of a poem. It is closer to a found object than to Emily Dickinson’s four-line poems in rhyme, which take “unorthodox, subversive, sometimes volcanic propensities” and channel them “into a dialect called metaphor.”

That’s what the poet Adrienne Rich found in Dickinson’s poetry—a hint as to how poems are made, a trace of their creation. Rich thought it was critically important that a poet’s imagination be followed back to her confining circumstances. For Dickinson, that was a house in Amherst in the 1860s and ’70s. For Rich, who wrote a century later, it was raising three children while questioning her sexuality and political commitments.

[Read: The encounter that revealed a different side of Emily Dickinson]

Not that the relation between the life and the poem is ever easy to make out: Indeed, Rich spent her career learning radically new ways to thread her experiences—as a mother, a homemaker in the suburbs, a lesbian, a feminist, a Jew—into language, changing the language in the process. She was like the poet she imagines in “Poetry: II, Chicago,” written in 1984:

Wherever a poet is born     enduring
depends on the frailest of chances:
Who listened to your murmuring
over your little rubbish     who let you be
who gave you the books
who let you know you were not
alone

Poems, she continues, are “fiery lines” that say, “This belongs to you     you have the right / you belong to the song / of your mothers and fathers     You have a people.” They are almost always precarious in their transmission, whether they get to the poet from a god via Plato’s chain of magnetized iron or from the “inconstant wind” of human inspiration that Percy Bysshe Shelley likened to a fading coal. Now is not the time to give up on that essential strangeness and fragility in favor of productivity and predictability. The world needs more poems, not faster ones.

ChatGPT cannot write poetry—or prose, for that matter—that is “the cry of its occasion,” as Wallace Stevens would have it, because there is no lived “occasion” other than the set of texts it can read. Neither can there be emotion recollected in tranquility. There’s no involuntary memory that’s stimulated by the taste of a madeleine. Creativity requires more than an internet-size syllabus or a lesson in syllables. So does essay writing, which is why, even though many acknowledge that ChatGPT can write passable high-school and undergraduate essays, I’m not concerned about that either.

The poems that ChatGPT writes are riddled with cliché and wince-worthy rhymes, but it isn’t just issues of quality that separate AI- and human-generated compositions. Poetry, whether in the style of Heaney or Dickinson or your journal from fourth grade, comes from the felt necessity to speak a truth, whatever kind of truth that might be, in a tongue that you’ve inherited or learned—or that has been imposed upon you by force or violence. That’s obvious to anyone who, for reasons they can’t fully explain, sits down and organizes their words into a pattern that’s slightly different from the language they use at the dinner table.

Whatever upgrades might come for ChatGPT, what it writes likely won’t emerge from the burning sense that something is missing from the world. Poetry speaks in the words of the dead, words sometimes borrowed from past poems—but the desire to use those words comes from an intuition that something is still hidden in them, something that needs to be heard in the harmony between our present voices and those earlier ones. The resemblance between AI-generated writing and human-generated writing is surface level. We know a little more now about how computers arrange words into patterns. The real question—the question that we keep trying to answer with vital metaphors of “fiery lines” and fading coals—is how humans do.