Itemoids

News

Don’t Be Misled by GPT-4’s Gift of Gab

The Atlantic

www.theatlantic.com › newsletters › archive › 2023 › 03 › dont-be-misled-by-gpt-4s-gift-of-gab › 673411

This is an edition of The Atlantic Daily, a newsletter that guides you through the biggest stories of the day, helps you discover new ideas, and recommends the best in culture. Sign up for it here.

Yesterday, not four months after unveiling the text-generating AI ChatGPT, OpenAI launched its latest marvel of machine learning: GPT-4. The new large-language model (LLM) aces select standardized tests, works across languages, and can even detect the contents of images. But is GPT-4 smart?

First, here are three new stories from The Atlantic:

Welcome to the big blur. Ted Lasso is no longer trying to feel good. How please stopped being polite A Chatty Child

Before I get into OpenAI’s new robot wonder, a quick personal story.

As a high-school student studying for my college-entrance exams roughly two decades ago, I absorbed a bit of trivia from my test-prep CD-ROM: Standardized tests such as the SAT and ACT don’t measure how smart you are, or even what you know. Instead, they are designed to gauge your performance on a specific set of tasks—that is, on the exams themselves. In other words, as I gleaned from the nice people at Kaplan, they are tests to test how you test.

I share this anecdote not only because, as has been widely reported, GPT-4 scored better than 90 percent of test takers on a simulated bar exam, and got a 710 out of 800 on the reading and writing section of the SAT. Rather, it provides an example of how one’s mastery of certain categories of tasks can easily be mistaken for broader skill command or competence. This misconception worked out well for teenage me, a mediocre student who nonetheless conned her way into a respectable university on the merits of a few crams.

But just as tests are unreliable indicators of scholastic aptitude, GPT-4’s facility with words and syntax doesn’t necessarily amount to intelligence—simply, to a capacity for reasoning and analytic thought. What it does reveal is how difficult it can be for humans to tell the difference.

“Even as LLMs are great at producing boilerplate copy, many critics say they fundamentally don’t and perhaps cannot understand the world,” my colleague Matteo Wong wrote yesterday. “They are something like autocomplete on PCP, a drug that gives users a false sense of invincibility and heightened capacities for delusion.”

How false is that sense of invincibility, you might ask? Quite, as even OpenAI will admit.

“Great care should be taken when using language model outputs, particularly in high-stakes contexts,” OpenAI representatives cautioned yesterday in a blog post announcing GPT-4’s arrival.

Although the new model has such facility with language that, as the writer Stephen Marche noted yesterday in The Atlantic, it can generate text that’s virtually indistinguishable from that of a human professional, its user-prompted bloviations aren’t necessarily deep—let alone true. Like other large-language models before it, GPT-4 “‘hallucinates’ facts and makes reasoning errors,” according to OpenAI’s blog post. Predictive text generators come up with things to say based on the likelihood that a given combination of word patterns would come together in relation to a user’s prompt, not as the result of a process of thought.

My partner recently came up with a canny euphemism for what this means in practice: AI has learned the gift of gab. And it is very difficult not to be seduced by such seemingly extemporaneous bursts of articulate, syntactically sound conversation, regardless of their source (to say nothing of their factual accuracy). We’ve all been dazzled at some point or another by a precocious and chatty toddler, or momentarily swayed by the bloated assertiveness of business-dude-speak.

There is a degree to which most, if not all, of us instinctively conflate rhetorical confidence—a way with words—with comprehensive smarts. As Matteo writes,“That belief underpinned Alan Turing’s famous imitation game, now known as the Turing Test, which judged computer intelligence by how ‘human’ its textual output read.”

But, as anyone who’s ever bullshitted a college essay or listened to a random sampling of TED Talks can surely attest, speaking is not the same as thinking. The ability to distinguish between the two is important, especially as the LLM revolution gathers speed.

It’s also worth remembering that the internet is a strange and often sinister place, and its darkest crevasses contain some of the raw material that’s training GPT-4 and similar AI tools. As Matteo detailed yesterday:

Microsoft’s original chatbot, named Tay and released in 2016, became misogynistic and racist, and was quickly discontinued. Last year, Meta’s BlenderBot AI rehashed anti-Semitic conspiracies, and soon after that, the company’s Galactica—a model intended to assist in writing scientific papers—was found to be prejudiced and prone to inventing information (Meta took it down within three days). GPT-2 displayed bias against women, queer people, and other demographic groups; GPT-3 said racist and sexist things; and ChatGPT was accused of making similarly toxic comments. OpenAI tried and failed to fix the problem each time. New Bing, which runs a version of GPT-4, has written its own share of disturbing and offensive text—teaching children ethnic slurs, promoting Nazi slogans, inventing scientific theories.

The latest in LLM tech is certainly clever, if debatably smart. What’s becoming clear is that those of us who opt to use these programs will need to be both.

Related:

ChatGPT changed everything. Now its follow-up is here. The difference between speaking and thinking Today’s News A federal judge in Texas heard a case that challenges the U.S. government’s approval of one of the drugs used for medication abortions. Credit Suisse’s stock price fell to a record low, prompting the Swiss National Bank to pledge financial support if necessary. General Mark Milley, the chair of the Joint Chiefs of Staff, said that the crash of a U.S. drone over the Black Sea resulted from a recent increase in “aggressive actions” by Russia. Dispatches The Weekly Planet: The Alaska oil project will be obsolete before it’s finished, Emma Marris writes. Up for Debate: Conor Friedersdorf argues that Stanford Law’s DEI dean handled a recent campus conflict incorrectly.

Explore all of our newsletters here.

Evening Read Arsh Raziuddin / The Atlantic

Nora Ephron’s Revenge

By Sophie Gilbert

In the 40 years since Heartburn was published, there have been two distinct ways to read it. Nora Ephron’s 1983 novel is narrated by a food writer, Rachel Samstat, who discovers that her esteemed journalist husband is having an affair with Thelma Rice, “a fairly tall person with a neck as long as an arm and a nose as long as a thumb and you should see her legs, never mind her feet, which are sort of splayed.” Taken at face value, the book is a triumphant satire—of love; of Washington, D.C.; of therapy; of pompous columnists; of the kind of men who consider themselves exemplary partners but who leave their wives, seven months pregnant and with a toddler in tow, to navigate an airport while they idly buy magazines. (Putting aside infidelity for a moment, that was the part where I personally believed that Rachel’s marriage was past saving.)

Unfortunately, the people being satirized had some objections, which leads us to the second way to read Heartburn: as historical fact distorted through a vengeful lens, all the more salient for its smudges. Ephron, like Rachel, had indeed been married to a high-profile Washington journalist, the Watergate reporter Carl Bernstein. Bernstein, like Rachel’s husband—whom Ephron named Mark Feldman in what many guessed was an allusion to the real identity of Deep Throat—had indeed had an affair with a tall person (and a future Labour peer), Margaret Jay. Ephron, like Rachel, was heavily pregnant when she discovered the affair. And yet, in writing about what had happened to her, Ephron was cast as the villain by a media ecosystem outraged that someone dared to spill the secrets of its own, even as it dug up everyone else’s.

Read the full article.

More From The Atlantic

“Financial regulation has a really deep problem” The strange intimacy of New York City Culture Break Colin Hutton / Apple TV+

Read. Bootstrapped, by Alissa Quart, challenges our nation’s obsession with self-reliance.

Watch. The first episode of Ted Lasso’s third season, on AppleTV+.

Play our daily crossword.

P.S.

“Everyone pretends. And everything is more than we can ever see of it.” Thus concludes the Atlantic contributor Ian Bogost’s 2012 meditation on the enduring legacy of the late British computer scientist Alan Turing. Ian’s story on Turing’s indomitable footprint is well worth revisiting this week.

— Kelli

Isabel Fattal contributed to this newsletter.

Mike Pence Is Warning Us About Trump

The Atlantic

www.theatlantic.com › newsletters › archive › 2023 › 03 › mike-pence-trump-january-6 › 673402

This story seems to be about:

This is an edition of The Atlantic Daily, a newsletter that guides you through the biggest stories of the day, helps you discover new ideas, and recommends the best in culture. Sign up for it here.

A former vice president of the United States identified a sitting president as a mortal danger. In another time, it would have been the Story of the Century. Instead, it was the Kerfuffle of the Week, and it is already dissolving away in the new media cycle.

But first, here are three new stories from The Atlantic.

Is Ron DeSantis flaming out already? NFL owners are making an example of Lamar Jackson. ChatGPT changed everything. Now its follow-up is here.

Broken Sycophants

Mike Pence stunned Washington at this weekend’s annual Gridiron Club dinner and gained the attention of the media and the ire of the White House by making an offensive joke about the Cabinet member Pete Buttigieg.

At the same event, by the way, Pence affirmed that on January 6, 2021, Donald Trump—at the time, the president of the United States—endangered his life along with the lives of his family, the members and staff of Congress, and numerous law-enforcement officers. Trump did this by inciting a mob to attack the Capitol, stop our constitutional process by force, and allow him to remain in office.

“Donald Trump was wrong,” Pence said at the white-tie event, which was attended by journalists, politicians, and other D.C. insiders. “I had no right to overturn the election, and his reckless words endangered my family and everyone at the Capitol that day, and I know that history will hold Donald Trump accountable.” He continued:

What happened that day was a disgrace. And it mocks decency to portray it any other way. For as long as I live, I will never, ever diminish the injuries sustained, the lives lost, or the heroism of law enforcement on that tragic day.

Yet here we are, three days later, talking about inappropriate jokes. This is the story now? That Pence tried out a dumb gag line aimed at Buttigieg? Make no mistake, the joke was stupid and disrespectful, but perhaps we might zero in on the more important point: Pence told us something horrifying this weekend about the condition of our democracy. The national underreaction to his comments, however, is a warning that we have all become too complacent about the danger my former party now represents.

Let us stipulate here that Pence is shamefully late to this criticism and has no obvious intention of going further. He had his one moment of courage, and there will be no others. My friend Neal Katyal, the former acting solicitor general, was present at the dinner, and he rightly lambasted Pence for posturing while refusing to answer a subpoena about what happened on January 6. “There are great actors at the gridiron,” he tweeted after the dinner. “But no one, and I mean no one, could pretend to be [Mike Pence] with a backbone.”

Nevertheless, we should not lose focus. I am still almost vertiginous at hearing a former constitutional officer of the United States government say what Pence said out loud. After all the violence, all the court cases, all the horrific videos (the stuff that will never air on Tucker Carlson’s show), and all the needless deaths, I am almost relieved that I’m still capable of being shocked. I was a boy during Watergate—I delivered the local newspaper that announced President Richard Nixon’s resignation, in 1974—but that long-ago scandal now seems like a polite comedy of errors next to the conspiracy fueled by Trump’s monstrous narcissism.

Even before Pence’s Gridiron-dinner speech, I had a conversation last week with Tom Joscelyn, one of the principal authors of the House’s January 6 committee report. Joscelyn is worried, as am I, that Americans don’t really yet grasp the degree to which the Republicans have been taken over by their most extreme wing. “The American right is overrun with grievance politics now,” he told me. “And they’ve married that approach to an authoritarian movement and cult of personality” around Trump.

Joscelyn is not a man who rattles easily: He was Rudy Giuliani’s senior counterterrorism adviser back in 2007, when “America’s mayor” was gearing up to run for president. He thinks Giuliani’s sad decline, in which he has become a kind of political Dorian Gray right before our eyes, is emblematic of the Republican collapse and surrender to Trump. He argues, and I agree, that Trump’s opponents, especially those running against him in the GOP, are not taking this threat as seriously as they should. Trump “puts the auto in autocrat,” Joscelyn said, because Trump sublimates everything to his personal needs, including his party. (I would argue that this is why Trump, despite his fascist rhetoric and Mussolini-like strutting, is incapable of the consistency and discipline required to build a truly fascist movement, but that’s an argument for another day.)  

Today, as Joscelyn notes, the GOP has ceased to function as a normal political party. There is no consistent ideology or set of policies, no internal mechanisms to check the power of the Trump cult. Even the people who want to dislodge Trump as the leader of the party and the 2024 nominee dare not to take him on in a direct confrontation. Trump’s critics are often accused of having “Trump Derangement Syndrome,” an irrational hatred of Trump that forces disagreement with Trump on everything, but Joscelyn rightly points out that Trump’s Republican enablers are the ones who have had to betray all of their deepest beliefs merely to avoid being cast out. Trump, he says, “broke his sycophants, not his critics.”

Which brings us back to Pence. It might not sound like much for Pence to admit what millions of people already know, but within the Republican Party, this is about as close as you can get to open heresy; Pence’s team deliberated making even this small move against Trump. Yet Pence’s comments have been shrugged off by both the press and the public.

To put into perspective how numb we’ve become, let’s do a thought experiment. Imagine, for example, if Hubert Humphrey, after the riots that broke out in 1968 at the Democratic National Convention, said later, “Lyndon Johnson encouraged those anti-war protesters and put me and hundreds of other people in danger. History will hold President Johnson accountable.” Those two sentences would have shaken the foundations of American democracy and changed history.

But not today. Instead, we’ve already moved on to whether Pence should apologize for a clumsy and offensive joke. (He should.) This, however, is the danger of complacency. What would have been a gigantic, even existential political crisis in a more virtuous and civic-minded nation is now one of many stories about Donald Trump that rush past our eyes and ears.

Voters are tired, and the national media are committed to treating the GOP as a mainstream party. Trump and his coterie are counting on this exhaustion to return to national power, but so are people such as Florida Governor Ron DeSantis, who is using Trump’s themes of bigotry, grievance, and cultural panic to harness that same authoritarian energy for his own purposes. Republican leaders have no intention of speaking truth—or decency—to their base, and until someone in the party of Lincoln is able to muster even the tiniest fraction of Lincoln’s courage, we will indulge our complacency about the Republicans at our peril.

Related:

Anne Applebaum: History will judge the complicit. (From 2020) The January 6 whitewash will backfire.

Today’s News

A Russian military jet hit the propeller of an American drone, causing the drone to go down over the Black Sea, according to U.S. officials. Russia has denied contact with the drone. Meta, Facebook’s parent company, plans to lay off another 10,000 workers—its second round of job cuts in recent months. Ohio is suing Norfolk Southern after one of its trains, carrying hazardous chemicals, was derailed in the state last month.

Dispatches

Work in Progress: The end of Silicon Valley Bank is also the end of a Silicon Valley myth, Derek Thompson writes.

Explore all of our newsletters here.

Evening Read

CBS Photo Archive / Getty

How Not to Cover a Bank Run

By Brian Stelter

On September 17, 2008, the Financial Times reporter John Authers decided to run to the bank. In his Citi account was a recently deposited check from the sale of his London apartment. If the big banks melted down, which felt like a distinct possibility among his Wall Street sources, he would lose most of his money, because the federal deposit-insurance limit at the time was $100,000. He wanted to transfer half the balance to the Chase branch next door, just in case.

When Authers arrived at Citi, he found “a long queue, all well-dressed Wall Streeters,” all clearly spooked by the crisis, all waiting to move money around. Chase was packed with bankers too. Authers had walked into a big story—but he didn’t share it with readers for 10 years. The column he eventually published, titled “In a Crisis, Sometimes You Don’t Tell the Whole Story,” was, he wrote this week, “the most negatively received column I’ve ever written.”

Read the full article.

More From The Atlantic

China plays peacemaker. The failed promise of having it all Photos: Winners of the 2023 Sony World Photography Awards Open Competition

Culture Break

Arsh Raziuddin

Read. Our editors suggest 10 poetry collections to read again and again.

Listen. Start Holy Week, a new narrative podcast by Vann R. Newkirk II about the revolutionary week that followed Martin Luther King Jr.’s assassination.

Play our daily crossword.

P.S.

Now that The Last of Us, HBO’s series based on the game of the same name, has aired its finale, I’ll write about the show later in the week. I hope The Last of Us, which has been remarkable in every aspect, illustrates how, for many years, computer games have had plots more intricate and more involving than much of the stuff Hollywood has been cranking out now for decades. (I say this fully aware of the creativity of this year’s Best Picture, Everything Everywhere All at Once. But I will remind you that it is also the 30th anniversary of The Beverly Hillbillies, a terrible movie full of great actors that I think was an early sign of American cultural exhaustion.)

I have particularly high hopes—that I fear will be dashed—for Amazon Prime’s upcoming Fallout series. Unlike The Last of Us, the Fallout games, set long after a global nuclear war, leaven the despair and violence of postapocalyptic survival with outrageous humor. If you’ve been watching Hello Tomorrow!, the Apple TV+ series that features the always excellent Billy Crudup selling lunar condos in a reimagined 1950s full of robots and floating cars—and yes, we are living in a golden age of television—you have a taste of what the world of Fallout looks like. I can only hope that Amazon’s series about life after the Bomb doesn’t turn out to be a bomb itself.

— Tom

Isabel Fattal contributed to this newsletter.