Itemoids

SAT

Don’t Be Misled by GPT-4’s Gift of Gab

The Atlantic

www.theatlantic.com › newsletters › archive › 2023 › 03 › dont-be-misled-by-gpt-4s-gift-of-gab › 673411

This is an edition of The Atlantic Daily, a newsletter that guides you through the biggest stories of the day, helps you discover new ideas, and recommends the best in culture. Sign up for it here.

Yesterday, not four months after unveiling the text-generating AI ChatGPT, OpenAI launched its latest marvel of machine learning: GPT-4. The new large-language model (LLM) aces select standardized tests, works across languages, and can even detect the contents of images. But is GPT-4 smart?

First, here are three new stories from The Atlantic:

Welcome to the big blur. Ted Lasso is no longer trying to feel good. How please stopped being polite A Chatty Child

Before I get into OpenAI’s new robot wonder, a quick personal story.

As a high-school student studying for my college-entrance exams roughly two decades ago, I absorbed a bit of trivia from my test-prep CD-ROM: Standardized tests such as the SAT and ACT don’t measure how smart you are, or even what you know. Instead, they are designed to gauge your performance on a specific set of tasks—that is, on the exams themselves. In other words, as I gleaned from the nice people at Kaplan, they are tests to test how you test.

I share this anecdote not only because, as has been widely reported, GPT-4 scored better than 90 percent of test takers on a simulated bar exam, and got a 710 out of 800 on the reading and writing section of the SAT. Rather, it provides an example of how one’s mastery of certain categories of tasks can easily be mistaken for broader skill command or competence. This misconception worked out well for teenage me, a mediocre student who nonetheless conned her way into a respectable university on the merits of a few crams.

But just as tests are unreliable indicators of scholastic aptitude, GPT-4’s facility with words and syntax doesn’t necessarily amount to intelligence—simply, to a capacity for reasoning and analytic thought. What it does reveal is how difficult it can be for humans to tell the difference.

“Even as LLMs are great at producing boilerplate copy, many critics say they fundamentally don’t and perhaps cannot understand the world,” my colleague Matteo Wong wrote yesterday. “They are something like autocomplete on PCP, a drug that gives users a false sense of invincibility and heightened capacities for delusion.”

How false is that sense of invincibility, you might ask? Quite, as even OpenAI will admit.

“Great care should be taken when using language model outputs, particularly in high-stakes contexts,” OpenAI representatives cautioned yesterday in a blog post announcing GPT-4’s arrival.

Although the new model has such facility with language that, as the writer Stephen Marche noted yesterday in The Atlantic, it can generate text that’s virtually indistinguishable from that of a human professional, its user-prompted bloviations aren’t necessarily deep—let alone true. Like other large-language models before it, GPT-4 “‘hallucinates’ facts and makes reasoning errors,” according to OpenAI’s blog post. Predictive text generators come up with things to say based on the likelihood that a given combination of word patterns would come together in relation to a user’s prompt, not as the result of a process of thought.

My partner recently came up with a canny euphemism for what this means in practice: AI has learned the gift of gab. And it is very difficult not to be seduced by such seemingly extemporaneous bursts of articulate, syntactically sound conversation, regardless of their source (to say nothing of their factual accuracy). We’ve all been dazzled at some point or another by a precocious and chatty toddler, or momentarily swayed by the bloated assertiveness of business-dude-speak.

There is a degree to which most, if not all, of us instinctively conflate rhetorical confidence—a way with words—with comprehensive smarts. As Matteo writes,“That belief underpinned Alan Turing’s famous imitation game, now known as the Turing Test, which judged computer intelligence by how ‘human’ its textual output read.”

But, as anyone who’s ever bullshitted a college essay or listened to a random sampling of TED Talks can surely attest, speaking is not the same as thinking. The ability to distinguish between the two is important, especially as the LLM revolution gathers speed.

It’s also worth remembering that the internet is a strange and often sinister place, and its darkest crevasses contain some of the raw material that’s training GPT-4 and similar AI tools. As Matteo detailed yesterday:

Microsoft’s original chatbot, named Tay and released in 2016, became misogynistic and racist, and was quickly discontinued. Last year, Meta’s BlenderBot AI rehashed anti-Semitic conspiracies, and soon after that, the company’s Galactica—a model intended to assist in writing scientific papers—was found to be prejudiced and prone to inventing information (Meta took it down within three days). GPT-2 displayed bias against women, queer people, and other demographic groups; GPT-3 said racist and sexist things; and ChatGPT was accused of making similarly toxic comments. OpenAI tried and failed to fix the problem each time. New Bing, which runs a version of GPT-4, has written its own share of disturbing and offensive text—teaching children ethnic slurs, promoting Nazi slogans, inventing scientific theories.

The latest in LLM tech is certainly clever, if debatably smart. What’s becoming clear is that those of us who opt to use these programs will need to be both.

Related:

ChatGPT changed everything. Now its follow-up is here. The difference between speaking and thinking Today’s News A federal judge in Texas heard a case that challenges the U.S. government’s approval of one of the drugs used for medication abortions. Credit Suisse’s stock price fell to a record low, prompting the Swiss National Bank to pledge financial support if necessary. General Mark Milley, the chair of the Joint Chiefs of Staff, said that the crash of a U.S. drone over the Black Sea resulted from a recent increase in “aggressive actions” by Russia. Dispatches The Weekly Planet: The Alaska oil project will be obsolete before it’s finished, Emma Marris writes. Up for Debate: Conor Friedersdorf argues that Stanford Law’s DEI dean handled a recent campus conflict incorrectly.

Explore all of our newsletters here.

Evening Read Arsh Raziuddin / The Atlantic

Nora Ephron’s Revenge

By Sophie Gilbert

In the 40 years since Heartburn was published, there have been two distinct ways to read it. Nora Ephron’s 1983 novel is narrated by a food writer, Rachel Samstat, who discovers that her esteemed journalist husband is having an affair with Thelma Rice, “a fairly tall person with a neck as long as an arm and a nose as long as a thumb and you should see her legs, never mind her feet, which are sort of splayed.” Taken at face value, the book is a triumphant satire—of love; of Washington, D.C.; of therapy; of pompous columnists; of the kind of men who consider themselves exemplary partners but who leave their wives, seven months pregnant and with a toddler in tow, to navigate an airport while they idly buy magazines. (Putting aside infidelity for a moment, that was the part where I personally believed that Rachel’s marriage was past saving.)

Unfortunately, the people being satirized had some objections, which leads us to the second way to read Heartburn: as historical fact distorted through a vengeful lens, all the more salient for its smudges. Ephron, like Rachel, had indeed been married to a high-profile Washington journalist, the Watergate reporter Carl Bernstein. Bernstein, like Rachel’s husband—whom Ephron named Mark Feldman in what many guessed was an allusion to the real identity of Deep Throat—had indeed had an affair with a tall person (and a future Labour peer), Margaret Jay. Ephron, like Rachel, was heavily pregnant when she discovered the affair. And yet, in writing about what had happened to her, Ephron was cast as the villain by a media ecosystem outraged that someone dared to spill the secrets of its own, even as it dug up everyone else’s.

Read the full article.

More From The Atlantic

“Financial regulation has a really deep problem” The strange intimacy of New York City Culture Break Colin Hutton / Apple TV+

Read. Bootstrapped, by Alissa Quart, challenges our nation’s obsession with self-reliance.

Watch. The first episode of Ted Lasso’s third season, on AppleTV+.

Play our daily crossword.

P.S.

“Everyone pretends. And everything is more than we can ever see of it.” Thus concludes the Atlantic contributor Ian Bogost’s 2012 meditation on the enduring legacy of the late British computer scientist Alan Turing. Ian’s story on Turing’s indomitable footprint is well worth revisiting this week.

— Kelli

Isabel Fattal contributed to this newsletter.

The Librarians Are Not Okay

The Atlantic

www.theatlantic.com › ideas › archive › 2023 › 03 › book-bans-censorship-librarian-challenges › 673398

The line for the tattoo station at the annual conference of the New York Library Association in Saratoga Springs was already snaking through the hotel lobby, and I hadn't even had my first morning cup of coffee yet. Harry Potter motifs, ghost dogs, angelic hearts, and, of course, books were just some of the tats of choice. These weren’t temporary tattoos or the kind that eventually fades away. These were the real deal. If getting inked seems an act of gritty rebellion more suited to a bikers’ rally than a librarians’ convention, it’s only because we haven’t been paying attention.

Across the country, Republican politicians and right-wing groups such as Moms for Liberty have been waging war against books—their battlefield: the shelves of libraries. “Book challenges”—attempts to ban or restrict titles—have hit a record high. In August 2022, Missouri passed Senate Bill 775, which made distributing “explicit sexual material” to minors illegal and resulted in the removal of nearly 300 titles from school libraries in the state. People everywhere are targeting books that deal with questions of race and sexual identity or expression.

As I puttered around the conference, I thought about the fact that although books don’t have feelings, the librarians forced to remove them from the shelves definitely do. America’s librarians are under enormous pressure, and they need to blow off some steam.

I practically grew up in the Brooklyn Public Library. It served as an after-school center, an SAT training school, and a place to get help filling out my financial-aid forms for college. So when I was invited to give a talk at the conference, I immediately said yes. The night I arrived, I stopped in the hotel bar for a glass of wine before dinner. The place was already packed; the librarians, the bartender told me, knew how to party. He was anticipating a late night.

But at dinner, the conversation was subdued and serious. Reading about all the attacks on books, Angela Gonzalez, a librarian from Penn Yan, New York, told me, “you get nervous. You’re like, Oh my gosh, they’re coming for us.”

Nearly every tumultuous movement in American politics has coincided with a call to ban books. “This piece of it is nothing new to librarians,” Allison Grubbs, the director of the Broward County Libraries in Florida, told me. “What I think is new is some of the pathways that people are choosing to take.” Protests in and outside libraries and library board meetings have become more dramatic. Online, in Facebook groups such as Informed Parents of California and Gays Against Grooming, the language is more and more incendiary. And the librarians themselves are being personally attacked.

They told me about getting hate mail and harassing phone calls on their private lines, about being verbally attacked while on the job over things as seemingly banal as book displays. “You can’t do a pride display—forget about it,” Shirley Robinson, the executive director of the Texas Library Association, told me. “That’s not gonna work.”

“​​I’ve been called a pedophile. I’ve been called a groomer. I’ve been called a Communist pornographer,” Cindy Dudenhoffer, a former president of the Missouri Library Association, told me. “I’ve been called all kinds of things. And I know many of my colleagues have been as well. It’s very hurtful.”

Robinson recounted the story of a Texas library worker who had facilitated a children’s story hour while wearing rainbow-flag Pride socks; a patron filed a complaint to the city accusing the individual of grooming children. Grubbs said she had heard angry patrons in Florida call library staff pedophiles too.

Maybe Americans have gotten ruder, but it’s not only that. Online groups are coordinating protests of Drag Queen story hours, compiling lists of books to challenge, and strategizing ways to amend laws in order to censor books. “They might organize a protest and not even live in the state that that library serves,” Grubbs told me.

Moms for Liberty honed this playbook. The group was founded in 2021 to protest mask requirements for kids and later turned to keeping LGBTQ issues and critical race theory out of schools. Their efforts are part of a larger “parents’ rights” movement that includes many other groups. No Left Turn, for example, offers a list of “aberrant books” on its website, under the “Exposing Indoctrination” tab, just above a link exposing “Woke School Staff & Board” members.

It isn’t just that the attacks are getting more personal for librarians; the laws are as well. Missouri’s S.B. 775 holds librarians (along with teachers and school administrators) criminally liable for distribution of materials deemed inappropriate. A librarian found guilty can face up to a year in prison and up to $2,000 in fines, not including legal fees.

In Texas, Jonathan Mitchell, the attorney behind S.B. 8, the law enabling citizens to sue individuals who violate the state’s abortion ban, is now going after books. Last month, Axios reported that he was allegedly writing draft ordinances for local governments that would use the same strategy, allowing private individuals to sue librarians over the books they choose to stock or even for just expressing LGBTQ support. “There’s a lot of fear,” Robinson told me, “which is what these groups were after from the beginning.”  

The graduate degree for librarians is not, typically, a master of arts, but a master of science—in library and information sciences. Librarians may adore books, but they are trained in the technical and data-driven work of running libraries. Unlike a privately owned bookstore, where the stock might reflect the tastes and preferences of the proprietor, at the library, books are acquired based on information about what its particular community wants and needs.  

“Librarians love data,” Dudenhoffer, who now coordinates the information-science program at the University of Missouri, told me. “Knowing how to analyze your community, knowing how to look at data, knowing how to look at circulation numbers, knowing how to look at population movement, those things are becoming increasingly important in what we do, and that drives all of this.”

Public librarians, she said, are looking at such things as regional household income, age, education level, and racial and ethnic backgrounds while making their selections. They also consider patron requests. In a school library, this analysis might include information shared by students or teachers about the needs and interests of the current student body.

Librarians who showcase books about underrepresented groups, including LGBTQ people, surely believe that these stories are valuable. But the librarians I spoke with insisted that they’re making these choices because an assessment determined that there was a patron need for these books, not to push some personal social agenda. Those controversial book displays? Many, Dudenhoffer said, are a means of letting patrons know that material they might be too shy or embarrassed to ask for is in stock.

“It’s really unfair to characterize displays or programs as ‘woke,’” Dudenhoffer lamented. “That’s just such a terrible word to use right now. But it’s not about that. It’s about serving our community, and everyone in the community, to the best of our abilities.”

What seemed most painful to the librarians I spoke with—even more than the personal attacks and fear of litigation—was the way in which book bans hinder their ability to connect their patrons to information that might help them.

Senate Bill 775 requires the removal of any materials deemed sexual in nature (which is subjective), with exceptions for “works of art” or of “anthropological significance” (also subjective). The law’s rollout was tumultuous at best: The list of books to be removed varied wildly across the state. One place banned more than 200 titles; others, just two or three. This was partly because the methodology, if you could call it that, also varied—not just county by county, but school district by school district. In some places, the choices were made by a school administrator; in others, an attorney for the district chose. Sometimes the librarians themselves were told they had to decide, which also meant deciding how much personal risk each book in their collection was worth.

“This is chaos,” Tom Bober, a vice president of the Missouri Association of School Librarians, told me. “When this law was put into effect, there was no process. There was no procedure. Everyone had to figure it out for themselves.” He said that one librarian in his association was told, essentially, “You figure that out.” In other words: “We’re not going to give you any support with this, because we don’t want to be liable for anything. So this is all on your shoulders.”

The state finally pushed the librarians too far. Last month, the Missouri ACLU filed a lawsuit on behalf of Bober’s organization and the Missouri Library Association challenging the law, arguing that it suppresses their members’ First Amendment rights. It also, they argue, exposes school personnel to prosecution based on what they teach their own children at home. The Moms for Liberty don’t want the government dictating what their children learn, but neither do parents who happen to be Missouri personnel.  

The Texas Library Association is fighting back, too, with a campaign called Texans for the Right to Read, which raises awareness about efforts to censor books. They also started a volunteer-staffed help line to offer support to librarians being intimidated into censorship.

Censorship is hardly the only challenge librarians face. Budget cuts mean that many librarians, particularly in smaller communities, are also tasked with things like plunging clogged toilets and taking on programs previously offered by schools. As public-facing professionals, they are on the front lines of the masking wars, the homelessness crisis, the opioid epidemic, and the general rise in public rage. The library, Grubbs told me, is “often the last place, the last opportunity” for people who have nowhere else to go.

In Los Angeles, librarians will soon be trained in administering Narcan to patrons overdosing on opioids. Gonzalez told me she’s seen mentally ill people strip their clothes off in the library, throw things. “A guy died on us,” she said. “He died right at the computers.”

About 50 librarians got tattoos at the New York conference. I asked what else they did to let off steam. Unsurprisingly, I heard a lot about reading: fantasy, romance, literary fiction. Grubbs had just taken a “bookcation” with friends; they rented an Airbnb for a long weekend and read, cooked, and talked about books together. Other attendees told me about therapy and yoga and socializing with other librarians. Dudenhoffer does needle felting—sculpting animals and dolls and other objects out of felt—which she described as a “stabby” craft.“I stab things,” she said. “We all have our outlets.”

The attacks on books aren’t letting up anytime soon, but luckily, the librarians aren’t either. “One of my mantras,” Dudenhoffer said, “is ‘I am smart and well intentioned.’ And so I just have to always go back to that, and I swear I say it 25 million times a day. ‘I am smart and well intentioned. This is the work I do. My work is important. My work is good.’”