AI, Search and the Future of News

Once again, distinctiveness is the best defense

Welcome to Second Rough Draft, a newsletter about journalism in our time, how it (often its business) is evolving, and the challenges it faces.

I could write almost every week about artificial intelligence and the impact it is having, and will have, on the news business. I don’t. AI remains one of the biggest stories of our time, but has been the subject of more heat than light. Its effects on our industry are also emerging more slowly than I might have guessed two years ago, when generative AI exploded into the consciousness of the general public.

To date, most of the visible experimentation with AI by newsrooms has focused on internal processes, and remarkably little has been reflected in new products or services. My own hunch is that newsrooms have been excessively spooked by the inevitable, deeply embarrassing errors that occur if AI is employed without effective quality controls. The lesson shouldn’t be not to use the tools, but to use them more carefully. AI doesn’t publish copy that hasn’t been checked unless you let it.

But if newsrooms have been slower to innovate, you can’t say that about the big platforms, which are rushing ahead into the AI future, some with their characteristic recklessness, others merely as quickly as possible. Among them, of course, is Google, which last year was officially adjudged to have illegally monopolized web search, and which maintains an overwhelming market share there.

An overwrought headline on a very good story

I am moved this week to wade back into the AI discussion by a recent, very good piece in the Wall Street Journal on how AI is changing search, and its implications for news sites. The piece contained a headline (generated using AI?) that was eminently clickable and seemingly dramatic: “News Sites Are Getting Crushed by Google’s New AI Tools.” The lede of the piece was similar: “The AI armageddon is here for online news publishers.”

As happens far too often these days, the body of the story was less alarming-- the dictionary definition of “armageddon” is a final and decisive conflict, pretty much the end of the world-- but actually more interesting. The piece also usefully detailed how some thoughtful publishers, such as the Atlantic, are consciously moving toward more enduring relationships with readers. (By the way, don’t people responsible for newsrooms understand at this point that headlines and ledes that over-sell stories may garner more clicks in the moment, but also diminish the trust on which subscriptions—the heart of their modern business model-- depend?)

The more interesting angle on the story was left for readers to tease out, but it’s this: AI in search is indeed “crushing” traffic to sites like Business Insider and HuffPost, but having much more modest effects at the New York Times and the Journal itself. Why? The story didn’t really say, although we’ll get to that. It did note that the smaller declines at the Times and Journal also came off lower bases as a proportion of total traffic; that is, these publications were less dependent on search in the first place. The Washington Post has been very hard hit, and also from a higher base, out of line with the Times and Journal, its natural peers. Again why? The higher base may provide a clue, and both it and the larger decline do add to specific worries about the Post.

Optimizing what?

In considering how the impact of AI in search will differ among sites, it’s useful to think about how and why we use search itself. I first wrote about this 14 years ago in a piece that I think stands up quite well. It talked about the conflict between optimizing search, more precisely answering the questions users pose (which Google has continued to do, most recently with AI) and search engine optimization (SEO), which was never intended primarily to help users answer their questions, but instead to drive traffic to publishers’ sites. The archetypal story that resulted asked, “What time does the Super Bowl start?” and generated a pathetic click. Google long since grabbed most of this traffic for itself—to users’ benefit.

Not to be mean about it, but the sites suffering most from AI in search are still publishing far too many stories that veer perilously close to that Super Bowl clickbait. If there is one overriding lesson of publishing in the digital age, it remains that distinctive content remains the most unassailable, the least vulnerable.

In drafting this column last week, I asked both Google search and Microsoft Copilot, “How much of a threat is AI search to news sites?” Each instantly produced a few bullet points summarizing the Journal story, along with a link. The bullet points felt telegraphic rather than thoughtful. If I hadn’t already read the Journal piece, I would almost certainly have clicked through.

But other links were provided as well, far less prominently, to standalone stories from TechCrunch, Yahoo Finance, the New York Post and ChannelNews from Australia. All of these merely followed the Journal story, and not one of them added anything to it substantively. Why are we still doing this? And if search stops rewarding this behavior, will anything of real value be lost?

What may come next

I do not mean by any of this to say that AI in search does not pose a threat to news publishers. It does, at a number of levels, and well beyond the modest losses at the better sites. First, there is the legal issue slowly making its way to the Supreme Court—assuming Congress continues to be unable to resolve it, as seems almost certain—about how much the AI models can appropriate from publishers without their (compensated) consent. The resolution of this issue will importantly re-set the economic rules of the game in publishing, one way or another.

Even beyond that, there are questions of how far the AI models may progress from where they stand today. How good can those bullet points summarizing the Journal piece get, again while remaining on the right side of copyright law? Most worrisome, in the long run, would be if AI could evolve to the point where it provided the answer to a question like, “what’s the news in [fill in the blank]?” That might actually yield Armageddon for online publishers.

We’re far from that point right now, however, and I hope (and expect) that the Supreme Court, and our own innovativeness and continuing drive to create distinctive news products and services, can hold off such an outcome. It’s a mission on which we should all be actively at work.

Second Rough Draft may be off next week. See you soon.