Humans are still writing the vast majority of guest columns published in the opinion pages of The Wall Street Journal, The Washington Post, and The New York Times, a Semafor analysis found — but large language models were creeping in well before a high-profile Journal op-ed provoked controversy this week.
Over the past month, 10 out of 310 guest submissions to the three publications were labeled as at least 80% “AI” by AI detector Pangram, suggesting AI is not yet making a meaningful dent in human contributions to the most prestigious publications.
Another 40 articles were partially AI-generated. Pangram uses custom AI models to analyze text and other content for AI and, according to a spokesperson, has a .01% false positive rate. If Pangram finds more than 80% of the text is AI generated, it is labeled as AI. In informal testing by Semafor, Pangram was very accurate in detecting AI-written text, though some complain of false positives.
I’m not too confident in the AI detection tools myself. They’re just as flawed as the rest of it. I used to have a boss who “wrote like an AI” years before Generative AI took off, we just didn’t know to describe it back then.
It’s up to editors to discern whether an op-ed is good for publication. Often times, it’s based on the reputation and credentials of the author. A well credentialed author can use AI as a reference and still create a piece that has their voice, and gets their point across, even if some analysis bot somewhere thinks it finds a familiar pattern.