Humans are still writing the vast majority of guest columns published in the opinion pages of The Wall Street Journal, The Washington Post, and The New York Times, a Semafor analysis found — but large language models were creeping in well before a high-profile Journal op-ed provoked controversy this week.
Over the past month, 10 out of 310 guest submissions to the three publications were labeled as at least 80% “AI” by AI detector Pangram, suggesting AI is not yet making a meaningful dent in human contributions to the most prestigious publications.
Another 40 articles were partially AI-generated. Pangram uses custom AI models to analyze text and other content for AI and, according to a spokesperson, has a .01% false positive rate. If Pangram finds more than 80% of the text is AI generated, it is labeled as AI. In informal testing by Semafor, Pangram was very accurate in detecting AI-written text, though some complain of false positives.
Semafor has come out of no where and became one of the best news organizations as of late. They even seperate reporting from analysis within the same article.