When it comes to writing, LLMs have won

As a middling author, I find it fairly easy to sniff out LLM-generated articles. I see such content almost daily on Hacker News — and mind you, that’s the home of a fairly discerning, tech-savvy crowd; the proportion of generative AI on my other social media is higher than that. I understand the economics of it and I know the output is good enough for most readers. Still, from the perspective of a writer, it stings to see that the robots are winning the zero-sum contest for human attention on the internet.

There are some activists who try to stigmatize the use of generative AI for content creation, but they often end up chasing ghosts; so far, I had two people accuse me of using AI on this blog, and I’m sure there’s more to come. The controversy around em dashes (—) is instructive. I think it’s important to realize that language model output doesn’t differ from human writing per se; there’s just no single tell. What’s true, however, is that if you go to chatgpt.com and ask for an essay, it will by default assume a certain writer persona. All articles written by that persona will have similarities, just as all articles written by the same human would.

The quirks start with basic formatting. For example, the current crop of chatbots has a preference for book-style capitalization of section headings (“Status in Academic Departments”) versus the more common Wikipedia style (“Status in academic departments”); in the same vein, LLMs opt for bolded text instead of italics for emphasis. The models also overuse various stylistic cliches, including negative parallelisms (“not only … but …”, “not …, not …, just …”), vague appeals to significance (“illustrates lasting influence”, “emphasizes the importance of …”), and superfluous outline and summary paragraphs.

Again, there’s no single bulletproof indicator, but contrary to some nerd lore, the totality of these statistical patterns can be detected with automated tools: if Pangram flags something as LLM output, you probably shouldn’t dismiss the finding out of hand. At the same time, such tests are not enough. First, there’s a relatively small proportion of people who also write that way; it’s not good prose, but it’s also not a reason to burn them at the stake. Second, a clever user can prompt an LLM to use a different voice. Ask a chatbot to write as a grizzled sailor — a man of few words who’s still grieving the loss of his wife and only child — and the mathematics get out of whack.

Because of these gotchas, I tend to lean on three additional, higher-level heuristics. First, on the topic of style, I ask myself if the author’s works exhibit any unusual yet consistent writing traits. Every person has some: a preference for overly long sentences, a tendency to overuse semicolons, a propensity for colorful asides or dry wit, or maybe a recurring odd idiom or a mixed metaphor. If a given pattern isn’t common in the training data, the chatbot won’t replicate it unless given an oddly specific prompt (e.g., “always hyphenate the word ‘co-worker’“). If you go through several pages of writing and can’t identify any rare traits, it’s a reason for concern.

Another good question to ponder is why the article exists in the first place. Long-form text takes effort and time. Setting aside compelled writing such as school essays, we write when we have something interesting to say. On an online blog, you should be able to discern the author’s interests and pet peeves. The path of the reverse engineer is to ask what’s the shortest LLM prompt the article could be compressed into. If the prompt is vague and aimless — “generate an article about the importance of Immanuel Kant in the age of AI” — something might be amiss.

The final experiment is to plug this hypothetical prompt into an LLM of your choice. If the model produces something functionally similar to the original article — the same outline, the same metaphors, the same conclusions — it’s a fairly clear sign of LLM chicanery.

All this leads to a more fundamental question: is it even useful to know? I have my misgivings about the origins of the tech: years of my hard work ended up getting pulled into the training datasets without any acknowledgment, let alone consent. I’m sure this is also the fate of any future books I publish and web articles I write, no matter the license or the contents of robots.txt. At the same time, the ship has sailed: most people don’t enjoy the writing process and see a “get it done” button as a godsend. No one cares about the hurt feelings of bloggers who refuse to get with the times.

In the end, if you employ a chatbot as a copyeditor but stand by the result, I’m happy for you. Conversely, if the article you’re posting on the internet doesn’t represent any real cognitive effort and has no discernible goal, then something’s broken — but that’s true no matter if the text is LLM-generated or not.