Last week I ran our article on formal methods in practical programming through Pangram. Guess what, it was deemed 100% AI-written.
We had put a disclaimer at the end: the article was written with the help of a coding agent. All ideas and mistakes are ours.
We’ve been thinking about formal methods, and about how software development should be organized in general, for decades. So we expressed our thoughts, and the machine put them on paper. Neither of us is a native English speaker. We both basically hate writing. So explaining the idea to an agent, seeing what comes back, and iterating until it looks right is very compelling. It produces better English than we could come up with on our own, with the same ideas. Pangram only reacts to the first part.
Here’s the funny thing. Had we typed and edited it ourselves, a’s and the’s missing and Russian word order poking through, it would almost certainly have passed. The older detectors notoriously flagged non-native writing. Pangram reportedly fixed that. So our clumsy English is safe. The English we get by asking a machine to make us clearer is not. The tool punishes exactly the workflow a non-native writer with (hopefully interesting) ideas would pick.
This is not a new arrangement. It has existed for centuries.
There is already a class of books where the author supplied the ideas, and someone else produced every sentence. They are called translations.
A reader of Dostoevsky in English has no option of judging him by his original prose. The prose is Constance Garnett’s. It’s arguably smoother than anything he wrote in the original Russian. Going in the opposite direction, Vonnegut in Russian is better than Vonnegut in English. I’ve read them both, and this is my sincere personal opinion. “Норки нараспашку” in Breakfast of Champions has more layers than “wide open beavers.” But this is not just my opinion. I later found that Dovlatov, a prominent Russian author in his own right, put this in Gore Vidal’s mouth: Vonnegut’s novels lose a great deal in the original. That’s really well put, and way above my level.
People argue about translations endlessly; Nabokov called Garnett’s a crime. But the argument is always about whether the translator got it right, not about the righteousness of having a translation in the first place. Even Nabokov, when he translated his own books, didn’t just transcribe. He rewrote. Languages don’t map word to word, so translation fundamentally is rewriting. In the end the reader gladly pays for the original author’s ideas expressed in translator’s words.
Dostoevsky is a good illustration. In my opinion, he was a mediocre writer with a very recognizable voice, and some of the most interesting ideas. He wrote for money, he wrote fast. He dictated The Gambler to a stenographer in about a month. So the words an English reader consumes were produced, twice, by someone else at his instruction. Nobody thinks the ideas were the stenographer’s. Or Garnett’s.
And yet the voice survives translation. Somewhat, but it does. Which tells you where voice mostly lives: in the ideas. Pangram does not measure ideas. It analyzes the structure and the statistical content of the sentences.
The difference between what we did with our article and what the wonderful Russian translator Rait-Kovaleva did with Vonnegut is that our translator was a machine, and we were translating out of our own heads rather than out of a book. Readers already accept machine translation. I doubt it’s easy to tell it from a human one in a blind test. The second part is the real reason to read anything.
In July, Substack added a badge based on Pangram. Anyone can scan a Substack post and see an estimate of how much of it was written by a human. I don’t think Substack did this for the readers’ benefit. Its revenue comes from a few thousand paid writers. The loudest of them proudly write everything themselves. Good for them! For those writers, voice is the moat. It’s what lets them charge for a newsletter when the information in it might be free elsewhere. A model that produces voice on demand threatens that moat. A detector guards it.
“But readers deserve to know!” Sure. The badge looks like a forced disclosure for people who won’t disclose otherwise. Except it doesn’t catch them. The people producing actual slop will learn within weeks what to prompt for. The model can already plausibly write in any style you show it. Their scores will go green. What’s left is false positives: careful writers, second-language writers, writers whose prose happens to be even. Even at one in ten thousand, over millions of posts, that’s a steady stream. Their only recourse is a postscript like ours, which now reads as a plea rather than a disclosure. The badge punishes the honest end hardest and the end it was built for least.
So, what will be the effect of the badge?
If a “human” score affects who reads you, people will write toward the score. Leave in typos. Rough up the sentences. Avoid the tidy parallel structures models love (where did they get them in the first place?!).
I believe that this is a form of audience capture: writers bending themselves to what their audience rewards. But the audience in this case is a classifier. A classifier is easier to game than a room full of people. The writers it rewards are the ones best at performing humanness, which is not the same group as the humans.
This also brings up Goodhart’s law. When a measure becomes a target, it stops being a good measure. Worse, the regularity Pangram relies on holds only because nobody was optimizing against it. With the badge, that ends.
Also, Substack is teaching millions of readers to trust a signal which will inevitably fade after a few model generations. There’s no good way to withdraw it. When the false negatives become obvious, the score will mean nothing, but people will keep paying attention to it. Being the platform that trained people to trust a broken instrument is a worse position than never having offered one.
How it ends, I’d guess: a few well-known writers get flagged, there’s an outrage, and the score quietly retreats into a menu.
What should be used instead? The same thing translations use: provenance. Not a number computed from the text, but a statement from the author about how the text was made, and the author’s willingness to stand behind every sentence. Translated books put the translator’s name on the title page right below the author’s name. Substack could do the same: let the author declare it. Written by hand. Translated. Drafted with AI, checked by the author. Our formal-methods article put the agent’s role in a postscript. I’d rather that be the norm than a badge, and I suspect the translators would agree.
P.S. This text was produced with the help of an AI agent. Pangram says it is 70% AI. All opinions and mistakes are mine.