The Substack AI Detection System

This blog goes all over the place. I write about sport, photography, literature, philosophy and Gregg’s. Anyway, recently Substack has introduced their AI detection system.

Powered, rather ironically, by AI.

Panagram, the company who provides Substack’s AI detection system proudly states on their homepage “AN AI DETECTOR THAT ACTUALLY WORKS”

Let’s put that claim to the test.

Here is an excerpt from one of my unpublished essays about Dostoevsky’s White Nights (currently scoring 0% AI written):

I had this pretty strict teacher in Year 5 at primary school. I remember he used to sit bolt upright in assembly. I’d say something like “Sorry Mr Fletcher.” and he’d go: “It’s not an apology if your behaviour doesn’t change.” I can’t really remember the wording exactly, but you get the point.

I’m going to add some AI generated text.

Except I’m going to write it myself and channel my internal ChatGPT.

At primary school, I had a strict teacher called Mr Fletcher. He sat bolt upright in assembly like a Victorian gentleman obsessed with his humours. I’d say: “Im sorry, Mr Fletcher.” He’d say, sternly — “It’s not an apology if your behaviour doesn’t change.” Sadly, I can’t remember the exact wording, but the point remains.

Actually, I kind of like that second version, maybe I’ll use it. Oh, wait…

15%… How ridiculous is that? Ok, I’ll revert to the old version.

Let’s really undermine Panagram’s claim that this system “really works.” This thing gets really wonky if I remove just one or two words (the White Nights essay sits at a total word count of 820).

I’m going to remove “I remember” from my paragraph about Mr Fletcher.

I remember he used to sit bolt upright in assembly.

Becomes:

He used to sit bolt upright in assembly.

10% — I removed two words, mate.

So my conclusion is that this tool is fucking useless. There are endless Substack notes claiming back the em dash, the Oxford comma and the word “quiet” from the clutches of AI. Now, we’ve somehow arrived at the ultimate problem, AI detection is taking perfectly good human writing away from us.

No one wants something they’ve spent hours writing labelled “10% AI generated”. People slave away over meaningful essays, choosing words and rearranging sentences, only to have a machine cast suspicion on the finished work because it happens to recognise some statistical pattern.

Perhaps even more concerning is Pangram’s claim that it exists to “protect authenticity”. How exactly is this protecting anything when some bloke on Substack can fool the thing by deleting two words?

More reavealingly, I then scanned the exact same text again. Now I’m at 6% and I’ve changed nothing from the above.

Which person at Substack QA’d this tech? I’d like to meet them.

Anyway, I then added at the end of the essay:

I’m really annoyed that Substack’s new “AI Detection System” is saying this text is whatever percent AI generated. So I’ve deliberately dumbed down some of this piece to sound more human, which is fucked up. I’m going to write about it when I figure out how.

Hilariously, adding this at the end has just tipped it to 0%, what the hell.

Phew, now we’re back at 0%.

I’m laughing at you, Alex Imas.

It’s probably worth explaining how a tool like this AI detector might work.

Broadly speaking, an AI detector is likely an LLM (large language model). Which means it has to look for patterns in writing that it has learned are associated with AI generated content. Somewhere it has been “trained” using examples of human writing and AI writing and has learned that certain combinations of words make one more likely than the other.

So instead of asking a machine to produce text that looks human, it’s essentially working backwards. We’re asking another machine to look at our finished text and decide whether a machine produced it. All it can truly say is: this looks more like the AI generated examples I know about than the human ones.

What it can’t actually know, from the text alone, is how those words came into existence. Obviously, that is key, because us humans are clearly quite capable of writing sentences that statistically resemble AI writing, as I’ve demonstrated above.

Yet again we’re outsourcing our critical thinking skills to a computer. All whilst using another computer (and its many faults) to generate a random percentage that somehow changes our perspective on what’s written.

I’m not going to sit here and tell you not to use AI. I use it for work all the time, it can be incredibly useful. But this paradoxical merry-go-round of AI checking AI has got to stop.

If somebody writes something beautiful, funny, strange, moving or interesting and your first instinct is to ask a machine whether another machine might have helped them write it, I think we’ve lost the plot.

You should probably just read the thing. If it resonates with you, leave it there and move on.