Slop Was Never About AI

A working definition of slop, and why the detectors can't find it

Image credit Magnific.com

They set up at the fords of the Jordan and made every man crossing say one word.

Shibboleth.

The meaning had become irrelevant. Depending on who you ask, it meant ear of grain or flowing stream. But that was not what the soldiers were listening for. They were listening for the sound.

The Ephraimite dialect had no “sh.” So a man from Ephraim could stand there in the shallows with soldiers on both banks, open his mouth, and out came sibboleth instead.

That was enough.

They killed him at the water’s edge.

Forty-two thousand, according to Judges 12.

That is what makes the story unsettling. The word did show something. It identified a genuine difference in speech. Humans have always searched for these markers. A phrase someone uses. A tiny signal that lets us sort the world into categories.

Sometimes those signals tell us something real.

The danger is what we decide to do with them.

Hi, I’m Neela,

Happy Thursday!

This wasn’t a scheduled post. I just had a weekend with some slack in it and couldn’t stop thinking about the word slop, and then I fell into a rabbit hole and never came out for about 4.5 hours.

I want to be clear about something. This isn’t the “is it okay to use AI” debate. That conversation is REAL, and it’s bigger than one essay, and I’ll get to it when I can do it honestly.

Too many people are using the word slop to mean “anything that touched a machine,” and that’s wrong, and wrong definitions make bad tools, and bad tools ruin people’s lives.

I’m a COO in the tech industry. I KNOW what I’m talking about.

If this clarified something for you, send it to someone who's confused. If you're not subscribed yet, now will be a good time.

welcome!

A Word Worth Rescuing?

Before we go any further, I think we need to agree on what we mean by the word itself.

Right now, people use “slop” to describe almost anything they dislike. Writing that feels generic. Writing that feels like it came from a machine. “Delve” “Em Dashes” Writing that somehow misses the mark.

But that is a very old problem.

We have always known when something feels rushed, lazy, or like someone checked out halfway through making it. A restaurant meal can be slop. A corporate presentation can be slop. A book can be slop. None of those things required an AI tool to identify.

The thing we already had was discernment.

Slop originally meant something much more literal.

It was hog feed. Kitchen scraps. Sour milk. Whatever was left over and thrown into a trough.

The interesting thing is that the issue was never the quality of the food. Pigs were perfectly happy to eat it.

The problem was that nobody had really considered the pig.

There is another meaning of the word, and this one is even more interesting.

Slops were cheap, ready-made clothes sold to sailors from the slop chest aboard ships and later through slop shops in ports across the English-speaking world from the sixteenth century onward.

By the nineteenth century, that trade had become connected to the factory and sweatshop systems that shaped modern clothing production. Cheap clothing was not automatically the problem. For many working men, these clothes were the only affordable way to dress.

The issue was what happened when a person became a category.

The clothing was made for “a sailor,” not a particular sailor.

A size, not a person.

A group, not an individual.

That difference matters.

So here’s my definition of slop. Slop is anything made without imagining anyone on the other end of it

---

Which Brings Me to 2009…

Demand Media.

You may not remember the name, but you have read their work. eHow, Livestrong, thousands of pages telling you how to boil an egg or clean a hairbrush.

Well…at their peak, they were pushing out somewhere between 4000-7000 pieces of content per day, all of it reverse-engineered from search queries.

ALL of it typed by underpaid freelancers who were told what to write by an algorithm that had noticed people were searching for it.

Every word of that was human.

They IPO’d in January 2011 at over a billion dollars. Google released the Panda update the following month and burned it all down. Demand Media reported a $6.4 million loss the following year.

Run any of it through an AI detector today. All of it will probably come back clean. Seven thousand pieces of pure slop a day and the crime-fighting tool has nothing to say about it, because the tool isn’t looking for slop.

It never was.

---

The Robot Looking for Robots

Here’s what the detectors measure.

Perplexity, mostly.

How surprised a language model is by your next word. Predictable prose reads as machine. Jagged, weird, unexpected prose reads as human. That’s the shibboleth.

They are not measuring effort, or care, or fraud, or whatever they claim on their website. They are measuring how statistically smooth your sentences are, and then they are telling your editor you’re a liar.

Now look who that catches.

In 2023, researchers at Stanford tested seven commercial GPT detectors against essays from two groups: 91 TOEFL essays written by non-native English speakers and 88 essays written by American eighth graders.

Then they flagged 61% of the TOEFL essays as AI-generated. At least one detector flagged 97.8% of them. All seven detectors agreed on about a fifth of them, unanimously, wrongly.

Why? Lower perplexity.

People writing in a second language use a smaller, safer vocabulary. They sound smooth because they’re being careful.

Then the researchers did something that should have made everyone stop and pay attention. They gave the TOEFL essays to ChatGPT and asked it to improve the word choices so, they sounded more like those of a native speaker.

The false positive rate dropped from 61% to 11.77%.

Enshitification defined.

Using AI made the essays register as more human. The tool inverts itself when you touch it.

The paper is right here, published in Patterns, and it has been sitting in plain sight for three years while people got expelled and lost their shit.

Meanwhile, OpenAI, which built the thing everyone claims to be detecting, tried to build a detector themselves. Theirs caught 26% of AI text and falsely accused humans 9% of the time.

They killed it in July 2023 and said so on the original announcement post: one sentence, low rate of accuracy, goodnight.

The people with the most information in the world about how this text is generated looked at their own tool and said fuck it.

Everyone else kept selling.

---

The OG Hallucination Detector Had a Retractable Blade

The last time we built an industry on detecting an invisible property of a person, the professionals were called prickers.

Scotland, mid-1600s.

The theory was that a witch carried the Devil’s mark somewhere on her body, a spot that wouldn’t bleed and wouldn’t hurt. So you’d bring in a specialist to strip her and stick a long brass pin into her, over and over, until he found the place that didn’t bleed.

Then she’d hang, and they’d burn the body after.

John Kincaid of Tranent was the most famous of them. He worked from about 1649 to 1662. He paid £6 per confirmed witch plus expenses, which was several months’ wages for a laborer during that time.

A rival named John Dickson had a contract with six shillings a day plus £6 a head, and within no time he had two servants and a very good horse.

Many of them used a bodkin with a retractable blade, so the pin appeared to go in and no blood came out, and the mark was found, and the fee was collected. Reginald Scot had come up with the trick in 1584.

Everyone had access to the debunking the whole time.

In 1662, the Privy Council finally jailed Kincaid and Dickson for fraud. When they opened up Dickson’s history, they found “he” was a woman named Christian Caddell, who had spent years traveling Scotland in disguise, detecting other women’s secret identities for money.

And here is the fact that matters most.

Once the prickers were exposed, the panics tailed off.

Funny how that works, eh?

---

So What Isn’t Slop?

I think the answer is less complicated than we have made it.

It is something where a human made choices. Where you can feel that someone struggled with a sentence, cut a paragraph, changed their mind, or decided that the obvious version was not good enough.

There is usually a cost attached to that.

The writer’s name is on it. They have to stand behind it. They have to walk around the internet afterward knowing that other people can read it, disagree with it, or call bullshit on it.

That matters.

Good writing has always required that kind of risk.

Writing is supposed to be fucking hard.

A writer who used an LLM to check her citations and cut her third paragraph is not producing slop.

A media company generating 7,000 SEO pages a day with an army of human freelancers is producing nothing but.

Any tool that gets those two backward is a FRAUD.

---

Thank you so much for reading.

You can thank Maria Marella for this essay.

She’s keeping me caffeinated.

Smashing that ❤️ button or sharing this post keeps the wheels on this greasy squirrel wheel.

You can also follow me on LinkedIn and Medium.