How to tell if you’re writing slop: measure it against your own hand

Using AI to write has been a game changer for my life, and it wasn’t one until recently. I use it for my creative work and I use it for my professional work. A few things I still write by hand from the first word to the last, but for most of what I make, the tool has let me augment my voice, my ideas, and my stories in a way I could never have imagined. I believe the pieces are real. I believe they are mine. I put my name on every one of them, proudly.

The other day I was listening to the AI Daily Brief, my favorite podcast, and they were doing another episode about slop: how to make sure you aren’t producing it, and how to tell. I have heard that conversation a dozen times this year, and this time I heard it as a question aimed at me. If I am going to put my name on work a tool helped draft, I owe that work a standard, and the standard I landed on is plain enough to say in one sentence. What goes out under my name should be as close to me as I can make it.

The way I made myself check was a deep dive into my own writing, looking for patterns. I pulled everything I could find, the work I wrote by hand before the tool and the work I have made with it since, and I measured all of it the same way. This guide is the steps I took, in the order I took them, with what each step is for, what it showed me, and the mistakes left in, so you can run the same check on your own folder tonight. There is no detector in it. The baseline is you.

What this comes down to

Here is the whole method in plain words, before any of the mechanics.

You have two piles of writing. One pile is what you wrote by hand before the tool. The other is what you have made with the tool since, and it helps to keep that second pile sorted by the kind of writing it is, because posts, letters, and emails drift at different speeds. If the tool changed your writing, the second pile will differ from the first in ways you can count: how long the sentences run, how much the lengths vary from one sentence to the next, and how often a handful of particular moves repeat. So you count those things in both piles, the same way, and lay the numbers side by side. Then one more step, the one that turned out to matter most: talk a piece out loud, let the tool draft from the transcript, and compare the draft to what you said. If the draft came out shorter, cleverer, and less varied than your own spoken words, you have found the pattern, and you found it with nothing but your own writing.

✶ The method

1. 1
   Two pilesWhat you wrote by hand before the tool, and what you have made with it since.
2. 2
   Count the same thingsSentence length, the mix of short and long, and a few repeated patterns.
3. 3
   Compare the shapesLay the two piles side by side and see where the sentences live.
4. 4
   Dictate one pieceTalk it out, let the tool draft it, and measure the draft against what you said.

What you need before you start

Three kinds of material, and you probably have all three already.

Writing you did by hand before the tool. For me that was a memoir draft, thirteen years of blog posts, and a folder of personal essays, about fifty thousand words together. Ten thousand is enough to see a shape.

Writing you have made with the tool, kept in one file per kind of writing: posts, letters, long pieces for clients, client emails, messaging documents. Keep the kinds apart, because they drift at different speeds and you want to see which ones moved.

One dictation. You, talking a piece out loud into your phone, transcribed, and never cleaned up.

Each of these goes in a plain text file. For the no-code version at the end, a spreadsheet does the job.

Step 1. Sort what you have into before, during, and after

Everything downstream depends on having a clean “before” pile to compare against. This step is about deciding which writing counts as you before the tool, which counts as the tool in the loop, and keeping the two from mixing.

I used three tiers and named the files so the tier is the first word.

• Anchor is by hand, before the tool: anchor-memoir.txt, anchor-blog-2012-2023.txt, anchor-essays-2016-2023.txt.
• Mid is the year the tool was around but did not draft: mid-linkedin-2024.txt.
• Test is the tool in the loop, one file per kind of writing: test-linkedin-2026.txt, test-letters-2026.txt, test-client-longform.txt, test-client-emails.txt.

Inside each file I put a marker line between pieces, ===== title =====, so I could measure per piece later without splitting the file. The first mistake arrived here: my splitter dropped the first piece in every file because the file did not begin with a blank line, and I did not notice for a day. Start every file with an empty line, or check that the piece count matches what you put in.

For client work, combine it by format before anything leaves your machine, and never by name. The question is whether the process flattens the writing, and that question has the same answer whoever’s name is on the piece.

Step 2. Clean the text, and the three ways it bit me

Counting is only as good as the text you count, and a raw export is full of things that are not sentences: links, headings, bullet lists, the Subject line of an email. This step strips those out, and it is where most of my mistakes happened.

Strip, in this order: markdown links down to their text, then any bare URL, then headings, list markers, image embeds, code blocks, and email scaffolding like Subject and Preview lines. I did links after URLs the first time and the URL pattern ate the link text with it, which took forty percent of one file out of the count until I noticed the word total was wrong. Check the word count against the raw file every time you change the cleaner.

Bullets never end in a period, so a sentence splitter reads a whole bullet list as one thirty-eight-word sentence and your long-sentence share goes up for the wrong reason. Strip the markers and add a period to any bullet over twelve words. Drop anything shorter than that without end punctuation; it is a headline or a label.

Do not split sentences on line breaks. I tested this on the memoir, which has dialogue and scene breaks, and splitting on newlines moved the average from sixteen words to thirteen and a half by cutting lines of dialogue in half. Split on end punctuation followed by a space and a capital letter or an opening quote mark. The rule misses a few cases and is wrong the same way on every file, which is what you want.

Step 3. Count eight things, per thousand words

Step 3 is where the counting happens. Eight numbers, each one there because it answers a question I actually had: are my sentences shorter than they were, do they vary less, and am I leaning on moves I never used to lean on. Everything is per thousand words so a three-hundred-word post and a thirty-thousand-word essay archive sit on the same row.

1. Average sentence length, in words, which is the number everyone already knows how to read.
2. Share of sentences of five words or fewer, the short end of the range.
3. Share of sentences of thirty words or more, the long end, and the measurement that turned out to matter most.
4. Spread, the standard deviation of sentence length, which is the single number that stands in for variety.
5. Em dashes.
6. The reversal, counted three ways together: “not X, it’s Y” in one sentence; “Not X. Y.” across two; and the tail form, “X, not Y.” The tail form is the one most checkers miss.
7. The verdict line and the setup line. A sentence that opens “That’s” or “This is” and lands in under forty words; a sentence that opens “Here’s the”.
8. Questions, as a count of question marks.

The whole measurement, in the form I actually ran it:

import re, statistics as st
text = open("anchor-essays-2016-2023.txt").read()
sents = [s for s in re.split(r'(?<=[.!?])\s+(?=[A-Z"“])', text) if s.split()]
L = [len(s.split()) for s in sents]
k = sum(L) / 1000
print("avg", round(st.mean(L), 1))
print("≤5 words", round(100 * sum(l <= 5 for l in L) / len(L)), "%")
print("30+ words", round(100 * sum(l >= 30 for l in L) / len(L)), "%")
print("spread", round(st.pstdev(L), 1))
print("em dashes /k", round(text.count("—") / k, 1))
rev  = re.findall(r"\b(?:not|isn't|never|don't)\b[^.!?]{0,50}[.;,] (?:It's|That's|Just|But)\b", text)
tail = re.findall(r",\s(?:not|never|rather than)\s[^.!?,]{1,40}[.!?]", text)
notopen = re.findall(r"(?:^|[.!?]\s+)Not\b[^.!?]{0,40}[.!?]", text)
print("reversals /k", round((len(rev) + len(tail) + len(notopen)) / k, 1))
kick = re.findall(r"(?:^|[.!?]\s+)(?:That's|This is|Here's the) [^.!?]{1,40}[.!?]", text)
print("verdict + setup /k", round(len(kick) / k, 1))
print("questions /k", round(text.count("?") / k, 1))

Twenty lines. Run it on the hand-written file first, write the numbers down, and only then run it on this year’s work. The order matters, because once you know what the tool does you start seeing it everywhere and you want your baseline recorded before that happens.

Step 4. Read the shape of the sentences

Now the reading. The average is the first number anyone looks at and the least useful one here, because one writer in two different years can keep the same average and share nothing else. What you are looking for is the shape: how the sentence lengths are distributed from very short to very long, and whether the long sentences thinned out while the short ones piled up.

The average told me a little, since the letters run about twelve words and the essays twenty and I could have guessed that much without counting. What I could not have guessed was the shape, so I binned every sentence into six buckets of width six and looked at where the sentences live.

✶ Figure 1

Where the sentences live

Personal essays, by handWeekly letters 2026, tool in the loop

The hand-written shape is a hump in the middle with a long tail to the right. One sentence in five in the essays runs past thirty words. The tool-in-the-loop shape is a wall at the short end and almost nothing past thirty: about one in twenty in the letters. Same writer, same subjects, same year for some of it.

Two things follow from reading it this way. A rule like “write longer sentences” would miss, because you can lengthen everything to seventeen words and have less variety than before. And the number that moved the most, the thirty-plus share, is the one that most style advice tells you to cut. The long sentence, the one that keeps going a clause past where a careful writer would stop because that is where the thinking is, was the first thing to go.

See every pile, all eight numbers

As of November 2026, with the client rows combined by format so no one is named.

Corpus	Words	Avg sentence	≤5 words	30+ words	Spread	Em dashes /k	Reversals /k
Memoir draft, by hand	7,485	16	11%	10%	11.4	0.1	0.1
Blog 2012–2023, by hand	12,134	19	13%	17%	15.8	3.2	0.7
Personal essays, by hand	32,239	20	8%	19%	12.7	2.6	0.2
Dictation, one piece	331	18	0%	11%	8.6	0	0
LinkedIn 2024, before the tool drafted	9,130	18	11%	12%	11.2	9.4	0.1
LinkedIn 2026, tool in the loop	20,293	12	28%	5%	10.0	15.2	3.7
Weekly letters 2026	12,289	12	29%	5%	8.6	4.0	1.4
Client long form	9,201	17	7%	9%	9.0	0.4	0.9
Client emails	6,406	12	23%	4%	8.5	16.7	1.1
Client messaging docs	4,498	13	14%	3%	7.3	7.8	3.8

Step 5. The moves, and the one that hides

Sentence length is half the picture. The other half is a handful of sentence shapes the tool reaches for, and once you can name them you will start seeing them in your own drafts within a day.

Four moves showed up in the tool-drafted work at ten times the rate of the hand-written work.

The reversal: “It isn’t the tool. It’s the habit.” By hand I used that under one time per thousand words, and with the tool in the loop it ran close to four.

✶ Figure 2

“It isn’t X. It’s Y.” per thousand words

Written by handTool in the loop

The verdict line: a short sentence that opens “That’s” and closes a paragraph by telling you what you just read.

The setup line: “Here’s the thing.” A sentence whose only job is to announce the next one.

The piece narrating itself: “Read that again.” “Let that sit.”

Then the one that hides. The tail form of the reversal, “X, not Y.”, has no “it’s” in it, so the search most people run to catch reversals passes right over it. It was the one a careful reader kept hearing in client work after the obvious tells were gone, and when I went back and counted it, the client messaging documents ran it at the highest rate of any file I had.

None of these are banned. Every one of them is a real sentence that real writers use. Count them, quote the sentences that produced the count, and compare the rate to your own hand. If you used the reversal once a page before the tool and once a paragraph after, you did not change your mind about the reversal. Something else did.

Step 6. Dictate one piece and compare

Everything up to here compares this year to earlier years, and someone could fairly say that I simply changed as a writer. This step closes that gap, because the dictation is me, this year, with no tool in it. It is the step I would keep if I could keep only one.

Talk a piece out into your phone the way you would explain it to a friend. Save the transcript next to the draft as dictation.txt and never clean it up. The hedges, the tangents, the sentence that restarts twice: that is the data.

Let the tool draft from it the way you normally would. Then measure both files and ask three questions.

1. Did the draft come out shorter than it was said? Mine did: an eighteen-word average in the dictation became under twelve on the page.
2. Did the draft add moves that were not in the dictation? Mine had none in the dictation and about six per thousand in the drafts.
3. Did the range shrink? Look at the thirty-plus share and the under-five share side by side.

✶ Figure 3 · the same piece, said and then drafted

Average sentence

18→12

words, dictatedwords, drafted

Repeated patterns

0→6

in the dictationper 1,000 words, drafted

The tool did not shorten my sentences because I asked it to. It shortened them because that is its habit, and I approved the result one draft at a time, each one reading fine on its own. The dictation is the only version of the piece that nobody but me touched, which makes it the one reference the comparison can trust.

I check for orphans too: sentences in the draft whose four-word phrases appear nowhere in the dictation. Some orphans are good editing. A paragraph of them is the tool talking.

Step 7. Do it for the people you write for

If you ghostwrite, or write under a brand, the same check works and the baseline changes. Find each person’s own words with no tool and no you in them: an email they wrote themselves, a memo they dictated, their lines alone from a call transcript.

What I found, with the names left out: every person I measured, composing alone, spread wide. Sentences in the twenties and thirties, few under five, no verdict lines. Their call transcripts were short and choppy, and so were mine, because conversation is short for everyone. Do not use a call as the composing baseline. Use something they wrote or dictated to be read.

I keep a baselines/ folder per person with a README that holds the table, and one short ear file in their own words about what they hear as machine writing and what they do not. Keep that file in their words, because yours will drift toward the tool’s, and the thing they hear is usually something you stopped hearing a while ago.

The read before send

Once I had the table, the question was what to do with it every week without turning it into a rulebook. The answer is a printout.

Every long piece now gets one before it goes out. The script prints the draft’s row, a soft band for the format, the three files from the table it most resembles, and a one-line verdict about where the draft sits: with the hand-written work or with the tool-drafted work. Under that it prints the six-bin histogram, every run of three sentences under eight words, every reversal and verdict line quoted in full, and the closing sentence on its own.

The bands are per format, because an ad is allowed thirty percent short sentences and an essay is not. Long form asks for an average between fifteen and twenty-one, under fifteen percent short, at least one in ten past thirty, a spread of ten or more, and the moves under one per two thousand words. When a piece I like lands outside a band, I widen the band and write down why.

Two rules make this work. It never blocks. And the model that drafts never sees the bands, which are read after the draft exists and never fed into it. I learned the second one the hard way in September, when I gave the drafting tool a rulebook instead. It obeyed every rule. Em dashes went to zero in a week in every kind of writing I checked. The writing got worse, because the sentences kept getting shorter and the rulebook had nothing to say about that.

The second read

After the numbers, three cold readers. I work alone, so they are personas with their own one-page briefs, and they return a combined read under two hundred and fifty words.

• A clarity reader who stops where they stop and says where, and is not allowed to suggest a shorter sentence as the fix.
• The actual reader, on a phone, who says whether the first screen earned the second.
• An editor holding my ear file, who asks three questions: does it build, does the close land on something still moving, and did a hedge get cut that I would have kept.

They quote and never paraphrase. They never rewrite. The output is what the second read caught, the one decision in front of me, the fix if there is one, and what is being left alone on purpose. Then I decide.

What my folder looks like now

_tools/rhythm-check/
  rhythm_check.py         the twenty lines above, plus bands, histogram, quotes, dictation compare
  bands.json              soft bands per format: longform, linkedin, email, memo, ad
  SLOP_GATE.md            300 words: dose and spread readings every piece gets, any length
  LONGFORM_GOVERNANCE.md  one page: what every 600+ word piece does, whoever's name is on it
  CHANGELOG.md            every rule change, dated, with the numbers before and after

<brand>/voice/
  EAR.md                  one voice file: lines that are me, moves that are not, no numbers
  anchors/                links to the hand-written files and every dictation
  disliked/               lines I have ruled out, in my own hand; only I add to it

<brand>/voice-audit/
  corpora/                anchor-*, mid-*, test-* text files
  monthly/                one page a month: the table, re-run

<piece>/
  dictation.txt           what I said, unedited
  post.md                 the draft
  rhythm-read.txt         the printout
  board-read.md           the second read

The changelog is the part I would not skip. The one rule change I made in September showed up in the numbers within a week in every kind of writing, and if the rule had not been dated I would have spent a month looking for a cause. If you can date the rule, you can trace the drift.

Mistakes I made, in the order they cost me

1. Stripped URLs before links and lost forty percent of a file for a day.
2. Dropped the first piece in every file because the splitter wanted a leading blank line.
3. Split on newlines and cut the memoir’s dialogue in half.
4. Measured a call transcript as if it were composing, and concluded for an afternoon that the person I write for naturally writes in eight-word sentences.
5. Wrote the rulebook first and handed it to the drafting tool. Every rule got obeyed and the writing got even less varied.
6. Answered a “sounds like a machine” note by making the sentences shorter, which moved the draft toward the tool’s shape and away from the person’s own.
7. Looked for the em dash and stopped there. Em dashes were the easiest thing to remove and the least informative thing I measured.

Run it on your own folder tonight, no code

Thirty minutes and a spreadsheet.

Open something you wrote by hand years ago and something you made with a tool this month. Paste twenty consecutive sentences from each into two columns. In a third column, count the words in each sentence; =LEN(TRIM(A2))-LEN(SUBSTITUTE(TRIM(A2)," ",""))+1 does it.

Now for each column: how many sentences are over thirty words? How many in a row are under eight? Circle every “not X, it’s Y”, every “X, not Y.”, every sentence that opens “That’s” or “Here’s the”. Compare the two columns, and you will hear it before you finish counting. I did.

Questions I get asked

What counts as slop here?

Writing that is less varied than the person whose name is on it. The podcasts use the word for AI output nobody shaped, and I use it for the drift that happens when a tool shapes your sentences a little more every week and you keep approving the result.

Isn’t this just AI detection?

No, because there is no classifier and no training set anywhere in it; the baseline is your own earlier writing, and the output is a table you read yourself.

Won’t the tool learn to write to the numbers?

Only if it sees them. The drafting model never sees the bands or the table; the check runs on the output, after. The one time I gave the tool the rules instead, it hit every number and the writing got worse.

What about formats that are supposed to be short?

Bands are per format. An ad gets thirty percent short sentences and no long-tail requirement. Email gets more than long form. The question is never “is it short” but “is it less varied than you are.”

So em dashes are fine?

In the dose you used before the tool. By hand I ran about three per thousand words, and the tool-drafted posts ran fifteen, so what changed was the dose, and the mark itself was never the problem.

What about the people who will call it machine writing no matter what?

They will. I wrote an essay about that in October. This guide is for you, so that when they do, you know what is on the page and why.

Does dictation have to be long?

No. My first one was 331 words and it told me more than any other file in the table.

The script

The twenty lines in Step 3 are the measurement. The full script adds the bands per format, the histogram, the quoted evidence, the nearest-files table, and the dictation comparison. I am deciding whether to put it on GitHub; if I do, the link goes here. Until then everything in this guide can be rebuilt from what is on this page.

— Joe