I Let AI Rewrite 130 Blog Posts. Clicks Went Down

Hey Builder 👋

In August, Claude Code and I rewrote titles and summaries on 130 Careerkit blog posts. In two days.

Last Tuesday I finally got the real grade back.

Oof.

The short version:

Quickly, the rest of my week:

I want to talk directly to the founder who ran a batch of AI rewrites, felt very productive, and moved on.

You're not careless. Search Console makes checking annoying. Results lag by weeks, and by then you've shipped ten other things. So "it felt better" quietly becomes the result.

Quick disclaimer: this doesn't prove AI rewrites are bad. It's one site with no A/B test, the batches share pages, and new pages went live in the same weeks. But it's a real grade, and most people never get one.

Here's everything below 👇

It's a Claude Code skill called /growth, with a mode called ctr:

On Aug 18 and 19 that produced 3 batches: 60 cleanup rewrites on titles, 8 bigger title rewrites and 90 summary rewrites. Some posts got more than one, which is how it comes to 130.

Every change adds one line to a file, at the moment it ships. A real one:

That's the whole trick. The page, what changed, the before numbers, and a date not to look before (about 3 weeks, because Google is slow).

Steal this: you can't grade a change you didn't log. One line per change, written the moment it ships.

verify.mjs goes through the ledger. For every row past its check date, it asks the Search Console API for that page over a window of the same length, then applies four dumb rules:

UNDERWATER exists because the grader once called my cover-letter fonts post a WIN for +17 clicks, on a page turning 83,968 impressions into 69 clicks. The comment in the code says it best:

Steal this: write the verdict rules before you see any numbers, and keep them dumb enough to fit in four lines.

The August rows stored each page as url. The script only read slug.

So all 150 of those rows were measured at /blog/undefined. Zero clicks. Graded FLAT.

Every rewrite: flat. Nothing to see here. It looked like boring news, which is the most convincing kind 🙃

It got fixed on Oct 27, with the missing baselines re-pulled. That's when the real grades showed up.

Steal this: test your grader on one page you know went up. If it says flat, the grader is wrong, not the page.

Same 28-day windows, Jul 19 to Aug 16 vs Sep 27 to Oct 25:

| Batch | Posts | Clicks |

| Didn't touch (already live in July) | n/a | +20% |

| Cleanup rewrites (titles) | 60 | +25% |

| Bigger title rewrites | 8 | -27% |

| Summary rewrites | 90 | -39% (195 to 118) |

Across the 130 posts: 13 wins, 76 flat, 35 regressions, 6 underwater.

What I read in that:

Quick math break, on my own tool's forecast:

That "predicted lift" was never a forecast. It's arithmetic: impressions times a gap. It assumes the page can ever reach a normal click rate. The fonts post now sits at position 5.8 with 97,552 impressions and 77 clicks, so it's underwater, which is a page-type problem and not a title one. (Yes, the same fonts post from two issues ago. It keeps teaching me things lol.)

The plan: before any more text rewrites, look at what those 90 pages lost, query by query. October's changes get graded on Nov 7.

The lazy version, this week:

If you take one thing away: AI made shipping a rewrite almost free. It did not tell me whether the rewrite worked. Build the grader before the next batch.

Okay, that's it from me. Reply with your question, I answer one every week. And if you've ever graded your own AI rewrites, tell me what you found. I can't be the only one 👀

Keep shipping,

Nishant