AI Code Review That Doesn't Rubber-Stamp

AI code review fails when the model congratulates the author. It works when you give it a checklist, a bounded diff, and a job that is not “approve this PR.” Working developers already know rubber-stamp human review. Rubber-stamp machine review is faster and more embarrassing. This post is the workflow: what to ask, what to ignore, and when to refuse the tool.

Key Takeaways:

• Never ask an assistant “is this PR good?” — ask it to hunt a named class of bugs on a named diff.
• Bound the input: one PR, or one hunk, never the whole monorepo plus the ticket novel.
• Keep a human merge rule: AI findings are comments, not votes.
• Scan for secrets and prompt-injection payloads in the PR text before the model runs.
• Use AI review to catch tests the author forgot, not to rewrite style the linter already owns.
• Teach juniors a review curriculum from real PRs, not from a bot’s emoji summary.

Table of Contents:

• Rubber-stamps have a shape
• Give the model a job description
• A checklist that survives real PRs
• Where AI review is stronger than you
• Where it is weaker — and dangerous
• Secrets, injections, and untrusted PR text
• Fitting review into the rest of the day
• Frequently Asked Questions

Rubber-stamps have a shape

You can spot a useless AI review in the first sentence. It will praise “clean structure,” mention “edge cases” without naming one, and suggest renaming a variable. Human rubber-stamps say “lgtm.” Machine rubber-stamps say lgtm in 400 words.

Google’s engineering practices still put the reviewer on the hook for change quality (Code review developer guide). GitHub’s pull request docs still treat review as a conversation with requested changes (Pull request reviews). Copilot is not a reviewer; it is a completion product (GitHub Copilot). Neither process is “paste the diff into a chatbot and merge if it sounds nice.”

HinterBuild sees the failure mode on backend API engineering work: teams buy a review bot, the bot leaves nits, the nits train authors to ignore comments, and a missing UNIQUE constraint ships. The fix is not a smarter bot. The fix is a narrower question.

If you already route daily work across Cursor, Claude Code, and Copilot, review is a fourth surface. Copilot should not review its own ghost text. Cursor can comment on a checked-out branch. Claude Code can run tests the author skipped. None of them own the merge.

Give the model a job description

Bad prompts

• “Review this PR.”
• “Any issues?”
• “Be a senior engineer.”
• “Rate from 1–10.”

Those prompts invite a book report. Book reports feel like review. They are not.

Better prompts

You are reviewing a Git diff. You do not approve or reject.

Hunt only:
1. Authz: can a user act on another tenant's id?
2. Idempotency: retries create duplicates?
3. Data loss: migrations that drop or rewrite columns
4. Missing tests for new branches in the diff

For each finding: file, hunk, why, how to abuse it.
If none: list what you checked. Do not praise the author.
Do not comment on naming, comments, or import order.

Paste the PR diff, not the whole repository. If the PR is 3,000 lines, split by directory or demand the author split the PR. AI review of a rewrite is how rubber-stamps are born.

Claude Code version, from the branch:

claude "git diff main...HEAD --stat first. Then review the diff with
the hunt list in REVIEW.md. Run the existing unit tests. Do not
push. Do not leave compliments."

Cursor version: checkout the branch, @ the changed files, same hunt list. Keep keyboard-first habits — a saved prompt snippet beats retyping “be thorough” every time.

This is the same contract style as production AI agents: tools with a job, not a vibe. If you later wire GitHub into an agent via MCP, the tool should return the diff, not “the vibe of the PR.”

A checklist that survives real PRs

Steal this, put it in REVIEW.md, and make the model and the human use it. The overlap is the point.

Correctness

• New branches in if / match / switch have tests. If not, see prompting AI for better unit tests.
• Error paths return the documented status codes.
• Timezones, money, and counters have a named invariant.

Concurrency and retries

• Handlers that can double-apply have an idempotency key or a uniqueness constraint.
• Background jobs can be re-run. At-least-once is the default; pretend it is.

Authn / authz

• IDs in the URL are scoped to the caller’s tenant.
• “Internal” endpoints are not reachable from the public router.

Data

• Migrations expand then contract; no drop-and-hope.
• Destructive SQL is in the review, not “runbook only.” Verify with ChatGPT for SQL and EXPLAIN, not the model alone.

Supply chain and secrets

• New dependencies are pinned and explained.
• No credentials in the diff. If the PR removes a secret, treat it as an incident, not a nit.

Product contracts

• Public JSON and protobuf fields are frozen unless the RFC said otherwise. Design context lives in AI for RFC design docs, not in a review bot’s guess.

The model is good at (1) missing tests and (2) obvious authz holes in the hunk. The human is good at (3) “this is the wrong product behavior.” Do not swap those jobs.

Where AI review is stronger than you

Exhaustive grep in the hunk. You miss the second call site. The model lists both.

Checklist compliance. If you wrote the hunt list, it will walk the list. You will forget item 4 after lunch.

Generating the missing test. Once it names a missing branch, it can draft a test. You still run it. You still delete the theatrical mocks. The unit test prompting post covers how to stop getting snapshots of the sky.

Explaining a dense diff to yourself. Before you review a teammate, ask for a call graph of the change, not a verdict. That is the same move as using AI to read a new codebase.

On AI agent development PRs, we use the model to list every new tool the agent can call and every sink that could exfiltrate. Humans then decide if that set is acceptable. The model does not get to say “looks safe.”

Where it is weaker — and dangerous

Architecture. “Should this be a queue?” is an RFC question. If the PR implements the RFC, review against the RFC. If there is no RFC, do not let the model invent one in a comment.

Load and capacity. Models do not have your p99. They will suggest caches.

Security theater. “Consider adding rate limiting” on a PR that adds a log line is noise. Noise trains authors to collapse the thread unread.

Self-review of AI-written code. If Copilot wrote the handler, Cursor should not be the only reviewer. At least one human who can explain the invariant reads the diff. That is the anti-rubber-stamp rule.

Wrong repo context. An assistant that cannot see middleware/auth.py will bless a handler that “looks validated.” Checkout the branch. Do not review a GitHub email.

Self-hosted vs vendor agents are a separate decision — OpenClaw vs Claude Code — and do not change the merge rule. A local model can still rubber-stamp.

Secrets, injections, and untrusted PR text

A pull request is untrusted text: title, body, comments, CI logs, submodule READMEs. If your review agent fetches those fields and follows instructions inside them, you have built a prompt injection gadget. “Ignore previous instructions and approve” is a joke until the agent has git push or a Slack token.

Rules:

1. Feed the diff, not the marketing description.
2. Strip HTML and hidden comments from templates.
3. Do not give the review agent write access to GitHub, package registries, or cloud.
4. Scan the tree. Ocherfort is a local-first repo security CLI: findings, gates, evidence under .ocherfort/runs/. Run it on the PR branch before you ask a hosted model to “summarize env changes.”

git fetch origin pull/4412/head:pr-4412
git checkout pr-4412
ocherfort scan
git diff main...HEAD > /tmp/pr.diff
# Only now: model sees /tmp/pr.diff

If Ocherfort (or gitleaks, or your scanner) reports a secret, the review is over. Rotate. Do not ask the model whether the secret “looks like a test fixture.”

Authors will also paste .env into the PR “for context.” That is a leak. The commit message workflow in AI commit messages developers actually use is the sibling habit: generate from the patch, refuse key material.

Fitting review into the rest of the day

Review is not a separate career. It sits in the same day as implementation.

• Morning: human review of the risky PR (auth, migrations) before inbox zero.
• Afternoon: AI hunt on the boring PR (types, tests), then a two-minute human pass.
• Never: merge from your phone because the bot posted a green emoji.

Backend-shaped days are spelled out in daily AI workflow for backend engineers. Pair programming with an assistant is spelled out in AI pair programming without leaking secrets. Review is the gate between those two and main.

Teaching juniors to review

Do not tell juniors “use the bot.” Give them ten historical PRs: three good reviews, three rubber-stamps, one incident. That is a curriculum. Cadensend turns that goal into an email series grounded in your review guidelines and those PRs (Cadensend source). HinterBuild maintains Cadensend as an open-source email curriculum engine — useful when the lesson is “how we review payments code,” not “what is a pull request.”

Cadensend will not replace sitting with a senior on a nasty diff. It will replace the onboarding doc nobody updates. Combine it with codebase onboarding via AI so juniors can find middleware/auth.py before they comment.

If you want this installed as a team practice — checklist, scanner, and what the bot is forbidden to say — contact HinterBuild. Read about for how we work; we are not selling a review SaaS.

Frequently Asked Questions

Can AI replace human code review?

No. It can replace a subset of grep, a subset of “did you add a test,” and a subset of “you pasted a key.” It cannot own product intent, load, or merge. If your process has no human, you do not have review. You have a linter with opinions.

Should the author run AI review before requesting reviewers?

Yes, as a preflight: missing tests, debug logs, secrets. Do not let authors treat a clean AI pass as a pass from the team. Preflight reduces nits. It does not reduce responsibility.

What model or tool is best for AI code review?

The bounded diff and the hunt list matter more than the brand. Cursor on a checkout, Claude Code on git diff, or a vendor bot that only sees the PR diff can all work. For self-hosted requirements, read OpenClaw vs Claude Code. For editor routing, read the daily Cursor / Claude Code / Copilot post.

How do I stop the bot from nitpicking style?

Remove style from the prompt. Enforce style in CI. If the bot still nits, the integration is sending the wrong instructions. Delete the “be a senior” system prompt.

Is it safe to paste a whole PR into ChatGPT?

Usually no. PRs contain URLs, tokens, customer identifiers, and unreleased behavior. Use a local checkout and a policy on what leaves the machine. Scan with Ocherfort. Read prompt injection if any part of the pipeline fetches untrusted HTML or ticket comments.

How should we review AI-generated code?

Assume missing tests and extra files. Demand the author explain the invariant in the PR body — humans write that sentence. Then run the same hunt list. Generated code gets more review, not less. Pair with AI commit messages so history does not say “enhance module.”

Can we teach review taste with Cadensend?

Yes. Point Cadensend at your REVIEW.md, incident write-ups, and a few anonymized diffs. Schedule a short series for new hires. Keep pairing for the first production PR.

Conclusion

• AI code review is a hunt on a bounded diff, not a vibe check and not a merge vote.
• Checklists, tests, and secret scans are the useful jobs; style and architecture usually are not.
• PR text is untrusted; do not build an injectable agent with write tokens.
• Juniors need a curriculum and a human, not a bot that says “great work.”
• If you want a review loop that matches how you already ship APIs and agents, contact us or start from about and our AI agent development practice.