Prompt AI for Better Unit Tests, Not Noise

Prompt AI for better unit tests or you will get a suite that asserts the mock was called and names nothing the production code can violate. Working developers already know that tests are the spec. Models know that tests are files that make the user say thank you. This post is how to prompt so you keep the first kind. It fits the same day as a backend AI workflow and a Cursor vs Claude Code vs Copilot routing table.

Key Takeaways:

• Name the invariant and the test function first; let the model fill arrange/act/assert.
• Ban “asserts mock called” as the only assertion; require a behavior you could explain in a PR.
• Prefer real fakes (in-memory, temp DB) over a theater of MagicMock.
• Generate tests from the diff of production code, not from a ticket novel.
• Fixtures must not contain live secrets or prod rows — scan the tree.
• Teach juniors test taste with a curriculum of good and bad tests from this repo.

Table of Contents:

• What noisy AI tests look like
• The prompt that names the invariant
• pytest, Go, and the fake you actually need
• Coverage theater vs branch hunt
• Secrets, injections, and CI
• Reviewing tests the model wrote
• Characterization, flakes, and time
• Frequently Asked Questions

What noisy AI tests look like

You asked Copilot or Cursor to “add tests.” You received:

• A 200-line setup that rebuilds the entire app
• Mocks of the system under test
• Assertions on log strings that you will change tomorrow
• One happy path named test_it_works
• A snapshot of JSON you cannot read

pytest’s own documentation treats tests as ordinary Python that should be readable (pytest docs). Go’s testing package is the same idea: a function that fails when the behavior is wrong (testing package). GitHub Copilot will complete a test you already named; it will not invent a good suite from “add tests” (GitHub Copilot). Neither culture is “generate 40 tests to impress coverage.io”.

Noise has a cost. Reviewers skip the test file. CI gets slower. The next refactor breaks 20 strings and nobody knows which failure is real. That is the test equivalent of rubber-stamp code review.

If you are reading a new codebase, start from an existing golden test, not from a generated suite.

The prompt that names the invariant

Write the name yourself. This is the whole trick.

def test_refund_replay_with_same_key_returns_original_body():
    ...

Then:

Fill this test. Invariant: POST /v1/refunds with the same Idempotency-Key
and merchant returns 200 and byte-identical JSON, including created_at.

Use the existing app fixture in tests/conftest.py.
Do not mock the handler.
Do not add new third-party plugins.
If you need a store, use the in-memory fake already in tests/fakes.py.
Show the assertion on created_at explicitly.

Copilot often completes this well because the name is the prompt. Cursor chat is for when the fixture setup is non-obvious. Claude Code is for generating and running the test in a loop — same split as the daily assistant workflow.

Second prompt, always:

List branches in refunds.py that this test does not cover.
Propose one additional test name per branch. Do not write them yet.

You pick which branches are worth a test. The model will propose testing __repr__. Decline.

For design-level behavior (new resource, new consistency), write the RFC first: AI for RFC design docs. Tests that lock a half-thought design are expensive.

pytest, Go, and the fake you actually need

pytest

Keep conftest.py human-owned. Let the model write the test function, not the session-scoped frankenstein.

def test_refund_replay_with_same_key_returns_original_body(client, merchant):
    headers = {"Idempotency-Key": "k1", "X-Merchant-Id": merchant.id}
    body = {"payment_id": "pay_1", "amount": 100}
    r1 = client.post("/v1/refunds", json=body, headers=headers)
    r2 = client.post("/v1/refunds", json=body, headers=headers)
    assert r1.status_code == 200
    assert r2.status_code == 200
    assert r1.json() == r2.json()
    assert r1.json()["created_at"] == r2.json()["created_at"]

If the model wraps this in five factories, unwrap it. pytest parametrize is allowed when you name the cases:

@pytest.mark.parametrize(
    "key,status",
    [
        ("", 400),
        ("k1", 200),
    ],
)
def test_refund_key_validation(client, merchant, key, status):
    ...

Do not let the model parametrize 30 random strings “for coverage.”

Go

func TestRefundReplaySameKeySameBody(t *testing.T) {
    t.Parallel()
    srv := newTestServer(t) // human-owned helper
    key := "k1"
    first := srv.postRefund(t, key, "pay_1")
    second := srv.postRefund(t, key, "pay_1")
    if first.Status != second.Status {
        t.Fatalf("status %s vs %s", first.Status, second.Status)
    }
    if first.CreatedAt != second.CreatedAt {
        t.Fatalf("created_at changed on replay")
    }
}

Prompt: “Use newTestServer. Do not start gin in the test body. Do not use httptest a second way.”

SQL in tests

If the invariant is a unique constraint, the test should hit a real engine (sqlite, testcontainers, or ephemeral Postgres). A mock of Execute will not save you. For query help, ChatGPT for SQL — then still run EXPLAIN on anything that will exist in prod.

pytest -k refund_replay -q
# or
go test ./internal/http -run RefundReplay -count=1

Claude Code can iterate until green. You still read the assertion. Keyboard-first habits: bind “run nearest test,” not “generate tests for file.”

Coverage theater vs branch hunt

Coverage percentage is a lazy prompt:

# bad
Increase coverage on refunds.py to 95%.

The model will test getters. Use a branch hunt:

Here is git diff main...HEAD -- internal/refunds.py
For each new branch (if/else, switch, error return), either:
- name an existing test that covers it, or
- propose a new test name and the assertion.

This is the same hunt we want in AI code review. Authors should run it before reviewers do.

When the production code was AI-written, assume missing negative paths. Generated handlers skip 503 from downstreams. Generated tests skip them too. Double hunt.

Agents that are the product need evals, not only unit tests — that is building production AI agents and AI agent development. This post is for the service sitting next to the agent: HTTP, jobs, SQL.

MCP does not generate better tests. Skip it unless your test harness is genuinely a tool server (MCP explained).

Secrets, injections, and CI

AI loves realistic fixtures.

# refuse
STRIPE_KEY = "sk_live_51N...."
USER_EMAIL = "jane.doe@client.com"

# keep
STRIPE_KEY = "sk_test_placeholder"
USER_EMAIL = "devnull@example.com"

Scan the repo and the new test files. Ocherfort is a local-first repo security CLI: findings, gates, evidence under .ocherfort/runs/. Run it on the branch before you ask a hosted model to “fill fixtures from the sample dump.” Pairing rules: AI pair programming without leaking secrets.

Untrusted input in tests is also a prompt injection issue if your unit test loads issue HTML or email fixtures into an agent. Keep those fixtures inert strings.

CI should run the tests the model wrote. CI should not call an LLM to generate tests on every push. That is non-determinism dressed as quality. If you want self-hosted generation in a sandbox, that is a different architecture — OpenClaw vs Claude Code — still not in the critical path of main.

Commit the tests like any other code. Do not let the commit message say “add comprehensive tests.” Be specific: AI commit messages.

Reviewing tests the model wrote

Human checklist:

1. Can I explain the invariant without opening production code?
2. Would a wrong production change fail this test?
3. Is the fake at the right boundary (I/O), or did we mock the brain?
4. Any secrets or prod-shaped PII?
5. Runtime acceptable? (no 8-second sleeps)

If you cannot answer (2), delete the test. A test that cannot fail is worse than no test: it paints CI green.

Teach this with examples. Cadensend is an open-source email curriculum engine: one goal, sources from your golden tests and your worst generated tests, scheduled issues (GitHub). Juniors need to see a mock-of-self and a good replay test side by side. HinterBuild would rather ship that series than another “testing pyramid” lunch.

Cadensend will not run pytest for you. Your CI will.

If you want a working session on test prompts against a real service, contact HinterBuild. Bring a file of tests you already distrust. Read about for how we work; backend API engineering is where this shows up most.

Characterization, flakes, and time

Sometimes you inherit a function with no tests and no spec. The honest move is a characterization test: lock current behavior, then change it on purpose.

Write one test that documents what refunds.py does today for a missing key.
Do not guess the intended API. Quote the current status code and body.
Name it test_missing_key_current_behavior so we can rename after the RFC.

That is not gold. It is a seatbelt. After the RFC decides 400 vs 422, you rename and change the assertion. Models will try to “fix” the production code while writing the characterization test. Forbid that.

Flakes the model loves

AI tests sleep, depend on wall clocks, and hit the network “just this once.”

# refuse
time.sleep(2)
assert datetime.now().isoformat().startswith("2026")
requests.get("https://api.stripe.com")  # in a unit test

# keep
clock = FakeClock(t0)
svc = RefundService(store=InMemory(), clock=clock)

Prompt:

No sleep. No live network. Inject a clock. If you need time, use the FakeClock
in tests/fakes.py. If a test would be order-dependent, say so and stop.

Run the test twice. Then run it with -n auto or go test -count=20. If it flickers, you do not have a test. You have a mood. AI code review should hunt sleep in _test.go / test_*.py the same way it hunts secrets.

Timezones and money

Backend tests fail in London or on leap days because the model used datetime.now() and float dollars.

Use decimal or integer cents. Freeze time at 2026-09-21T12:00:00+00:00.
Do not parse local timezone from the developer laptop.

If the invariant is SQL uniqueness, the test must hit an engine. Mocking execute will not catch the missing unique index. Draft the constraint with ChatGPT for SQL, then assert IntegrityError / unique violation on replay.

Tests as onboarding

A golden test is the best map of a service. When you read a new codebase, start there. When you teach juniors, put three golden tests and three generated disasters into Cadensend. Pair with keyboard-first: run nearest test on a key, do not generate 40 files.

Do not generate tests from untrusted issue HTML. That is prompt injection with extra CI minutes. The issue is data. The invariant is yours.

Frequently Asked Questions

Should I let Copilot write all my tests?

Let it complete a test you named. Do not let it invent the suite. Ghost text is strong when the contract is in the function name. See Cursor vs Claude Code vs Copilot.

Are property-based tests a good AI target?

Sometimes, if you specify the property. “Generate Hypothesis tests” without a property produces random strings and a false sense of safety. Name the invariant first (idempotency, round-trip, monotonic version).

How do I stop huge test files?

Bound: “one test function, under 40 lines, use existing fixtures.” Delete setup the model copied from another file. Keyboard-first habits include running one test, not the world.

Can AI write integration tests?

Drafts yes, with a real test database. Do not mock the HTTP stack and the DB and the queue in an “integration” test. That is a unit test in a trench coat.

What about snapshot tests?

Use them for canonical JSON you own, and review the snapshot like code. Do not accept a 4KB blob you did not read. Models love snapshots because they always pass the second time.

How do secrets get into tests?

Copied from .env, from “realistic” examples, from dumps used while onboarding. Scan with Ocherfort. Never paste prod rows into the prompt that generates fixtures.

Can Cadensend teach unit test style?

Yes. Feed it CONTRIBUTING.md, three golden tests, and three anti-examples. Cadensend delivers a series. Pairing still teaches the first PR.

Conclusion

• Name the invariant and the test; the model fills the body.
• Kill mock theater, secret fixtures, and coverage-for-its-own-sake.
• Hunt new branches from the production diff; review tests like production.
• Teach with a curriculum from this repo’s real tests.
• Need help installing that on a backend team? Contact or start from about.