A Bug Is Not Fixed Until the Protection Fails When It Returns

For a consequential, recurring or supposedly systemic defect, the strongest closure package contains four artifacts — failure evidence, the repair, a protection, and controlled-mutation evidence showing that a deliberate reintroduction of the failure mode makes that protection fail. Deleting the suspicious line does not prove a bug is fixed. A passing test is useful evidence, but it is stronger when you have shown that the test fails for the defect it is meant to catch.

The four artifacts, stated plainly:

• Failure evidence. A reproducible case demonstrating the original defect, captured before the code changes underneath you.
• Repair. The smallest coherent implementation change that restores the broken rule at the layer that owns it.
• Protection. A test, lint rule, static check, schema constraint or runtime guard that expresses the rule.
• Mutation evidence. A controlled reintroduction of the defect that makes the protection fail, for the reason you intended.

We adopted this after broad AI reviews repeatedly reported that a whole bug class had been eliminated because a repository search returned no matching pattern. That evidence is real, but it only proves absence in the snapshot that was inspected. It says nothing about whether the repository would detect the next occurrence — which is the only property that matters once other people, or other agents, keep committing to it. This is the same failure mode as trusting a constant because a comment above it looks authoritative, which is the subject of part 12 of this series.

The full loop, which the rest of this guide unpacks step by step: reproduce the defect, add a protection that fails, implement the repair, show the protection passing, deliberately reintroduce the defect, confirm the mutated program still builds and still reaches the relevant path, show the protection failing for the intended reason, then restore the repair and show green again. It is not ceremony. It tests the test.

A test can pass for the wrong reason, and that can be more dangerous than an obvious coverage gap because it converts an unknown into false assurance. Every item in the list below is something we have watched happen on real code.

• It never executes the changed branch. The fixture takes the happy path; the defect lives in the retry path.
• It asserts an unrelated outcome. The function returns success, and success was never the thing that broke.
• Its fixture cannot trigger the defect. The bug needs a boundary value; the fixture uses a comfortable mid-range one.
• It mocks away the failing dependency. The network timeout that caused the duplicate write is stubbed into a clean success.
• It recomputes the expected output using the same flawed implementation. The oracle and the subject share the bug, so they always agree.
• It passes because the mutation no longer compiles. Green from a build that never ran the test is not green.
• It depends on ordering or cached state. It passes in the suite and fails alone, or the reverse.
• It asserts only that no exception was thrown. Silent data corruption throws nothing.
• It verifies the UI label while persistence remains wrong. The screen says one record; the database holds two.

This is not a niche concern. Google runs diff-based mutation analysis across its monorepo on roughly 30% of the diffs that have statement coverage calculated, surfacing the results to around 6,000 engineers inside ordinary code review, on the stated grounds that mutation testing is the strongest available test criterion and subsumes other coverage criteria. That subsumption is the point for our purposes: coverage records that a line ran, not that anything would notice if the line were wrong. A controlled mutation is a direct instrument for telling those two apart.

Do not begin with the line of code. Begin with what became untrue — because the implementation will move and the invariant will not. A bug stated as an invariant violation stays valid across every refactor that follows; a bug stated as a file and a line expires the first time someone renames the file.

Here is the difference on a real defect shape from a growth-tracking product we worked on:

Invariant: Retrying a failed measurement save must not create a second record.

Observed violation: A network timeout after local persistence leaves the UI in
a failed state; manual retry writes a duplicate with a new ID.

Compare that with "fix duplicate save in the view model". The second version names a symptom and a location. It gives a reviewer nothing to check, gives a future maintainer nothing to preserve, and gives an AI agent an invitation to patch the nearest visible cause. The first version tells all three of them exactly what must remain true.

Alongside the invariant, record the consequence, because consequence is what determines how strong the protection needs to be. A cosmetic misalignment and a silent duplicate financial record deserve very different amounts of engineering. Capture six things:

• User-visible impact — what the person using the app sees or cannot do.
• Data impact — what is written, lost, duplicated or corrupted.
• Downstream impact — which reports, charts, exports or syncs inherit the bad value.
• Privacy or financial impact — whether the defect moves money or moves personal data.
• Recoverability — whether the user can undo it, whether you can repair it server-side, or whether it is permanent.
• Scope of affected versions — which builds shipped with it, and how many installs sit on them.

That last one carries more weight in emerging markets than teams building for a single high-income market expect. In India and comparable markets, a substantial share of an Android install base commonly sits several releases behind for months, because updates get deferred on metered data and on devices short of storage. That is an observation from the version distributions we watch across our own portfolio rather than a published figure, but the direction of it holds consistently enough to plan around. A defect you fixed in the current version is still running in production for those users, which changes the answer to a question the consequence record should force you to ask: does this need a client-side repair, or a server-side or data-layer guard that protects the old builds too?

Write the invariant down before you touch anything. It becomes the assertion in step 3, the mutation target in step 7 and the rule in the project bible at the end.

A useful reproduction is deterministic, small, fast enough to run routinely, independent of systems the defect does not need, representative of the real failing condition, and able to show a clear before and after. Anything that fails only sometimes, or only on one machine, is a lead — not yet a reproduction.

For the retry defect above, the reproduction is expressible in six lines:

Given a known profile and no existing measurement
When a save persists locally but the remote acknowledgement times out
And the same operation is retried
Then exactly one measurement exists with the intended identity
And the UI reaches the defined recoverable outcome

Notice that the reproduction names the outcome for both the data and the interface. A great many defects of this shape get half-fixed because the team verifies only the record count and never checks that the screen leaves the user somewhere sensible, or verifies only the screen and never opens the store.

If reproduction genuinely requires a full manual journey — and for lifecycle, permission and background-execution defects it sometimes does — capture the manual journey first, then move the essential condition into an integration test where the platform allows it. Android's instrumented tests and Apple's XCTest both let you drive real persistence and real lifecycle transitions rather than mocking the exact layer where the bug lives. That distinction matters: mocking the failing dependency is the single most common way a regression test ends up unable to see its own defect.

Then preserve the original evidence, because the moment you patch the code the failure becomes unreproducible and memory is not a record. Keep the finding ID, the build or commit hash, the fixture, the exact commands or steps, the failure output, a screenshot or log where the defect is visual or asynchronous, and your root-cause hypothesis at the time. In our portfolio the most expensive rework is almost always a defect that was fixed, regressed four weeks later, and had no preserved reproduction — so the second investigation started from zero.

One discipline is worth adding here. Write down what you expected to see as well as what you saw. A reproduction that records only the failure lets a later reader assume the current behaviour was always intended.

Choose the lowest reliable layer that can express the invariant, and add higher-level tests only where the user-visible behaviour is itself part of the rule. The lower the guard sits, the more paths it covers without anyone remembering to invoke it.

The available layers, from lowest to highest:

• Database or schema constraint. Use it when data integrity can be enforced at the storage authority. A unique index refuses the duplicate no matter which caller attempts it — from the retry path, the import path, or a future feature nobody has written yet.
• Type or domain model. Use it when the invalid combination can be made unrepresentable. A value that cannot be constructed cannot be persisted.
• Static check or lint rule. Use it when the bug is a prohibited coding pattern detectable structurally — a raw colour outside the token layer, a direct network call from a view, an unguarded force-unwrap.
• Unit test. Use it for deterministic transformation and local business logic, where the invariant is a pure function of inputs.
• Integration test. Use it when repositories, persistence, networking, concurrency or lifecycle interactions are involved — which is where most genuinely damaging defects live, per Android's own framing of what each test type can and cannot prove.
• UI or journey test. Use it when the defect is the feedback, the navigation, the focus order, the accessibility semantics or the recovery path.
• Runtime guard and monitoring. Use it for conditions that cannot be fully prevented statically and need to fail observably rather than silently.

These are not alternatives to choose between. For the duplicate-retry defect, the correct answer is usually three of them at once: a uniqueness constraint at the storage layer so the second write cannot land, an integration test proving the retry path is idempotent, and a UI test proving the user reaches a recoverable state rather than a dead end. Each protects a different failure mechanism, and each would survive the other two being deleted.

The question to ask when choosing is not "what is easiest to write?" but "if this defect returns through a plausible path, which of these stops or exposes it?" If the honest answer for a given guard is "only if the developer happens to change this one file", that guard is too high in the stack.

A protection you have never seen fail is an untested assertion, and writing the test while the bug is still live is the cheapest opportunity you will ever get to watch it work. Red-green ordering is not a ritual borrowed from test-driven development. It is the only moment when the defect is available as a free, guaranteed source of failure.

The sequence is short:

1. Add the test while the bug still exists in the code.
2. Run it.
3. Confirm the failure reflects the defect — not a broken fixture, a missing permission or an unconfigured test host.

That third line is where most of the value sits, and where most teams skip. A red test tells you something failed. It does not tell you the right thing failed. Capture the four facts that distinguish them:

Command:
Expected failing assertion:
Actual failure:
Why this demonstrates the defect:

If the fourth line is hard to write, the test is probably not exercising the invariant, and finding that out now costs minutes rather than months.

There is a common real-world complication: the repair often lands before anyone thinks about the test. An agent fixes the bug in the same pass that finds it, or a developer patches it locally and only later writes the regression case. When that happens you have lost the original red state permanently. This is exactly when mutation stops being optional — it is the only remaining way to manufacture a red state and confirm the protection is sensitive. A test written after a fix, never observed failing and never mutated, has an unknown value that will stay unknown for as long as it stays green.

One more caution about red states. Confirm the intended test actually ran. Suites that fail during setup, on a simulator that failed to boot, or on a module that did not compile will report failure while never reaching your assertion at all — a false red that reads exactly like a true one in a truncated log.

The smallest coherent repair fixes the invariant at the layer that owns it — not the smallest diff, and not the nearest visible symptom. Those two often point in opposite directions, and choosing the smaller one is how a defect comes back through a door nobody was watching.

For the duplicate-retry defect, a coherent repair might involve some combination of:

• A stable operation identity generated once, at the point the user commits the action, rather than at each attempt.
• An idempotent repository write that recognises the identity and updates rather than inserts.
• A transaction boundary that makes local persistence and status update atomic.
• An explicit local/remote status so "saved locally, not yet acknowledged" is a real state rather than an absence.
• A retry that reuses the identity rather than starting a new operation.
• A UI state representing committed-but-unacknowledged, so the user is not invited to retry something that already succeeded.

Now consider the tempting alternative: disable the retry button faster. It is a two-line change, it makes the reported symptom disappear, and it protects nothing. Background retries still duplicate. A second caller still duplicates. An import still duplicates. The bug did not move — the report did.

Once you know the root cause, audit the sibling paths before you decide the repair is complete. Search deliberately for the same shape in edit, import, restore, sync, undo, other data objects and the other platform. Cross-platform siblings are the ones teams miss most reliably, because the two codebases are usually reviewed by different people in different weeks and share nothing but a specification.

Do not expand the implementation blindly on the strength of that search. Record the siblings you found, apply the invariant where the evidence justifies it, and file the rest as scoped follow-ups with an explicit status. An unbounded refactor triggered by one bug report is how a one-day fix becomes a three-week regression risk. What makes deferral safe rather than quiet abandonment is the findings register described in our post-mortem on shipping an AI-built iOS app, where every finding carries an ID and a mandatory status and "not mentioned" is not one of the permitted values — roughly 70 findings on that project evaporated for want of exactly that discipline.

Run the new protection, the closest relevant suite, the build and type and lint checks, the original runtime path, and every downstream consumer of the data you touched — because the test matrix should follow data and behaviour, not the list of files in the diff. A local fix routinely causes a distant regression, and the distance is measured in data flow rather than directory structure.

The order that catches the most, fastest:

1. The new regression test. It should now pass, for the reason you expect.
2. The nearest suite. The module, feature or repository the change lives in.
3. Build, type and lint. Cheap, and they catch the class of mistake that produces a false green later.
4. The original runtime journey. Perform the manual reproduction again on a real build if the defect was ever user-visible. A green suite and a broken screen coexist more often than anyone likes.
5. Downstream consumers. Charts, exports, sync payloads, notifications, widgets — anything that reads the value you changed the shape of.

Step four is the one that gets dropped under time pressure, and it is the one that most often finds something. A repair that changes what is written also changes what everything downstream reads. We have seen a correct idempotency fix produce a chart that silently stopped plotting the most recent point, because the chart was keyed on the insertion identifier that the repair stopped regenerating. Both the fix and the chart were individually defensible. The integration between them was never checked.

Record what you ran, not just that you ran something. "Tests pass" is not a check a reviewer can evaluate. The command and the summary line are, and they cost nothing to paste. On Android that is typically a Gradle invocation of the command-line test runner; on Apple platforms an xcodebuild test action against a named scheme and destination.

A valid mutation reintroduces the essence of the defect while keeping the program compilable and the protection reachable — anything that merely breaks the build proves the compiler works and proves nothing about your guard. This single distinction separates a real proof from a performance of one.

Mutations that carry real information, drawn from defect classes we see repeatedly:

• Remove normalisation from an import path so unnormalised values reach storage again.
• Change an inclusive comparison to exclusive at a boundary the reference data depends on.
• Generate a new identifier on retry instead of reusing the committed operation identity.
• Skip cache invalidation after an edit so consumers read a stale value.
• Remove a consent gate before a payload is constructed.
• Restore a forbidden hard-coded colour that the token rule is supposed to reject.
• Move a reference-table boundary selector by one row so a small band of inputs resolves to the wrong coefficient.

Every one of those compiles. Every one changes behaviour. That is the bar.

Before you run the target protection, confirm the mutation is actually valid — four checks, none of which take more than a moment:

1. The code parses and compiles, as applicable to the language.
2. The changed branch is reachable from the protection's fixture.
3. The mutation genuinely represents the original bug class, rather than some adjacent defect that happens to be easier to introduce.
4. No unrelated failure prevents the protection from executing at all.

Then run the narrow protection — the single test, the single rule — rather than the whole suite. A full-suite run produces a wall of output in which the failure you care about is one line among thousands, and it invites the reviewer to accept a summary instead of reading the evidence. Narrow output is reviewable output.

If you want the mutation step automated across a whole module rather than performed by hand on one guard, dedicated tooling such as PIT for JVM code will generate and run hundreds of mutants for you. That is a different instrument for a different question — suite-level sensitivity, rather than proof about one specific regression guard — and the two complement each other rather than competing.

There are five possible outcomes of a mutation run, and only one of them is evidence — reading the other four correctly is what stops the method degrading into a box-ticking exercise. The result is not binary, and treating it as binary is how weak protections get certified as strong.

• The protection fails as intended. This is the evidence you were after. Record the exact assertion or rule that detected it, and the failure message, because that message is what a future maintainer will read when the guard fires for real.
• The protection still passes. The guard does not exercise the defect, or does not assert the invariant. This is a finding about your test, not about your fix. Strengthen the assertion or move the fixture closer to the real failing condition, then mutate again.
• The build fails before the test runs. The mutation is invalid for this purpose. Construct a compiling one. Resist the temptation to count this as a pass — it is the most common way the method gets faked, usually unintentionally.
• A different test fails instead. Useful, but ambiguous. Determine whether that other check is in fact the real protection, in which case your claim should name it, or whether your intended guard is simply inadequate and the other test caught the mutation for an incidental reason.
• The result is flaky. Do not treat it as protection at all. A guard that fails half the time on a defect will be muted or deleted by whoever is on call when it fires spuriously. Stabilise the setup and the timing first.

The second outcome is the one worth pausing on, because it is genuinely good news wearing bad clothes. A mutation that survives has told you, at a cost of about ten minutes, that a test you were relying on protects nothing. Without the mutation you would have found that out during an incident.

Interpretation also requires asking whether the failure message is useful. "Expected true but was false" is a weak signal that tells the next maintainer nothing. "Retry created 2 records; expected exactly 1 for operation ID X" tells them the invariant, the observed violation and where to look, in one line. Rewrite weak assertions while you are already in the file.

Return the repository to the repaired state, run exactly the same commands again, and record the whole sequence — because the proof is the sequence, not any single run inside it. A failure without a matching pass is inconclusive, and a pass without a matching failure is unsupported.

The evidence sequence a reviewer should be able to read in five lines:

Original defect:              FAIL
Protection before repair:     FAIL
Repair applied:               PASS
Deliberate valid mutation:    FAIL
Restored repair:              PASS

Store the commands and the concise output alongside the finding or in the pull request. Not the full log — the specific lines that show the assertion firing and the assertion passing. A reviewer who has to scroll through four thousand passing lines to find your evidence will stop scrolling and trust the summary, which defeats the point of producing evidence at all.

The restoration itself deserves care. Restore only the lines you deliberately mutated, then inspect the diff before you run anything. Broad restoration commands are the danger here: a wholesale checkout or reset will happily erase unrelated work in progress alongside your mutation, and that is a genuinely destructive outcome for a step whose entire purpose is safety. Where a version control system offers a scoped mechanism — stashing to isolate a working tree, or a saved baseline diff for comparison — use the narrow tool rather than the broad one.

Finish by confirming the working tree contains nothing but the repair and the protection. No mutation markers, no temporary fixtures, no commented-out experiment, no debug logging left behind from the investigation. The final diff and the repository status are the last two things to look at before the change becomes a pull request.

The protection must match the failure mechanism, not the surface where the bug was noticed — a defect reported as "the chart is wrong" may need a storage constraint, an oracle comparison or a repository test depending entirely on why it is wrong. Mapping class to guard before you start writing saves the rework of building the wrong protection well.

The mapping we work from:

• Wrong boundary or formula — a full-range oracle comparison against an independent source, not spot checks at three convenient inputs.
• Duplicate retry — an idempotency integration test plus a storage-level uniqueness constraint.
• Missing state branch — a state-machine or unit test enumerating every state, plus runtime capture for states you cannot easily construct.
• Forbidden pattern — a lint or static rule, proven with a deliberately violating fixture.
• Privacy payload leak — a schema or allowlist test plus a payload snapshot assertion.
• Stale downstream UI — a repository-to-consumer integration test that follows the value all the way to what renders it.
• Visual drift — an approved visual baseline with automated comparison where it is stable, plus human review for hierarchy and intent.
• Accessibility regression — a semantics test against WCAG 2.2 success criteria, plus inspection with the actual assistive technology.
• Navigation break — a journey test that covers entry, back behaviour and deep link, because those three break independently.

Two of these deserve a warning. Visual drift and accessibility regressions are the classes where teams most often write a guard that cannot fail usefully — a pixel-perfect baseline that fires on every legitimate change until it is disabled, or an automated accessibility scan that passes while the screen is unusable with a screen reader. Neither one is a defect a code diff can evaluate, which is why both need a proven mutation and a human in the loop rather than one or the other.

The other recurring mismatch is choosing a UI test for a data defect because the bug was reported from the screen. The screen is where it was noticed. The storage layer is where it happened.

Lint rules, schema allowlists and static checks need mutation proof more than tests do — because a rule that matches nothing looks identical to a rule that is working perfectly. A clean lint run is exactly what you would see if the rule were misconfigured, scoped to the wrong directory, or silently disabled.

Take a project that prohibits raw hard-coded colours outside the token layer. The proof sequence is eight steps:

1. Add or configure the rule.
2. Confirm the current source passes.
3. Add a temporary violating fixture, or mutate one approved file.
4. Confirm the file still parses.
5. Run lint.
6. Confirm the expected rule fires, at the expected location.
7. Remove the mutation.
8. Confirm lint passes again.

Step six matters as much as step five. A rule that fires with the wrong identifier, or reports a file you did not touch, is telling you it matched something other than what you meant. Searching the repository and finding zero violations is a snapshot of today. The mutation is what demonstrates enforcement tomorrow.

The same shape applies to privacy protection, where the stakes are higher and the false confidence is more dangerous. For a payload sent to an external service — an AI feature, an analytics pipeline, a crash reporter — define an explicit allowlist schema, then test that the payload contains only approved fields. Temporarily add a forbidden identifier such as the user's display name to the builder, confirm the mutated target still builds, run the payload protection, confirm it fails naming that field, then restore and rerun.

That sequence is meaningfully stronger than a policy statement asserting the name is never sent, and it is the version that survives a reviewer asking how you know. It also matters commercially. Your listing already carries a published account of the data your app collects — App Privacy details on the App Store, the Data safety section on Google Play — and a payload that quietly exceeds that account is a compliance problem as well as a privacy one. An allowlist test that has been mutation-proven is the cheapest evidence that your declaration matches your code.

The general rule: any guard whose healthy state is silence needs a mutation, because silence is also what failure looks like.

A reference table is verified only when a single mutated row or coefficient makes the check fail and name the offending entry — anything less proves you compared something, not that you would notice corruption. Lookup tables are where quiet, high-consequence wrongness lives, because a wrong number produces a plausible answer rather than a crash.

The proof sequence for a table your product depends on:

1. Compare every supported row against an independent oracle — the published source, not a copy that shares your import path.
2. Mutate one boundary row or one coefficient by a small, realistic amount.
3. Confirm the fixture still loads and the calculation still executes.
4. Run the full-range verification.
5. Confirm the mismatch report identifies the specific metric, group, age band and field.
6. Restore and rerun.

Step five is the whole point. A check that reports "reference data mismatch" has told you almost nothing; a check that reports the exact row has told you where to look. Realistic mutation size matters too — mutating a coefficient by an order of magnitude proves only that your check can detect an absurdity, which was never the risk. The risk is a transcription error in the third decimal place, or a boundary row that is off by one age band. Mutate at that scale.

This is the class of defect where a screen-by-screen review is structurally incapable of helping, which is the argument of part 11 of this series. A wrong percentile, a wrong tax band or a wrong dosage threshold renders beautifully. Nothing about the interface looks broken. The only instrument that finds it is executing the table against an authority and having a check sensitive enough to fire on a single row.

Two practical notes. First, version the oracle — reference data gets revised, and a check that passes against the wrong edition is worse than no check. Second, run the comparison over the entire supported domain rather than sampling. Sampling is how a defect confined to one age band or one boundary condition survives a verification that reports full coverage.

Write the contract once, then reuse it for every consequential defect — because the value of this method comes from it being the default shape of a fix, not something you remember to do when a bug feels serious enough. The template below is the one we hand to developers and to AI agents unchanged.

BUG FIX CONTRACT — [FINDING ID]

INVARIANT
- Rule that must always hold:
- User/data consequence:

ORIGINAL FAILURE
- Build/commit:
- Fixture/preconditions:
- Reproduction steps/command:
- Observed output:
- Expected output:

ROOT CAUSE
- Owning layer:
- Why current protection failed:
- Sibling paths to inspect:

PROTECTION
- Type: unit / integration / UI / lint / static / schema / runtime
- Exact assertion or rule:
- Why it matches the invariant:
- Expected red failure:

REPAIR
- Smallest coherent change:
- Behaviour preserved:
- Paths affected:

VALID MUTATION
- Change that reintroduces the bug class:
- Why it remains compilable and reachable:
- Intended protection failure:

EVIDENCE SEQUENCE
1. Original or pre-fix protection fails.
2. Repair passes target and neighbouring checks.
3. Mutated implementation builds and reaches target.
4. Protection fails for intended reason.
5. Restored repair passes again.

COMPLETION REPORT
- Files changed:
- Commands and results:
- Mutation output:
- Runtime verification:
- Remaining uncertainty:
- Project Bible update:

Two fields do disproportionate work. "Why current protection failed" forces a diagnosis of the process, not only the code — the answer is usually that no protection existed at that layer, which is itself the argument for where the new one belongs. "Remaining uncertainty" gives the honest answer somewhere to live. Without that field, uncertainty either gets suppressed to make the report look clean, or inflates the report with hedging that nobody can act on.

The last field connects this guide to the next one. A contract completed and then discarded protects one defect. A contract whose rule and command get written into a durable project instructions file protects the class, which is what part 14 of this series is about.

AI agents are genuinely good at this workflow — locating sibling patterns, writing the protection, performing a controlled mutation — but only when the instruction forbids the shortcuts, because every shortcut available to an agent produces a confident narrative instead of evidence. The constraint that matters most is refusing to accept prose in place of output.

The requirements we put in the instruction, each of which exists because we watched its absence cause a problem:

• Preserve unrelated work. No broad restoration commands, no reverting files outside the mutation.
• Explain the invariant before editing. An agent that cannot state the rule is about to patch a symptom.
• Show the failing case with the command and the output, before the repair.
• Name the protection explicitly — file, test or rule identifier, and the assertion.
• Make one mutation at a time. Two simultaneous mutations produce an uninterpretable result.
• Confirm mutation validity — that it compiles and that the branch is reachable — before running the guard.
• Restore the repository and show the diff.
• Show final status rather than asserting cleanliness.
• Avoid destructive commands by name, listing the ones that are prohibited.
• Report inability when the environment cannot execute a required check, rather than describing what the check would have shown.

That last one is the highest-value line in the whole instruction. An agent with no simulator available will otherwise produce a fluent account of a test run that never happened — not from dishonesty, but because completing the narrative is the path of least resistance. Give it an explicit, approved way to say "blocked" and it will use it.

Do not accept a narrative claiming a mutation was performed. Require the command and the relevant failure output. This is the same discipline the whole series rests on, developed at length in part 8, on prompts that produce evidence rather than confidence: an agent asked for a verdict returns a verdict, and an agent asked for evidence returns evidence.

One project-specific note. Agents are markedly better at this loop when the invariant is written down somewhere durable rather than restated in each conversation. In our own AI-assisted iOS build, the rules that were captured in the project instructions file held across hundreds of commits; the rules that lived only in chat history did not survive a single context reset.

A reviewer should never have to infer the proof from a large diff — give them a compact sequence they can challenge, in a fixed order, at the top of the description. The reviewer's job is to evaluate whether the evidence supports the claim, and that is only possible if the evidence is legible.

Bug/invariant:
Original reproduction:
Root cause:
Repair:
Protection:
Pre-fix or mutation failure:
Post-fix pass:
Neighbouring checks:
Runtime evidence:
Remaining uncertainty:

Attach only the relevant output. Thousands of passing lines do not strengthen the case; they obscure the two lines that carry it, and they train reviewers to skim. If the failure output is long, quote the assertion and link the full log.

Then review the failure reason rather than the failure itself. A red state is useful only when it is red for the right cause. For each one, ask five questions:

• Did the target test actually execute? Or did the run die during setup?
• Did the intended assertion fail? Or a different one earlier in the same test?
• Was the fixture valid? A malformed fixture fails convincingly and means nothing.
• Did the environment fail first? Network, simulator, emulator, missing credential.
• Would the failure message guide a future maintainer? If not, improve it now.

This changes review culture more than it changes engineering. It replaces the argument about whether a change "probably covers it" with a visible sequence any reviewer can attack on its own terms. When the mutation survives, the protection is weak and everyone can see it. When the wrong assertion fails, the signal is noisy and everyone can see that too. When the intended guard fails and the restored repair passes, the team has evidence proportional to the claim — which is the entire objective.

It also shortens reviews. A reviewer who can verify the proof in thirty seconds does not need to read every line of the diff looking for reasons to worry.

Perform one controlled mutation at a time, save a baseline diff before you make it, restore only the lines you changed, and verify the working tree afterwards — because a deliberate defect that escapes into a shared build is a self-inflicted incident with an embarrassing root cause. This risk is small, but it is entirely preventable and the prevention costs almost nothing.

The discipline, in order:

1. Capture the baseline. Save the current diff before mutating, so unrelated working-tree changes are visible and distinguishable from your mutation.
2. Mutate one thing. Record the exact file and the conceptual change in the contract, not just in your head.
3. Restore only the mutated lines. Never a broad revert of the whole tree.
4. Inspect the diff. Read it, do not assume it.
5. Re-run the target protection and the required neighbouring checks.
6. Confirm nothing remains — no mutation markers, no temporary fixture files, no disabled assertions, no debug output.

The failure mode to design against is the interrupted session. A mutation made at the end of a working day, left in the tree overnight, and committed the next morning alongside unrelated work is exactly how this goes wrong. If a mutation must survive a break, leave it obviously broken rather than subtly broken — a failing check is a much better reminder than a comment.

Keep mutations out of commits by default. There are two legitimate exceptions: a dedicated mutation fixture that a lint rule or schema test is designed to reject, and configuration for a mutation-testing framework. Both are permanent parts of the protection rather than temporary defects, and both should be named clearly enough that nobody mistakes them for real code.

Never allow a temporary defect to reach a shared branch, a build server or a beta channel. The whole method exists to reduce risk; a mutation in a release build would be the one way to make it a source of risk instead.

When the invariant can be enforced by a mechanism that makes the wrong change impossible rather than merely detectable, that mechanism beats any test — because it protects paths nobody thought to test, including the ones that do not exist yet. A test remembers to check. A constraint cannot forget.

The mechanisms worth reaching for, and what each one buys:

• Unique indexes prevent duplicates at the storage authority, regardless of which caller attempts the write.
• Foreign keys protect relationships, so an orphaned row cannot be created by any code path.
• Types and domain models restrict invalid combinations to the point where the wrong state cannot be constructed.
• Schemas reject forbidden payload fields before they leave the process.
• Lint and static rules forbid dangerous patterns at authoring time, when the fix costs seconds.
• Continuous integration prevents merging when verification fails, which is what turns every guard above from advisory into binding.

Constraint-based guards do come with real operational cost, and pretending otherwise leads teams to add one and then remove it during the next incident. A unique index will reject a write at runtime, so the application has to handle that rejection gracefully rather than crash. Adding a constraint to an existing table needs a migration that deals with the violating rows already there — the duplicates a unique index will reject, the orphans a foreign key will reject — and on a large table the verification scan is itself an operational event. PostgreSQL's ALTER TABLE documentation is explicit that adding a constraint normally scans the whole table and locks out other updates until it commits, which is why it offers a NOT VALID option that defers that scan for check and foreign-key constraints. Plan the migration as part of the repair, not as a follow-up.

And still mutation-test the mechanism. Attempt the invalid write and confirm it is rejected. Compile the invalid type use in a fixture and confirm it fails. Add the forbidden payload field and confirm the schema rejects it. Introduce the lint violation and confirm the rule fires. A constraint that was never exercised is a configuration you believe in, which is precisely the state this whole method exists to eliminate.

The question stays the same at every layer: if the defect returns through a plausible path, which protection stops or exposes it?

Visual and content defects need protections that detect the failure class without freezing every pixel or every word — a baseline that fires on legitimate changes gets disabled within a month, and a disabled guard protects nothing. This is the hardest category to guard well, and the one where over-engineering fails as reliably as under-engineering.

For a visual regression, the workable combination is four parts:

• A controlled fixture and an approved baseline — fixed data, fixed device class, fixed text size, so the only variable is the thing you are protecting.
• Automated comparison where the surface is stable. Static components and layout structure compare well; animated, data-driven or dynamically sized surfaces do not.
• A mutation to prove detection. Change the token or the layout constraint and confirm the comparison fails. A baseline that has never rejected anything is unproven.
• Retained human review for hierarchy, emphasis and qualitative intent, none of which a diff can evaluate.

For a content or privacy disclosure defect — a required consent line, a prohibited health or financial claim, a missing attribution — the equivalent set is: test for the required semantic elements or structured content, add a prohibited-claim or required-field check, mutate the copy or the structure and confirm the check fails, then review the final runtime context rather than only the string. A required disclosure that exists in the string catalogue but is clipped, hidden behind a fold or displayed in the wrong locale is still a missing disclosure.

Localisation deserves a specific note, because it is where content protections most often prove insufficient in practice. An app shipping across Indian languages will see string lengths vary substantially between English and Devanagari or Tamil renderings, and a disclosure that fits comfortably in English can truncate in three other locales while every test stays green. Guard the presence of the element with an automated check; verify the rendering at the widest realistic text length and in the longest supported language, using the accessibility text-size settings the platforms already provide.

The bar for this category is lower than for data defects, and it should be. Aim for a guard that would catch the defect class if it returned, not one that catches every change.

Eight mistakes account for nearly every case where this method is followed in form but produces no protection in substance — and most of them are shortcuts that feel efficient in the moment. Recognising them is cheaper than discovering them during an incident.

• Writing the test after the fix and never seeing red. The most common of all. The remedy is not guilt, it is a mutation — it is the only way to recover the missing red state.
• Mutating into a compilation error. Proves the compiler works. Create a valid behavioural regression instead.
• Asserting implementation details. A test coupled to internal structure breaks on every refactor and stops proving anything about the invariant. Assert outcomes, unless a prohibited code pattern is itself the risk.
• Testing only the main path. The root cause frequently also exists in import, sync, restore, undo or retry — and those paths are where nobody looks.
• Running only the new test. Run the neighbouring suites and the original runtime journey. A local repair with a distant regression is a net loss.
• Keeping mutation changes accidentally. Use explicit diffs and a final status check. Restore only your controlled mutation, never unrelated work.
• Calling manual verification permanent protection. Manual evidence closes the current claim and nothing else. Only an automated guard protects future changes.
• Protecting symptoms. Disabling a button does not enforce idempotency. Put the guard at the authority that owns the invariant.

If your team is deciding what warrants this treatment, the practical filter is consequence plus recurrence: apply the full loop to defects that are expensive if they return, and to any bug class you are about to declare eliminated. Those two categories are also, in our experience, the ones that do the most damage to a young product — quiet data defects that nobody guarded, discovered by users rather than by tests.

A single occurrence is repaired when its evidence sequence passes; a class is closed only when the scope was inventoried, every current occurrence was resolved or justified, a guard covers future occurrences, that guard was mutation-tested, the exceptions are explicit, and the rule is written down. Six conditions, and skipping any one of them turns "we fixed the file we saw" into the much larger claim that the issue can never happen again.

Those conditions in order:

1. The pattern's scope was inventoried. You know how many occurrences existed, not just that you found some.
2. All current occurrences were resolved or justified. A documented, reviewed exception counts; an unexamined one does not.
3. A guard covers future occurrences. Not a code review habit — a mechanism.
4. The guard was mutation-tested. You have seen it fail on a realistic reintroduction.
5. Exceptions are explicit and reviewed. Every suppression has a named reason and an owner.
6. The project instructions record the rule and the command. So the next developer, and the next agent, inherit it.

When the same class keeps returning, the answer may be an earlier guard rather than another test at the same layer. A domain type is earlier than a runtime error. A lint rule is earlier than a code review. A storage constraint is earlier than a corrupted report. Each move earlier can make the wrong change harder to write and any remaining violation louder when it happens. In practice, combining an earlier preventive control with a sensitive regression check is more durable than adding another downstream assertion.

Do not use mutation as theatre. Choose a realistic return path for the defect, preserve the output that demonstrates detection, keep the mutation narrow and reversible, and inspect the final diff and repository status before anything is committed. A fix changes the current implementation. A protection changes the future risk — and only the second one compounds.

Define the invariant. Reproduce the failure. Add a guard that fails. Repair at the correct authority. Deliberately reintroduce a valid version of the bug and prove the guard catches it. Restore and show green. That evidence is what gives the word "fixed" a meaning beyond confidence — and if you want help building this discipline into a team that is shipping fast with AI assistance, talk to us. The final guide in this series covers how to preserve these invariants, commands and protections in a project bible that future agents will actually read.