I use Claude Code every day, and these 17 skills are the habits I got tired of re-teaching it. The library covers building and operating AI systems in production: adversarial and tiered review, validation gates, cost and safety guardrails, deploy campaigns, debugging playbooks, and change control. In July 2026 I distilled 26 skills from two production codebases I run Claude Code against, and every one went through four review passes plus an adversarial pass by Codex at extra-high effort, run twice. Eight were then A/B tested against pre-registeredmust-hits and all eight passed, and the library was consolidated into the 17 portable skills on this page, an open instrument that ships with its own eval harness.
A skill here means a Claude Code skill, a written instruction file that changes how Claude behaves while it works, nothing to do with your own tools. Learn more →
The evidence behind them is measured, not asserted: the July 2026 study and the effort lattice both graded these skills against pre-registered criteria, and the whole library is open source at github.com/HamzaYM/reliable-ai-skills.
01The library
The 17 skills
Each skill is a self-contained folder with one SKILL.md: a description that tells an agent when to reach for it, the failure mode it prevents, and the pattern as decision gates and checklists rather than prose. The groups below are the library’s real category directories. Every card links its skill file and its folder in the repo, and every card’s From the skill section quotes or closely paraphrases lines that are actually in that skill’s file.
Click any card for the full skill file, rendered in place.
No skill matches that search.
Three to start with
The three richest skill files in the library, each opened up to one real pattern from inside it.
adversarial review
multi-model-adversarial-review
Asking a model to critique its own work mostly gets you agreement, because its blind spots are its reviewer’s blind spots too. This skill runs a second model from a different vendor, then makes every finding survive a deliberate attempt to knock it down before it gets reported. A long list of nitpicks with no real issue is read as nothing substantive found, not as lots of problems.
Some stacks have no real local setup: what feels like a scratch database on your own machine is shared infrastructure the whole team depends on, one destructive command away from damage. The skill’s rule is to confirm where you are by the real hostname or account ID, never by an environment label. Some shared environments deliberately label themselves production to pick up production’s safety defaults, so the label misleads by design.
Every mature codebase has approaches that were tried, didn’t work, and were abandoned on purpose, and the reasoning usually lived in a conversation that is gone. This skill keeps a settled-battles record: what was tried, why it was dropped, and what the codebase does instead. Its most honest rule: when the reason for a revert is lost, the entry says so plainly, because an admitted unknown beats a plausible-sounding guess.
Two pre-registered studies grade this library, and a second vendor’s model then re-checked the judging behind the second one. Each study lives on its own page with full methodology, and the harness that ran them is open in the repo’s eval/ folder.
How the July 2026 study ran
Freeze gateThe task list locks firstNothing runs until the list is frozen: 16 tasks, 50 pre-registered must-hits.
↓
One task from the frozen listEach of the 16 tasks walks this same path.
↓↓
Cold armAnswers with no skill loaded
Loaded armAnswers with the skill loaded
↓↓
Answer A
Answer B
↓
Order scramblerShuffles which answer comes first, so position gives nothing away.
Order key, kept secretThe map of which answer is cold and which is loaded. The judges never see it.
↓
Blind judgesScore both answers against the pre-registered checklist.
↓
The resultCold 26/50 (52%)→Loaded 46/50 (92%)
Honest note: these judges were order-blinded, not content-blinded; a few answers mentioned their own skill in passing. The sequel's harness closed that gap by scrubbing those tells before judging.
The July 2026 study
Baseline 26 of 50. With the skills loaded, 46 of 50 (92%).
An order-blinded A/B across 50 pre-registered must-hits in 16 tasks. All 8 tested source skills passed the pre-registered rule.
Effort raises the floor. The skills still add on top.
Three Claude models at five reasoning-effort levels, each run with the skill library installed and without it. More effort raised the no-skills baseline on all three models, and it did not shrink what the skills add on any of them.
A second vendor agreed on 383 of 394 marks (97.2%).
After the lattice run, Codex, a model from a different vendor, re-scored a fixed sample of the committed judge inputs in one run and agreed with the panel’s judging 97.2% of the time. How the check ran is on the lattice page.
A single model reviewing its own work is not an adversarial review, even if you ask it to "be critical." One model's blind spots are usually its reviewer's blind spots too. The pattern that actually catches things: run two models from different vendors or of different sizes, then reconcile their findings instead of concatenating them.
Why this works
Different models (or the same vendor's different model tiers) are trained differently enough that their errors are only partially correlated. Agreement between them is a real confidence signal. Disagreement is where the interesting findings live. But they are still both language models trained on overlapping data: agreement reduces stochastic misses, but it does not clear a shared blind spot. For anything safety- or correctness-critical, pair this with at least one non-LLM check (run it, compute it, grep for it).
The pattern
Identify the artifact. A diff, a file, a plan, a claim. Decide whether you're reviewing code or prose: the framing changes what "finding" means.
Run your primary reviewer's own pass first (one or several review lenses; see below). Don't skip this: the second model adds diversity, it doesn't replace your own critique.
In parallel, run a second model (a different vendor's CLI, or a different tier of your own vendor) with read-only access to the same artifact. Give it a specific focus question, not just "review this." Never send secrets, regulated data, or embargoed material to a third-party model you don't already trust with that data.
Synthesize. The actual value is here, not in step 3. Never just paste the second opinion into your output; reconcile it against your own.
Run the two passes concurrently rather than serially; on a large artifact, split it into chunks and review each chunk separately, since most review wrappers warn about size but silently truncate rather than erroring.
Multi-lens fan-out (when one reviewer isn't enough on its own)
For anything merge-bound or high-stakes, don't rely on a single reviewer even within one model family. Fan out several independent reviewers, each briefed on a distinct lens (e.g., correctness, security, product-intent fidelity, completeness) and, if your tooling allows model overrides, running on different model sizes so they don't share a blind spot. Two stages, not one: a genuine correctness/simplicity pass first, then the adversarial pass. A same-model, same-lens panel is not a real panel.
Refute-verification: the load-bearing step
Every candidate finding must survive an attempt to refute it before it gets reported. In practice: route each finding to two or three independent reviewers whose job is specifically to try to knock it down. A finding survives only if it withstands that attempt. This single rule is what keeps a review from turning into a pile of plausible-sounding but wrong "findings." Fewer real findings beat a long list of maybes, and a wall of nitpicks with no real issue means "nothing substantive found," not "lots of problems."
Add one more pass after the fan-out: a completeness critic whose only job is "what did we not look at?"
Synthesis contract
Tag every finding by source (which lens, or which model raised it).
Agreement across independent reviewers = high confidence. Surface these first.
Single-source findings get adjudicated, never rubber-stamped and never silently dropped. State agree / disagree / uncertain with a one-line reason.
On factual disagreement about something material, run one rebuttal round. Give the dissenting side the specific counter-evidence and ask it to withdraw (if genuinely refuted) or hold and sharpen its reasoning. Cap it at one round (two for genuinely high-stakes cases). A model that withdraws immediately just because it was told the other side disagrees is a known failure mode, the rebuttal prompt should explicitly say "withdraw only if the evidence refutes you," not "agree to be agreeable."
Matters of taste are the lead reviewer's call. Tone, styling, severity, record the dissent in one line, decide, and move on. Don't escalate taste to a human.
Escalate to a human only when a finding is material and genuinely unresolvable by the reviewers themselves, not merely because two models disagree.
Asymmetric veto for self-checks. If a second opinion would, if true, mean an error in your own prior work or output, don't dismiss it on your own authority. That's precisely the case you're least equipped to judge fairly. Verify it with a non-LLM check or escalate, regardless of how confident you feel.
Weigh the sampling asymmetry: if you ran several review lenses and the second model ran one pass, "the second model didn't flag it" is weak evidence either way.
Fold-back
Fixes for confirmed findings should be test-first where feasible (write the failing test, then the minimal fix) and re-run the full validation gate before calling the finding closed. Standing practice worth adopting: after a PR opens, dispatch one more independent reviewer against the live diff and fold any real findings back the same way. Reviewers should return their verdicts to whoever is orchestrating the review, not post directly to external systems (PR comments, tickets). That keeps a human or lead model as the actual editor of what gets reported.
Graceful degradation
If the second model or reviewing tool is unavailable (not installed, unauthenticated, timed out, empty output), proceed with the single-model review and say so explicitly in your report. Never block a review pass on tooling being down, and never silently downgrade to single-model without disclosing it.
A related, prospective use of the same idea
The pattern above is retrospective: it critiques finished work. The same two-model idea also works prospectively: before a consequential, hard-to-reverse decision (an architecture choice, a migration, a release), ask a second model for its take on the options and risks before you commit. Same rule applies: it advises, you decide. Reserve it for genuinely high-stakes forks. Overusing it just adds latency and becomes a way to avoid deciding.
Tiered consultancy review
Most review setups are flat: one reviewer, one pass, done. That catches surface errors but misses the things a real firm catches before a partner-level deliverable goes out the door: weak framing, an unearned claim, a tone mismatch for the audience, a competitor argument nobody stress-tested. The fix is to run review as an actual escalation ladder, the way a consultancy staffs a real engagement: juniors first, then managers, then specialists, then partners, and only then does it reach the person who actually owns the decision.
The ladder
Each tier has a distinct job. Don't collapse them into one giant "review this" pass, the value is in the separation.
Analysts (parallel, blind to each other). Two or three independent first passes, each reading the draft cold through a different frame (e.g., one for the core argument, one for factual/technical grounding, one for how a skeptical outsider would read it). Blind means they don't see each other's notes yet: that's what prevents one loud opinion from anchoring the rest.
Managers (synthesis). Merge the analyst passes into one coherent draft. Resolve overlaps, keep the sharpest version of each point, and drop redundant feedback. This is where "several good ideas" becomes "one good draft."
Specialists (targeted passes). One or two reviewers with a narrow, specific mandate, e.g., "is every factual claim actually true and not overstated," or "does this read naturally for the intended audience, not like a machine wrote it." Specialists catch what generalists miss because they aren't trying to catch everything at once.
Partners (three angles, not one). This is the tier that decides if it ships:
Internal partner: reads for whether it serves the requester's actual goal and asks for anything that strengthens their position.
External partner: reads as the actual audience would, with zero sympathy for how the sausage was made. Cuts throat-clearing, hedging, and anything that reads as unearned.
Adversarial partner: actively tries to find the argument's weakest point, the way a skeptic or competitor would attack it. This tier catches structural problems the earlier tiers were too close to see (e.g., a draft that led with the wrong achievement because the "obviously best" evidence actually undercut the core claim).
Final polish. One last pass for length, redundancy, and voice consistency after all the substantive edits have landed.
Rules that make this actually work
Each tier can do one of three things with what it receives: fix it directly, confer with a peer at the same tier, or kick it back down with specific, actionable feedback. A tier that can only pass things up is not adding review, it's adding latency.
Keep a review ledger. One line per material change: what was flagged, at which tier, and what changed as a result. This is what lets you defend the final version later and lets the next review start from where this one left off instead of re-litigating settled points.
Escalate to the actual owner only for calls only they can make: a factual claim only they can verify, a framing choice that depends on context the reviewers don't have, a tradeoff between two legitimate options. Don't escalate taste; the partner tiers exist to make taste calls.
Calibrate the depth of the ladder to the stakes. A five-tier pass with three independent adversarial angles is for something that ships externally and is hard to walk back. An internal working doc gets one honest pass, not the full ladder. Running the whole pipeline on low-stakes material is how review theater happens.
The adversarial tier is not optional for anything customer- or decision-facing. It is the one most likely to be skipped because it feels redundant after four other passes already said "looks good," and it is the one that catches structural problems the others structurally cannot, because everyone before it was working from inside the same frame.
A parallel version for finished code or docs
The same escalation idea works as a fixed-panel variant rather than a strict ladder, useful when you want speed over depth: run several review lenses in parallel and blind (fidelity to spec, completeness against a checklist, clarity for a reader with zero context, honesty about alternatives considered, and a red-team pass), then route every material finding to an independent verifier whose only job is to try to refute it against the actual artifact (the live code, the actual source document). Report confirmed vs. refuted counts explicitly, fix every confirmed finding, and re-verify the fix against the real artifact rather than trusting the diff. This variant trades the sequential ladder's depth for running everything at once. Pick it when the artifact is well-defined and you want thorough coverage fast rather than a genuinely escalating argument.
When not to use this
For a quick correctness check on a small diff, use a single adversarial pass instead (see the adversarial-review pattern). The full ladder is overkill and will just slow you down without adding real signal.
Pre-merge validation gate
"The build passed and the unit tests are green" is not the same claim as "this works." Treating them as equivalent is the single most common way agentic work ships a real bug that automated checks structurally cannot see: a UI regression that a type-checker and a passing test suite both miss because neither one looks at a rendered screen.
Core rule: a UI-touching change is not done until it has been driven live
Build passing plus unit tests passing is necessary, never sufficient, for anything that changes what a user sees or interacts with. If there's no CI safety net on the branch you're targeting, treat your local gate as the only one that exists. Don't assume a downstream check will catch what you skipped.
Scale the gate to the change
Not every change needs the full gate. Match the check to the blast radius:
Change
Minimum required
Docs/config only
None of the below
Backend-only, no client-visible behavior change
Backend test suite
Backend change to an endpoint a client calls
Backend suite + integration/e2e suite
Anything in the UI layer
Full suite + a live, driven check of the changed screen
Shared/high-traffic UI chrome
All of the above + any byte-identical or snapshot fixtures that depend on it
New or changed user-facing strings
Land in every locale in the same commit, then run the test suite (a locale-parity check, if you have one, is the only thing that catches a string landing in one locale only)
Live verification, not just green CI
Driving the actual feature (clicking it, submitting it, refreshing it, observing the real output) is not optional for a UI change, even when an end-to-end suite exists and passes. End-to-end coverage tests what someone thought to write a test for; live verification catches what nobody thought to test yet. Concretely: bring up the real stack, exercise the specific thing you changed, and report what you actually observed (screenshot as evidence when you can produce one), not just "the suite passed."
Gate paid or costly test suites behind an explicit switch
Any suite that spends real money (a live LLM call, a paid third-party API) should be opt-in, not run-by-default. Keep it deselected in your default test config and only enable it explicitly when asked. Don't "fix" that deselection just because it looks like a suite is being skipped: that's the intended behavior, and removing the opt-in gate is how you get a surprise bill.
Baselines drift: measure, don't assume
Lint/type-check error counts, warning counts, and "known-flaky" baselines are a snapshot from whenever someone last measured them, not a permanent constant. State the bar as "zero new problems in the files you touched" rather than an absolute number, and re-measure the baseline yourself before citing it. Don't copy a number from an old doc or a stale comment.
Byte-identical / snapshot fixtures: recapture is not a rubber stamp
If your project pins some UI surface to committed fixtures (golden screenshots, byte-identical content baselines), a failure right after your intentional change is expected, but recapturing over it is not automatic. Before committing a recaptured fixture: read the diff against the old fixture and confirm every changed hunk traces to your intended change. Anything you can't explain is a regression, not a fixture update. Investigate it; don't launder it into the new baseline. Recapture, then re-run in assert mode (no capture flag) at least twice to confirm the new fixture is actually deterministic before committing it.
Reporting: unambiguous, not optimistic
Report the gate as a table: one row per check, with exit status and counts, new-vs-baseline where applicable. Then:
List every applicable stage you did not run, and why ("environment wasn't up" is a finding: go start it, don't skip the step and stay silent about it).
Never write "tests pass" if any applicable stage was skipped or any failure is unexplained.
Quote failures verbatim and label each one as pre-existing (already failing before your change) or introduced by it.
For anything verified live, state exactly what you exercised and what you observed.
Environment write discipline during verification
If verifying a change requires writing to a shared environment (a staging database, a real third-party service), positively identify the write target before you write anything. A working credential proves nothing about which environment it's pointed at; check the actual resource identity (account ID, hostname, connection string marker), not an environment-name variable that might be misleading by design. Never report a verification write as a "no-op": if it created, published, or mutated anything, say so. Some records are legally or operationally material the instant they're created (a published policy version, a config version, anything append-only). There is no "test" scope that makes writing one of those safe; if a verification step seems to require it, stop and escalate rather than improvising a workaround.
When not to use this
This is the "is it done" gate, not the review-quality gate. Pair it with an adversarial or tiered review for anything merge-bound (see the adversarial-review and tiered-review skills). It's also not a debugging guide: if the gate surfaces a failure, that's a handoff to a systematic debugging pass, not a reason to loosen the gate.
Evidence: both source versions of this gate passed the pre-registered, order-blinded A/B of July 2, 2026 (one source: must-hits 1/3 cold to 3/3 loaded on one task and a 4/4 tie on the other; the second: 0/3 to 3/3 on a consent-scenario task and a 2/3 tie); this merged rewrite was later tested in the July 2026 effort lattice, where the full 17-skill library ran against every model and effort level (results/matrix/MATRIX.md; evidence note updated 2026-07-12).
Architecture contracts as law
Large systems accumulate implicit invariants ("this module never imports that one," "only this file talks to the LLM SDK") that everyone half-remembers and nobody has written down. The fix is a single, explicit contract document that is treated as merge-blocking law, updated in the same commit as any change to what it governs: not a wiki page that quietly goes stale.
The contract document is a source of truth, not documentation
Whatever you call it (a CONTRACTS file, an architecture decision record, an invariants doc), it needs one property to actually work: drift between it and the code is treated as a bug, not as "the doc needs updating eventually." Concretely:
Any change to schema, API/wire shape, route surface, or a documented invariant updates the contract document in the same commit, and the change is called out explicitly in the summary of that commit/PR.
If the contract is ambiguous or looks wrong for what you're trying to do, that's a stop-and-surface moment. Don't silently improvise an interpretation and move on.
Locked algorithms or formulas (see the llm-eval-harness-and-scoring-pipeline skill for a concrete example) live here, with an explicit note on who has to sign off before they can change.
Enforce a module dependency direction, and name the exceptions
If your system has a layered or DAG-shaped dependency structure (module A may depend on B but never the reverse), write the intended direction down explicitly, and separately maintain a short list of the real, current exceptions. In any system old enough to matter, reality has drifted from the original diagram at least a little. An exception list that's honest about existing deviations is far more useful than a diagram that's aspirationally clean but wrong. New code should follow the documented direction; an existing exception is not a license to add a new one elsewhere without the same scrutiny the original one presumably got.
Verify programmatically, don't just trust the doc:
# generalized pattern, adapt the paths/import syntax to your language
grep -rn "^from your_module\." path/to/leaf_module | grep -v "expected internal imports"
Isolate third-party SDK / vendor boundaries to one module
If your system calls an external LLM provider, payment processor, or any other vendor SDK, funnel every call through a single module that owns that boundary. Domain code gets a thin accessor function, never the raw SDK client. This buys you two things: a swappable provider (changing the default model or backing vendor is a one-line change in one place) and an enforceable invariant you can literally grep for:
Keep an explicit, current list of the rare legitimate exceptions (e.g., a batch-API client that needs its own SDK instance, or a module that only imports the SDK's error types for exception handling). Anything not on that list is a violation.
Version anything whose meaning changes over time
Prompts, scoring rubrics, rule configurations: anything read at runtime whose content changing changes behavior should be versioned as explicit files or records, not edited in place. Every row/record produced using one of these should record which version produced it, so you can answer "what changed, and which past results does it affect" without guessing. New versions get added as new files; old versions stay around for as long as anything produced under them needs to remain comparable.
Keep an explicit deferred-work ledger with pickup triggers
For deliberately-postponed infrastructure (a queueing system you don't need yet, an auth provider swap, a caching layer), don't just leave a "TODO: maybe someday" comment. Keep one ledger with, per deferred item: what's deferred, what the interim shortcut is, and the explicit, concrete trigger that means it's time to build it now ("when scoring consistently takes longer than N minutes," "when a second external identity provider is needed"). This does two things: it stops people from building speculative infrastructure early ("check the ledger: has the trigger actually fired?"), and it stops the deferred item from being silently forgotten once the trigger does fire.
Freeze identifiers that other systems depend on for comparability
If an ID (a category, a rubric dimension, a taxonomy key) is used anywhere that compares values across time or across records, treat renaming or removing it as almost never acceptable once real data exists under it. A rename breaks every historical comparison silently. Add new identifiers for new concepts; don't repurpose old ones. Where a decision like this has already been made, document it as settled rather than leaving it to be re-litigated by whoever encounters the temptation next.
Physical infrastructure names can be permanent even after logical renames
If you rename a logical reference to a piece of infrastructure (a Terraform resource address, a code-level identifier) but the actual deployed resource has a physical name baked in at creation (an IAM role name, a fixed identifier some other system already depends on), renaming the physical name usually means destroy-and-recreate, not a clean rename. Decide explicitly whether that's worth doing as its own planned operation. Don't let it get bundled unintentionally into an unrelated change just because the logical name looks inconsistent.
Don't cite a number: read the source
Any place your documentation states a count that can drift (a count of error codes, capability flags, dependency-free modules) is guaranteed to go stale, and multiple docs disagreeing about the same count is worse than no doc at all. It hands the reader three wrong answers instead of one honest "go read the code." Where you need to communicate something like this, either point at the single source of truth to compute the number, or drop the number entirely and describe the property instead of the count.
When not to use this
For the specific technique of keeping a symptom-driven token/permission reference for auth systems, see the multi-tenant-auth-reference skill. For which document wins when several pieces of documentation disagree with each other, see the docs-of-record-and-arbitration skill.
Multi-tenant auth and tenancy reference
Auth bugs in a multi-tenant system are expensive to debug by guessing, because the failure modes look identical from the outside (a 401, a blank page, a row that "should" be there) but have completely different root causes. The fix is to maintain a ground-truth reference for your own system's token/role model instead of re-deriving it from scratch every time, and to never guess which token or role kind is in play.
Document every token/credential kind as a table, not prose
If your system mints more than one kind of credential (a full user session, a scoped short-lived token, a share link, an account-level token distinct from a session token), keep one table: what claims discriminate it, what mints it, what verifies it, and its TTL. The single highest-cost mistake in this space is guessing which kind of token a code path expects instead of looking it up. Treat two wrong guesses in one session as a sign the reference is missing or stale, not a sign to guess harder.
If tokens are stored client-side across multiple keys (a global session token, a per-resource token, an account-level token that must never get swapped into the global slot), document the full key set and both directions of any swap that can happen. The classic bug here is exactly that: an unrelated code path swaps the global slot back to the wrong credential mid-flow, and everything downstream 401s in a way that looks unrelated to the actual cause.
Build a symptom → cause → fix decision table
Keep a running table of "if you see X, it means Y, do Z" for your own system's actual auth errors, e.g.:
Symptom
Likely cause
Fix
401 "wrong token kind"
A route expects one credential type and received another
Check the discriminator table above; don't retry with a different token blindly
401 after a previously-working flow
A credential got clobbered by a parallel bootstrap/session-refresh path
Check the storage-key swap directions, not just the immediate call site
403 on a role that should have access
A role guard is narrower (or a permission model is more granular) than the role name suggests
Check whether access is actually gated by a capability/permission, not the role string; see below
Cross-tenant read returns zero rows instead of an error
Isolation is enforced by a session-scoped context variable that wasn't set on this code path
Confirm the isolation context is pinned before the query runs, not after
This table is worth more than any individual fix, because most "new" auth bugs in a mature system are re-encounters of an already-diagnosed failure mode.
Prefer capability checks over role-string checks
role === "admin" (or any literal role-string comparison) is a trap the moment your permission model has more nuance than the role name suggests: for example, an "admin" role that shouldn't automatically get every admin capability, or a narrower role that legitimately needs one specific elevated permission. Gate behavior on an explicit capability/permission check (requireCapability("doTheSensitiveThing")), not on the role label. When you widen a role guard to "just make the 403 go away," you are very likely re-opening a boundary that was deliberately drawn. Treat any urge to widen a role guard as a signal to go find out why it was narrow, not a green light.
Row-level isolation has a silent-bypass failure mode: know it
If tenant isolation is enforced by a database-level policy (row-level security or equivalent) keyed on a session-scoped variable, there is usually a role or connection mode that silently bypasses it entirely: for example, a superuser or elevated-privilege connection role skips row-level security unconditionally, with no error and no warning at query time. This is the single most dangerous failure mode in this category because it doesn't fail loud: the query just returns everything, and nothing tells you the policy didn't run. Concretely:
The application's runtime connection must use a least-privilege role that cannot bypass the isolation mechanism. Never the same role that owns the schema or runs migrations.
Add a boot-time or startup check that asserts the connected role does not have bypass privileges, and treat any warning from it as a stop-the-line issue, not a log line to ignore.
Defense in depth: also filter by tenant ID in application code, even though the database policy should already be doing it. Isolation should not depend on exactly one mechanism working.
Invite / redemption flows: enumerate every path explicitly
If your system has more than one way to redeem an invite (a user-account invite vs. a scoped single-use invite tied to a specific resource), they usually need genuinely different handling: different token shapes, different consumption semantics. Document both paths side by side and treat "what happens when someone re-invites an existing user, or a role mismatch shows up on redemption" as a product decision, not an engineering one. Escalate it rather than picking a direction that changes user-facing semantics unilaterally.
Maintain provenance, not just facts
Auth references go stale fast: file:line citations rot the moment the branch moves. Date-stamp what you verified and how (a specific grep, a specific test name), so the next reader can re-verify in seconds instead of re-deriving the whole model from scratch.
LLM eval harness and scoring pipeline
Any pipeline that turns LLM output into a number or a structured decision (a grade, a rank, a composite score) needs three things most first drafts skip: locked-down math that can't silently drift, an explicit policy for what happens when part of the pipeline fails, and a way to measure quality changes before they ship. This skill covers all three, plus the shadow-comparison technique for testing changes safely.
Lock the aggregation math, and gate changes on it explicitly
Whatever formula turns multiple sub-scores into one number (a mean, a weighted composite, a rank with a tie-breaking rule) should be written down as an explicit, versioned contract: not just "whatever the code currently does." Once real scores exist that people compare over time, changing this math retroactively changes the meaning of every past score. Treat any change to it as requiring the same sign-off as a database migration: explicit, documented, and updated in one commit alongside whichever doc is the source of truth for it.
Partial failure: degrade, never silently substitute
The most expensive bug class in a multi-call scoring pipeline is emitting a comparable-looking number that quietly lost an input. Concretely: if one sub-call in a multi-call scoring flow fails,
If the failed piece is not load-bearing for the final number (a non-critical axis, an optional embellishment), renormalize over what succeeded and keep the result flagged as complete.
If the failed piece is load-bearing (a required axis, or the piece that determines pass/fail), do not emit a number that looks comparable to a fully-scored case. Mark the result as degraded/incomplete, still show whatever partial signal you have (clearly labeled as partial), and queue the missing piece for retry.
If nothing scoreable survived, emit nothing. Not a zero, not a placeholder value. A zero is a real, comparable data point; a missing score is not, and conflating them corrupts every downstream aggregate (rankings, cohort averages) that reads the number.
This matters because the alternative failure mode is silent and expensive: a partial result presented as complete has, in practice, produced real mis-rankings that only surfaced once someone noticed the underlying scores looked off.
Version your prompts like schema migrations
Store prompts as versioned files (not inline strings), with an explicit version marker at the top of each one. Every scored/generated row should record which prompt version produced it, so that a prompt change is diffable after the fact: you can answer "which rows were scored under the old wording" without guessing. Keep old versions around rather than overwriting them; re-scoring needs to stay comparable across versions.
Cost and concurrency controls belong next to the pipeline, not bolted on later
Concurrency limit on fan-out calls. If a single unit of work triggers several parallel LLM calls, cap the fan-out with an explicit semaphore. If you start seeing rate-limit errors or timeouts under load, lower the concurrency limit first. Raising per-call timeouts just delays the same failure.
Retry only genuinely retryable errors (rate limits, 5xx, timeouts) with exponential backoff; don't retry a definitive rejection (bad input, auth failure, content-policy block). That just wastes calls and, if the call carries sensitive data, can also reroute it somewhere it shouldn't go (see the cost-and-safety-guardrails skills for the full fallover-safety version of this rule).
Price every model you can call. Any model string that can appear in a response needs a corresponding entry in your pricing table. An unpriced model call should never silently report as free: that blinds your cost dashboard exactly when a new/experimental model starts getting real traffic.
Make demo/seed data provably free. If you seed a demo environment with synthetic scored data, run it through the real scoring code path with a deterministic stand-in for the LLM call, not a separate hand-authored shortcut. That way the seed data has the same shape and honesty as real output, costs nothing, and reruns byte-identically.
Eval harness: real calls, opt-in, checked before every prompt change
An eval harness that makes real, costed LLM calls against a fixed set of test inputs should be deselected by default in your test runner (an explicit marker/flag), never run accidentally in a normal test pass. Run it deliberately after any prompt edit, before merging: assert on output shape and value ranges, not exact free-text content. Free text will legitimately vary between runs even with a stable prompt.
Shadow comparison: how to test a model or prompt change safely
To answer "would a different model, prompt, or extraction strategy behave differently in production" without flipping anything live: run the candidate in parallel with the production path, on real traffic, but never let its output affect what the user sees or what state gets persisted. Concretely:
The production path always wins and always determines behavior; the shadow path is purely observational.
Log a structured comparison (exact-equality booleans, or a diff) between what production did and what the shadow candidate would have done, and aggregate those logs to make the go/no-go decision.
Shadow calls should be excluded from cost caps and rate limits that apply to real traffic, and tagged distinctly in your cost ledger, since they're overhead, not user-facing work.
Never resolve "would X behave differently" by actually flipping X on production to see what happens. That's an experiment on real users, not a test.
When not to use this
If the question is "should this specific PR be reviewed before merge," that's the validation-gates or adversarial-review skill. This skill is about the scoring/eval pipeline's own architecture and invariants, not about reviewing a change to it.
AI cost tracking and safety guardrails
Every LLM call in a production system costs real money and, in a lot of real systems, also carries sensitive or regulated data. Both problems get solved the same way: make the safe/tracked path the only path that's easy to write, and enforce it with an automated check rather than a code-review convention that erodes over time.
Guardrail 1: every call is tracked, with an explicit opt-out
The failure mode to design against is a new LLM call that quietly bypasses your cost/usage tracking because it was written against the raw SDK client instead of your tracked wrapper. Enforce this with a static check (a script that greps for raw provider-client usage and diffs against an allowed baseline), wired into your test suite so it runs on every change, not just something a reviewer might remember to check. Two things make this actually hold up:
The check needs to catch direct instantiation, not just direct function calls. A grep for "raw call" typically misses new ProviderClient(...). Wrap the instance yourself immediately after constructing it.
The opt-out must be explicit and local. If a script, test, or one-off tool genuinely can't build the tracking context, require an explicit { untracked: true } (or equivalent) on the same line as the call, never a silent bypass. That keeps the opt-out visible to the same grep that catches accidental misses.
Tracking failures should fail loud, not swallow silently. If writing a usage record fails, that should surface as an error on the request, not a silently-dropped tracking row. A silent swallow defeats the entire point of a cost cap, because the cap logic reads the same table that just failed to get written to.
Guardrail 2: cross-provider fallover is a denylist, not a guess
If you fail over from a primary LLM provider to a secondary one on error, classify errors as definitively non-retryable (bad request, auth failure, not-found, a content-policy rejection) vs. everything else (rate limits, timeouts, 5xx, connection errors). Retry/fail over on the second category; never fail over a definitively-rejected payload to a different vendor. If the primary vendor rejected it outright, rerouting it to a second vendor can mean sending sensitive data somewhere it was never approved to go. Write this as an explicit denylist function with tests, not an inline if in the retry logic, because "everything else fails over" is the correct default and it's easy to accidentally invert it while refactoring.
Guardrail 3: rate limiting fails closed, everywhere, by default
A rate limiter (or any safety-relevant gate) should fail closed in every environment unless a narrow, explicitly-named override is set for local development. Do not gate fail-open/fail-closed behavior on an "is this production" check: see the config-and-secrets-hygiene skill for why environment-name checks are unreliable in staging-like environments that deliberately mirror production defaults.
Guardrail 4: cost-cap state must survive the transaction that triggered it
If a cost cap is checked inside a database transaction (e.g., under an advisory lock) and the cap is exceeded, the "we blocked this" record often needs to be written to the database after that transaction has already rolled back, because the very error that trips the cap is what's causing the rollback. Writing the "blocked" record on the same connection, inside the same transaction, means it gets thrown away by the rollback that triggered it. Persist it on a separate connection/transaction, in the caller's exception handler, after the original transaction has fully unwound. Test this specifically (start a session, force the cap, verify the block record survived). It's the kind of bug that looks fine until the first real production cap trip.
Guardrail 5: no session/call without a resolved tenant or cost-attribution context
If cost caps and tracking are keyed on a tenant/organization/account ID, any code path that can start a billable session must refuse to proceed without that ID resolved. Never fall through to an untracked, uncapped path just because the ID lookup came back null. A missing tenant ID should block session creation with a clear error, not silently downgrade to "untracked call, no cap enforced."
Guardrail 6: price every model that can appear in a response
Any model identifier that can show up in your usage records needs a matching row in your pricing table, including less common presets and foundation-model IDs from a secondary cloud provider. An unpriced model should never silently report as free-of-cost: that blinds your spend dashboard exactly when a new or experimental model starts taking real traffic.
Handling sensitive or regulated data specifically
If the system processes health data, financial data, or other regulated personal information:
Fail closed on anything that gates handling of that data. A missing required secret (a salt, a signing key) should make the dependent route fail hard, never silently fall back to an unsalted or unguarded path.
Keep sensitive data out of logs and audit metadata by allowlist, not denylist. Maintain an explicit allowlist of metadata keys that are safe to log; drop anything not on the list rather than trying to enumerate everything unsafe. A denylist of "known-bad" keys reliably misses a new field the next engineer adds; an allowlist fails safe by default. Long or free-text content that must be referenced in a log should be hashed/truncated, never logged verbatim.
Never claim data residency or handling guarantees your infrastructure doesn't actually provide. If you're not certain every code path for a given data category stays within a specific region or provider, say so explicitly rather than asserting compliance. This is very often a live open question for counsel or compliance, not an engineering fact you can assert unilaterally.
New AI action that touches sensitive data → it needs a place in the audit-metadata allowlist and a place in the cost/fallover guardrails above, in the same change, not as a follow-up.
When not to use this
For the actual consent/retention/regulatory-lifecycle side of handling personal data (not the AI-call-specific guardrails), see the consent-and-regulated-data-reference skill.
Budget-aware model allocation
If you have access to more than one model or vendor, each with its own separate rate-limit or spend budget, treat that as a resource-allocation problem, not just a cost line item. The failure mode this guards against: an offhand, unscoped request (a wide fan-out, a big batch job, ingesting a large corpus) detonates a large fraction of a budget window that was already running low, at the worst possible moment.
When a budget signal says a window is getting low
Be deliberate before large spends. If the next action is inherently large (a wide parallel fan-out, ingesting a big corpus, a long agentic loop), say so explicitly before doing it: state the intended scope, either in your own reasoning or to whoever you're working with. If no one is available to check with, default to the smallest scope that actually accomplishes the task rather than blocking entirely.
Shift heavy work to whichever provider has headroom. If one provider is the constraint and another has budget to spare, route the heavy or parallel execution and review work there. You orchestrate; the other provider spends its own budget. This is the same idea as the multi-model-adversarial-review skill: a second vendor gives you diversity of opinion. It also gives you separate capacity.
Trim before you cut scope. Lower the effort/reasoning setting, batch requests, and avoid re-reading large context you've already processed, before you resort to skipping work outright.
When budgets are healthy
Work normally. Don't pre-emptively ration based on a hypothetical future shortage: that just produces worse answers for no real benefit.
Rules
Don't trust a stale signal as current. If a budget snapshot is flagged as possibly out of date, re-check it or proceed conservatively rather than trusting a number that might be minutes or hours old.
This is judgment, not a hard gate. Never hard-block work because a budget looks low; degrade gracefully instead (smaller scope, lower effort, defer to the other provider).
If every available provider is constrained, proceed with the smallest scoped action and say so. Don't stall waiting for headroom that may not come.
Config and secrets hygiene
Configuration bugs are rarely about the value being wrong. They're about the value landing in a layer nobody reads, a precedence rule nobody remembered, or a flag whose null/missing state does the opposite of what its name implies. This skill is a checklist for avoiding all three.
Pick the right layer, on purpose
Most non-trivial systems end up with several config layers (server-side environment variables, client-side/build-time variables, and a runtime database-backed settings table for things end users or admins can tune). Choose deliberately:
Server-side env var: per-deploy, infrastructure-shaped, or secret.
Client-side/build-time var: the client needs it and it is not a secret. Anything that ends up in a client bundle ships to every browser that loads the app, full stop.
Runtime, tenant/admin-tunable setting: an admin should be able to change it without a deploy. This is the highest-overhead option (needs validation, an admin surface, auditing); don't reach for it by default.
Precedence traps that waste hours
A misspelled env var may be silently ignored rather than erroring, if your config loader is configured to ignore unknown keys. If a new variable "does nothing," check the spelling against the exact field name before assuming a deeper bug.
**Process-level environment variables usually beat a checked-in .env file**, which usually beats a local-only override file. Know your own stack's order, and check the override file closest to "wins" first when a value seems wrong. A stale local override file is one of the most common causes of "it works for everyone except me."
A local override pointing at a dead port/host is a classic silent trap. If a client is healthy per every log but every request fails at the network layer, check for a stale local override before assuming a code bug.
Boot-time guards exist to be obeyed, not relaxed
If your app refuses to boot on a placeholder/weak secret, or refuses to start without a required credential, treat that as working as intended, not as an obstacle. Fix the actual input (generate a real secret, set the real credential); never widen the list of "acceptable" values or delete the validator to unblock yourself locally. The entire point of the guard is to make it structurally impossible to accidentally run production, or anything production-adjacent, on a placeholder secret.
Runtime feature flags: verify a consumer actually exists
A flag that renders correctly in an admin UI is not proof that anything reads it. Before relying on a runtime-tunable setting, or citing it in a fix, grep for its actual consumer in the codebase outside the settings-registration code itself. It is common, in a system that's grown organically, for a meaningful fraction of registered settings to have zero runtime consumer: the value is stored, displayed, editable, and completely inert. If you're asked to "make X configurable," check whether the field already exists unconsumed before adding a new one; the real work is usually wiring an existing dead field to its intended consumer, not creating a new one.
Per-tenant flags: the null/missing semantics is a product decision, not a coin flip
When a new tenant-level flag can be null, missing, or malformed, decide its default deliberately based on blast radius:
Low-risk, convenience-only behavior (an authoring nicety, a UI preference) → grandfather existing tenants in as "on" by default; new capability should not require every existing tenant to opt in manually.
Anything that can trigger outbound messaging, spend money, or take an automated action on a user's behalf → invert the default: null/missing/malformed means off. The asymmetry matters because grandfathering a convenience feature on is low-cost if wrong, while grandfathering an automated outbound action on can mean silently enrolling every existing tenant into behavior they never opted into.
Never copy-paste one resolver's null-handling into a new flag in the other category without re-deciding this explicitly.
Rollout flags: check the exact literal, and check both halves
If a rollout flag is read as a strict string literal (=== "true", not any truthy value), and it's gated by an env-level master switch AND a per-tenant/per-record value, a flag flip that "does nothing" is almost always one of:
The literal doesn't match ("1" vs "true", case, whitespace).
The other half of the pair is off (the master gate can override the per-tenant value in either direction; know which way).
The variable was set in the wrong deployment target, so the specific component that reads it never got it.
The change hasn't actually been redeployed yet.
Check in that order before assuming the flag's logic is broken.
Adding a new flag: a repeatable recipe
A pure, testable resolver function, not an inline check scattered across call sites.
Default OFF / no-op everywhere, until step 4 explicitly turns it on somewhere.
If it's per-tenant, combine the tenant value with an env-level master (AND them together) and choose grandfather-on vs. invert-off per the blast-radius rule above.
Wire it into deploy config with a paragraph-length description and a genuinely no-op default, so adding the wiring itself ships as a no-op.
Document it in your env-var reference with a ticket/issue reference for why it exists.
Write down a rollback affordance before you ship the "on" state: know exactly what flips it back off.
Live hygiene: things to check, not just things to fix
Keep a short automatable check (a script, or a documented set of greps) for the traps that recur in practice: a real secret sitting in a gitignored-but-present local env file, a client-side variable that would leak a secret if anything ever reads it, a stale local override pointing at a dead host/port, and any registered-but-unconsumed flags. Run it before trusting your own assumptions about current config state. These things drift the moment anyone edits a local file, and stale assumptions here waste real debugging time.
Staging-to-prod cutover campaign
Standing up real infrastructure (a first staging environment, or a greenfield production environment separate from staging) has a small set of traps that recur across almost every stack, plus one discipline that matters more than any individual trap: probe the actual live state before acting on what a doc says, because infra docs rot the moment anyone applies something out of band.
Step 0: probe, don't assume
Before touching any infrastructure, determine the real state with live commands, not by trusting a doc's last-updated date:
Has the relevant infra branch/config actually been applied anywhere, or does it only exist as authored-but-unapplied code?
Does a CI/CD pipeline actually exist and run on the branch you think is live, or does automation live only on an unmerged branch?
Can you actually authenticate to the target cloud account right now? (Session tokens expire silently and mid-task. Check this before starting any work that touches live infrastructure, not after you hit an auth error partway through.)
Has the state-backend/bootstrap step (e.g., a Terraform state bucket) actually been created, or does nothing exist yet?
Never proceed on "the doc says X" without a live check backing it up. Treat a doc's own staleness disclaimer as a warning that it needs re-verification, not as ambient permission to trust it anyway.
Hard rule: never apply, push, or activate automation without an explicit go-ahead
plan/validate/diff/dry-run commands are always safe to run. Anything that creates billable resources, pushes an image, or activates a pipeline that will auto-deploy on future pushes needs an explicit, in-the-moment go-ahead from whoever owns the account, not an inference that "it's probably fine because the branch looks ready." Merging an infra branch that arms an auto-deploy pipeline is itself the action that needs that go-ahead, separate from any individual apply.
First-apply traps (these recur across almost every stack)
DNS/certificate validation can time out on the very first apply, before the domain's nameservers have actually been pointed at the new infrastructure. This is expected, not a failure: the fix is finishing the manual DNS delegation step and re-applying after propagation, not debugging the certificate module.
An empty container registry blocks the first deploy in a chicken-and-egg way: the compute layer can't start a task with no image to pull. Push a trivial placeholder image first, purely to prove the wiring, before your real image is ready.
Placeholder secrets seeded during infra creation are often NOT caught by your weak-secret boot guard, because that guard's denylist typically only knows about your own dev-environment placeholders, not the placeholder string a cloud provider auto-generates. Replace every placeholder secret with a real value immediately after apply, before the first real workload starts. Don't rely on the app refusing to boot as a safety net here.
Automated deploy pipelines commonly don't run your schema migrations for you on the very first setup. Confirm explicitly whether migrations run automatically or need a manual first pass, and don't assume "the pipeline succeeded" means the schema is current.
The do-not-inherit scrub, before promoting anything to a stricter environment
Config that's safe in a lower environment and dangerous in a stricter one survives naive copy-paste more often than you'd expect. Build an explicit checklist and review it every time you promote, at minimum:
Any "developer convenience" flag (debug tools, relaxed auth, preset test accounts): these should be omitted entirely from the stricter environment's config, not merely set to false, because "omit vs. false" often has different failure behavior if someone later copies the block wholesale.
Any shadow/observation-mode AI or experimental feature flag should be fully absent, not just toggled off, since a stray non-empty value elsewhere in the same config can silently re-arm it.
Any allow-list of internal/test accounts with elevated privileges.
Any secret injection that's a known dead fossil from a previous architecture: don't recreate it in the new environment just because the old config still references it.
The deploy-role trust policy and CI/CD authorization: a stricter environment needs its own scoped trust condition, never an inherited or OR'd-together condition shared with a looser environment.
Rebuild per environment; don't promote a build artifact wholesale
If any client-facing configuration (a base URL, a feature toggle) gets baked into a build artifact at build time rather than read at runtime, promoting the exact same artifact between environments ships the wrong environment's baked-in values with no runtime fix available. Rebuild for each environment's actual config instead of promoting a digest/artifact, and add an automated check that scans the built output for the wrong environment's identifiers before it ships.
Ops/seed scripts: verify they actually exist in the deployed image
A script that "works locally" is not proven to exist in a containerized deploy unless you've verified it's actually registered in every place the build needs it: a container's file copy allow-list, a pipeline step's explicit file list, and (if it imports sibling files) each of those transitively. This has broken real deploys more than once because deploy-time file inclusion is easy to get subtly wrong even when local execution works perfectly. Write a small static checker that verifies a new script's full import closure is actually included in the deployed image, and run it before merging.
Separate go-live gates from infra bring-up
Infrastructure can come up completely dark (no real user traffic, every risky feature flag off) well before the business/legal/compliance gates that allow real traffic are satisfied. Don't hold infra work hostage to gates it doesn't actually depend on, and don't let "the infra is ready" become an implicit argument for skipping a compliance gate that's still open. Track them as genuinely separate checklists, and start the longest-lead-time gates (legal sign-off, third-party vendor approvals) as early as possible since they typically take far longer than the engineering work.
When not to use this
Local development environment setup (not a shared staging/prod environment) is covered by the environment-and-build-hazards skill. Ongoing operational triage of an already-live pipeline (a specific failed deploy, a stuck rollout) is a debugging-playbook problem once the environment itself is established.
Evidence: one of this skill's three source versions was tested in the pre-registered, order-blinded A/B of July 2, 2026 and passed (must-hits 1/3 cold to 3/3 loaded and 2/3 to 3/3); this merged rewrite was later tested in the July 2026 effort lattice, where the full 17-skill library ran against every model and effort level (results/matrix/MATRIX.md; evidence note updated 2026-07-12).
Environment and build hazards
Most local-environment pain falls into two buckets: a privilege/role mismatch that only shows up as a confusing runtime error, and "local" actually being a shared remote resource in disguise. Both are worth documenting explicitly rather than re-discovering per session.
The two-role model, if your database enforces row-level isolation
If your database enforces tenant isolation via row-level security (or an equivalent per-tenant policy), you need two distinct database roles, not one:
A schema-owning role, used only for running migrations.
A least-privilege application role with no bypass privileges, used for everything the running app does.
The reason this has to be two roles: a superuser or bypass-privileged connection skips row-level security unconditionally and silently. No error, no warning: every isolation policy just quietly does nothing. If your app ever connects with the migration-owning role "because it was easier," every isolation guarantee in the system is inert without any visible signal that it's broken. On first-time setup, the app-only role is often created by an init script that runs once, at cluster creation. If you ever reset the underlying data volume, that role won't exist until you either re-run the init script by hand or fully recreate the environment.
Seed data: know exactly what's idempotent and what isn't
Document precisely what a seed script does on a second run: does it add more data, or does it drop and recreate a known synthetic set? Getting this wrong wastes real debugging time (assuming stale seed data is old when it's actually freshly regenerated with new IDs, or vice versa). If a seed script exists specifically to avoid real API costs (a deterministic stand-in swapped in for a real LLM/API call), don't "fix" it by making it call the real service. That defeats its entire purpose.
The defining hazard when "local" isn't actually local
Some stacks don't have a real local environment at all: "local" development runs against a shared remote database and shared remote auth, typically because standing up a fully local equivalent (a local auth provider, a local database matching production extensions) is more effort than it's worth for a small team. If that's your situation, treat every local database command as a remote operation against shared state, and:
Never run a schema-mutating command "locally" if the connection string actually points at shared infrastructure. A destructive migration command run against what you think is a disposable local database can silently mutate the shared environment everyone else depends on.
Confirm which environment you're actually pointed at by resource identity (the actual hostname or account ID), not by an environment-name variable. A shared environment may deliberately set its environment-name variable to look like production, precisely to get production's safety defaults (see the config-and-secrets-hygiene skill for why).
A stack-bringup tool (docker-compose, a bootstrap script) that looks current may be a dead fossil from a previous architecture. If it hasn't been touched in a long time and references services you know were replaced, verify it's actually still wired to anything before spending time debugging why it "doesn't work."
Cloud auth: verify it before you need it, not when you hit the wall
If any part of a task will touch a shared cloud resource (a remote database, a secrets manager, an infrastructure tool, a deploy), verify you're actually authenticated at the start of the task, not when the first API call fails partway through. Session credentials expire silently and often mid-task; a single preflight check at the top saves discovering an expired session in the middle of something you can't easily unwind.
# generic pattern, substitute your own provider's identity check
your-cloud-cli sts get-caller-identity --profile <profile>
If it fails, re-authenticate immediately (this usually opens a browser: tell whoever you're working with and wait for it, rather than silently retrying) and re-check before proceeding.
Ports, processes, and "it's already running somewhere else"
A surprising fraction of "the stack won't start" reports are actually "something is already listening on that port." Check for a squatting process before debugging application code. Keep a short list of the ports your stack needs and a one-line command to check each one.
When not to use this
Deploying to, or standing up, a genuinely shared staging or production environment (not your own local dev loop) is the staging-to-prod-cutover-campaign skill. Running the actual test/validation suite once the environment is up is the validation-gates skill.
Systematic debugging playbook
The most expensive debugging mistake is skipping straight to a hypothesis and a fix. In any codebase with real history, most "new" bugs are re-encounters of something already root-caused, and most fixes that get re-reviewed introduce a second bug in the process of fixing the first. This playbook is the discipline against both.
Step 0: look up ground truth before forming a hypothesis
Before writing any code:
Search prior art first. Grep commit history for the symptom (git log --all -i --grep="<keyword>"), check any existing investigation/post-mortem archive your project keeps, and check file history for the specific file involved (git log --follow -- <path>). A surprising number of "new" bugs already have a full root-cause writeup sitting in history.
If a prior investigation doc exists, check whether it was later corrected or resolved. A doc that opens with a "RESOLVED" or correction banner should have that banner treated as overriding the body. Verify the recorded fix is actually still present in current code before concluding a fixed issue "recurred": it's easy to mistake "I'm looking at the pre-fix version" for "this regressed."
Don't trust a finding's classification blindly, including your own project's tracker. A finding marked resolved, retracted, or not-reproducible should be re-verified via the real path that would trigger it, not assumed correct because a doc says so. Post-mortems have real examples of a "confirmed" finding turning out to be an artifact of how it was tested (a guessed URL producing an expected 404, not an actual broken link).
If the symptom touches auth/permissions, resolve it against your actual token/role model (see the multi-tenant-auth-reference skill) before guessing which credential or role is involved. Guessing here is exactly the kind of thing that burns hours for no reason, because the answer is almost always already in the code.
The core loop: reproduce, then write the failing test, then fix
Reproduce at the lowest layer that actually shows the bug. Pure logic belongs in a unit test with a minimal mock, not a full end-to-end run. A user-facing flow bug needs an end-to-end reproduction; a runtime-only issue (something that only shows up under production-like load or logging) may need a read-only look at real logs before you can even reproduce it locally. That's a legitimate outcome, not a failure to reproduce.
Then:
Write the failing regression test before the fix, and make sure it actually reproduces the real failure shape. A test using an unrealistic stand-in (e.g., a simplified stub that doesn't match real-world framing/encoding) can pass while the real bug remains, which is worse than no test because it looks like coverage.
Fix the root cause, not the symptom. If the same logic exists in more than one entry point (a streaming and non-streaming version of the same operation, for instance), the fix has to land in both: "twin" code paths are a common place for a fix to land in one and not the other.
Write a commit message that states symptom → root cause → fix → how you verified it. This is what makes the "search prior art" step above actually work for the next person (including future-you). A vague commit message breaks the whole mechanism.
Danger zones: track your own hot files
Keep a short, explicit list of files with unusually high fix-churn in your own project (measurable with git log --format= --name-only | sort | uniq -c | sort -rn). Treat any edit to one of these as needing extra care and, if one exists, a guard test run before and after the change. These files have earned their reputation, and a "quick fix" to one of them is disproportionately likely to be the next entry in this list.
Multi-round fix loops: the regression rule
When a review pass (see the adversarial-review or tiered-review skills) surfaces multiple findings and you fix them in rounds, the proven failure mode is the fixer introduces the next round's bug. Concrete rules that prevent this:
Every fix in round N ships with its own regression test in round N, not a follow-up.
Before declaring round N done, re-run every test added in rounds 1 through N, not just the new one. A fix late in the loop can silently break an earlier round's guard.
Treat each round's diff as the next round's review target. A reviewer's job in round N+1 is to diff round N's changes specifically, not just re-check the original findings.
One commit per round, with the round labeled in the message, so that if a regression does slip through, it bisects cleanly to the round that caused it.
Calibrate how many rounds and how much review-fleet weight you throw at this to the actual stakes: a shipped, customer-facing change warrants the full loop; an internal working doc doesn't.
Never "flip it to see what happens" in a shared environment
If the question is "would a different model, flag, or config behave differently," resist the urge to just flip it on a shared/production-like environment to observe. Use a shadow/observation mechanism instead (see the llm-eval-harness-and-scoring-pipeline skill) that runs the candidate in parallel without it ever affecting real behavior or real users.
Writing up a nontrivial investigation
If you spend real effort investigating something, write it down in a consistent, searchable format: what triggered the investigation, your conclusion (labeled clearly as confirmed root cause vs. best hypothesis), and the evidence trail (log lines, file:line references, session/request identifiers). If you found nothing and changed nothing, say so explicitly at the top: "read-only, no changes" is a valid and useful outcome to record. If a later finding invalidates an old writeup, prepend a dated correction banner; never silently edit or delete the original. The original's evidence trail still has value even when its conclusion turns out to be wrong.
When not to use this
If you're specifically trying to avoid re-attempting something that was already tried and abandoned (not "already fixed," but "already tried this exact approach and it didn't work"), see the failure-archaeology skill. It's a distinct, complementary discipline.
Evidence: both source versions of this playbook passed the pre-registered, order-blinded A/B of July 2, 2026 (one source: must-hits 1/3 cold to 3/3 loaded and 2/3 to 3/3; the second: 0/3 to 2/3 and a 2/3 tie); this merged rewrite was later tested in the July 2026 effort lattice, where the full 17-skill library ran against every model and effort level (results/matrix/MATRIX.md; evidence note updated 2026-07-12).
Failure archaeology
Every mature codebase has a set of approaches that were tried, didn't work, and were deliberately reverted or abandoned. Without a record of them, every new contributor (human or agent) is doomed to rediscover the same dead end at the same cost. This skill is about building and using that record.
Why this is worth maintaining as its own artifact
A revert commit in history is a strong signal, but revert commits often don't explain why in the commit body itself: the reasoning lived in a conversation, a ticket, or someone's head, and is gone unless it's captured somewhere durable. The fix is a standing "settled battles" reference: one entry per abandoned approach, each with what was tried, why it failed or was reverted, and what the actual settled alternative is.
What belongs in an entry
For each settled battle:
What was tried: concretely, not vaguely ("switched the ORM's hosting adapter to X").
Why it failed or was reverted: the actual technical reason if known (a specific limitation, an incompatibility), or "reverted, reason not recorded; re-derive before retrying" if the reasoning is genuinely lost. Don't invent a plausible-sounding reason after the fact; an honestly-unknown reason is more useful than a fabricated one, because it tells the next reader to actually investigate rather than trust a guess.
The settled alternative: what the codebase actually does today instead, and where to find it.
A commit hash or reference so the claim is independently checkable, and a note that hashes/details should be re-verified before being cited, since branches move.
Categories worth keeping a battle log for
Hosting/platform migrations that were tried and reverted. These are expensive to re-litigate because the original evaluation (cold starts, bundling incompatibilities, operational opacity) took real time to discover and is easy to forget once the codebase has moved on.
Framework-specific footguns that don't fail until build/deploy time. Some config syntax (e.g., certain regex features in a framework's routing-matcher config) parses fine as a string but fails to compile at build time in a way that's easy to reintroduce if the constraint isn't written down anywhere near the code.
Product/UX decisions that were reverted specifically because they weren't the engineer's call to make: a copy or behavior change that got reverted with a note like "this is a product decision, not a refactor target," independent of whether the change itself was good. The lesson here isn't about the specific change; it's that anything in this category needs the actual decision-owner's sign-off before it lands, even when it looks obviously correct.
A feature that shipped, got reverted, and came back reshaped. Document the reshaped, currently-live version as the settled outcome, not the original attempt, so nobody re-proposes the original shape without knowing it already failed once.
Schema/migration naming footguns: for example, a mismatch between a model's logical name and its actual mapped table/column name has, in practice, broken a migration chain more than once in the same codebase. Write down the exact rule for looking up the real physical name before hand-writing any migration SQL.
Infrastructure teething problems that were each fixed individually but look like a pile of unrelated hacks if you don't know the underlying root cause connecting them (e.g., several container-networking symptoms that were all actually one binding/interface issue, fixed piecemeal over several commits). Document the root cause once, with pointers to where each individual fix lives, so nobody "cleans up" a fix whose purpose isn't obvious from the code alone.
How to use the archive
Before proposing a change, check whether it matches a settled battle. If it does, read the entry, and if you still think it's worth reattempting, that's a decision for whoever owns the system, not something to silently re-attempt. The fact that it was tried before is itself information they need.
Before deleting "weird" code, check if it's the fix for a settled battle. Code that looks unnecessarily defensive or oddly specific is disproportionately likely to be exactly that. Read any comment near it, and check the archive, before assuming it's dead weight.
Treat a revert commit you find in history as a prompt to go looking, not as a fact you already understand. If the archive doesn't have an entry for it yet, that's a gap worth filling once you understand it.
Keeping the archive honest
An archive entry is only useful if it's trustworthy. Re-verify a cited hash or file reference before relying on it (git show <hash> --stat costs nothing and confirms the entry hasn't drifted). If digging through history turns up other settled battles not yet documented, add them. A quick sweep of git log --oneline -i --grep="revert" across a mature repo reliably turns up more of these than anyone remembers.
When not to use this
This is about already-attempted-and-abandoned approaches specifically. For "here's a bug we already root-caused, here's the fix," that's the settled-battles table inside the systematic-debugging-playbook skill: related discipline, different artifact.
Git change control for agents
Agentic git work fails in specific, recurring ways that human muscle-memory usually avoids by accident: trusting a branch's state instead of checking it, building a PR on a base that's secretly already dead, and treating "clean up the working tree" as safe when it isn't. This skill is the checklist against all of them.
Step 0: determine current state every time, never trust what you inherit
Never assume the checkout you're starting from reflects the true remote state, and never act on remembered sequencing from an earlier session. At the start of any branch/PR work:
git fetch origin
git status --short --branch
git log --oneline -1 origin/main
git log --branches --not --remotes --oneline # local-only, unpushed work
gh pr list --state open --json number,title,headRefName,baseRefName
git worktree list
Decision gates:
On the main/trunk branch → never work here directly; branch first.
Your current branch is already merged into the trunk (its tip is an ancestor of the trunk's tip, and isn't the trunk tip itself) → don't commit more here; cut a fresh branch from the trunk instead.
Local trunk is behind the remote trunk → never branch from local trunk; branch from the fetched remote tip.
Unrecognized dirty or untracked files → don't stash or discard them; they may not be yours. Ask before touching anything you didn't create.
The dead-base PR trap
Stacking a PR on another feature branch as its base, instead of the trunk, creates a real hazard: if that base branch later gets merged via a different PR (or an earlier snapshot of it does), your stacked PR silently orphans into repeated merge conflicts. The base still technically exists, which proves nothing about whether it's still the right target. Rules:
Base every PR on the trunk. For coordinated parallel work, merge individual branches locally into one integration branch first, then open a single PR from that integration branch to the trunk.
Before opening a PR, confirm your branch actually contains the current trunk tip. A stale branch needs a rebase/merge first, not a PR.
If you must target a non-trunk base, verify it's actually still alive (not merged, not closed) before you do: "the branch exists on the remote" is not sufficient evidence.
Migration-number and other serialization collisions
If your schema-migration tool numbers files sequentially, two branches independently adding a migration can collide (duplicate numbers, multiple heads). Before and after merging any two branches that both touch migrations, check for exactly one head using your migration tool's own "list heads" command. Never assume it merged cleanly just because git didn't report a conflict (numbered migration files often don't textually conflict at all, which is exactly what makes this collision easy to miss). Fix by renumbering the later migration and re-chaining it, or by adding an explicit no-op "merge" migration if either colliding migration might already be applied somewhere you can't rewrite.
Working-tree discipline
Never stash, pull, or switch branches mid-fix. Work only on the current branch, and stage only the files your actual change touched.
Stage by explicit path, never a blanket "add everything." A repo used as a workspace for more than just tracked product code (scratch notes, screenshots, generated artifacts) will sweep unrelated files into your commit if you add everything blindly.
Never run a destructive git cleanup command (removing untracked files/directories) without first checking whether anything untracked is actually load-bearing. Some untracked directories are deliberately kept out of git because they're a live work queue or contain material that shouldn't be committed without asking. Treat any untracked directory you don't recognize as "ask before touching," not "safe to clean."
When a mistake lands on the trunk directly, recover in preserve-first order: pin the mistaken commits on a new branch first, then figure out whether to reset (only if strictly local/unpushed, and prefer a non-destructive reset mode that aborts rather than discards if there's anything else going on in the tree). Never reset-first on a shared or already-dirty checkout.
Parallel-work collision rules
When more than one agent or contributor is working simultaneously:
Any single serialized resource (a migration chain, for example) needs one coordinator, not independent parallel writers. Race a schema change against a scratch/disposable copy of the database before merging, never against the first real shared instance.
High-churn shared files (a shared translations/localization dictionary, a shared types file) need one clear owner per work session, with everyone else handing that owner their diffs rather than editing the shared file directly.
Before opening a PR from a long-running branch, confirm your diff is actually scoped to what you meant to change (git log origin/main..HEAD -- <file> should be non-empty only for files you actually intended to touch). A stale base can make your diff quietly include changes you didn't make.
Product decisions surfacing mid-task
If a task reveals a decision that changes user-facing behavior or semantics and isn't already settled by existing documentation, don't decide it yourself and don't guess to keep moving. Log it somewhere durable (a queued-decisions doc, a ticket) with the concrete options and your own recommendation. Park the affected work and continue on independent parts of the task. Note clearly, wherever you report progress, which parts were parked and why.
Hard gates before anything merges
Never self-merge; get an actual review (see the adversarial-review and tiered-review skills for how to make that review substantive).
Follow the repo owner's attribution policy for anything user-visible (commit trailers, PR bodies, changelogs), and never let tooling set that policy for you. Where the owner's stated convention is no attribution artifacts, strip them and check before pushing (git log <base>..HEAD --format=%B | grep -i <attribution-marker> should return nothing); where there is no stated policy, ask rather than silently strip. Attribution is the repo owner's call, not the default your commit or export tooling happens to inject.
If merging to your trunk is itself a deploy trigger, don't merge a schema or seed change you aren't prepared to see applied to the shared environment minutes later.
When not to use this
Executing a large multi-PR backlog as coordinated batches is the campaign-execution skill. It builds on this one but adds wave planning and checkpoint discipline for work that spans many sessions.
Evidence: both source versions of this skill passed the pre-registered, order-blinded A/B of July 2, 2026 (one source: must-hits 2/3 cold to 3/3 loaded plus a 3/3 tie; the second: 1/3 to 3/3 on both of its tasks); this merged rewrite was later tested in the July 2026 effort lattice, where the full 17-skill library ran against every model and effort level (results/matrix/MATRIX.md; evidence note updated 2026-07-12).
Multi-agent batch campaigns
A single large backlog (dozens of review findings, a multi-part feature) executed by parallel agents over multiple sessions fails in specific, avoidable ways: agents redo already-finished work because a status doc rotted, two batches collide on the same file, and a session that dies mid-campaign takes unrecoverable context with it. This skill is the execution model that avoids all three.
Step 0: re-establish ground truth. Status docs rot, git doesn't
Never trust a backlog's own status annotations at face value. Before planning or resuming anything:
Check the actual current tip of your integration branch, not what a tracking doc says it is.
Read the tail of any checkpoint/progress file for the real "what's done, what's next," and never redo an entry already marked done.
Check for open PRs: a finding already owned by an open PR isn't yours to pick up.
Check for unpushed, local-only branches: these are invisible to any remote-based check and are one disk failure from gone.
For each individual finding you're about to work, re-read the actual current code at the cited location before trusting the backlog's description of it. Backlogs go stale the moment anything gets fixed, refactored, or moved, and citations (line numbers especially) rot fast.
A campaign that trusts a stale status doc instead of live state will, with some regularity, spend real effort redoing work that's already shipped.
Planning a campaign
Re-baseline every candidate item against the current integration tip. Drop anything already fixed (with a one-line reason), and dedupe near-identical items across different sources into one canonical owner.
Flag anything that would change user-facing behavior and isn't already a settled decision. Escalate these as a batch, don't block the rest of the campaign on an answer, and don't guess.
Group remaining work by area and primary file, so each batch owns a disjoint slice of the codebase. This is what makes true parallelism possible without collisions.
Sequence in waves: test-only and genuinely file-disjoint work first (fully parallelizable), then area-local application code, then heavy-overlap areas last and sequenced rather than parallelized.
Use one integration branch. Batches branch off the integration branch's tip (not the trunk directly), get folded back in as they're reviewed and green, and only the single integration branch ever goes to the actual trunk as one PR. This keeps "many small changes landing continuously" from becoming "the trunk is red half the time."
Maintain an explicit file-contention map
For any file that multiple planned batches might touch, write down in one place which batch owns it, and route every other batch's related changes through that one owner rather than letting them collide. High-contention files in most systems are predictable in advance (a routing hub, a shared type/contract file, a heavily-used service file), and calling them out explicitly before work starts avoids a lot of merge pain later.
Parallel execution mechanics
One workspace/worktree per agent or batch, as a sibling checkout, not a shared working directory. This is what actually enables true parallelism without agents clobbering each other's uncommitted state.
A backend batch that runs its own test suite needs its own isolated test resource (a separate test database, a separate scratch environment) so parallel test runs don't collide with each other.
Every fix stays test-first (see the systematic-debugging-playbook skill), and gets reviewed before folding into the integration branch (see the adversarial-review skill). A campaign is not an excuse to skip either discipline just because there's a lot of work to get through.
Checkpoint-to-file discipline: mandatory for anything long-running
Long autonomous sessions die (hit a length limit, get interrupted, crash) more often than anyone plans for, and an in-progress session's context does not survive that. The only thing that reliably survives is what got written to a file. For any campaign expected to span more than one sitting:
Maintain a single progress file with unambiguous resume instructions at the top ("if resuming: read this file, find the next unfinished item, do not redo finished ones").
Append an entry after every meaningful event (a unit fixed, a batch gated, a PR opened, a review returned, a fold-back merged), with enough detail (branch, commit, what passed) that a fresh session could resume from just this file with zero other context.
If a past entry turns out to be wrong, append a correction; never edit history in place. The point of the file is to be a trustworthy trail, and silently rewriting it defeats that.
Keep individual work small enough to checkpoint frequently; detail belongs in the file, not in a single giant in-memory transcript that might not survive to the next session.
Guardrails for autonomous/looped execution specifically
Whoever owns the trunk merges the final integration PR. An autonomous loop should never merge anything into the trunk itself.
Every fix stays test-first and gets reviewed; a long autonomous run is not a reason to relax either discipline.
A new product/behavior decision surfacing mid-loop gets batched and flagged, never decided by the loop itself.
Folding a reviewed batch branch into the integration branch with a local merge is not the same action as merging a PR to the trunk. The first is the required mechanism for a multi-batch campaign; only the second is the one action reserved for the trunk's owner.
When not to use this
For a single bugfix or small feature, plain git-change-control-for-agents is enough. The wave-planning and checkpoint machinery here is specifically for backlogs and campaigns large enough to span multiple sessions or genuinely parallel agents.
Evidence: the source version of this skill passed the pre-registered, order-blinded A/B of July 2, 2026 (must-hits 2/4 cold to 3/4 loaded on one task and 2/3 to 3/3 on the other); this rewrite was later tested in the July 2026 effort lattice, where the full 17-skill library ran against every model and effort level (results/matrix/MATRIX.md; evidence note updated 2026-07-12).
Docs of record and arbitration
Mature projects accumulate overlapping documentation: several versions of the same spec, a "why we built it this way" doc that's now describing a stack nobody uses anymore, a README that was accurate on the day it was written and hasn't been touched since. The failure mode is building from whichever doc you happened to open first. The fix is an explicit arbitration order, applied consistently.
Classify every doc as authoritative or historical, explicitly
Keep one short table: which documents currently govern which decisions, and which are retained only as history and should never be built from. A doc that's superseded should say so in its own header if possible ("superseded by vN, kept for history"), but don't rely on that alone. Maintain the classification independent of whether the old doc admits it's stale.
Arbitration order when documents disagree
A reasonable default order; adapt it to your own project's actual sources of truth:
A settled-decisions ledger, if you keep one (a running log of "we decided X, here's why, here's where it's implemented"). This beats every other document, because it records an actual decision made with full context, not a snapshot of intent.
The contract/invariants document (see architecture-contracts-as-law), for anything about schema, API shape, or system invariants.
The current product/behavior spec, for what the system is supposed to do, as of now.
Everything else: older specs, README sections, "why we chose this stack" docs. Useful for historical context, never for settling a current disagreement.
Concretely: if a doc contradicts a settled decision, or contradicts code that already correctly implements that decision, the doc is wrong. Fix the doc and cite the decision when you do. If code contradicts the contract document, that's a bug, full stop, regardless of which one you assume is "obviously" right. And if resolving the discrepancy would itself create a new product decision that isn't already settled, that's an escalation, not something to resolve by picking whichever reading is more convenient.
Docs can be wrong even when they look official
Even a formally-versioned spec has, in real projects, been caught inventing behavior that never shipped and never matched an already-settled decision. The lesson isn't "specs are untrustworthy," it's "verify against the actual settled decision and the actual code before treating any single document, however official-looking, as ground truth." A living review corpus (open findings, triage notes) can itself go stale the moment a finding gets fixed: a "still open" claim from a finding-tracking doc is worth a quick check against current code before you act on it.
How docs of record get properly updated
Contract-relevant changes land in the same commit as the code change that necessitates them, never as a promised follow-up.
When a document needs a substantial rewrite rather than an amendment, prefer cutting a new version that explicitly supersedes the old one over silently rewriting history in place. Within a given version, small amendments get folded in as dated addenda rather than spawning a new version for every tweak.
Screenshot-based or visually-captured documentation (a team guide, a walkthrough) should treat every image as a regenerable slot tied to a specific route/state, not a hand-made asset. If a slot's capture fails because the underlying UI moved, that's a signal to fix the capture definition, not to hand-edit or fake the image.
Producing a new business/team deliverable: confirm scope before building
Before generating any nontrivial document, screenshot showcase, or report, confirm three things if they aren't already pinned by the request: the actual output format, the intended audience (their technical depth changes the right level of detail enormously), and where it's meant to live. Building the wrong format for the audience is expensive to redo and is a very avoidable mistake. One clarifying question up front is cheaper than a full rebuild.
Calibrate review effort to the deliverable's stakes: an internal-only draft needs one honest verification pass (facts, links, do the screenshots match what they claim to show); a customer-facing or high-stakes deliverable earns the full tiered or adversarial review (see those skills). Running a heavy review pipeline on a low-stakes internal doc is not "being careful," it's wasted effort that could go somewhere more useful.
Fact-check before you polish
For any deliverable built substantially from claims about a real system (a feature showcase, a technical write-up), write the underlying fact sheet first, every claim paired with the concrete evidence backing it (a specific file, a specific measured number, a specific verified behavior), before writing the polished prose. Let the prose only ever say what the fact sheet actually supports. This ordering matters: polishing prose first and fact-checking after tends to preserve whatever the first draft assumed, even when it's wrong.
Screenshot-heavy documents need their own integrity check
If a deliverable includes before/after image pairs or a set of captured screenshots:
Verify every referenced image file actually exists, and every captured file is actually referenced somewhere (an automatable check, not a manual scan).
For every before/after pair specifically, open both images and confirm they show the same surface in a comparable state. This is the single most common way this kind of document ships something quietly wrong (an "after" that's actually an unrelated screen), and no automated tool catches it reliably.
Caption honesty: if a shown surface has placeholder or non-final content, the caption needs to say so. Don't let a polished screenshot imply something is finished when it isn't.
Follow the owner's attribution policy in anything user-visible
The same rule as in git-change-control-for-agents applies here with a wider blast radius: a changelog entry, a report footer, or embedded document metadata should carry attribution according to the repository or organization owner's stated policy, not according to whatever your export tooling defaults to. Where the owner's convention is no attribution artifacts, strip them; where no policy is stated, ask rather than silently strip. Check the actual output, not just the source you wrote, since export tooling (document generators, static-site builders) sometimes injects its own metadata you didn't type. The point is deliberate policy-obedience: attribution is the owner's call, never the tool's.
When not to use this
For the specific discipline of handling regulated data (consent records, retention) inside these documents or systems, see the consent-and-regulated-data-reference skill.
Consent and regulated-data reference
Systems that handle regulated personal data (health records, financial data, or personal data generally under a privacy regime) share a small set of hard rules regardless of jurisdiction or industry specifics. Get these wrong and the failure is not a bug ticket, it's a legal or compliance incident. The unifying principle: fail closed everywhere, and never let a "test" or "verification" action create a legally material record.
Fail closed, structurally
No consent record → no processing, no send, no session. Don't let a missing consent check degrade to "allow, and log a warning."
A missing secret required for a compliance-sensitive path (a hashing salt, a signing key) should make that path fail hard (an error), never silently fall back to an unsalted or unguarded equivalent. This is one of the few places where "fail loud and ugly" is strictly better than "fail soft and available."
If your consent/communication model has more than one generation (an old model being phased out, a new authoritative one), the gate logic should read from the current authoritative model only. A legacy model kept around for backward-compatible reads should never be treated as satisfying a current-model check.
Never mix communication scopes
If your system distinguishes necessary/operational communication (e.g., service-related messages a user needs to receive) from optional/commercial communication (marketing, upsell, review requests), keep those two consent scopes completely separate, in both directions:
An opt-in for necessary communication never authorizes commercial communication.
An opt-out from commercial communication must never also silently revoke necessary/operational consent.
Any new message type needs its scope declared in exactly one place, so "which scope does this belong to" is never re-derived ad hoc at each call site.
Commercial-category sends should sit behind their own explicit kill switch, off by default, and that switch must gate every stage of the flow (rendering the opt-in UI, capturing the opt-in, and the actual send). A gap at any one stage defeats the whole switch.
A hard, global opt-out signal (e.g., a "stop all messages" reply) should override every per-service or per-relationship opt-in: global always wins.
Delivery failures (bounces, spam complaints) are a deliverability signal, not a consent signal. Don't let your bounce-handling code accidentally mutate consent state.
Age and minors: gate deliberately, everywhere self-consent matters
If your product allows self-directed sign-up or data submission and your jurisdiction has an age-of-consent rule, every entry point that can create a session or consent record on someone's own say-so needs the same age gate. A new unauthenticated entry point added later needs the identical check, not a "we'll add it eventually."
Retention and erasure: automate it, and never touch the audit trail
Automated retention/erasure sweeps (a scheduled job that purges expired data, processes a deletion request) need to actually run on a real schedule in your deployed environment. Verify the scheduling mechanism is live, not just that the code exists. A retention job that only exists on an unmerged branch is not compliance, it's a draft.
An append-only audit trail (anything designed as tamper-evidence) should never be purged, updated, or reordered, ever, including as part of a retention sweep. If a genuine legal need to purge audit history arises, that's a decision for whoever owns legal/compliance risk, never a routine engineering action.
Distinguish between data that must cascade-delete with a user's record (their raw data) and a durable, PII-free proof-of-consent record that should survive deletion specifically to prove consent was once properly obtained. Conflating the two means you either fail to actually erase someone's data, or you lose your own evidence that consent was handled correctly.
Audit metadata: allowlist, not denylist
Any logging or audit path that records metadata about an action should filter what it stores through an explicit allowlist of safe keys, dropping anything not on the list. Never try to enumerate everything unsafe and block that instead, because a denylist reliably misses the next new field someone adds. Long or free-text content that needs to be referenced in an audit record should be hashed or truncated, never stored verbatim. Adding a new logged action means adding its specific new metadata keys to the allowlist as part of that same change, not as an afterthought.
Publishing anything legally material is a real write, never a "check"
If a change publishes a new version of a legally material document (a consent-text version, a policy version, anything that other records will reference going forward), that publish action is real and irreversible-in-effect the moment it happens: subsequent records point at it. This means:
It is never an acceptable "verification" or "no-op check." There is no test-environment exception for this specifically, because in a lot of real systems test/staging environments share the same tables as production for exactly this kind of record.
Any minimum-content or validation guard on a publish action (a minimum length, a required review step) exists specifically to block low-effort accidental publishes. Never weaken it to unblock a test.
If a genuine test of the publish mechanism is required, use an isolated, disposable copy of the data store, never a shared environment.
Breach notification: build it to actually fire
A breach-notification path should always record that a breach was reported (an audit entry), independent of whether the notification email actually got sent. Don't let a missing notification-destination configuration silently mean "nothing happened." If the destination isn't configured, that should surface clearly (at minimum a loud warning, ideally a hard failure at the point notification is attempted), not a silently-skipped send.
Legal/regulatory status is often a live open question, not a fact you can assert
Whether a specific data-handling posture (e.g., "all sensitive data stays within a specific region," or "this consent language is sufficient") is actually compliant is very often something that's still pending a formal legal or compliance sign-off, even after the engineering work is done. Present open regulatory questions as open, explicitly, and point at wherever your organization tracks real status (a ticket, a legal review doc) rather than asserting compliance yourself. Never let engineering confidence in the implementation substitute for an actual compliance sign-off.
When not to use this
For the AI-call-specific version of handling sensitive data (cost tracking, provider fallover safety), see the ai-cost-tracking-and-guardrails skill. The two are complementary but distinct.