What a skill is
A skill here means one very specific thing: a Claude Code skill. It's a written instruction file, a SKILL.md, that changes how Claude behaves while it's writing or reviewing code for me. It works like a pilot's checklist: the pilot already knows how to fly, the checklist just makes sure nothing gets skipped under pressure. A skill does that for an AI coding agent.
It isn't a general "AI skill," and nothing here installs anywhere for you. This is how I make my own coding assistant behave more reliably.
The library of 17 skills built this way is at /lab/skills/, each one a short, packaged playbook the agent loads before it starts working: which traps to check, and what a finished answer has to contain.
Here's a real one, exactly as it's written. Every skill starts with two lines that matter more than anything below them:
name: pre-merge-validation-gate description: Use before declaring any change "done," before opening or updating a PR, after any UI-touching change, or whenever you're about to report test results. Defines what "done" actually means and how to report it without overstating what was checked.
That description line is the whole mechanism for how a skill gets picked. Claude reads every skill's description before starting a task, decides which ones actually apply, and only then loads the full file. Nobody switches a skill on by hand: the agent reads a one- or two-sentence pitch and decides for itself whether this particular checklist is relevant to what it's about to do. Write the description too narrow and the skill never fires when it should; too broad and it fires on everything, which is its own kind of unreliable.
Below that is the body: the actual instructions, written the way you'd brief a very capable but very literal new hire. This one opens with the core rule in one sentence (build passing plus tests passing is necessary, never sufficient, for anything a user actually sees), then a table scaling how much checking a change needs to its size, then the specific traps: gate paid test suites behind an explicit switch, don't assume a stale baseline is still true, recapture a snapshot fixture only when the change is real. Nothing abstract. Every line is there because skipping it once caused a real problem.
Writing one is just that: notice a mistake worth not repeating, write down the rule that would have caught it, and give it a description specific enough that the agent reaches for it at the right moment. The full file for this one, and the other 16, is at /lab/skills/.
How a skill gets tested
A claim that a skill "helps" is cheap on its own, so every skill on this site is graded, not just described. The method is an order-blinded A/B: every task ran twice, once with the skill loaded and once without. Judges graded both reports without being told which was which, and the two versions were shown in alternating order.
Each task carries its own short list of things a correct answer must do, called must-hits: pass/fail checks written before any run. A task's score is just the share of its must-hits the output actually hit. No style points.
The grading itself is done by a model acting as judge, not by me reading the two answers and picking a favorite. Two judges grade each pair of reports against the task's written must-hits, never knowing which answer came from the skill-loaded run. When they disagree on a mark, a fixed third judge breaks the tie, and the two-of-three majority is final.
The full run of this method against the skills library is the skills study, and the version of it that gates client work before it ships is stage 03 of the Assurance Chain.
What the effort lattice study is
Newer Claude models expose a reasoning effort dial: a setting that controls how long the model thinks before it answers. Higher settings buy more thinking per run, at a higher cost. The obvious question is whether turning that dial up makes a packaged skill unnecessary, since the model might think its way to the same result on its own.
The effort lattice study ran three Claude models at five effort levels each, once cold (no skill) and once with the skill loaded, with the rules and pass criteria locked before any run. The short answer: turning up the dial made every model better on its own, but it never closed the gap a skill added on top, at any effort level, on any model. The full study, with the numbers, is at /lab/lattice/.
The remaining terms
A few more words show up across the lab. Here's what each one means.
- Eval harness
- The program that runs a model on fixed tasks and scores the answers against pre-written checklists, so claims get measured instead of asserted.
- Pre-registered
- The rules and pass criteria were written down and posted publicly before any run, so the test could not drift toward a wanted result.
- Complete-case set
- Each model's two endpoints are scored on exactly the same tasks, only the tasks valid in both runs, so a missing task can't tilt the comparison one way or the other.
- MIT licensed
- Free to use, copy, modify, and ship commercially. Just keep the license notice.