Guardrails

Process discipline usually degrades under deadline pressure. Pre-registration is the kind of practice that gets skipped when a release is close, and the skipping is invisible; nobody logs the hypothesis they declined to write down. Worse, a methodology that only ever confirms its predictions is indistinguishable from one that measures nothing. The design problem was to make the disciplined path the easiest path, and to define in advance what failure would look like.

The intervention was behavioral design applied to process rather than product, built into GitHub issue templates. Friction: blank issues were disabled, so structure could not be skipped. Commitment devices: an EPIC could not open without a linked pre-registration, which would lock the hypothesis before the intervention existed. Defaults: INVEST checkboxes were pre-filled to nudge completion. Forced functions: required fields would block submission without a JTBD statement and acceptance criteria. Traceability: every artifact would link to its parent EPIC and to the pre-registration it tests. Timeboxing: SPIKEs would get an explicit timebox, and SLICEs were capped at eight hours to prevent scope creep. No issue was ever filed, so none of these templates was put to use.

The study fixed four failure conditions before the work it measures. Here is each one against the repository on October 23, 2026.

Every registration confirmed, which would mean the predictions were too safe. It has not happened. Of the 4 closed, three came back null under their own rules: prerendering the study pages did not move their largest paint over HTTP/2, inlining the study stylesheet lowered the score on every page it touched, and a performance pass left the TerraQuotes page below the bound it named. The fourth split, supported on the study page and null on the home page.

More than 20% of EPICs shipping without a registration. This one cannot be tested as written, because no EPIC was ever filed: the repository has no issues at all. The work went through pull requests instead. Of the 29 feature pull requests merged, 15 cite a registration; 14 do not, 13 of them from one day of design passes on September 30.

Zero decision-log entries citing an unexpected finding. The nulls changed what happened next: the stylesheet inlining was dropped, and the prerendering result sent the work to headers that tell the browser what to fetch, which is where the decisions were logged.

Blank issues appearing, which would mean someone found a way around the friction. Blank issues never appeared, but the way around was found: the label that exempts a pull request from registering went on 78 of 111. Most of those are content edits that make no claim, which is what the label is for; it is also the easiest path, and nothing checks that a labeled pull request really makes no claim.

The method is teaching: three of four closed predictions were wrong in ways that changed the next decision, which is the surprise the study was looking for. The guardrails are not holding where they were built. The issue templates went unused, the exemption label became the common path, and 11 of 15 registrations are still waiting to be closed by evaluation. A registration closed by measurement is the standard the delivery notes set; most of these have not met it yet.

The evidence keeps accumulating with every pull request, and the registrations the site added on October 23 name the read-depth comparison that will close them four weeks after deploy.

Put the friction where the work goes. A check on the pull request that asks a labeled change to say in one line why it makes no claim, and a weekly list of registrations still open past their evaluation date, would guard the path I actually use instead of the one I designed.