The Hypothesis-Driven Design Kit

A form, a check and a log for writing down what you expect before you build, and a forecaster for what it will take, drawn from how TerraQuotes has worked since its first issue

My role

Designer and builder

Company
Good Citizens
Method used on
TerraQuotes, since its first issue
Years
March 2025 to now
Team
Sole developer
  1. Built the method into a GitHub repository template: Prediction, Observation and Decision forms, a check that won't merge a pull request without a prediction, and a trace rebuilt from the links.
  2. Added an agent skill that turns briefs, PRDs and decks into predictions that can be proven wrong.
  3. Ran the method on TerraQuotes from its first issue, on March 9, 2025, and added a design history file, borrowed from medical-device design controls, in January 2026.
  4. Built the kit's effort forecaster, and held it to its own predictions, PRE-004 and PRE-005.
  • 340 build-cost forecasts in TerraQuotes' ledger when it was read on October 29, 2026; 146 have an actual cost to check against
  • 82% of actual costs inside the kit forecaster's 80% range, backtested forward across 131 of those issues
  • 167 design-history records in TerraQuotes, as of October 17, 2026
  • 9 of 40 predictions in TerraQuotes' registry can be measured fully from its warehouse; 16 only in part, and 15 from CI and code instead

Summary

Before I build something, I write down what I expect it to do, the number that will settle it, and the result that would prove me wrong. The Hypothesis-Driven Design Kit makes that habit hard to skip: three GitHub issue forms, a check that won't merge a pull request without a prediction, and a log that traces each prediction to what was decided.

TerraQuotes' first epic, in March 2025, wrote down what it expected and the measure that would settle it, and most of its epics since have done the same. Since September 2026 TerraQuotes has also forecast what most issues will cost to build, and the kit's own forecaster was built and tested on that record.

Who was involved

  • Product and design teams Who the template is for: teams that want to decide what will count as working before they ship it.
  • TerraQuotes The first product run this way, from its first epic to the issues it works today.
Read the record 4 sections, about 2 min

Context

The method came first: TerraQuotes' first epic, in March 2025, wrote down what it expected, and most of its epics since have done the same. The kit started in February 2026 as a GitHub repository template: issue forms and a check that won't merge a pull request without a prediction. In March 2026 it gained an agent skill that turns briefs, PRDs and decks into predictions. In October 2026 its issue forms were cut to three, and its trace was rebuilt from the links. The forecaster came after, as a script.

TerraQuotes has kept a design history file since January 2026: 167 records as of October 17, 2026. Records dated before January 11, 2026 were written afterward from git history; the change records say so.

Why most predictions stay open

Writing a prediction is quick. Settling one takes data that arrives on its own schedule. TerraQuotes' registry holds 40 predictions; as of March 31, 2026, its table counts 9 that can be measured fully from the warehouse, 16 only in part, and 15 that need CI or code evidence instead. Predictions about pricing and conversion settle only as real jobs close, won or lost. TerraQuotes is live with one pilot operator, Quiet Village Landscaping, but those outcomes come back slowly, and a lost bid reaches the record only when the contractor's sales software sends it. Those predictions stay open, and the record says so.

Forecasting the work

Since September 2026, TerraQuotes has forecast the build cost of most issues it worked: 340 forecasts in its ledger when it was read on October 29, 146 of them on issues whose actual cost is in. The kit's forecaster reads a range from your own past misses, and a backtest says whether to trust it. Run forward in time over those issues, each forecast only from issues finished before it, its 80% range held 82% of actual costs across 131.

PRE-004 expected ranges half as wide as TerraQuotes' own estimator's, and batch forecasts that held. Its observation found neither: misses arrive in streaks, so batches aren't offered. PRE-005 was registered after a simulation showed ranges falling short with only 10 to 30 finished items, and before the fix was run on TerraQuotes' record. Neither decision has been recorded yet.

One column per TerraQuotes issue, in the order they were estimated, from September 8 to October 26, 2026: 131 issues that had at least ten finished by the day before they were estimated. Each column is the 80% range the kit's forecaster would have given that issue, using only issues finished before it was estimated, drawn as a share of the issue's own estimate on a log scale from a tenth to ten times. A dot marks what the issue actually cost. 107 of the 131 land inside their range, 82%. The 24 outside come in runs. Two land above their range on September 9 and 10, nine below it on September 11 and 12, twelve above it from September 13 to 21, and one below it on September 30. None land outside after that.
107 of 131 inside. Each issue's range was drawn from the issues finished before it, then checked against what it cost. The misses come in runs, which is why batch forecasts aren't offered.

What it taught me

Writing a prediction was never the hard part. Being in a position to settle it is. Build costs are recorded every time an issue ships, so the forecaster could be checked against TerraQuotes' record the day it was written. Pricing predictions settle only when real jobs close, so they haven't moved. Writing down the ones still waiting keeps them from being quietly forgotten, or quietly closed.

Questions about this project?