Skip to main content

Quality Gates

Quality gates are automated checkpoints that catch defects at three progressively more expensive stages. The earlier a defect is caught, the cheaper it is to fix. A lint error caught immediately after editing costs seconds. The same error caught during code review costs minutes (context switch, review cycle, re-review). The same error caught in production costs hours or days (incident response, root cause analysis, deployment).

Gate admission rule (meta). Every gate or ritual in this methodology must declare: (a) its per-occurrence cost, (b) its enforcement mechanism — a hook, CI check, or script; "the agent will remember" is not a mechanism — and (c) its retirement condition. A gate that cannot name its mechanism is a wish, and the 2026-06-11 assessment found wishes recur as incidents (meta-pattern B: every prose-only rule was violated, some within minutes of being written; every mechanized rule held). New gates without all three fields are rejected at canon review — the PR template in this repo carries the fields.

The quality gate system is designed so that each layer is independent and no single gate is the sole defense against any category of defect.

Gate: live contract smoke. Any feature whose runtime path crosses an external SDK/API boundary ships with a marker-gated live smoke (3-5 calls, real credentials, ~cents) that runs before deploy. Mocked-only coverage of an external boundary is an unverified boundary: the 2026-05-22 voice outage shipped behind 9/9 green AsyncMock tests (assessment I13, BP-3).

Gate: migrated-schema oracle. Integration-test databases are built via the project's migration chain (alembic upgrade head), never via ORM metadata create_all. A create_all-built schema is a proxy oracle — it lacks migration-only DDL (partial indexes, constraints), so tests can pass against a database that production will never run (assessment I35).

Review line: name the oracle. Every review checklist includes: "for each verification claimed, name the production artifact it observes." A check that observes a proxy (a mock, an ORM-created schema, an intermediate frame, a spot-check subset) is recorded as a proxy, and the gap is either closed or accepted in writing (assessment meta-pattern A: at least 10 incidents passed green gates that measured proxies).