There is a failure mode in CI that is worse than a red build, and it is invisible by construction.
A red build at least tells you something. It says: this change, on this commit, in this environment, did not work. It is annoying, it costs you minutes, and it is honest.
The failure mode I am describing tells you the exact same thing, and it is a lie.
The week nothing ran
On 21 October 2026, the largest repository in the workspace had 28 failing runs out of the last 30.
Every one of them failed in about six seconds.
Six seconds is the tell. A test suite takes as long as the test suite takes. A compile error takes as long as the compile takes. Six seconds is how long it takes a runner to allocate, start a job, decide it will not start it, and give up.
The annotation said exactly what had happened:
The job was not started because recent account payments have failed or your spending limit needs to be increased. Please check the 'Billing & plans' section in your settings
GitHub had refused to start the jobs. The same annotation was on the failures in a second repository.
But at the level of the pull request, a red build is a red build. The developer assumes their change broke something. They fix a problem that does not exist, or they push again and wait. What had actually happened was that nothing was verified at all, and the reason was one line of text in a log nobody reads.
The repository was not failing. It was unverified. And for a week, nobody could tell the difference.
The bill
The cost of that week was not the compute. No runner minutes were billed, because no runner started. The cost was the guarantee.
| Remote CI | Local gate | |
|---|---|---|
| Feedback on a push that breaks the build | 4–12 min, billed | ~100 s, free |
| Feedback when the runner never starts | 6 s, and it says why — if you read it | ~100 s, and it says why |
| Feedback when the runner never starts and you are not reading the log | never | ~100 s |
That last row is the one I keep returning to. The gap is not between a fast answer and a slow answer. It is between an answer and no answer at all, and the difference between those two states is whether anyone happens to be reading a log.
The bill nobody sees
The instinct of most engineering leaders, when a bill gets uncomfortable, is to look at the number next to it: the cloud invoice. Runner minutes. Storage. Egress. That number is real, it is attributable, and reducing it produces a clean chart. It is also, in my experience, the smallest and least interesting part of the problem.
The expensive part is the waiting. Not the minutes billed — the minutes spent.
Consider the mechanics of a slow pipeline. A developer makes a change, pushes it, and then waits. If the wait is short, the change stays in their head and the next change is a refinement of the same thought. If the wait is long, the developer's mind wanders, opens a ticket, answers a message, or starts a completely different piece of work. When the pipeline finally reports back, the context required to interpret it has partially decayed.
That reconstruction cost is invisible. But it is paid on every commit, and it is the reason a two-minute pipeline and a nine-minute pipeline are not merely different in comfort but different in output.
What a local gate is actually for
I want to be careful here, because the marketing for this category is relentless and most of it is wrong.
A local gate is not faster than a CI runner. I measured it. On identical work — npm ci, two type checks, vitest, the build — the local gate took 99 seconds on a 2022 Intel laptop against 43–48 seconds on a GitHub runner. A modern runner has more cores and faster storage, and for work that is mostly npm ci and TypeScript, it wins. The local gate on that machine was roughly twice the wall clock of the remote job.
The first attempt at that measurement was discarded: the machine had a load average of 212 from an unrelated build, and the same gate took 194 seconds under that load against 105 without it. Both numbers are published, because the first one is what a measurement looks like when nobody checks the load.
So the local gate is not a speed optimisation. It is a guarantee of feedback, and it buys that in three ways:
- It always runs. The remote one did not, for a week, and nobody noticed what that meant.
- It runs before the push. A broken commit never reaches the branch.
- It is the same script, so its verdict and the workflow's verdict agree by construction rather than by luck.
If your CI is fast, healthy and free, a local gate is a convenience. If your CI bills by the minute, or is red for reasons unrelated to your code, it is the only signal you have.
Drift is real, and it is not what bit us
I should say this plainly, because the category I work in is sold on drift, and drift is a real problem. A runner image gets a patch upgrade. An action is referenced by tag and the tag moves. A cache restores a different lockfile resolution than the commit was written against. Each of those is individually invisible and collectively fatal: the build is not wrong in an obvious way, it is wrong in a plausible way, which is the worst way to be wrong. The failure points at your code, so your code gets debugged. Your code is fine.
But drift was not what happened in my workspace. Drift is the more interesting failure and it announces itself over weeks. What happened was one line of text about billing, and it took a week to notice.
The tell for drift is different. A genuinely flaky test is not fixed by trying again. A job that goes green on the second attempt is telling you the environment changed between the two attempts. If you are not measuring that rate, you are not measuring drift.
What the gate found that CI could not
This is the part that made the case for the tool, and it is not about speed at all.
Running the local gate against repositories whose CI was in the state above found three real defects, in under four minutes, on a laptop, with no runner minutes spent:
- A stale secret-scan allowlist.
.gitleaksignorenamedLOCALDEVELOPMENTGUIDE.md, but the file had moved todocs/product/LOCALDEVELOPMENTGUIDE.md. The fingerprint never matched, so a tracked documentation file — acurlsample containing the literal stringYOUR_TOKEN— was failing the secret scan. The file is tracked, so the CI scan was failing on it too. The CI had been red for the same reason the jobs were not starting, and nobody could tell the two apart. - Ninety-three dependency advisories. 53 high, 8 critical. The same command CI runs.
- Build artefacts tripping a working-tree scan. Playwright traces under
test-results/contain a token, and a working-tree secret scan reads them.
The first one is the story. There were two independent causes of red in that repository, and they were indistinguishable from the pull request view. One of them was a documentation file that had moved.
The fix is dull, and that is the point
The remedy is not a new CI vendor, a build cache service, or a migration to a faster runner. It is a discipline:
- One copy of the setup. The gate scripts live in the repository and both the Makefile and the workflow call them. There is no second implementation to drift.
- Make the failure loud. Classify it. A job that did not start is not a test failure, and it should not look like one.
- Move the fast gate local. The checks that run on every commit should run on the machine in front of the developer, in the same scripts CI uses.
- Know whether you are faster or guaranteed. Measure both, publish both, and do not confuse them.
Step four is the one that requires actual courage, because it is the one where the honest answer is usually less flattering than the pitch.
What to measure this month
If you take one thing from this: find your job failure rate, and read the annotations on the failures rather than assuming they are test failures. If jobs are failing in seconds, something is failing before the job starts, and your pipeline is not telling you it is red — it is telling you it is unverified.
That is the Silent Bill. It does not appear on the invoice, because nothing was billed. It is paid in unverified commits.
The implementation is open source: CI Kit, with the full measurement — including the number I threw away — in its evidence document.