Most systems optimize for getting an answer. Some have to optimize for never getting the wrong one. Here's how building for asymmetric error costs changes everything about your architecture.
A note before we start: this is an architecture essay, not a product teardown. There's no proprietary anything in here — just a design philosophy I've had to live inside for the last couple of years, and that I think more engineers should be deliberate about.
Two kinds of "wrong"
Most of the systems we build are quietly optimized around a comfortable assumption: that a missed answer and a wrong answer cost about the same. A search result you didn't surface and a search result that's slightly off are both just "not great." You tune for accuracy, you ship, you move on.
Then occasionally you build something where that assumption is not just wrong — it's dangerous.
I've spent the last couple of years building a system where the cost of the two errors is wildly asymmetric. Failing to produce an answer is mildly disappointing. Producing the wrong answer is unrecoverable — it doesn't just degrade the experience, it damages trust in a way you can't apologize your way out of.
Once you internalize that asymmetry, almost every default in modern system design starts to look subtly miscalibrated. This is a tour of what changes.
Accuracy is the wrong headline metric
The first thing that has to go is "accuracy."
Accuracy blends two very different failures into one number. A model that's 95% accurate might be making its 5% of mistakes by staying quiet — or by confidently asserting falsehoods. Those are not the same system. One is cautious; the other is a liability.
The metrics that actually matter when errors are asymmetric are precision and the shape of your failure distribution:
- Precision: of the answers you did commit to, how many were right?
- Abstention rate: how often did you correctly decline to answer?
- False commit rate: how often did you assert something wrong? (This is the one you're really managing. It should be the metric on the wall.)
Recall — how many answerable cases you actually answered — becomes something you sacrifice on purpose. That feels deeply uncomfortable the first time you do it, because we're trained to think coverage is the goal. It isn't. In a precision-first system, coverage is a dial you're allowed to turn down to protect correctness.
"I don't know" is a first-class result
In most codebases, "I couldn't determine an answer" is an afterthought — a null, an empty array, a fallthrough else. It's treated as the absence of a result rather than a result in its own right.
In a precision-first system, abstention is a designed, named, fully-supported outcome. It has its own code path, its own logging, its own downstream handling, its own success criteria. "We looked and chose not to commit" is a correct behavior, and your system should be able to say it as clearly and confidently as it says anything else.
Practically, that means your core decision function doesn't return Answer | null. It returns something closer to:
Decision =
| Committed(value, confidence, supporting_evidence)
| Abstained(reason)
Both branches are first-class. Both are tested. Both are observable in production. The moment "I don't know" becomes a real return type instead of a missing value, the rest of the design gets much easier to reason about — because you've stopped pretending every input deserves an output.
A confidence floor you do not cross
The heart of the thing is a gate, and the gate has a non-negotiable floor.
Below a certain confidence threshold, the system does not commit — full stop. Not "commits with a warning." Not "commits but flags for review later." It abstains. The floor is a hard architectural boundary, not a soft suggestion, and it is the same for everyone. No special case, no VIP path, no "just this once because the demo is tomorrow" gets to lower it.
The reason this has to be structural rather than cultural is that confidence floors are exactly the thing that erodes under pressure. Someone will always have a very reasonable-sounding argument for why this case should squeak through. The floor only means something if it's enforced by the system, not by the discipline of whoever is on call that week.
def decide(signals):
score = evaluate(signals)
if score < FLOOR:
return Abstained("below confidence floor")
if not corroborated(signals):
return Abstained("insufficient independent support")
return Committed(resolve(signals), score, signals)
Notice there are two ways to abstain there, which brings us to the second principle.
One strong signal is not enough
A single source being very, very sure is not the same as being right. Confident-but-wrong is the entire failure mode you're trying to eliminate, and a lone high-confidence signal is precisely how it sneaks in.
So the gate asks for more than a high score — it asks for independent corroboration. Two signals that don't share a failure mode, both pointing the same way, are worth far more than one signal shouting. The key word is independent: two measurements derived from the same underlying source aren't corroboration, they're an echo. Designing for genuine independence — making sure your "second opinion" can't fail in the same way as your first — is most of the real work.
This is an old idea in disguise. It's quorum. It's defense in depth. It's why critical systems use multiple sensors that fail differently. The novelty isn't the pattern; it's the discipline to apply it to decisions, not just to availability.
Graceful degradation, not graceful guessing
When part of the system is unavailable — a dependency is down, a signal is missing, something times out — there's a strong temptation to "do your best with what you have." In a precision-first system, that temptation is the enemy.
Degradation should make the system more cautious, not more creative. Fewer signals available means a higher bar to commit, not a lower one, because you have less ability to corroborate. The correct behavior under partial failure is to abstain more often, not to fill in the gaps with optimism. A system that gets bolder as it gets blinder is a system that will eventually hurt someone.
The hardest part isn't technical
Here's the thing nobody warns you about: the engineering is the easy half. The hard half is that a precision-first system will, by design, do nothing in a large number of cases — and "did nothing, correctly" is a genuinely difficult thing for an organization to celebrate.
Stakeholders see the abstentions and read them as missed opportunities. There's relentless gravity toward "can't we just lower the bar a little?" Every conversation, every dashboard, every incentive nudges toward more coverage. And every one of those nudges is asking you to trade away the exact property that makes the system trustworthy in the first place.
So part of the architecture lives outside the code. You have to make the asymmetry legible: show people the cost of a false commit in the same frame as the cost of an abstention, so that "we chose not to answer" reads as the system working, not the system failing. The floor survives only if everyone understands why it's there.
The takeaway
If you're building something where being wrong is worse than being silent — fraud decisions, safety interlocks, anything that touches a real person's trust — consider designing around these explicitly:
- Measure precision and false-commit rate, not accuracy. Put the scary number on the wall.
-
Make abstention a first-class, designed outcome — a real return type, not a
null. - Enforce a hard confidence floor in the system, not in the discipline of your on-call engineer.
- Require independent corroboration, and do the hard work of making "independent" actually true.
- Degrade toward caution. Less information should raise the bar, never lower it.
- Make the asymmetry visible to the org, so "correctly did nothing" can be recognized as a win.
Recall is a dial. Trust is a ratchet — it only turns one way, and it turns slowly. Build like the wrong answer is the only one you can't take back, because usually, it is.
If you've built systems like this, I'd love to compare notes on how you keep the confidence floor from eroding over time — that's the failure mode I find myself defending against most.