← All articles

Governance and refusal

A caller must not choose its own policy

A defect family that shows up in almost every gateway, proxy and policy engine, including ones built by careful people.

Manah Khalil/

Here is a bug you can find in a surprising number of systems that enforce policy on traffic. It is not exotic, it does not require a clever attacker, and it survives review because every individual line of it looks reasonable.

A caller sends a request. Somewhere in that request it says something about itself: what kind of traffic this is, which service it belongs to, which user it is acting for. The gateway reads that, uses it to select which rules apply, and enforces them faithfully.

Everything in that sentence is fine except the middle. The caller chose which rules would be applied to it.

How it gets built

Nobody designs this. It accumulates.

You start with one kind of traffic, so a label on the request is just a hint and it does not matter much. You add a second kind, and now the label picks between two sets of rules, which is convenient and works. By the time there are five, the label is load-bearing and nobody remembers deciding that.

Then it hardens in ways that are hard to see:

A default that is really a choice. An unlabelled request has to be treated as something. Whatever you pick becomes reachable by omitting the label, and omission is the easiest thing in the world for a caller to do.

Observe mode that trusts the caller. You add a check comparing what the caller claimed against what you can determine yourself. Sensibly, you ship it in observe-only first, logging disagreements without refusing. But look closely at what observe mode usually does: it logs the disagreement and then governs by the caller's claim anyway. The switch was supposed to control whether you refuse. It ended up controlling whose answer is used.

Absence read as agreement. A caller that says nothing has not disagreed with you, but a comparison written as "does the claim match?" answers no for an empty claim just as it does for a wrong one. So either you refuse callers who were honest, or you accept a claim that was never made.

Escape through a weaker category. If one category is governed less strictly than the others, and the caller influences which one applies, then that category is the policy. Every other rule is optional.

The rule that prevents it

Where you can determine something yourself, your determination governs. Always. Not when a switch is on.

The caller's claim keeps its place. It is useful as a hint for things you cannot determine, and it is useful as a cross-check, because a caller whose claim disagrees with reality is worth knowing about. What it must never be is the authority.

Squidder resolves who is calling from the verified session rather than from anything the caller says about itself.

Which means separating two decisions that feel like one:

  • Whose answer governs? Yours. There is no mode where this is otherwise.
  • Do you refuse a caller whose claim was wrong? That is a rollout decision, and it can absolutely be a switch.

Collapse those and you get a system that is correct in enforcing mode and trusting in observing mode, which is the opposite of how a safe rollout should work.

How to check your own

Three questions, and they take about an hour on a codebase you know.

Take the field that selects which policy applies. Where does it come from? If a caller can influence it, follow that path all the way to the decision.

What happens when it is absent? Find the default. Ask whether a caller who wants that default can get it by sending nothing.

In observe mode, whose answer is used? This is the one that catches people. Read the code, not the comment. If the log says "we would have refused" and the enforcement uses the caller's value, you have this defect and it is invisible in every test you have.

There is a fourth, and it belongs to a neighbouring experiment: whether a caller that has never been seen before can obtain a policy simply by arriving. That one is testable in ten seconds and worth running on the same afternoon.

Why it is worth the hour

Because this class of bug does not produce an incident. It produces a system that appears to be enforcing policy and is in fact enforcing whichever policy each caller prefers, and the audit trail looks completely normal throughout.

The uncomfortable part is that it is most likely in the systems built by teams who took the rollout seriously enough to add an observe mode.