Forty-one of three hundred and forty-seven alerting rules carried no instruction for whether to act

Partway through building the fleet’s alerting rules, a convention was adopted. Every rule carries a label saying whether it is there to inform a diagnosis or to page a human. Everything written after that day followed it. Nothing went back to check whether everything written before it did too, because there was no mechanism that would force that check. A missing label is invisible: the rule still evaluates, still fires, still routes somewhere. It just carries no answer to the one question any tooling reading it needs to ask.

The audit of 20260920 counted 41 of 347 rules without the label, roughly 12%, spread across five different rule groups rather than concentrated in one forgotten corner. The backfill itself was unremarkable. Each rule took the label its already-labelled peers in the same group used, derived from its severity and its existing routing, and no expression, threshold or severity was touched.

The fix worth copying is the shape of the second half. Rather than trusting a future author to remember the convention, the team added a test that makes an unlabelled rule a build failure. Note why this needed a dedicated script. The standard tool for testing alerting rules tests one rule’s behaviour against sample data, and cannot check a property that has to hold across every rule in the system. Cross-cutting properties need their own cross-cutting test.

Any project accumulating rules, policies or metadata over time under a convention introduced partway through should assume a backlog exists until a script, not a person, proves otherwise. The coverage is otherwise only as good as the newest third of the rules.