You cannot see your own front door from inside your own home
- date
- 20260820 to 20260907
- what happened
- A network renumbering on 20260820 left an administrative panel reachable from the open internet for an unrecorded part of a day, undetected, found only because someone happened to check from an unrelated external host that morning. Nothing caught it because every monitor in the fleet looked outward from inside the network, where a correctly firewalled address and an exposed one are indistinguishable.
- what it cost
- An unrecorded part of a day with an administrative interface exposed to the internet, closed about ten minutes after it was found, and found by luck rather than monitoring, plus the near-miss of a replacement check that would have been permanently and wrongly red from its first minute.
- what changed
- A dedicated external vantage point, hosted outside the network with no credential and no path back in, running a periodic full-port scan of the site's public address range against a version-controlled list of exactly what should be reachable.
- the check now
- An unexpected open port on the public range pages at the highest priority. The declared list of what should be reachable is a committed, reviewed change, never something the checker updates on its own. Every scan first proves it can see a known, always-scannable reference host.
A renumbering left an administrative login page answering on a public address for most of a day. Nothing in the fleet noticed, and the reason is structural rather than an oversight. Every monitor ran from inside the network. From inside, a probe against the site’s own public address returns the same result, no answer, whether that address is correctly firewalled to silence or is serving an unauthenticated administration panel to the entire internet. Those two states were genuinely indistinguishable from every vantage point the fleet possessed. It was found because someone happened to be on an unrelated external host that morning and looked.
The argument generalises to any network. Monitoring built entirely inside a perimeter is mathematically incapable of telling correctly closed from wrongly wide open at that perimeter’s own edge, because both look like silence from inside. The only fix is a vantage point that is genuinely outside, which for this site meant a small host in a datacentre in another country, scanning the public range on a schedule against a committed list of what is supposed to be reachable. That list is deliberately not self-updating, because a checker that quietly accepts whatever it finds has stopped checking anything.
Standing up that host raised its own question, which is why it has no credential and no route back into the fleet. A new checking capability must not become a new way in, because a vantage point built to catch exposure is the worst possible thing to expose itself.
And the near-miss is worth keeping as its own small warning. An early version of one of the new host’s uptime checks used a plain ping against public addresses that, by design, do not answer ping. It was permanently red from the moment it was switched on, which would have taught its reader to ignore it within minutes. Test the checker against a known state before you trust what it says about anything else.
Source: node0 lessons v0.1, lesson 2.21. Sanitized: checklist v0.1, 20260921; address ranges, service names; voice pass 20260921. Part of oznog.com/node0.
