You cannot see your own front door from inside your own home
- date
- 20260820 to 20260907
- what happened
- A network renumbering on 20260820 left an administrative panel reachable from the open internet for an unrecorded time, undetected, found only because someone happened to check from an unrelated external host that morning. Nothing caught it because every monitor in the fleet looked outward from inside the network, where a correctly firewalled address and an exposed one are indistinguishable.
- what it cost
- An unrecorded time with an administrative interface exposed to the internet, closed about ten minutes after it was found, and found by luck rather than monitoring, plus the near-miss of a replacement check that would have been permanently and wrongly red from its first minute.
- what changed
- A dedicated external vantage point, hosted outside the network with no credential and no path back in, running a periodic full-port scan of the site's public address range against a version-controlled list of exactly what should be reachable.
- the check now
- An unexpected open port on the public range pages at the highest priority. The declared list of what should be reachable is a committed, reviewed change, never something the checker updates on its own. Every scan first proves it can see a known, always-scannable reference host.
A renumbering left an administrative login page answering on a public address for a time nobody recorded; it was closed about ten minutes after it was seen. Nothing in the fleet noticed, and the reason is structural rather than an oversight. Every monitor ran from inside the network. From inside, a probe against the site’s own public address returns the same result, no answer, whether that address is correctly firewalled to silence or is serving an unauthenticated administration panel to the entire internet. Those two states were genuinely indistinguishable from every vantage point the fleet possessed. It was found because someone happened to be on an unrelated external host that morning and looked.
On Node0’s network, as tested, monitoring built entirely inside the perimeter could not tell correctly closed from wrongly wide open at the perimeter’s own edge, because both looked like silence from inside. Whatever a probe inside the network sees, it arrives by a different path, through different rules, than a visitor from the internet, so no check from inside can show what the internet can reach. The only fix is a vantage point that is genuinely outside, which for this site meant a small host in a datacentre in another country, scanning the public range on a schedule against a committed list of what is supposed to be reachable. That list is deliberately not self-updating, because a checker that quietly accepts whatever it finds has stopped checking anything.
Standing up that host raised its own question, which is why it has no credential and no route back into the fleet. A new checking capability must not become a new way in, because a vantage point built to catch exposure is the worst possible thing to expose itself.
And the near-miss is worth keeping as its own small warning. An early version of one of the new host’s uptime checks used a plain ping against public addresses that, by design, do not answer ping. It was permanently red from the moment it was switched on, which would have taught its reader to ignore it within minutes. Test the checker against a known state before you trust what it says about anything else.
The incident itself, and the firewall rule behind it, is lesson 4.3 .
Source: node0 lessons v0.1, lesson 2.21. Sanitized: checklist v0.1, 20260921; address ranges, service names; voice pass 20260921. Part of oznog.com/node0.