Skip to content
Oznog

3.16 · storage and backups · after redaction

The bucket that was declared purged and was not

date
20260905, discovered stale and deleted 20260911
what happened
An old S3 bucket on the site's own object storage, left over from a superseded backup design, was noted in a design document as being purged server-side, slow, ignore it. Six days later a follow-up document found the purge had never run: no lifecycle rule existed to make it happen, and 9.97 TiB across about 614,000 objects were still there.
what it cost
At least six days, and in truth longer, of an assumed-completed cleanup silently not happening, consuming roughly 10 TiB that nobody was tracking against any budget.
what changed
The bucket was found and manually deleted once someone actually checked rather than trusting the earlier note. The older design document's claim was left in place as a now-known-stale record, with the correction living in the newer document instead.
the check now
Any "this cleanup is in progress, ignore it" note in a design document is treated as a claim requiring re-verification, not a closed item, especially across a design change.

Migrations and redesigns leave behind old artefacts, and the natural response is to note that this will clean itself up and move on to the actual new work. That note is a prediction, not an observation, and predictions about background processes are exactly the kind of thing that silently fails to come true with nobody watching.

Here, an old object-storage bucket from a replaced design was assumed to be draining itself through a purge operation that had never been configured to run at all. No lifecycle rule existed for it. The bucket sat there for at least six days, continuing to accrue storage cost, discovered only because a later, unrelated piece of work looked at storage usage again and noticed the number had not moved.

Nobody had done anything wrong in the moment. The original note was a reasonable thing to write when the redesign happened. The gap was that nothing in the new design’s own checklist inherited the old design’s unfinished cleanup task, so it fell into a crack between two pieces of work, each of which reasonably assumed it was the other’s concern. The durable habit this suggests is plain. Any note that defers a cleanup to the background should carry its own re-check date, or be folded explicitly into whatever supersedes it, rather than being trusted indefinitely on the strength of having been written down once.

Source: node0 lessons v0.1, lesson 3.16. Sanitized: checklist v0.1, 20260921; bucket name removed; voice pass 20260921. Part of oznog.com/node0.