All notes
    15 September 2026·5 min readMonitoringLessons

    We found the same failure three times in our own delivery. Here's the fix.

    An unrun price change, a dead email queue, a checkout that refused every payment. All found by customers, none by us. What we changed, and why every build now includes monitoring.

    Daniel Huthwaite

    Daniel Huthwaite

    Founder, 88 Automate

    I sell businesses a layer they don't have: something that watches their systems and tells them when a piece has quietly stopped working. This summer I found out I didn't have it either.

    Three incidents, one cause

    A price change that never happened. In July I wrote the update that corrected a client's whole catalogue. Every price, every size, every currency. It sat there, ready, and never ran against the live database. For three weeks the business sold at the old prices. A customer noticed before we did.

    An email queue that was quietly dead. Every transactional email in one system goes into a queue, and a scheduled job drains it. The job was never scheduled. A guard that was meant to protect a rollout skipped it silently instead. For weeks, booking confirmations and login emails went nowhere. Nobody got an error. They just didn't get the email.

    A checkout that refused every payment. In September a change to how a deposit was calculated read the wrong field. Every online payment failed for several hours. The first we heard was a customer trying to book a party.

    Three different systems, three different mistakes. The same root cause: nothing was checking. In two of the three, the publishing tool reported success while the change never reached the live system. I believed the green tick.

    What it cost

    Nobody lost data and every customer was looked after. But I'd sold every one of those clients a system that was supposed to remove worry, and each time the worry came back through their inbox rather than mine. That's the wrong way round, and it's the part I minded most.

    What changed

    The fix was the one I'd sell a client, applied to myself.

    1. A health check on every integration I own. A small script asks each connection a simple question every few minutes: are you there, and did the last thing you were meant to do actually happen? Migrations get checked against the live database, not against the tool that claims to have run them.
    2. A deployment checklist that ends with proof. A change isn't done when it's pushed. It's done when I've seen it working on the live system and the check has turned green.
    3. A weekly digest across every client. One page, every Monday, that says what ran, what failed, and what I fixed. If that page is boring, the week was good.

    Why this is now in every build

    Since then, monitoring goes into every system I build. It isn't an upgrade. It's the reason every monthly plan, from Protect upwards, starts with it: automatic checks on every connection and agent, and a person who notices and fixes it before you do.

    I could have written this as a clean success story. The honest version is more useful, and it's the one I'd want to read before hiring someone to look after my systems. If you've ever found out from a customer that something broke, you already know the feeling. The fix is simpler than you think, and it's cheap compared with the alternative.

    Sound like your business?

    Thirty minutes on a call with me. You tell me what's frustrating you, I tell you straight what I'd do.

    A note from Daniel, once a month

    One real lesson from a business I work with, and one thing you can fix this month. No newsletter padding. Unsubscribe any time.