Making every money-moving endpoint safe to retry, on a network where retries are the normal case rather than the edge case. A request that times out on a Cameroonian mobile network has, from the phone's point of view, an unknown outcome. Retry and you may move real money twice. Don't retry and a user's cash-out silently fails.
What I did about it
Idempotency became a structural property of the data model rather than a check in each handler. Balances are not a mutable column — they are derived from an append-only ledger_entries table.
Every entry carries an idempotency key that is unique at the database level, so a replay hits a uniqueness constraint and returns the original result. Postgres enforces the guarantee, not a remembered if already_processed branch.
External rail calls are wrapped identically via processed_webhook_events, retry_jobs and webhook_deliveries, so a duplicate MoMo callback is absorbed rather than double-credited.
Money is held in integer minor units throughout, with floats banned past the display layer, so replay and reconciliation compare exact values.
What I rejected, and why
Optimistic locking on a balance column would have been simpler to write and impossible to audit. It makes the current balance authoritative and the history advisory — exactly backwards for a payments system. Every balance read is now a SUM, mitigated with a cached value plus periodic invariant checks that re-derive and compare. Worth paying.
The coverage gate is set at 80% but the actual measured number was never recorded, so I can say the gate passes and not what it passes at. I would wire coverage reporting into CI output from day one — a gate you cannot read is a gate you eventually stop trusting.