Getting a live order to three different parties on three different devices in real time, over Cameroonian mobile networks — and behaving correctly when one of them doesn't get the message. Android push delivery there is genuinely unreliable: Doze mode, aggressive OEM battery managers on cheap devices, patchy connectivity.
What I did about it
Layered redundancy with server-side authority, rather than trusting any single delivery channel. Each surface gets a persistent WebSocket for live state, with dedicated E2E tests per channel.
FCM push over HTTP v1 acts as a wake-up channel — and where a missed message costs money, a native attention-grabber that doesn't depend on the notification tray takes over.
The partner app plays audio through audioplayers on the order:new event, because a restaurant tablet sits on a counter and a silent banner is invisible. The rider app draws a full-screen overlay over other apps, the only reliable way to interrupt someone on Android.
Acceptance is resolved server-side, not by whichever UI reacts first — the order state machine lives in the backend with order_status_history recording every transition, so two riders tapping accept in the same second produce one winner and one clean rejection.
What I rejected, and why
Polling is far simpler and would have worked. Rejected because at Cameroonian data prices, three apps polling every few seconds is a real cost passed to riders who are least able to absorb it — and it still wouldn't solve the 'phone is asleep' problem that made the native overlay necessary.
Only 14 of 161 pages are server-rendered — the thinnest ratio here — and for a consumer marketplace that needs restaurant pages to rank, that is a growth problem rather than a technical one. I would server-render the whole storefront and keep the client rendering to the authenticated portals.