Incident Timeframe: May 13-14, 2025 Impact: Delayed onchain transaction confirmations and stuck redeem operations
Summary: On May 13, we experienced a nonce management issue in our backend, which led to a backlog of stuck transactions. This was triggered by a combination of a UX change on the CBW side and a temporary spike in Base gas prices. A user attempted to redeem the same link multiple times within milliseconds, leading our system to submit duplicate transactions before recording the first operation in our database. The second transaction failed due to network congestion and was evicted from the mempool, resulting in a nonce gap. This prevented subsequent transactions from being processed correctly.
Resolution: We have resolved the nonce issue and all systems are now fully operational.
Next Steps:
Investigate root cause of the nonce management issue
Add caching to prevent duplicate redeem transactions
Improve resilience against network-related nonce gaps