Event Recap: Unexpected Fault During Database Upgrade
Recently, Paradex encountered a technical issue during a planned 30-minute database maintenance window. A race condition occurred during a service restart, leading to corrupted data being persistently written to the cloud and propagated to ParadexChain.
This error resulted in funding indices across multiple markets being reset to zero, distorting profit and loss calculations and triggering widespread liquidations.
- Funding index disruptions caused market instability
- Users' positions were exposed to unintended risks
- Immediate intervention was required to limit losses
Mitigation Actions and User Protection
To prevent further damage, Paradex implemented several emergency measures, including:
- Temporarily suspending platform access
- Cancelling all open orders except take-profit and stop-loss orders
- Rolling back the blockchain to a snapshot taken before the maintenance window began
In addition, Paradex has committed to refunding affected users totaling $650,000 across 200 accounts, ensuring no user suffers financial loss due to this incident.
Future Enhancements
Paradex has announced plans to conduct a comprehensive review of its upgrade protocols and enhance fault tolerance in database operations. This incident serves as a critical reminder for the industry to adopt more rigorous procedures and contingency planning during critical system maintenance.