Starting at 2025-12-05 3:47 AM ET, Block services experienced a service degradation resulting from a bad deployment in our third party provider's security tooling. This disruption impacted many features including the ability to take payments, checkout or log in to Block services.
In this postmortem recap, we’ll communicate the root cause of this degradation, document the steps that we took to diagnose and resolve it, and share our analysis and actions to ensure that we are properly protecting our customers from service interruptions like this in the future.
2025-12-05 03:47 - Provider implements a change on their system for all customers to protect against a critical security vulnerability. Impact starts
2025-12-05 03:48 - Provider changes fully propagate through their network.Â
2025-12-05 03:50 - Block teams are notified through alerting, team begins triaging
2025-12-05 03:50 - Provider declares incident
2025-12-05 04:01 - Square Status Page is activated
2025-12-05 04:04 - 04:06 - Block engineering team begins preparations to failover traffic in case of extended impact
2025-12-05 04:11 - Provider reverts changeÂ
2025-12-05 04:12 - Provider change propagation completes
2025-12-05 04:12 - Block systems start to recover
2025-12-05 04:14 - Payments, web and mobile activity is back up
2025-12-05 04:20 - provider declares incident as resolved
2025-12-05 04:24 - Block declares incident as stabilized
The root cause of this incident was a bad deployment in our provider's security tooling which propagated globally as they attempted to update a security vulnerability rule. The incident revealed areas of improvement where we are actively working with both internal teams and with our provider, namely:Â
Improve our tooling to automate the use of alternative providers during incidents
Increase efforts to onboard Block domains onto alternative providers
Improve offline payment enablement and user experience
Many sellers utilized Square’s Offline Payments feature to continue accepting payments throughout the degradation. While offline payments were operational during this time if sellers opted in, due to other portions of the system such as login and web being unavailable some sellers may not have had the chance to enable offline payments. We are continuing to expand support for and improve our communication around service disruptions.
We apologize for the disruption this outage might have created for you, your customers, and your employees. We appreciate your business and we are committed to continuing to improve reliability.