Issue with Buffer Not Loading
On September 25, an update stopped Buffer's dashboard from loading. The app partly opened, but account details failed to load, so customers couldn't use it. We're sorry for the disruption.
Root cause
The update had two parts that needed to go live in a set order. One went live before the other was ready, and while they were out of step, account details couldn't load. Our release queue was busier than usual that day, so the second part never caught up.
Steps to resolution
Within minutes, we traced the problem to the update and reversed it. Getting that fix live was slowed by the same busy queue, so we added capacity to push it through.
Key learnings and what we're changing
Blocking unsafe updates: our checks flagged this kind of change, but only as a warning, so the update still went out. We're turning that warning into a hard stop, so an update like this can't go live until it's safe.
Faster releases when it's busy: we're adding capacity to our release process for peak times, so fixes reach you quickly even on our busiest days.
