Issue with Buffer Not Loading
On October 9, Buffer showed a blank page instead of loading for about 20 minutes. During that time, customers couldn’t use the app. We’re sorry for the disruption this caused.
Root cause
We made a small change to how the Buffer app is packaged before it’s sent to customers’ browsers. Browsers run a security check to confirm each file is exactly what they expect, and after this change some files no longer matched. Browsers then refused to load the app. Our release checks didn’t test the app the way it’s actually delivered to customers, so the mismatch wasn’t caught before release.
Customer impact
From 12:16 UTC to about 12:35 UTC, everyone who tried to open the Buffer web app saw a blank page. Scheduled posts, our mobile apps and our API kept working normally.
Steps to resolution
Within minutes of the first reports, we traced the problem to the latest release and switched back to the previous version. Buffer was loading again by 12:35 UTC. We then fully reversed the change and monitored for about an hour before closing the incident.
Key learnings and what we’re changing
Testing releases the way customers get them: we’re adding automated checks that make sure the app’s files match what browsers expect before a release goes out.
Checking the app works before customers see it: we’re exploring ways to confirm each new version actually loads before it starts serving customers.
Recovering quickly: a recent improvement that lets us quickly switch back to a previous version and hold it there helped us restore service in about five minutes. We’ll keep building on it.
