Date: May 5, 2026 Duration: 1h 35m (16 minutes to mitigation) Severity: Major
On May 5, 2026, the calendar view in our Publish dashboard stopped displaying post previews for both queued and published posts. The List View kept working. We identified the cause within minutes, reverted the change that introduced it, and restored service.
For roughly an hour and a half, the calendar view rendered as a blank grid — both scheduled and already-published posts were invisible. The List View was unaffected, so customers could still see and manage their posts there, but the primary calendar planning experience was broken. The issue affected all customers using the calendar view, not a subset. The first customer report came in about 14 minutes before we declared the incident internally.
A recent backend change introduced a new limit on how many posts could be returned in a single request. The calendar view requests more items per page than the new limit allowed, so every calendar load was rejected. The List View stayed under the limit and was unaffected.
First customer report: ~14 minutes before formal declaration
Suspect change identified: within ~2 minutes of declaration
Revert merged and deployed: 16 minutes after declaration
Incident resolved: 1h 35m total
Once the suspect change was identified, we reverted it and confirmed the calendar was working again across both internal and customer accounts. A corrected version of the original change was prepared and verified the following day.
We need a step for checking downstream impact when changing shared parts of our backend. The change passed its own tests, but no one searched for other parts of the product that depended on the old behaviour. We're exploring tooling to catch this automatically at PR time.
End-to-end tests on critical views would have caught this in CI before it reached production. We didn't have one covering the calendar.
Faster detection. Better frontend and backend error alerting, along with post-deployment smoke tests on key views, would have caught this within minutes of deploy rather than waiting for a customer report.
What worked. From declaration to fix-merged was about 6 minutes. Roles were clear, severity was escalated proactively once customer impact was understood, and the retrospective focused on systemic gaps rather than individual fault.
