The detail that stuck with me isn’t the failed component in Central US. It’s that Copilot errors triggered a client-side retry loop that made recovery harder — GitHub had to mitigate their own clients before they could safely restore traffic. Every LLM wrapper I’ve written retries on failure. None of them have a retry budget.
So that’s my week: consistent retry limits and variable timeouts across anything that calls out, which is exactly the first fix GitHub named. The second is reviewing low-priority CPU and memory alerts for components that fail during sudden spikes — the ones I muted because they were noisy. Neither incident came from a code or config change. Both were capacity failures, which is the kind you can’t roll back.
The story — GitHub CTO Vlad Fedorov wrote up the August 17 outage: 7 hours and 47 minutes across github.com, authentication, Actions, APIs, pull requests, issues, and Copilot. A critical infrastructure component failed to scale with peak traffic. Monthly commits have grown from 1.4 billion to 2.9 billion since April; GitHub has added over 3 million CPU cores, 120 petabytes of storage, and moved roughly 58% of platform load to Azure. (Source)