On May 15, 2026, between 19:13 UTC and 19:34 UTC, users in the Central US and Sweden Central GHEC DR regions experienced failures when pushing to Git repositories. On average, Git HTTP Push Operations saw 10.9% errors, with a peak minute of 31.5%. Actions saw 10% rate of run start delays >5m a 5% failure rate in queueing new runs. The issue was caused by a configuration change that inadvertently stopped background jobs across multiple regions. When the fix was deployed and background jobs restarted, the backlog of queued jobs caused a thundering herd to backend systems, resulting in push failures. The team mitigated the issue by throttling the job queues, and all services recovered by 19:34 UTC. We are implementing queue depth monitoring and alerts for background job processing, as well as improving testing coverage to prevent similar issues.

Posted May 15, 2026 - 19:53 UTC

Impact has been mitigated in the Central US and Sweden Central regions. The underlying issue was correlated to background jobs which had stopped running earlier. When they were started again, repository storage hosts experienced high load, failing some pushes.

All background jobs have completed at this time and we've observed full recovery.

Posted May 15, 2026 - 19:37 UTC

The degradation has been mitigated. We are monitoring to ensure stability.

Posted May 15, 2026 - 19:34 UTC

We are currently investigating increased Git client push operations in the GHEC DR environments in the Central US and Sweden Central regions.

Posted May 15, 2026 - 19:21 UTC

The degradation affecting Git Operations has been mitigated. We are monitoring to ensure stability.

Posted May 15, 2026 - 19:20 UTC

We are investigating reports of degraded performance for Git Operations

Posted May 15, 2026 - 19:18 UTC