The Wayback Machine - https://web.archive.org/web/20241009112944/https://www.githubstatus.com/
GitHub header
All Systems Operational
Git Operations ? Operational
Webhooks ? Operational
Visit www.githubstatus.com for more information Operational
API Requests ? Operational
Issues ? Operational
Pull Requests ? Operational
Actions ? Operational
Packages ? Operational
Pages ? Operational
Codespaces ? Operational
Copilot Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Past Incidents
Oct 9, 2024

No incidents reported today.

Oct 8, 2024
Isolated Codespaces creation failures in the West Europe region
Resolved - This incident has been resolved.
Oct 8, 23:32 UTC
Update - Codespaces is operating normally.
Oct 8, 23:32 UTC
Update - Codespace creation has been remediated in this region.
Oct 8, 23:32 UTC
Update - We are once again seeing signs of increased latency for codespace creation in this region, but are at the same time recovering previously unavailable resources.
Oct 8, 22:54 UTC
Update - Recovery continues slowly, and we are investigating strategies to speed up the recovery process.
Oct 8, 22:10 UTC
Update - We are continuing to see gradual recovery in the region and continue to validate the persistent fix.
Oct 8, 21:39 UTC
Update - The persistent fix has been applied, and are beginning to see improvements in the region. We are still working on follow-on effects, however, and expect recovery to be gradual.
Oct 8, 21:06 UTC
Update - We are nearing full application of the persistent fix and will provide more updates soon.
Oct 8, 20:26 UTC
Update - Mitigations we have put in place are yielding improvements in Codespace creation success rates in the affected region. We expect full recovery once the persistent fix fully rolls out.
Oct 8, 19:51 UTC
Update - We are continuing to work on mitigations while the more persistent fix rolls out.
Oct 8, 19:17 UTC
Update - We are continuing to apply mitigations while we deploy the more persistent fix. Full recovery is expected in 2 hours or less, but more updates will be coming soon.
Oct 8, 18:44 UTC
Update - We have applied some mitigations that are improving creation success rates while we work on the more comprehensive fix.
Oct 8, 18:08 UTC
Update - We have identified a possible root cause and are working on the fix.
Oct 8, 17:43 UTC
Update - Some Codespaces are failing to create successfully in the Western EU region. Investigating is ongoing.
Oct 8, 17:11 UTC
Update - Codespaces is experiencing degraded performance. We are continuing to investigate.
Oct 8, 17:08 UTC
Investigating - We are currently investigating this issue.
Oct 8, 17:02 UTC
Oct 7, 2024

No incidents reported.

Oct 6, 2024

No incidents reported.

Oct 5, 2024

No incidents reported.

Oct 4, 2024

No incidents reported.

Oct 3, 2024

No incidents reported.

Oct 2, 2024

No incidents reported.

Oct 1, 2024

No incidents reported.

Sep 30, 2024
Incident with Codespaces
Resolved - On September 30th, 2024 from 10:43 UTC to 11:26 UTC Codespaces customers in the Central India region were unable to create new Codespaces. Resumes were not impacted. Additionally, there was no impact to customers in other regions.

The cause was traced to storage capacity constraints in the region and was mitigated by temporarily redirecting create requests to other regions. Afterwards, additional storage capacity was added to the region and traffic was routed back.

A bug was also identified that caused some available capacity to not be utilized, artificially constraining capacity and halting creations in the region prematurely. We have since fixed this bug as well, so that available capacity scales as expected according to our capacity planning projections.
Sep 30, 11:26 UTC
Update - Codespaces is operating normally.
Sep 30, 11:26 UTC
Update - We are seeing signs of recovery in Codespaces creations and starts. We are continuing to monitor for full recovery.
Sep 30, 11:25 UTC
Update - Codespaces is experiencing degraded performance. We are continuing to investigate.
Sep 30, 11:24 UTC
Update - We are investigating a high number of errors in Codespaces creation and start.
Sep 30, 11:09 UTC
Investigating - We are investigating reports of degraded availability for Codespaces
Sep 30, 11:08 UTC
Sep 29, 2024

No incidents reported.

Sep 28, 2024

No incidents reported.

Sep 27, 2024
Disruption with some GitHub services
Resolved - On September 27th, 2024 from 10:26 to 10:34 UTC, some users listing project releases may have encountered a 500 error.
Sep 27, 15:30 UTC
Sep 26, 2024
Degraded performance for some Copilot users
Resolved - Between September 25, 2024, 22:20 UTC and September 26, 2024, 5:00 UTC the Copilot service was degraded. During this time Copilot chat requests failed at an average rate of 15%.

This was due to a faulty deployment in a service provider that caused server errors from multiple regions. Traffic was routed away from those regions at 22:28 UTC and 23:39 UTC, which partially restored functionality, while the upstream service provider rolled back their change. The rollback was completed at 04:41 UTC.

We are continuing to improve our ability to respond more quickly to similar issues through faster regional redirection and working with our upstream provider on improved monitoring.
Sep 26, 05:08 UTC
Update - Monitors continue to see improvements. We are declaring full recovery.
Sep 26, 05:08 UTC
Update - Copilot is operating normally.
Sep 26, 05:03 UTC
Update - We've applied a mitigation to fix the issues and are seeing improvements in telemetry. We are monitoring for full recovery.
Sep 26, 03:51 UTC
Update - We believe we have identified the root cause of the issue and are monitoring to ensure the problem does not recur.
Sep 26, 02:34 UTC
Update - We are continuing to investigate the root cause of the latency previously observed to ensure there is no reoccurrence, and better stability going forward.
Sep 26, 01:46 UTC
Update - We are continuing to investigate the root cause of the latency previously observed to ensure there is no reoccurrence, and better stability going forward.
Sep 26, 01:03 UTC
Update - Copilot users should no longer see request failures. We are still investigating the root cause of the issue to ensure that the experience will remain uninterrupted.
Sep 26, 00:29 UTC
Update - We are seeing recovery for requests to Copilot API in affected regions, and are continuing to investigate to ensure the experience remains stable.
Sep 25, 23:55 UTC
Update - We have noticed a degradation in performance of Copilot API in some regions. This may result in latency or failed responses to requests to Copilot. We are investigating mitigation options.
Sep 25, 23:40 UTC
Investigating - We are investigating reports of degraded performance for Copilot
Sep 25, 23:39 UTC
Sep 25, 2024
Incident with Actions Runs
Resolved - On September 25th, 2024 from 18:32 UTC to 19:13 UTC, Actions service experienced a degradation during a production deployment, leading to actions failing to be downloaded at the start of a job. On average, 21% of Actions workflow runs failed to start during the course of the incident. The issue was traced back to a bug in an internal service responsible for generating the URLs used by the Actions runner to download actions.

To mitigate the impact, we rolled back the affecting deployment. We are implementing new monitors to improve our detection and response time for this class of issues in the future.
Sep 25, 19:19 UTC
Update - We're seeing issues related to Actions runs failing to download actions at the start of a job. We're investigating the cause and working on mitigations for customers impacted by this issue.
Sep 25, 19:14 UTC
Investigating - We are investigating reports of degraded performance for Actions and Pages
Sep 25, 19:11 UTC
Incident with Git Operations
Resolved - On September 25, 2024 from 14:31 UTC to 15:06 UTC the Git Operations service experienced a degradation, leading to 1,381,993 failed git operations. The overall error rate during this period was 4.2%, with a peak error rate of 12.5%.

The root cause was traced to a bug in a build script for a component that runs on the file servers that host git repository data. The build script incurred an error that did not cause the overall build process to fail, resulting in a faulty set of artifacts being deployed to production.

To mitigate the impact, we rolled back the affecting deployment.

To prevent further occurrences of this cause in the future, we will be addressing the underlying cause of the ignored build failure and improving metrics and alerting for the resulting production failure scenarios.
Sep 25, 16:03 UTC
Update - We are investigating reports of issues with both Actions and Packages, related to a brief period of time where specific Git Operations were failing. We will continue to keep users updated on progress towards mitigation.
Sep 25, 15:34 UTC
Investigating - We are investigating reports of degraded performance for Git Operations
Sep 25, 15:25 UTC
← Incident History Powered by Atlassian Statuspage