Thousands of Windows machines are experiencing a Blue Screen of Death (BSOD) problem at boot today, impacting banks, airlines, TV broadcasters, supermarkets, and voluminous businesses worldwide.
What Happened?
A defective update from cybersecurity provider CrowdStrike is knocking affected PCs and servers offline, compelling them into a recovery boot loop so machines can’t start properly. CrowdStrike is broadly used by many businesses worldwide for managing the security of Windows PCs and servers.
Affected machines are stuck in a recovery blue screen at boot. Image: Microsoft
Australian banks, airlines, and TV broadcasters first raised the alarm as thousands of machines started to go offline. The issues blowout fast as businesses based in Europe started their work day. UK broadcaster Sky News was unable to broadcast its morning news bulletins for hours this morning, and was showing a message apologizing for “the interruption to this broadcast.” Ryanair, one of the prevalent airlines in Europe, also says it’s experiencing a “third-party” IT issue, which is impacting flight departures.
CrowdStrike’s President & CEO George Kurtz says the global issues were caused by a single faulty content update.
“That update had a software bug in it and caused an issue with the Microsoft operating system,”
he says.
“We identified this very quickly and remediated the issue.”
“CrowdStrike is actively working with customers impacted by a defect found in a single content update for Windows hosts. Mac and Linux hosts are not impacted,”
CEO George Kurtz said in a statement on X.
What to Do?
It’s not easy to say what to do next, as there is a workaround, it’s not scalable as it would need to be applied manually, system by system. In a large company, this could mean it takes hours or more to get back up and running.
By its nature the issue is going to be very hard to resolve once systems are in a reboot loop, says Adam Harrison, managing director at FTI Cybersecurity.
“Manual fixes are going to take time for system admins to apply: CrowdStrike can’t push a new update remotely to fix. It’s going to need manual intervention on each system.”
While initial reports focused on a dodgy update, a user named Brody, who is director of CrowdStrike Overwatch posted on X, formerly Twitter that it is “a faulty channel file, so not quite an update.”
There is a workaround, he added.
1. Boot Windows into Safe Mode or WRE.
2. Go to C:\Windows\System32\drivers\CrowdStrike
3. Locate and delete file matching “C-00000291*.sys”
4. Boot normally.
CrowdStrike says the issue has been identified and a fix has been deployed, but fixing these machines won’t be simple for IT admins. The root cause appears to be an update to the kernel level driver that CrowdStrike uses to secure Windows machines. While CrowdStrike identified the issue and reverted the faulty update after “widespread reports of BSODs on Windows hosts,” it doesn’t appear to help machines that have already been impacted.
This is a breaking story. Keep your eyes peeled and check back to Techwrix.com site for updates.
Great Read. It’s essential for security vendors to have rigorous testing processes in place to prevent these widespread issues.
The article provided an insightful and timely overview of the recent CrowdStrike Windows outage, shedding light on the potential impact and offering practical advice for individuals and organizations affected by the disruption. The clear explanations of the incident and the suggested actions to mitigate its effects were particularly helpful. Thank you for sharing this valuable information.
Thank you Naindini for your kind words!