Global IT Outage: Crowdstrike Falcon Update Causes Windows Boot Issues

The Issue

A significant global IT outage is currently afoot, with early reports indicating that an update to the Crowdstrike Falcon agent has modified Microsoft Windows system files required for stable operation and boot.

If you're a @CrowdStrike customer and your machine is off, leave it that way.

Something has caused blue screen loops with csagent.sys and it's, um, not good... pic.twitter.com/PeYLH8qhGT

— Jake Williams (@MalwareJake) July 19, 2024

Outage Impact

There are emerging announcements of significant knock on effects due to the widespread use of the Crowdstrike software, in particular Microsoft’s 365 infrastructure, causing issues for businesses and users who are not direct customers of Crowdstrike.

The current scope of outages has been described to impact the following regions:

• EU-1

• US-1

• US-2

• US-GOV-1

Source: https://supportportal.crowdstrike.com/s/article/Tech-Alert-Windows-crashes-related-to-Falcon-Sensor-2024-07-19 (login required)

Solution

For those seeking immediate rectification, here is a functional solution to the recovery boot loop:

Workaround Steps:

1. Boot Windows into Safe Mode or the Windows Recovery Environment

2. Navigate to the C:\Windows\System32\drivers\CrowdStrike directory

3. Locate the file matching "C-00000291*.sys*, and delete it.

1. ‘cd \windows\system32\d4ivers\crowdstrike’

2. ‘del C-00000291*.sys’

3. ‘shutdown /r’

4. Boot the host normally

It should be noted that this action will require local administrative permissions and is largely unfixable in a remote support scenario without user interaction.

You will require the following to enact the above resolution steps:

• bitlocker recovery keys (where bitlocker in enabled)

• Local Administrative username and password, or LAPS password where in use.

The failed .SYS file update is no longer being distributed by the Crowdstrike platform.

For those organizations scrambling to resolve this issue and alleviate the business disruption, Datasolace LTD are currently offering consultation and IT surge support. Contact us at: enquiries@datasolace.com, or through the form below.

Name *

First Name

Last Name

Email *

Subject *

Message *

Thank you!

Implications and Lessons Learned

The disruption caused by this poor control of software testing and distribution is certain to substantial and should lead to all businesses and vendors to consider their single points of failure. Crowdstrike software operates within a position of escalated privilege and is intended to thwart cyber security incidents; this could be a pivotal moment in their future.