Written by: Aaron Pries, Technology Consultant

“When it rains, it pours.” – Early Friday, July 19th, 2024, reports started flooding in from technologists and business professionals regarding several large outages that began disrupting their business services. CrowdStrike, a prominent cybersecurity software giant that services half of the Fortune 500 companies, had confirmed that an update to their Falcon software contained a defect that caused Windows systems running Falcon to crash. To add to the confusion, Microsoft started reporting a massive territorial outage around this same period within one of their most populated Azure regions, which took many Microsoft services and applications offline. After further investigations, the conclusion was that the Microsoft outage was unrelated to the CrowdStrike incident.

The aftermath of the CrowdStrike update went on full display as Americans on the East Coast began noticing that all Windows systems that were running CrowdStrike were affected, as cash registers, airport check-in portals, hospital networks, coffee shop POS systems, healthcare systems, and more were down. The resulting downtime was so severe that the Federal Aviation Administration (FAA) was forced to ground all flights across the United States. CrowdStrike has since issued a statement and action plan to remediate the problematic update for affected customers. It is worth noting that several organizations reported struggling to implement the remediation, as the action plan involves booting into safe mode to remove the problematic file, which requires entering the Bitlocker recovery key for the affected system. This step unexpectedly stressed the preparedness of many organizations as their Bitlocker key management process was lacking or never established, which led to situations akin to self-inflicted ransomware attacks.

How Policies Help Businesses Navigate Through IT and Cybersecurity Disruptions

This major outage serves as a stark reminder of the critical role that strong corporate cybersecurity policies play in navigating the chaos of major disruptions and outages. By maintaining these policies, organizations can create a secure and protected environment, even in the face of unexpected events.

Policies to maintain include:

• Information Security Policy
• Incident Response Plan
• Business Continuity Plan
• Vendor Management Policies

These are critical for maintaining order and reducing the loss of downtime in outage scenarios. Far too often, organizations develop these policies initially but fail to revisit, review, and update them to align with their business goals, technology changes, and emerging threats, which leaves them vulnerable to cybersecurity incidents and unnecessary loss. Especially in circumstances where there are two competing failures, such as the Microsoft regional outage and the CrowdStrike faulty update, it is easy for IT and managerial staff to be overwhelmed with inquiries, communications, collaboration, and documentation while trying to piece together the scope of the incident. Without formal protocols, improper communications with clients may induce unnecessary confusion and mayhem that can result in financial or reputational losses. Maintaining strong corporate policies is essential for minimizing downtime and ensuring a swift path to regaining order.

Connect with Xamin

These Friday incidents, which resulted in major global outages, underscore the need for proactive Incident Response Planning, Vendor Management Policies, Risk Assessment, Contingency Planning, and strong corporate policy oversight. Proactive policy maintenance and oversight are critical for safeguarding customer data and confidence while restoring order in the chaos of major outages. Don’t wait until the next big storm to start preparing your corporate policies and incident response plans. By staying ahead of the curve, you can ensure your organization is always prepared and in control.