Over the last few days, I’ve been reading about outages at Microsoft, the FAA, United Airlines, Chase, and Amtrak. I’ve also been calling friends, family members, and employees caught in the disruption, checking whether they were okay and how they were getting home.
The word “outage” feels different when someone you care about is stuck at an airport. Behind the technical explanation are tired kids, older passengers waiting in line, missed connections, and plans falling apart. For a business, it can mean lost orders, delayed payments, and a day of work gone.
I am the founder and CEO of OPSWAT, a cybersecurity company focused on protecting critical infrastructure. I am also an instrument-rated pilot. Looking at these incidents, I keep coming back to the same question; why don’t we apply more of the thinking that keeps an airplane operating to the infrastructure we depend on every day?
An airplane packs power, navigation, communications, and flight controls into a remarkably small space. These systems work together, sometimes in difficult conditions, with people’s lives depending on them.
The design assumes that there will be failure. A great deal of pilot training is about recognizing those failures and knowing what to do next. You practice while you still have the luxury of making a mistake.
For instance, take electrical power. An alternator generates electricity for the airplane. Yes, it may help charge your iPhone, but I am more interested in keeping the radios and flight instruments working.
On an airplane equipped with two alternators, one may support the essential loads if the other fails. If both fail, batteries can provide limited operating time. You reduce the load and use that time to manage the emergency.
The important detail is that an alternator and a battery work differently. One generates electricity. The other stores it. The battery gives you another way to keep essential equipment running when the generating system is gone.
That thinking goes further. Airbus has documented flight control designs using different hardware and software in primary and secondary computers, including simpler secondary systems. The aim is to reduce the chance that one defect disables every way of performing a critical function.
I usually bring my iPad with ForeFlight alongside the instruments already in the airplane. It is a supplemental aid, with capabilities that depend on its connected sensors, and I need to understand its limits. Still, having another source of flight information, on different hardware with its own battery, gives me some comfort.
Now look at the same question in IT, in critical systems, and critical infrastructure. These systems need to support life or freedom, like an airplane, so we should have the same design mindset.
If my primary system and backup run the same software, receive the same update, and depend on the same network, how independent are they? Moving one to another building does not remove a defect in the code. A second cloud may still depend on the same identity service or data source.
We can clone a system and copy its weaknesses along with it.
If the same failure can take down both systems, do we really have a backup?
The recent incidents show why that question really matters.
Microsoft, November 12 and 13
Their preliminary report describes a hidden software defect, increased gateway traffic, and reduced capacity. Redirecting traffic from failing servers helped spread the failure to others with the same weakness. Microsoft already had different gateway architectures, but some services, including Exchange Online, could not move to the alternative at the required scale. I appreciate the detail Microsoft published. My takeaway is that independence and capacity need to be tested together. I believe that an alternative architecture needs to carry the essential work as a hot backup when it is actually needed.
FAA, November 13
The FAA described network flooding during a period of router replacement work. Blocking two suspected originating sites stabilized the network. Controllers and pilots maintained safety, but departures nationwide stopped. The root cause remains under investigation, and the FAA says SMART (Strategic Management of Airspace, Routes and Trajectories) was unrelated. The architecture question is how a problem at a small number of locations spread so widely, and what additional isolation and independent communications could preserve more operations safely. However, should we also ask what caused the existing architecture to crumble, and how SMART was ruled out?
Amtrak, November 13
Booking problems appeared while stranded airline passengers were looking for alternatives. Amtrak referenced a system update and increased demand, although the exact technical cause remains unclear. This is a scenario I would put into any recovery exercise: when one part of our infrastructure fails, people turn to another. The next system needs capacity for those people, too.
United, November 16
Reuters reported that an outage affected multiple crew applications and prevented United from providing final aircraft weights for departing flights. The underlying technical cause remains undisclosed. Think about that: the airplane, crew, and passengers can all be ready, yet the flight cannot depart without essential information. Any alternative must have current, accurate loading data. Yesterday’s backup would not be enough. We need to restore a functioning system with current data.
Chase, November 16.
An internal software bug delayed transactions, including payments involving other banks. Here, recovery has to preserve the financial record. Switching quickly helps only if payments are accounted for, balances are correct, and transactions are not processed twice. That needs to be demonstrated under realistic transaction volume.
These are different incidents, and we should be careful about conclusions while investigations continue. But there is already plenty here to bring into our own engineering discussions.
And now we are adding AI to the systems we depend on.
Recovering a conventional application can already be difficult. An AI-operated system adds a model, runtime, data lakes, and pipelines whose relationships can be hard to untangle when something goes wrong. Even when the model stays fixed, the data and operating context can change. Restoring an earlier version of the code may leave components that no longer work correctly together.
The software can be running again while the system is still making bad recommendations.
That is why I think of an AI system as a living operational environment. Recovery means understanding what changed, restoring compatible components, and checking the behavior before relying on it again.
People still need the service while that work is happening.
For our most critical functions, I believe we should consider a separate implementation on a different architecture, already running and ready to take over. That might be another AI platform. It might be a simpler conventional system that can keep essential operations going.
The independence has to extend underneath the application. Different software on the same failing network may still be unavailable. Two different models using the same corrupted data may still produce bad results. We need to examine the whole path the service depends on. We need to take the same approach I described in the Airbus cockpit architecture example.
I know this costs money and adds complexity. The alternative does not need every feature of the primary system. Like the battery in an airplane, it needs to support the essential functions for a defined period, using a different architecture.
Start by deciding what must continue, at what capacity, and for how long. If the operation can tolerate only milliseconds or seconds of interruption, that becomes an engineering requirement we must demonstrate. Restoring everything else can take longer.
Then pressure-test the design under controlled conditions. Make the AI unavailable. Disconnect the network. Introduce stale data, a bad patch, or a cyber compromise. Try it when the backup has already lost some capacity. Watch what happens when real people have to make the transition, and measure whether the service remains usable and correct.
In the cockpit, I need to know what still works after a failure, how long it will work, and what I must do next. I need those answers before I take off.
The families trying to get home and the businesses waiting on payments deserve that same preparation from us.
At your next leadership meeting, ask your team, “if our primary system failed right now, what would keep working, and could the same failure take that away too?”
Dig into the detailed design and architecture and try to poke holes in it. Server down, communications down, a cyber incident. If it’s an AI system like SMART, study its components. Assume it may go out of control, limit bandwidth, and have a second system with a different architecture ready to go if the entire system is down.
Then ask them to show you.
The next outage should not be the first time we find out whether the backup works.
