I didn’t plan on writing about the DHS blowing up a power generator on CNN, but I’m in my hotel room in Vegas waiting for a conference call and it’s all over the darn TV. Martin and Amrit also talked about it, and I hate to be late to a party.

That little video has started an uproar. Based on the press coverage you’ve got raving paranoids on one side, and those in absolute denial on the other. We’re already seeing accusations that it was all just staged to get some funding.

I’ve written about SCADA (the systems used to control power grids and other real-world infrastructure like manufacturing systems) for a while now. I’ve written about it here on the blog, and authored two research notes with my past employer that didn’t make me too popular in certain circles. I’ve talked with a ton of people on these issues, researched the standards and technologies, and my conclusion is that some of our networks are definitely vulnerable. The problem isn’t so bad we should panic, but we definitely need to increase the resources used to defend the power grid and other critical infrastructure.

SCADA stands for Supervisory Control And Data Acquisition. These are the systems used to supervise physical things, like power switches or those fascinating mechanical doohickies you always see on the Discovery Channel making other doohickies (or beer bottles). They’ve been around for a very long time and run on technologies that have nothing to do with the Internet. At least they used to.

Over the last decade or so, especially the past five years, we’ve seen some changes in these process control networks. The first shift was starting to use commodity hardware and software, the same technology you use at work and home, instead of the proprietary SCADA stuff. Some of these things were O L D old, inefficient, and took special skill to maintain. It’s a lot more efficient for a vendor to just build on the technology we all use every day; running special software on regular hardware and operating systems.

Sounds great, except as anyone reading this blog knows there are plenty of vulnerabilities in all that regular hardware and software. Sure, there were probably vulnerabilities in SCADA stuff (we know for a fact there were), but it’s not like every pimply faced teenage hacker in the world knew about them. A lot of new SCADA controllers and servers run on Microsoft Windows. Nothing against Microsoft, but Windows isn’t exactly known as a vulnerability free platform. Worse yet, some of these systems are so specialized that you’re not allowed to patch them- the vendor has to handle any software updates themselves, and they’re not always the most timely of folks. Thus we are now running our power plants and beer bottling facilities on stuff that’s on the same software all the little script kiddies can slice through, and we can’t even patch the darn things. I can probably live without power, but definitely not the beer. I brew at home, but that takes weeks to months before you can drink it, and our stash definitely won’t last that long. Especially without any TV.

Back to SCADA. Most of these networks were historically isolated- they were around long before the Internet and didn’t connect to it. At least before trend number two, called “convergence”. As utilities and manufacturing moved onto commodity hardware and software, they also started using more and more IT to run the business side of things. And the engineers running the electric lifeblood of our nation want to check email just as often as the rest of us. And they have a computer sitting in front of them all day. Is anyone surprised they started combining the business side of the network with the process control side? Aside from keeping engineers happy with chain letters and bad jokes, the power companies could start pulling billing and performance information right from the process control side to the business side.

They merged the networks. Not everyone, but far more companies than you probably think.

I know what you’re all thinking right now, because this is Securosis, and we’re all somewhat paranoid and cynical. We’re now running everything on standard platforms, on standard networks, with bored engineers surfing porn and reading junk email on the overnight shift.

Yeah, that’s what I thought, and it’s why I wrote the research.

This isn’t fantasy; we have a number of real world cases where this broke real world things. During the Slammer virus a safety system at a nuclear power plant went down. Trains in Sydney stopped running due to the Sasser virus. Blaster was a contributing factor to the big Northeast power outage a few years ago because it bogged down the systems the engineers used to communicate with each other and monitor systems (rumor has it). I once had a private meeting in a foreign country that admitted hackers had gained access to the train control system on multiple occasions and could control the trains.

Thus our infrastructure is vulnerable in three ways:

1. A worm, virus, or other flaw saturating network traffic and breaking the communications between the SCADA systems.
2. A worm, virus, or other attack that takes down SCADA systems by crashing or exploiting common, non-SCADA, parts of the system.
3. Direct attack on the SCADA systems, using the Internet as a vector

Some of these networks are now so messed up that you can’t even run a vulnerability scan on them without crashing things.

Bad stuff, but all hope isn’t lost. Not everyone connects their systems together like this. Some organizations use air gaps (totally separate, isolated networks), virtual air gaps (connected, but an isolated one-way connection), or air-locks (a term I created to describe two separate networks with a very controlled, secure system in the middle to exchange information both ways, not network traffic). NERC, the industry body for the power networks, created a pretty good standard (CIP, Critical Infrastructure Protection) for securing these networks that went into effect last year. When I talk to power guys these days about network separation, I don’t get nearly the strange looks I did five years ago.

Another thing in our favor is that to cause serious damage like we saw in the video, you really need to know what you’re doing. You have to gain access to the network, disable safeties, and know exactly what to do.

Well, more bad news. I’m not worried about Joe Hacker at Starbucks or whatever they use for Internet cafes in Russia (Starbucks?) taking down the North American power grid. But it’s very clear that foreign nations have the expertise to do this, especially over in China where they seem to be having all sorts of fun on our networks. Terrorists? They’re better off just blowing up a few major transformers. That will take out major parts of the grid, might blow up some generators (years ago the one at the University of Colorado blew up during a big blackout), and those transformers are both costly and may take years to replace. Besides, terrorists are blood-obsessed psychotics, despite their threats to attack our economy and infrastructure.

In summary we are definitely vulnerable to just the right kind of attack, but it’s a problem we can get our arms around and solve with a little investment and common sense. Not everything is vulnerable yet, and we’re early enough on the convergence trend that we can still stop and put the right security precautions in place.

I’m glad that video hit the news; maybe we’ll get the right amount of dollars in the right places so we can take this one off the table.

Unless the bad guys just get jobs at the power plants and flip switches during the midnight shift.

Not that I’m paranoid or anything.