Skip to main content

Securing Real Things

Here’s a podcast worth a listen. Brian Contos @ Imperva interviews Joseph Weiss on the topic of control system security.


  • "Running the operational systems comes before any security requirements."
  • "Power [industry] does not take this seriously"
  • "Are we more secure [than 10 years ago]? No."


  • Systems that run Windows 95 and NT 4.0 even after the upgrades.
  • Can't patch them without breaking them.
  • Custom written TCP stacks that crash when you ping them.
  • Field devices with built in bluetooth or wireless modems
  • Bolted-on security. Very old platforms, not designed with security in mind.
  • Two cyber incidents on systems with brand new control systems. Very significant equipment damage.
  • Significant environmental discharge. Three deaths.

Janke-CNC-1980I have no particular knowledge of this general topic, other than a couple decades ago I made a living programming CNC machine tools, wrote a textbook on the topic, and occasionally played with PLC's. Oh - and I tried to write a Windows 3.x app that communicated with HART field instruments at 1200 baud. That was pre-internet, pre-almost-everything. You can see from the 30 year old pic of me that we were hardly advanced past stone knives and bear skins.

Starting from the point of view of someone with almost no knowledge here’s my- 

Random thoughts:

A bunch of years ago when I lived in a small town, I woke up one winter morning to a cold house, no heat and no hot water. A bit of investigating and a knock on the door from a city utility worker cued me in to the cause. A couple teenagers thought it would be amusing to climb over a fence and put a big wrench on a big valve and shut off the main natural gas line coming into town. It didn't take long for the underground gas distribution pipes to empty out and run the whole town out of natural gas. This ended up being much more than an amusing prank. The city utility workers had to:

  • shut off and lock the gas meter at every house and business in town. Gas meters in older houses were still indoors, so the workers had to get access to those houses. (a couple days work)
  • re-fill the pipeline and city distribution network
  • go back to every house and business, contact the residents, locate every gas appliance in each house, turn on the gas to the house, re-light and test each appliance. (a couple more days work)

The tasks needed to be done serially. In other words, until all the gas was shut off, no gas lines could be re-pressurized. If memory serves me correctly, I had no gas, stove, heat or hot water for 3 or 4 days in late winter (in Minnesota, where winter is really winter) This was a minor incident, nothing more than a prank gone bad, but one with significant downstream consequences. Safely restoring service after a simple prank took significant resources and disrupted an entire community.

CNC machine tools and industrial robots, most of which are now networked, can be damaged easily by a simple bug in their programming, and when damaged can take weeks to repair (don't ask me how I know…).

Programmable road signs apparently are amusingly easy to re-program.

If a bad guy can embed a trojan at the heart of a payment processor network, then presumably doing the equivalent with the networks that control infrastructure like power, water, chemical, oil and similar facilities shouldn't be much more difficult. The consequences though, could be far worse. Nobody dies when their payment card gets caught up in fraud.

If you are going to declare cyber war on a nation, don't play around with network DDOS's. They are annoying and disruptive, but the damage is transient and whatever service was disrupted is easily restored after the attack. Servers reboot, routes heal, Big deal. 'Rebooting' a pipeline, refinery, or similar infrastructure is non-trivial, and repairing  physical damage caused by disrupting complex control systems is orders of magnitude more expensive and difficult than repairing virtual damage from  hacking web sites or DDOS'ing political entities.  Crank around on the PLC's that control the  valves that mix things together in a municipal water system, refinery or chemical plant and really bad things that hurt real people can happen.

In  a recent Schneier post, Bryan Singer commented:

I feel pretty confident in saying that any of us that have been working in this space for any time probably have the knowledge required to stop a significant amount of manufacturing, disable infrastructure, stop utilities, turn off the lights, water, etc without a lot of effort. If we know how to do it, so do the proverbial "bad guys" (or they shortly will).


  1. I haven't listened to the podcast yet, but it sounds great.

    It's always been easier to destroy than create. Trading bricks for bits doesn't change much in that equation.

  2. Except that in many cases, the brick are harder to fix than the bits. :-)


Post a Comment

Popular posts from this blog

Cargo Cult System Administration

Cargo Cult: …imitate the superficial exterior of a process or system without having any understanding of the underlying substance --Wikipedia During and after WWII, some native south pacific islanders erroneously associated the presence of war related technology with the delivery of highly desirable cargo. When the war ended and the cargo stopped showing up, they built crude facsimiles of runways, control towers, and airplanes in the belief that the presence of war technology caused the delivery of desirable cargo. From our point of view, it looks pretty amusing to see people build fake airplanes, runways and control towers  and wait for cargo to fall from the sky.
The question is, how amusing are we?We have cargo cult science[1], cargo cult management[2], cargo cult programming[3], how about cargo cult system management?Here’s some common system administration failures that might be ‘cargo cult’:
Failing to understand the difference between necessary and sufficient. A daily backup …

Ad-Hoc Versus Structured System Management

Structured system management is a concept that covers the fundamentals of building, securing, deploying, monitoring, logging, alerting, and documenting networks, servers and applications. Structured system management implies that you have those fundamentals in place, you execute them consistently, and you know all cases where you are inconsistent. The converse of structured system management is what I call ad hoc system management, where every system has it own plan, undocumented and inconsistent, and you don't know how inconsistent they are, because you've never looked.

In previous posts (here and here) I implied that structured system management was an integral part of improving system availability. Having inherited several platforms that had, at best, ad hoc system management, and having moved the platforms to something resembling structured system management, I've concluded that implementing basic structure around system management will be the best and fastest path to…

The Cloud – Provider Failure Modes

In The Cloud - Outsourcing Moved up the Stack[1] I compared the outsourcing that we do routinely (wide area networks) with the outsourcing of the higher layers of the application stack (processor, memory, storage). Conceptually they are similar:In both cases you’ve entrusted your bits to someone else, you’ve shared physical and logical resources with others, you’ve disassociated physical devices (circuits or servers) from logical devices (virtual circuits, virtual severs), and in exchange for what is hopefully better, faster, cheaper service, you give up visibility, manageability and control to a provider. There are differences though. In the case of networking, your cloud provider is only entrusted with your bits for the time it takes for those bits to cross the providers network, and the loss of a few bits is not catastrophic. For providers of higher layer services, the bits are entrusted to the provider for the life of the bits, and the loss of a few bits is a major problem. These …