Comment from Jake Brodsky
Time: February 26, 2008, 3:36 pm

Dale, I started to write this twice before and then each time I stopped to think.

What we’re dealing with here is are differences of perspective on a very large subject that no-one can wrap their minds around.

I think that’s why so many are reeling and saying “We’re DIFFERENT, it works, DON’T TOUCH IT!”

To work effectively, one needs strong backgrounds in cyber security, computer networking, computer software, process engineering, process operations, and management –just to name a few disciplines. Go ahead and try to find a candidate who meets all those criteria. I’ll claim a few of these creds, I’m sure you can claim others. But at the end of the day, we’re all missing some key parts to understand the whole issue. There is no “getting it.”

One thing we may want to consider is this: Perhaps it is time to divorce control engineering from the complexity that PC operating systems have become. The problem we’re having is that too many are seeking that “real time” data not knowing the implications and actual performance impact they may have by asking for something that a glossy CIO porn magazine suggested.

Computer security involves knowing the information flows, and understanding what the KPIs are, who needs them, how frequently they need the information, what data needs to be archived, what data needs to be trended, how the process data can be certified, and so on and so forth.

The symptoms here are also found in other parts of IT. I’ve always said you can tell how healthy an organization is by looking at their IT department. In many cases, complexity was added with little thought to ROI simply because it could be added. Then the people who wanted it wrote lovely papers for glossy magazines, and left for higher paying jobs elsewhere, leaving the complex mess for someone else to figure out.

It happens all the time, in virtually every segment of business that IT serves. People then look to the IT department to explain the extra complexity –and they can’t because they don’t remember who created it or why it was created in the first place.

Control Systems Engineers have been fighting against this sort of thing for years. SCADA security on the existing PC based HMI platforms is starting to raise mind blowing complexities to something they thought they once understood. Yeah, we’re all frazzled.

Now the glossy magazine readers are all clamoring for their piece of the system –and we’re all left screaming WAIT A MINUTE! TIME OUT! WHO NEEDS THIS STUFF? Where is the business case?

If you see a lot of cantankerous people in this business it’s because we really are up to our collective asses with alligators, and yes by the way, we still need to drain the swamp.

What we need to do is to try to make these systems comprehensible, and easier, not more secure and harder. The final success of this project will be “do the operators understand and use these tools?” If not, it’s back to the drawing board.

The problem is software bloat, feature bloat, and excessive abstraction of too many concepts. For example, Virtual Machines are really nice and all that –if you live in front of the computer all day. But if you don’t, understanding what an instance of an operating system is, is not common among typical users.

And the most dangerous thing in this business is someone who doesn’t understand the tools he uses. That’s why so many are so resistant to change. We know what we’re doing most of the time, but we don’t understand these new tools, policies, procedures, and so forth that everyone keeps pulling out of more and more bags. At some point, we have to say enough is enough, let’s simplify!

Jake Brodsky

Comment from Bryan Singer
Time: February 26, 2008, 5:28 pm

I think that is the reason so many spend so much time debating… Far too many people have only one hammer in their toolbox, and if all they have is that, every problem looks like a nail. Its why so many poor vulnerability assessments have been completed. Its why so many places I go to have spent all their money on IT and physical security because their “scans” didn’t show anything significant in controls… but we can flatten plants in seconds if called upon to do so (note, I do NOT advocate active penetration testing in controls). Its also why so many people have thrown up their hands in disgust because you have two competing interests…

IT wants to do things smaller, faster, cheaper with more standardization and less support. They are an overhead to the business and called upon every day to lower costs and do more with less. They don’t want process control desktop images to manage, additional firewalls, differing configurations for switches and firewalls, and other measures.. because they make their job more complex and costly, meaning less bonuses and less incentives.

Engineering is driven by ROI… downtime is serious money, safety is imperative, and uptime is king.

With competing interests, it is ESSENTIAL that we manage the problem together. It is important to articulate the differences in approach that are needed, and come up with a joint strategy for addressing the issues. All too often, one or both sides are completely insistent that one way is the only way, and that usually smacks of people that are unwilling or unable to sit down and identify and address issues.

There are differences we MUST address.. such as we don’t typically run the risk of killing people in IT type failures, and network functionality and connectivity doesn’t mean a thing if the device I/O and designed functions can’t occur.

There are differences that we should understand to realize they really aren’t differences: We manage network by requirements.. articulately communicate the requirements, and then the issue goes away. Patch management CAN be done, but it must be thought through and TESTED. Anti-virus is the same… Passwords may well not be appropriate for control devices, but if that is the case, we need to tighten the borders to entry into the environment (it always astounds me when someone says “NO PASSWORDS but we have to be SECURE!” Then they are unwilling to do what is required which is to keep unauthorized people out of the environment as a compensatory control

Anyway… the fight will go on… But no one is served by taking an absolute stand to the point that the differences become a barrier to moving things forward.

Comment from Matthew Franz
Time: February 27, 2008, 2:38 pm

Since nobody would smack me in the head, I was forced to unsubscribe from all the SCADA mailing lists.

Comment from Ralph Langner
Time: February 27, 2008, 6:48 pm

“NO PASSWORDS but we have to be SECURE!” — that was a good one. I could add: “We CAN’T firewall the connection to the corporate network, there are just to many interconnections… but we have to be secure”. (The plant floor network from this admin was so “secure” that his buddies from the IT department advocated for firewalling because they feared a malware intrusion from the production network.)

Besides… Dale, I haven’t seen you lose temper for so long, but this IT vs. SCADA thing really seems to get you. [quickly walking away]

Comment from Ron Southworth
Time: March 1, 2008, 3:19 pm

I am tired of it too guys. I am sad that the us and them debate still rages and that Matt’s sagely advice is lost to the SCADA community on the list as an outcome.

Bickering is not going to fix the problem and I would rather see energy being put into a solution.

Comment from cnioperator
Time: March 5, 2008, 8:15 am

Oh come on chaps! Wasn’t it more fun when we all blamed the IT department beacuse they didn’t understand us