Comment from Tim
Time: December 12, 2007, 7:15 am

I can see both sides of the argument. The side that you didn’t talk about was the justification for using SP 800-53: interoperability. As various technologies gain network “awareness”, their security communities collide with network security, resulting in horrendous (and sometimes humorous) mis-communications based on: 1) the lack of a common syntax and 2) conflicting paradigms.

Network security has to work with security types from: law enforcement, the building trades, utilities (SCADA), bankers, government & military, all of the manufacturing fields, etc., etc. It is much, much easier (it’s still not all that easy) to get a single group or industry to adapt to an overall standard than it is to get the other groups to pick up the new kid’s syntax. Even worse is having to live with the disparate lexicons, having to translate each and every time one group has to communicate with the other.

I agree that they should be given free reign in developing the NERC CIP. However, while it’s still quite young, it needs to be mapped to the “higher” common infrastructure syntax/lexicon/taxonomy so that when it’s needed, communication with the rest of the national infrastructure doesn’t require a translator and two engineer-priests.

(Hint: pick any three of the industries considered to be primary infrastructure by Infragard and look up their definitions of risk mitigation, residual (acceptable) risk, and incident.)

The goal of all this painful movement is overall infrastructure protection, not just a single industry. In other words, incident prevention and rapid (stress on rapid) incident response. When either of these two processes have to go through multiple levels of management in multiple organizations, incident prevention acquires blind spots (”I thought they were taking care of that”) and incident response devolves into arguments about who’s in charge, closely followed by who was responsible.

Comment from Jake Brodsky
Time: December 12, 2007, 3:46 pm

Speaking as one representing the interests of an asset owner in the Water business, we aren’t waiting for any standards before we take action. What the standards have to offer is a method to document what we do. It doesn’t tell us what to do first or how to go about it. So don’t get the idea that the standards are driving us. We use our judgement as Professional Engineers to select the immediate threats and we deal with them. The documentation will come later.

However, I have a specific problem with the CIP standards: For those who work on Federal Resources, as many water utilities do, NERC CIP isn’t acceptable. The FIPS and NIST 800 series of standards should apply. For the rest of the water utilities, yes, they could model themselves after the NERC CIPs, but why bother? A major portion of the industry is already committed to 800-53. What purpose would be served by aligning themselves with another standard?

Comment from CallBEFOREYouDig
Time: December 12, 2007, 9:33 pm

SANS has some breathless rhetoric on this in their latest NewsBites (Vol. 9 Num. 7). According to A. Paller: “This is a stunning development. NERC’s cyber security standards were coming to be seen as almost totally ineffective.” SANS, of course, is hyping their upcoming SCADA summit, which apparently will have a somewhat different agenda as a result of this announcement.

In particular, SANS is planning a session on “The Revolution in the CIP Standards for Control Systems Security In Electric Utilities: FERC’s new mandate and how best to navigate the changing landscape”, including this choice bit of background: “The CIP standards, under intense Congressional scrutiny in the fall of 2007, have come up short, being characterized as ‘inadequate for protecting critical national infrastructure’ according to a NIST-commissioned technical review). Now (on December 11, 2007) the FERC has changed the rules.”

Obviously, there’s some confusion here. I’m also kind of confused by the marketing strategy behind all this CIP-bashing. After all, CIP is a consensus-based industry standard, and it is the same industry that SANS is trying to attract to their summit.

Comment from Dale Peterson
Time: December 13, 2007, 10:52 am

A few comments:

- I’m not sure how Alan Paller came to those conclusions after reading the FERC document, and they were filled with such hyperbole that I decided to let them pass. You own what you write.

- Joe Weiss disagrees with my post in his aptly titled Unfettered blog (see http://www.controlglobal.com/unfettered/?p=32 ). Obviously we are not going to find common ground on this. Our approach is to reduce risk as quickly and efficiently as possible - - don’t let the perfect be the enemy of the good. And we can only go by what we see in our clients who are definitely dramatically reducing risk and improving security driven by upper level management support to meet regulatory requirements.

Maybe we are just fortunate to work with clients who get the importance of security - - and to be clear we do not offer NERC CIP compliance services so this is not a play for more consulting $$$. It does not affect our top or bottom line what standard, if any, is selected.

One last point - it is hard to judge the NERC CIP standards as a success, failure or somewhere in between because we have not gotten to compliance and audit dates. That said, it is not too early to look for how the standards and process can be improved by one or more of the suggestions in my para two of this blog entry.

Comment from Keith Stouffer
Time: December 13, 2007, 1:10 pm

Dale,

I agree with you. Throwing out the NERC CIP standards for a new standard (ISA 99, NIST SP 800-53, etc.) would be a great mistake. Asset owners not doing anything to secure their systems because it looks like the NERC CIP standards might change or be replaced is also a great mistake.

The NERC CIP standards are out there and are what are being used. If implemented, they will improve security of the bulk electric sector.

The NIST suggestion to FERC has always been to implement the NERC CIP standards and then create a planned transition to “enhanced” NERC CIP standards, with security controls added to the current standards to make them consistent with the level of security that federal organizations that own/operate bulk electric sector systems must adhere to. This has always been a suggestion to enhance the NERC CIP standards, rather than a plan to abandon them.

Comment from Jake Brodsky
Time: December 13, 2007, 3:33 pm

The fundamental argument here is whether to make “good” the victim of “better.”

The CIP standards are better than nothing. But they are flawed in a very fundamental way: CIP-002 is hardly worth downloading unless you want to see how NOT to set the scope of a security expenditures. It mostly ignores attack vectors in a global fashion, focussing instead upon whatever you deem to be a critical asset. I’m sorry, that’s not how I would conduct a survey where I work. I would analyze attack vectors and estimate the global expense of the attack versus the cost of defense against it. I’d make my choices for security that way.

The rest of the CIP standards are a good start. If someone chose to follow those parts, I wouldn’t argue.

So are the CIP standards good enough not to be made a victim of a better, perhaps newer standard? That’s not for me to say. I do not propose sitting around waiting for something better. But the way that CIP-002 reads, I can’t help thinking the exercise is an excuse to do as little as possible.

It’s better than nothing, but not by much…

Comment from Keith Stouffer
Time: December 13, 2007, 4:18 pm

Absolutely agree on the issue with CIP-002. My comments above are directed to the management, operational and technical requirements specified in CIP-003 – CIP-009.

Comment from Dale Peterson
Time: December 13, 2007, 6:34 pm

I think Keith and Jake are on the right path, and this does not have to be that difficult.

In the short run NERC/ERO provides additional guidance on the criteria for selecting Critical Cyber Assets (as FERC requested in their NOPR). The legal types would have to determine how this can be done in the framework of the Energy Act / FERC / ERO, but it would seem they could issue FAQ’s and audit guideline information without touching the standard and improve the situation. A lot of the asset owners are begging for guidance on this and other CIP requirements.

While most people focus on the Critical Cyber Assets, remember that a large number of the requirements apply to all cyber assets (critical and non-critical) within an electronic security perimeter.

At the same time the ERO begins the process of the next enhanced NERC CIP standards because the clock is ticking if you want to have a more rigorous next CIP to follow one or two years after this set hits the auditably compliant deadline.

Maybe the NIST SP800-53 document help with CIP-002 where we move from Critical / non-critical based solely on consequence to the 3×3 grid in the NIST document. There is a need for a more granular approach to risk and required protections, but it is not an unreasonable approach to identify your most critical cyber assets and protect them first.

Comment from Bryan Singer
Time: December 13, 2007, 9:47 pm

Left into the category of “a good start” I would agree that NERC CIP is ok. The challenge I see is that so many know that they are only scratching the surface. Too much is left to interpretation, is far too subjective, and up to the capabilities of the analyst. I have been behind several NERC CIP evaluations now where we uncovered significant deviation in the documents and also in how people quantify what they think their cyber risk is. I think its a good start, but people really can not afford to stop there.

Comment from Ron Southworth
Time: December 13, 2007, 10:09 pm

Hi gentlemen.

I lament this often and if I had more time I think it would be a great basis for a doctoral thesis. Below is based on the research I have been doing for a couple of years now but in this forum I have not identified sources etc. Consider it an executive summary of sorts definitely a work in progress.

Knowing you all reasonably well by your previous words and efforts I offer the following.

Business is about profit and loss. Security fundamentally is about humans and relies heavily on motivations, culture and ideology, not much money in either of these.

The technical stuff is what gets us technical types out of bed in the morning so I contend this is the “easy bit” of the puzzle.

Regulation is not going to make the utility or control system operator more profitable.

Implementing security improvements that give what Bryan Singer describes as, a ROIS will however.

My father used to say to me you can only flog the willing horse for so long before it will buck you off and trample you to death. This is a primal response worth remembering. It is still present in the modern human context although possibly more subtle to us most of the time.

The CIP standards as they sit and as proposed will result in incremental improvements over time but standards and technology are only half (probably less in reality) of the equation with security as I have already alluded to.

Keith the way forward may I respectfully suggest with the acceptance of the NIST standards you need to leverage on examples that can show the benefits of progressing from the CIP standards to the NIST standards.

There is nothing wrong with the technical aspects of the NIST standards. It is what the standard is going to be used for that makes “the sales pitch” so difficult and creates all the adversarial conflict. We need to limit conflict to the arena of dealing with threats amongst us and be directing effort and energy in co-operative efforts.

My simplistic outsider generalised perspective: The CIP standards started out with a higher degree of improvements but were amended to gain industry acceptance and adoption. It took so very long to become ratified. I can see the same if not more resistance to your efforts.

I hope that I can be proved wrong Keith but I suspect the present model, being used in the security change process in the USA as long as the process does not change itself I contend my prediction is a fairly safe wager to make.

I don’t think a 9/11 event will motivate the sort of cultural change required. Maybe the looming recession should be a wake up call but I suspect it will take the hurt of the scale of a recession to revitalise a nation that I do hold so dear, almost as dearly as my own.

I hope that this discussion however will change opinion perspective or culture but I feel sadly it will fall largely on deaf ears like so much of my earlier research of some 20 years ago.

A way forward…

A quality systems approach leveraging on the best practices identified in the standards may be an alternate process to follow.

What I find such a shame is that this philosophy is an American invention that revitalised the Japanese economy post WW2.

There are many success stories and information in the public arena. Some of the utilities that have implemented quality systems in Australia and other nations if contacted could provide in confidence evidence to support this suggestion.

Keep forging forward with your goal Keith, as it is an admirable one.

Comment from Joe Weiss
Time: December 14, 2007, 2:25 am

After testifying to the House Homeland Security Committee, I was asked what was the most egregious part of the NERC CIPs (you already have this on your blog). The answer is very simple - scope. The NERC CIPs allow utilities to exclude equipment to assess (many have a list of critical cyber assets in the 10-20 range - some have zero). If the scope were changed in CIP-002 to include all equipment that is electronically connected (which would mean the critical asset list would be in the thousands), the rest of the NERC CIPs would be acceptable. In fact, it wouldn’t be that different from NIST.

The NERC proposed phased transition is years away. If this is a real problem, we cannot wait.

Joe

Comment from Dale Peterson
Time: December 14, 2007, 9:17 am

Joe - All bulk electric system cyber assets are included in NERC CIP-002, see detour at the bottom of comment for distribution. The protection requirements are determined by a consequence based assessment which you know well but for others:

1) identify critical assets
2) identify critical cyber assets in critical asset (including the routable/IP or dial-up criteria)
3) apply CIP-003 to 009 protection to critical cyber assets
4) apply a subset of CIP-003 to 009 protection to all cyber assets inside an electronic security perimeter that has one or more critical cyber assets

As mentioned by many in this thread and elsewhere including the NOPR there is a need for more guidance from the ERO on the consequence assessment for CIP-002 as it exists today. Even with that guidance, this first pass is crude and designed to identify and protect cyber assets with an IP address that have a significant consequence if successfully attacked. A reasonable risk management and risk reduction approach, although it took about two years too long and I won’t repeat my comments on the danger of further confusion or delay from the blog entry.

The next gen CIP should move from a pure consequence to a risk based model with more security requirement levels than critical or nothing. Here you can look at a document like NIST SP800-53 that has three control levels: high, moderate and low and a set of security requirements for each level. Perhaps there should be an implementation schedule based on control levels so resources are placed on continuing to protect the highest risks first.

Joe your number of thousands is intriguing in two ways.

First - I don’t believe you mean all 1,000’s of cyber assets should have the same security requirements. This is almost unheard of in risk management or any security practice - - including NIST SP800-53. My guess is you mean some protection should be applied to all cyber assets.

Second - The 1000’s number implies to me that you believe that the exclusion of cyber assets that do not use a routable protocol (think IP) and are not dial-up accessible is wrong. I say that because there are many if not most transmission and generation systems that have Ethernet and IP in the control center and little elsewhere. Transmission early adopters may have some PLC’s, RTU’s, IED’s in the field with Ethernet cards, but they are typically small in number or little Ethernet islands that are not routable over any WAN. This will change in the future and NERC CIP as written has those planning projects thinking twice due the added cost of CIP compliance to a planned upgrade - - which is a whole other post.

I’m 100% on board with the focus on systems running IP. These are the systems that are susceptible to worms and easily attacked using the plethora of free hacking tools available on the Internet. Typical routing, without ACL’s or firewall rulesets, allows an attacker to go anywhere in the routed network.

Maybe in a future generation, I would argue not even in the 2nd gen, of the CIP standards security requirements for a special serial interface, but it would be a huge misallocation of resources to focus on the cyber risk to serial devices now.

Detour - I know you have an issue with distribution being excluded but one could argue either the jurisdiction issue or a reasonable case that loss of a distribution system is of less consequence than bulk electric assets. Distribution could be used as a source of attack, but the CIP-003 to CIP-009 are designed to protect this just as they protect communication to any other network outside an electronic security perimeter. It would not be a problem to include distribution, but my guess is the consequence based assessment would drop out the distribution assets.

Comment from Trey
Time: December 14, 2007, 11:43 am

What is the ‘Aurora vulnerability is THE problem’? i have not seen this anywhere. What is the issue and where is it researched?

Comment from Ralph Langner
Time: December 14, 2007, 12:21 pm

Dale, don’t be in a hurry to kick everything over board that doesn’t use IP.

First, there are several legacy products connected to Ethernet that use OSI-style protocols. While not IP and not routable, they may still be affected by DoS attacks and implementation bugs (funny Ethernet packets freeze them). By the way, some of the newer realtime Ethernet variants, such as EtherCAT and PROFINET IO, also are not IP in the strict sense (they require dedicated NICs because they don’t use CSMA/CD to achieve realtime).

Second, controllers and SCADA boxes not connected to the network account for some of the same risks as the networked stuff. An attacker, given the physical access, can simply walk in and steal or destroy the box. A harddisk may crash and it turns out that the admin didn’t care for a decent backup+restore regime. A standalone controller with no redundancy may fail and it turns out that a supportive process is affected that brings a complete plant to a halt. Been there, seen that, got that t-shirt. We always try to look at any asset, networked or not, as long as there is something in it that could remotely be identified as “cyber”. As it turned out, this worked quite well for our clients.

Comment from Elec User
Time: January 17, 2008, 5:10 pm

Why is no one pressing the obvious question? The ES ISAC order was classified as For Official Use Only (FOUO). Our company went to great lengths to protect information related to the vulnerability and the actions we took to secure our systems. Then, with 1 day notice, DHS provides the information to CNN! Why would the utility industry feel secure about giving Critical Energy Infrastructure Information (CEII) to NERC at this point in time? How does NERC propose to collect and secure this information? NERC has no secure transport mechanism for protected information.

Comment from Trey
Time: July 11, 2008, 5:11 pm

Does the ESIAC Advisory cover all plants or just ‘Critical Assets’?