- ISA SP99 Part 2 is out for ballot a second time with comments after addressing the first ballot comments.
- An opportunity for control system security research funding has opened in Europe. The EU will spend at least 20 million euro on this Critical Infrastructure Protection Research. Good news for our European researcher friends.
- Ron Gula from Tenable Network Security, the vendor that develops Nessus, blogged on our .audit file for OPC servers.
- Matt Franz has an interesting idea and project named PeerTab to use P2P to share threat information.
- Bryan Singer posted his review of Wurldtech’s Achilles QA Test Tool and Certification.
- ABB’s AC 800M controller has achieved Mu Security MUSIC certification. This is the second product that has achieved this certification.
- I’m sure you have heard about the DHS video and piece that CNN did. The full video is effective for gaining attention to various government officials and C-level executive. This video probably is no new news for regular blog readers. Two minor tidbits:”Can you say right now that this vulnerability has been eliminated” says Jean Meserve. “No. I can’t say it has been eliminated, but I can say a lot of risk has been taken off the table” answers Robert Jamison (DHS). Really?The focus on the meager Federal budget as the reason these vulnerabilities exist seems to be overplayed a bit both on CNN and other reporters chasing the story. Sure the government can do more to provide tools and guidance if the budget was larger, but most control systems are run by non-governmental organizations. They are the ones that need to take this seriously and allocate the money and other resources.
Archives for September 2007
The 90’s were filled with hope on the IT / SCADA front. Asset owners could save money by just moving to the Windows platform. Put web servers in most systems so the browser is the easy to use, universal GUI. Connect everything so information can be used throughout the organization and control can occur wherever the people are. Integrate SCADA and DCS applications with Active Directory for single sign-on and easier user management.
How many times have you thought life would be a lot easier if none of the above had happened? I have heard it numerous times in conferences and from our asset owner clients. This is often followed by an admission that the community has no one to blame but themselves because asset owners asked for most of this and vendors willingly obliged.
The exuberant adoption of IT in the 90’s had at least two major flaws.
1 . Little or no attention was paid to security
This is not surprising and perhaps can be forgiven when considering the level of threat at the time and the limited security efforts even in most organizations. IT security grew up in the 90’s.
The good news is wireless is not making this mistake. Security is a primary consideration in the development of protocols and systems. We can argue if the measures are adequate, what new threats are introduced and their impact on risk, if the standards and implementations are adequately vetted, and other questions. That said, it is clear the community has learned that security must be considered when deploying new technologies.
2. A gross underestimation of the resources required to appropriately maintain and manage the IT hardware and software.
Sure Windows is cheap, but it requires active anti-virus and patch management. There is work to develop an maintain a hardened configuration. Most asset owners thought they could install the Windows HMI and not touch it for at least five years like their previous HMI. Releasing this was not deploy and forget technology, many migrated to having an engineer have the added task of being an IT System Administrator which is not what the wanted or was trained to do.
More egregious examples are domain controllers and databases, and the vendors deserve a bit more blame for these examples. Being a domain administrator is an important job requiring special training. Look at the IT Departments. They have teams of domain administrators who spend time learning and staying current with the skills required for this role. The idea that you can drop a domain controller into a SCADA network and not have at least two skilled domain administrators is asking for trouble. Sure you may get by, but you may also get by not understanding and performing maintenance on your physical assets in your plant. Vendors’ sales staffs new this but often avoided raising this issue because it was an impediment to the sale. Instead they deployed the domain controller, often in the default config, got the system working, and walked away leaving a ticking security time bomb.
The community needs to avoid falling into this second trap again with wireless LAN’s. Lured by cost savings, ease of use, and the promise that security is addressed, we may be avoiding thinking of the lifecycle consequences of this new technology.
- What will you do when a new zero-day exploit is released for the wireless system you have selected? Turn it off until patched? Have security guards patrol all areas the signal reaches? ???
- Do you have a workable plan to patch these systems without causing an unacceptable outage? Have you build enough redundancy into the system?
- How will you verify that all your wireless access points are configured securely on an ongoing basis? Is there a centralized management application? Do you have adequate trained staff to manage this wireless network?
- How will you monitor your wireless access points – – from a NERC CIP standpoint are not these wireless access points a gate at the electronic security perimeter?
- What other lifecycle issues need to be addressed?
IT advances in the 90’s and this decade have significant benefits for control systems, but we need to make sure we consider, address and accept all the costs before falling into the trap again. Our clients can attest that I’m not a Luddite. Quite the contrary, I’m a technology enthusiast. But there is a reason why your IT Department is so large.]]>
Wireless for control systems has been a hot topic for a few years now, and recently we have been treated to the efforts of different groups, i.e. ISA 100 and WirelessHart, to develop a standard that includes security. Which leads to the question how does the use of wireless increase the risk to a control system?
Of course, many loyal blog readers would certainly point out that wireless WAN communications have been used for years. So when people use the shorthand term of wireless, they are typically talking about wireless LAN or MAN protocols, many of which are based on protocols commonly used in IT networks.
Sometimes it pays to go back to basics. Risk is a function of consequence, vulnerability and threat.
Much of the focus when discussing wireless is on the vulnerability factor. This is reasonable because wireless LAN protocols have a poor security history, and the technical security controls warrant a serious peer review even though that has failed in the past.
Threat is the factor that will cause the greatest increase in risk of wireless over wired networks. In wired systems a typical attacker needs to gain physical access to a port or at least a cable to launch an attack. With wireless, an attacker only needs access to the wireless signal to launch an attack. This increases the potential population of threat agents, to perhaps anyone who can get to the parking lot.
The argument can be made that the vulnerability factor can be reduced to such a low number that the risk is acceptable regardless of the threat and consequence. This same argument can be made for using the Internet for control system WAN communication. However, most people in the community recoil at the idea of using the Internet.
The key is to have management understand the risk and weigh this against the costs of not accepting the risk and benefits of accepting the risk. As long as the right level of management makes an informed decision on risk acceptance, wireless is fine.
Unfortunately, what we commonly run into, and it happened again with a client on Friday with an issue unrelated to wireless, is a focus almost solely on decreasing vulnerabilities without a focus on decreasing threat. One control is piled upon another. Two-factor authentication plus encryption plus IPS plus multiple firewall plus … It still is to great of a risk – – well what other security can we deploy?
And we are going to deploy these security controls perfectly and there are no zero-days in these security products. What are we going to do if a new vulnerability is found in the wireless protocol? Or maybe the protocol is fine, but a vendor implementation is vulnerable. It has been known to happen. What is the response? Pull out the wireless until it can be patched?
Consider taking a step back and determining are there ways of reducing potential threat agents, such as someone sitting in the parking lot. Consider if the benefits to wireless or other approaches that increase threat agents are truly required. What would be the hard cost and soft cost in using solutions that do not increase threat agents?
Just to be clear, this is not a blog saying do not use wireless. There are situations where the benefits would be worth the increased threat and an increased risk. What I would argue is a small cost savings or convenience might not be worth it and management needs to consider this.
This is not unique to wireless. Another area where this is common is routine access from the corporate network. It is too inconvenient to place people in a physically secure area or make them get up and walk to a secured HMI, so access is allowed from the corporate network, albeit with security controls, and the threat is increased.
To summarize this semi-rant, more security hardware and software is not always the answer. Sometimes the answer is to not expose yourself to the threat. This seems to be the knee-jerk answer to Internet use for control systems no matter how much security is in place.
We mentioned AppID’s in our introduction of the OPC Security .audit files for use in compliance testing with the Nessus Vulnerability Scanner.
While it is not difficult to find the AppID for your OPC server, we have started a SCADApedia page with the AppID’s to help you out. A lot of this information came from Lluis Mora and the team from Neutralbit. Subscribers can add new AppID’s directly to the SCADApedia page.
- ISA decided to go forward with a Security Division. Read the announcement and listen Bryan Singer talk about this at the Weiss Event Interview podcast (at 29:25).
- US-CERT has added a Control System Security topic area to their Build Security In efforts. There is a call for authors and reviewers. It will be interesting to see what type of submissions they get, and if there are limitations or censorship on what the submissions.
- Tipping Points DVLabs disclosed the Modbus TCP vulnerability that Ganesh had been discussing at Defcon and other events. The vulnerability is “a controllable heap corruption can occur which may result in execution of arbitrary code” on the Automation Solutions Modbus TCP Slave ActiveX Control. There is a patch available from the vendor.
Part 3 of the recently released OPC Security whitepaper series provided step by step instructions for implementing the available security measures for OPC clients and servers. It is complex, and we wondered if there was a simple way to audit OPC servers compliance with Part 3. We still are wondering, but we have a partial answer.
The Tenable’s Nessus vulnerability scanner has a Policy Compliance plugin family that is available for Tenable Direct Feed subscribers. Compliance is becoming a huge issue. In fact, many of my friends with SEM vendors say that accumulating log records for compliance is driving sales of SEM products more than the security correlation and analysis features. The Policy Compliance plugins allow you to compare registry, file and other settings on a host to a predefined policy. Tenable promotes these plugins for compliance with SOX, GLBA, FISMA and other regulations.
The good news is anyone can create their own audit template, in the form of a .audit file, and use Nessus to compare a host’s configuration against the template. So we did that for the OPC security recommendations in Part 3. We have developed .audit files for Windows 2000 Server, 2003 Server, and XP as well as a document explaining the audit checks and how to modify the .audit files. These are now available for Digital Bond subscribers.
- OPC Server Audit Document
- OPC Server .audit file for Windows 2000
- OPC Server .audit file for Windows 2003
- OPC Server .audit file for XP
At the beginning of this blog, I mentioned this tool was only a partial answer. It is very effective, but some of the compliance checks require customization for different OPC servers and the DCOM auditing capabilities of Nessus are not as full featured as we would like.
A simple customization example is the check on the services running on the OPC server. We performed a default install of an operating system and OPC server and eliminated all unnecessary services. Nessus will check that only those services are running on the OPC server. Your install or standard build may have additional authorized services, and these will need to be added to the .audit file – – or you can simply ignore the findings in the Nessus results related to that service.
Another simple customization example is auditing the opc accounts. The whitepaper recommends creating and using an opcadmin and opcuser account. If you choose different account names that section of the .audit file will need to be modified.
A more complex customization is related to auditing the DCOM permissions, which are one of the most critical security settings in OPC servers. The OPC server Application ID must be added to the .audit file for this part of the audit to work. It is unique to each OPC Server version. The OPC Audit Tool document describes where the Application ID is found and where to modify it in the .audit file.
This experience with Nessus Policy Compliance has been very interesting, and we see a lot of places it would be helpful. In fact, it is slotted into some of our research projects, and we are creating .audit files for HMI and other systems for our clients. OPC is not the best example of this audit feature, but try it out and let us know what you think.
When iccpsic was released to vetted subscribers, Matt Franz reminded me that other systems, such as VoIP, use part of the utility stack fuzzed by iccpsic. Siemens PLC’s use the portion of the stack that is fuzzed by iccpsic.
After my last post, I thought it was time for some good news. Ralph Langner of Langner Communications ran iccpsic against a Siemens PLC and said it continued to operate properly.
Frustration building . . . must keep civil tone . . . another silent fix in widely used control system application passes by our doorway . . .
This site has had a running series of blog entries on vulnerability disclosure including discussions on the dangers of the “silent fix”. A silent fix is when a vendor is aware of the vulnerability, has actually addressed or removed the vulnerability, but has chosen not to make any of their customers aware of the vulnerability or fix.
This is a huge problem in control systems where asset owners rarely implement upgrades, even free upgrades, unless there is no choice or an extremely compelling reason. By going with the silent fix, the vendor does not provide the asset owner with the information they need to make an informed upgrade decision. The asset owner may say I don’t need those new features and pass on the upgrade, not realizing they have a latent, easily exploited vulnerability.
I guess an argument can be made for a silent fix if it involves some new and novel attack never seen in the wild. This has not been the case in control system vulnerabilities to date. The vulnerabilities to date have involved rather simple attack methodologies taken from the IT world and applied to control systems. This is just what you would expect an attacker to do.
The silent fix may be due to the stigma of having a vulnerability. We need to get over that. Software has bugs and some of those bugs result in vulnerabilities. Hopefully we can move to a mode where vendors are evaluated on two factors:
1) There integration of security into the development lifecycle. To date this is an area where control system vendors are woefully behind.
2) The vendors response to identified vulnerabilities.
- Honeywell is first out of the box announcing founding membership in ISA’s Security Compliance Institute (SCI), background on SCI is in this blog entry. The target was 15 founding members by September 1. ISA has not announced the founding members yet.
- Emerson announced a marketing “collaboration” with Cisco wireless solutions. This type of announcement, whether it be Cisco with Rockwell, Symantec with Areva or Symantec with Telvent, typically result in some whitepapers, which are of some value, and sales collateral on how to use the products together. Nothing that exciting and Cisco did not even bother to issue a press release.
- I missed this article in June’s Automation World that describes an Industrial Defender red team engagement on a control system in entertaining detail. We tend towards informed assessments rather than red team exercises, but how can you beat camouflage garb and night vision gear if you are an IT type?
- Walt Boyes has been covering the SP100 v. WirelessHART contest and conflict in multiple posts in his blog the past two weeks.
- One more day to get your S4 abstracts in for consideration for S4 2008 in beautiful Miami Beach next January.
It was a very long time in the works, and I have to give Eric Byres a lot of credit for his diligence in getting reviews and incorporating feedback from a cast of thousands for Part III. The final part of the OPC Security Whitepaper Series written by Byres Research, Digital Bond and BCIT is now available on our site to subscribers and likely will be on Eric’s site soon as well.
As a reminder, Part I provided an overview of OPC and included interesting survey results on how OPC is being used. Part II described risks and vulnerabilities in OPC clients and servers. Part III completes the picture by provide specific and detailed guidance on how to harden OPC clients and servers.
Part III is 54 pages long, which represents the complexity of the OPC / DCOM security problem. Some of the possible security measures, such as configuring the Windows firewall, are quite frankly so complicated we would be hard pressed to recommend them, but they are a technical control that is available to OPC clients and servers.
Other portions, such as guidance on setting DCOM authentication and permissions to limit access to OPC servers as well as the RPC hardening recommendations to make OPC more firewall friendly, we consider essential. The good news is there finally is a step by step guide to a thorough set of security hardening recommendations for OPC clients and servers.
If the paper is not enough – – stay tuned for next week when we will release a tool to subscribers that will allow you to audit your OPC servers to the guidance provided in Part III.