More Granularity on Security Patching Strategy

Pumpkin Patch (image by wildcat_dunny)Eric Byres recently published a 4-part series on security patching for ICS. While I have a few minor disagreements with it and the emphasis/approach, it’s a good primer and important for those who are new to the ICS security space.

Owner/operators are struggling with security patching. It’s hard, even for IT organizations patching the corporate network. One of the keys to success is prioritizing the security patches and applying the high risk vulnerabilities in an expedited manner. Eric correctly identifies “┬áthe criticality of the system being patched and the criticality of the patch” as a factor in prioritization. We recommend including two other factors.

Think of the risk equation. It includes likelihood of attack, vulnerability and consequence (or impact).

  • Likelihood of attack: owner/operators should prioritize security patches related to vulnerabilities that can be exploited from an external network. For example a web service or other port accessible from the corporate network on a PI Server, GE Proficy Server or other historian should be prioritized. Database replication between security zones is another example.

Look what is allowed through your security perimeter. If the attack is allowed through your security perimeter this would almost always jump to your highest priority. In fact you should consider modifying the firewall rule or implementing an IPS rule to block this communication or attack until the security patch can be applied. Of course, good security practice would not allow any inbound access from an external network, but most owner/operators are not there yet.

Another important question is whether it is a client side or server side attack. The recent Mitsubishi MX Overflow Vulnerability is a great example. It was an easy to exploit vulnerability on an HMI / EWS software that is widely used in important systems. It would rate high on impact if compromised, but the likelihood of compromise is low. It is a client side vulnerability that requires the HMI/EWS to visit a compromised web site. If your HMI are allowed to access Internet, corporate and external web sites you have much bigger issues than this patch.

As an aside – this is a great example of how bad the ICS-CERT alerts are. I have access to the Critical Intelligence Advisories, and they provide a helpful chart, see below. There is accompanying text that clearly describes why the rating was selected for each category. This particular Critical Intelligence Advisory also noted that the MX Component Active X library was in some versions of CitectSCADA and CitectFacilities. I would rate the Critical Intelligence product consistently 3x better than the ICS-CERT product based on clarity and depth of information.

 

Critical Intelligence

Source: Critical Intelligence

  • Vulnerability: This is where we see a lot of errors in judgement, both in owner/operators and in CERT’s. The measure should not be based solely on the vulnerability that requires patching. The measure should be the how this new vulnerability adds to the vulnerability of the component or system. Most ICS protocols and many of our components are insecure by design. A vulnerability is not required to compromise the integrity or availability of the device. The accretion to the component or system risk caused by a new vulnerability, even a simple, remotely exploitable vulnerability, is minor in an insecure by design device.

PLC vendors that are patting themselves on the back for dealing with web server vulns when the PLC’s allow unauthenticated firmware and logic uploads are not really reducing the vulnerability component their device adds to risk.

  • Consequence: Or impact. This factor in the risk equation is usually the focus and was covered in Eric’s blog series. It is an important factor.

Security Patching As A Maintenance Activity

There should be a security patching interval for even low risk vulnerabilities as a regular maintenance activity. The practice of letting software get years or even a decade out of date needs to end. You need to do this to be running a supported and supportable system, and it will also make those high priority updates lower risk and possible.

As the ICS user community expects more from ICS vendors in terms of delivering and supporting secure systems, there is a parallel expectation that ICS users need to keep up to date and refresh their systems more frequently. An ICS vendor may only be able to provide the security support for two prior versions.

Minimizing the Attack Surface

If you are tired of security patching, one way to reduce the burden is to minimize your attack surface. Remove all unnecessary software and close unnecessary listening ports. Implement a least privilege firewall ruleset to reduce the likelihood of attack component of the risk equation.

Go the next step and modify your business processes to remove inbound access to the ICS from the corporate network. I can count on one hand the number of owner/operator systems that had a minimized the attack surface when we first visited over the last decade. This includes systems covered by NERC CIP. Documenting what is running is not the same as minimizing the attack surface.

2 comments to More Granularity on Security Patching Strategy

  • Excellent last point.

    Not only is patching tiresome – it can be costly without automated testing and deployment. Some times costs are hidden in service contracts but no less real.

    For data server assets based on Windows, it is better to start with Server Core Only add roles that are required. This approach also reduces the hardening effort.

    Core not supported == SDL failure to observe ASA during design.

  • Bob Huber

    Great points. One thing that caught my attention this week, was the fact that Microsoft actually redacted a security patch they released – http://www.computerworld.com/s/article/9238371/Microsoft_urges_Windows_7_users_to_uninstall_Blue_Screen_of_Death_patch
    This doesn’t undermine the fact that you need to patch. First, this doesn’t happen often. Second, it just means that you really need to test. Testing doesn’t just mean load a patch and make sure your system comes back up, it means you actually have a test plan to run through the operation of the device and ensure all runs well. This can even include, letting the patches “take” for a week or more under real world, or load conditions. My guess is, most organization don’t have a test harness to simulate load, events and real world operations, and of course the gorilla test.

    When I worked for a Fortune 50 bank, our patch cycle was defined based on severity. An example would be: Internet Facing System, Highest Risk Vulnerability, 24 Hours to patch. Same patch, internal system, 72 Hours. You get the idea.

    I am also aware of a few ICS vendors that have essentially contracted out the testing of patches against their ICS systems. Their contractor in essence, tests and documents the MS patches against every component of their system. Doubt many do that, but it’s a sign that at least some of them get it.

Leave a Reply