665 SCADA Bugs Presentation from DerbyCon

DerbyConTerry McCorkle’s presentation at DerbyCon, 100 Bugs in 100 Days: An Analysis of ICS (SCADA) Software is available online. He did this research in his spare time with Billy Rios, and it is informative technically and culturally.

The research focused on freely downloadable HMI’s from around the world. They found 380 HMI applications. They only had time to look at 76 of the 380 applications.

The results were staggering, but then again not too surprising for this type of software in the ICS space. McCorkle and Rios found 665 bugs that caused an application to crash. 75 of these bugs led to exploits, and McCorkle pointed out that they had so much information to go through that they focused on the easy exploits. There are likely more exploits in the 665 bugs.

A variety of fuzzing tools were used including Commraider, Sulley, FileFuzz and a new tool developed by the researchers called Blasty.py. Fuzzing found 590 or almost 90% of the bugs. They further broke the bugs down by:

  • 360 resulted from file fuzzing
  • 204 were ActiveX vulns
  • 90 were web vulns

“Most of the bugs were straight out of the 90′s” according to McCorkle. Skip to minute 28 if you want to see some examples.

Cultural Impact

It is striking how much the DerbyCon audience is laughing at ICS security, albeit the beer was flowing. The number and simplicity of the exploits shown as demonstrations deserved the laughter. Who knows what the result will be of the growing knowledge and consensus of ICS security being laughable in the all-hat-color hacking community.

On one hand the research and interest in the area could be diminished if it is viewed as a minor technical accomplishment to find a SCADA exploit. Or it could lead the IT security community to move beyond HMI’s, better understand ICS and create a large amount of nasty exploit code. Again after the lost decade, it is unclear what would be better for the ICS community.

ICS-CERT

Terry gives ICS-CERT high marks and a lot of thanks for coordinating the vulnerabilities Billy and he found. The researchers did not want to deal with all the vendor issues and just turned the whole mess over to ICS-CERT. This matched Digital Bond’s experience that ICS-CERT does a good job coordinating disclosures when the researcher does not want to deal with the issue.

This ties into part of my upcoming presentation at ICSJWG on vulnerability disclosure. Do we really want the smart guys at INL/ICS-CERT spending time coordinating large numbers of HMI freeware vulns? They need to focus on the vulns and other security issues that could have a major impact on the critical infrastructure and pass on some of this work.

Viewing Recommendations

Most loyal blog readers can skip the first 13 minutes of the 40 minute presentation as it is an explanation of ICS as background for the IT security audience. He is a bit off on the level of segration or separation between the typical ICS and corporate network, but other than that it is an adequate introduction.

Minutes 17 – 22 to is when Terry goes over the statistics mentioned above.

Minute 22 is full of thanks and good words for ICS-CERT.

Minute 25 is interesting to see what a smart guy in IT security knows about SCADA and DCS. He shows an attack path with a user on the corporate network going through a firewall, which he indicated earlier is not common, to reach an HMI. This is not a common scenario, at least in critical infrastructure ICS. Corporate access is typically to some data that is pushed out to a DMZ. Sometimes a view only HMI application on a terminal server could be found in the DMZ. There are other items that could be pointed out as inaccurate. It is not an IT v. Ops comment, but a view as to what level top-notch IT security researchers understand about critical infrastructure control systems.

This doesn’t negate the scenario of compromising a box on the corporate network that is allowed through the corporate/ICS firewall. It would just likely happen in a different set of device-to-device exploits and pivots.

The 28 minute mark is where McCorkle starts showing some of the bugs/vulns if you want some technical meat.

7 comments to 665 SCADA Bugs Presentation from DerbyCon

  • bryan owen

    “Most of the bugs were straight out of the 90′s” commentaryhighlights an important gap.

    Asset owners can’t patch out of fundamental design issues.

    We now know more inherently secure ICS applications are needed.

    I believe the gap will last for a few generations as legacy tech expires.

    In the meantime, the ICS community will wrestle with incident response – sometimes facing the uncomfortable issue related to technical end of life.

  • Shawn Merdinger

    Hi dale,

    You raise a great point about the “Do we really want the smart guys at INL/ICS-CERT spending time coordinating large numbers of HMI freeware vulns?”

    One thing I’d add to that line of questioning is how good/bad the bugs coming into CERTs these days actually are — that is, in light of the various bug-bounty options of every flavor (including black market), how severe are the bugs CERT folks typically deal with these days.

    And if it’s possible that someday CERTs won’t bother to handle vendors and disclosure of some bugs, then who will? If no independent resource is available for researchers, what will they do? Bury and forget the bug? Or disclose it without any vendor notice? Or sell it to whomever for whatever? Who knows?

    Cheers,
    –scm

  • Michael Toecker

    Don’t forget, according to ICS-CERT fundamental design issues aren’t problems to be reported. It’s even addressed at the end of the DerbyCon talk.

    If this design issue isn’t known to the asset owner, then how can you prevent it from being used maliciously? This is security by obscurity, and it doesn’t work. Regardless of whether it’s a bug, or a feature, it needs to be reported so that vendors know to fix it, and owners know to ask about it.

    If a fundamental design issue (like a hardcoded password) isn’t disclosed, we can’t build tools to prevent, detect, respond.

    Mike

  • Michael

    Dale, your reviews often point out inconsistencies between IT Security’s view of control system networks and what is actually happening. What are some resources that accurately depict real word control system networks? What would you say are the common misperceptions from IT?

  • Dale G Peterson

    Michael,

    Just to be clear again, I’m not making the IT v. ops argument. The contributions from the IT security community are valuable regardless of how much actual hands on ICS experience the research has.

    It is little things that indicate that sometimes the presenter hasn’t actually been at a number of plants or control centers. For example, in a large ICS the HMI rarely communicate directly with the PLC/RTU/controllers. You would not want multiple HMI’s polling for status, and a large ICS might have more than ten HMI and a bunch of engineering workstations. So they typically have a server that does this and then all the HMI communicate with the server for monitoring and control. Sometimes this SCADA/DCS server talks to a comms server, which is usually a very ugly, proprietary, semi-custom mess of software that takes the field comms and deals with all the different communication paths … dial-up, microwave, vsat, leased lines, … Luckily comms servers are going away with the move to IP in SCADA.

    Another misconception is with the corporate / ICS perimeter and what typically is allowed through that perimeter for an owner/operator who has passed SCADA 101.

    In terms of learning, it would be books on SCADA and DCS that don’t cover security. Or some of the large vendors have excellent white papers showing their architecture. That said, there is nothing better than wandering around a plant or visiting field sites. And this is one of my favorite things about this ICS security space.

    Dale

  • Terry McCorkle

    Dale,

    Thank you for the write up. It was a very fun and interesting project. The point we really wanted to get across is that people need to be aware that these systems have bugs, whether they are design or coding bugs, and that they need to be treated as such.

    The examples during the presentation were there to help people get some idea of the different components that make up an ICS environment. I do agree that they were very simplistic and do not model what you often find in the field, but they will help someone that is not familiar with these systems to get an idea of how they operate.

    The actual designs you find in the field vary greatly based on the type and function of the system, and of the systems we’ve seen, there doesn’t really seem to a unified approach to segmenting them. The main reason we went after the HMI’s is because they usually provide an avenue to the other systems such as comm servers.

    The biggest hope I have is that people look at their systems and determine what their risk is. In addition we wanted to encourage security researchers to look at these systems and recognize that it is an area of research that takes little investment financially while providing a lot of opportunity to the researcher.

    Thanks again

    -Terry

  • Plantae

    Smart guys at INL/ICS-CERT…

    ehhhhhhhhhh “OLE for Personal Computers?”
    http://www.us-cert.gov/control_systems/pdf/ICSA-11-279-03.pdf

Leave a Reply