We Stand Upon the Shoulders of Giants

giantThere has been a common theme in cyber security to have great discoveries follow on the heels of new tools. This situation exists in the sciences in general, and has been described by Isaac Newton, Stephen Hawking, and others as “standing on the shoulders of giants”, where the giants are those who have come before, often cutting their way through dense jungle, so that we can pick up the machete ourselves and continue, potentially making great discoveries in the process.

What’s even better is when a new tool comes along that opens up a new facet of cyber security to inspection and analysis. These tools transform complex concepts and science into commonplace analysis, allowing access to specialized science and technology that we had limited access to before.

A recent example is the Shodan search engine, a simple concept and large effort that allows researchers and others to search for internet facing systems, for whatever purpose they desire. Research using the Shodan search engine has uncovered unsecured SCADA, hundreds of thousands of vulnerable devices, potential misconfiguration, and other insecure situations. Without the Shodan engine, researchers that wished to look for these devices had to develop their own scanning and analysis infrastructure, not a simple task.

But, this is the past…  What future giants will help shape how cyber security, and ICS, research will move? And what tools should we be developing to move the needle on ICS Security progress?

A commonly shared opinion is that the opening of Radio spectrum to easy analysis will lead to new research on radio based devices, both in ICS and regular cyber security. With a large chunk of automation being supported by radio in the 900 MHz spectrum, the RFCat is a must. I’ve written up my review of my experience with RFCat training here, and @at1as does many training events during the year as well.

Second, bug hunting is often depicted in media as an ‘individual sport’ with an image of some researcher bent over a keyboard, hunting and pecking. Not only is this view false, it’s very damaging from a sustainability perspective; fundamentally, the vendors out there that don’t do much security testing think that decent testing requires lots of manpower. The real pros, like Billy Rios and Terry McCorkle, have an automated infrastructure where software goes in, is processed, and the results are easily analyzed after-the-fact for the really nasty issues. They find more bugs while drinking beer and playing with their kids than any hunt and peck search.

Last, we all know most automation devices are inherently insecure, with protocols that do not provide integrity, access control, or authentication to ensure commands issued are coming from appropriate sources.  So, we can make a finding like the one at the right, but what have we done to demonstrate this vulnerability? Well, we have some plugins with the Metasploit project for SCADA that can demonstrate this, much better than a simple one-liner in a report. (MTEdit: Uh, no production demonstrations, got it?)

So, my question to those who read: What tool should be developed for ICS Security that currently does not exist? What problem needs to be solved in a group manner before individual research can continue?

title image by Gage Skidmore

AGA’s McGurdy Says No To Regulation

ICS Legislation

Dave McGurdy, President & CEO of the American Gas Association (AGA), will testify before the US House Committee on Energy and Commerce. He has published his testimony (ht: patrick coyle). After singing the praises of their industries efforts on ICS security and the current public/private partnership, Mr. McGurdy takes potential US Government regulation head on:

In the recent past, concerns over increasing cyberattacks on critical infrastructure have led to legislative efforts to create a set of top-down cybersecurity regulations. AGA remains concerned that prescriptive cybersecurity regulations will have little practical impact on cybersecurity and, in fact, will hinder implementation of robust cybersecurity programs.

First and foremost, prescriptive cybersecurity regulations would fundamentally transform the productive cybersecurity relationship natural gas utilities have with the TSA Pipeline Security Division from a successful partnership to a more standard regulator-regulated mode, forcing companies to focus more resources on compliance activities than on cybersecurity itself. Also, from a practical perspective, it is unlikely that any set of cybersecurity regulations will be dynamic enough to help companies fight constantly changing and increasingly sophisticated threats.

I sympathize with AGA and other organizations trying to avoid regulation. NERC CIP actually harmed the cyber security posture of companies that were actively trying and succeeding in improving their ICS security. The gas sector, broadly viewed, has been one of the most proactive sectors, along with petrochem, in addressing ICS cyber security.

AGA membership is primarily natural gas distribution companies, but it also includes transmission companies, storage, and others involved in the industry. Some of these members are Digital Bond clients, and some have been actively working on SCADA security for over five years and are making great progress. Regulation would likely do little to improve their security posture.

The problem is the majority still have very poor SCADA security programs, and just like the electric sector will not spend money or resources on this until forced to by regulation or after something very bad happens. NERC CIP has at least forced some utilities to harden their security perimeters, employ user management controls, etc.

Two other points:

  1. I’m not sure if fracking falls under AGA, but those plant and field systems are being put up so fast because every day delay can be measured in dollars. Cyber security, while desired, is often pushed aside to get the operation running.
  2. Insecure by design – even those owner/operators who care and are actively working on security are stuck with a keep the bad guys out philosophy given the protocols and PLC/RTU available today. The defense-in-depth is really defense-in-security-perimeters.

I’m increasingly sympathetic to the contention that government or industry regulation is required. Developing effective and efficient regulation or certification programs is difficult, and I don’t have the answer, or strongly felt concepts, on how to do it. However, the lack of discussion and experimentation on what type of regulation or certification would be effective is disappointing.

Image by earthwatch

  • Critical Intelligence

S4x13 Video: Recovering Siemens S7 Level 3 Passwords

Eric Johansson from Management Doctors in Sweden brought over a great Siemens S7 demo rack to demonstrate some attacks on the Siemens S7 PLC family.

At 7:45 Erik shows the Level 3 (highest security that controls read/write access) can be recovered by capturing the packets sent to the PLC. This was actually known and included in Siemens documentation (July 2011) as a potential weakness — with advice to not let the bad guys on your network.

At 14:13 Erik shows a denial of service attack on the Siemens equipment he brought to S4x13. It requires a power cycle and administrator action to put the PLC back into Run mode.

The Q&A at 18:19 is quite interesting.

Arne Vidstrom of FOI was part of the team that did this work.

Friday News & Notes

ICS Security NewsOdd and troubling week.

DHS Secretary Napolitano announced Enhanced Cybersecurity Services — the US Government will share information on 0days and threats via a paid service offered by private government contractors like AT&T, Raytheon and Northrup Grumman. This would even include 0days purchased from researchers. Does this make or break the 0day market? How does this compare to a bug bounty? this is so odd it’s hard to even come up with a cogent argument for or against your tax dollars at work.

The US NIST published a document analyzing the request for information (RFI) responses to the upcoming cybersecurity framework. Respondents think it should be flexible, global, risk-based and leverage existing standards. Ok …

NIST issued Revision 1 of SP800-82 Guide to ICS Security. More importantly they announced an effort for a major update of this document to Revision 2 in the next year.

The NY Times and most other major media vaguely reported on cyber attacks on energy sector companies with the goal of sabotage or control of the ICS. The information is based on a non-public bulletin from ICS-CERT.

Anonymous announced Operation Petrol will start on June 20th against “greedy oil companies” and governments that support them.

The US Security and Exchanges Commission (SEC) reported that the 27 largest public companies sustained no major financial losses due to cyber attacks.

Tweet of the Week

the same public that says the #nist cyber security framework should be "risk based" says "baseball should include baseballs" #nistcsf #eo
@sintixerr
Jack Whitsitt

Don’t forget to subscribe to this blog RSS feed and follow @digitalbond.com on twitter.

Worth Reading Articles

Critical Intelligence’s ICS Security Event Calendar Updates

Critical Intelligence provides reports and other information products on  Cyber Situational Awareness and Threat Intelligence services for Industrial Control System Owner/Operators, Vendors and Government stakeholders.

Image by ChrisInPlymouth

Research and PR and ICSsec Frenzy

ICS Security News

If you had any doubts about the thirst for ICS security news in the press, this week’s articles on some research from NC State provided a vivid demonstration. NC State puts out a press release on some early research, far away from anything that can be purchased, questionable if it would be of value in and ICS, and it turns into articles such as New Software Protects Networked Control Systems From Cyber Attacks.

Let me be clear that is not a knock on the research itself. We have been advocating and highlighting ICS security research for almost a decade now with our S4 conference. The researchers at NC State have an interesting approach that we might consider worthy of a slot at S4x14 or some future event as it develops.

The paper Convergence and Recovery Analysis of the Secure Distributed Control Methodology for D-NCS is available online.

The base idea is to eliminate sensor data from a process if the sensor or device passing the sensor data has been compromised. Actually, this could be due to any fault, not just a physical or cyber compromise. The challenge is how does the system know the sensor or device has been compromised?

The researchers rely primarily on a consensus algorithms, which I believe would severely limit it’s practical use. The example given in Section V.A is easy to understand. Eight temperature sensors take a measurement, and sensors 5 and 6 vary significantly from the converged value of the other six sensors. They are considered compromised and excluded from the process.

The problem with using consensus algorithms to detect cyber attacks or other anomalies is it requires the deployment and maintenance of a large number of additional sensors that don’t exist today in most control systems. Many times sensors are not redundant (let alone in numbers to allow consensus calculations), but data smoothing/interpolation takes place to remove and replace flawed or missing data. In a sense the researchers suggest doing this in a brute force way rather than through intelligent use of surrounding state and data.

There are high risk / high value sensors that implement a simple variant of the consensus algorithm suggested by the researchers. For example, you will sometimes see three sensors used in the chem sector, and one of the sensor’s data discarded if it varies more than a certain percentage from the other two sensors’ data.

If this is pushed out to the field or plant, as the researchers envision and makes sense, then the consensus algorithm would be implemented in the PLC which is much more likely to be attacked than the sensor. And there is still the data integrity issue in the PLC demonstrated by Stuxnet.

A more promising and realistic and difficult approach is to combine the suggested removal of data from a process with process anomaly detection rather than consensus algorithms. There have been a few sessions at S4 where researchers have tried to model possible states of a process and detect when impossible or unlikely states or state chains occurred. The most detailed have involved substation processes, and it has worked. The problem is it was very time consuming to develop the model to detect anomalies.

One minor item in  the paper particularly worried and bothered me. The researchers used the term Distributed Network Control System (D-NCS). DCS is a very common term if they wanted to focus on plant implementations, or the could have used ICS. Did they not know what terminology is commonly used by the people they hope will use the research?

The NC State press release was reasonable, not sensational, and sensible marketing of their research capabilities. The press are simply feeding the reading public what they want. The oddity is the interest in control system security is stronger outside the control system community than inside the community where the majority still hope the issue just goes away.

Image by chrissam42

Scanning PLC Devices – PLCScan

PLCScan is a utility that was released by scadastrangelove to help identify PLC devices. It does so by acting as a port scanner to see if two common ports are open and then decides what to do based on the availability of the ports. Documented within The Rack is PLCScan, a set of python scripts that will help gather information from PLC Devices.

First uses of a utility like this, could be fast scans of large subnets to identify PLCs. This could be PLCs of your own networks or PLCs of the internet, after all the blog post from scadastrangelove was titled “PLCScan the Internet”. It is only a matter of time before functionality of this sort gets added to searches like Shodan. A problem is that even this utility can cause issues on a production system if one does not know what kind of sensitivity comes along with these types of scans.

The more we use the control system protocols within to help identify the systems the more accurate information we will be able to get from the devices, also the safer the utilities will be to run. In recent blog post about practicing tools, Ralph Langner said “the device should be considered fragile by default, period”. I agree with this statement, especially for utilities like PLCScan, we should assume that we will bring the device offline if we are using a tool against a production network.

PLCScan could add great abilities to the assessment team. The utility can pull information from a PLC that then can be used as a reference point to validate information. This information could be information that the assessment team would had to manually pull from devices and configurations with screenshots. Information from the output from utilities like PLCScan are a lot easier to parse and utilize the data then reviewing screenshots.

Even with the risk of bringing the device offline the benefits to knowing what type of information that utilities like PLCScan can provide are very important to understand. Based on the testing we have performed with PLCScan, the script is well written and does take some errors into account to be the safest with the device as possible. It should be used against hosts and not subnets to start with as the only way to truly stop the scan may leave some devices in state that might crash the device.

Control system specific utilities like PLCScan will provide good information and a great value to the community if we keep helping projects like this get the most amount of information in the safest way possible. The more information we are able to gather about the systems on line we will be able to have accurate information about what is truly connected to the internet. If we can correctly identify the devices attached, we can remove them from the internet and protect them from malicious uses.

Image from Threatpost

Friday News & Notes

ICS Security NewsI asked Eyal Udassin of C4-Security in Israel to comment on the ICS hack disclosed this week. “The hack isn’t something for the books. It’s of small kibutz named Sa’ar in the northern part of Israel, indeed from a year ago. The operator had a remote access software with no password on it, so not surprisingly it was hacked. He found out that someone moved the screen view the same morning, so he understood immediately that something is fishy and changed the remote access method to a secure one.”

ISA99 released another draft standard for comment this week – ISA-64432-4-1 Product Development Requirements. I’ll write up my thoughts on it in an article next week.

Kim Zetter of Wired covered another vulnerable ICS connected to Internet story this week. It normally wouldn’t warrant mentioning as loyal readers have certainly heard enough of this Shodan / Internet search story. However, it was a Google Building Energy Management System.

Tweet of the Week

1) Hack Google's Building Management System2) Report it to the Google Vulnerability Rewards Program3) …4) Profit?http://t.co/VvLJrjrr9N
@mikko
Mikko Hypponen ✘

Don’t forget to subscribe to this blog RSS feed and follow @digitalbond.com on twitter.


Worth Reading Articles

  • Dan Goodin’s article on the use of Honeywords.

Critical Intelligence’s ICS Security Event Calendar Updates

Nothing this week.

Critical Intelligence provides reports and other information products on  Cyber Situational Awareness and Threat Intelligence services for Industrial Control System Owner/Operators, Vendors and Government stakeholders.

Image by ChrisInPlymouth

John The Ripper – S7 Password Cracking

At S4x13, Scadastrangelove (@scadasl) released a offline brute force password cracking script (http://pastebin.com/0G9Q2k6y). Shortly after the script was released the functionality from that script was added into John The Ripper. Documented in The Rack is how John The Ripper is capable of cracking S7 password hashes using the Scadastrangelove technique of offline password cracking from a packet capture.

John The Ripper has been around for many years, and is one of the most common password cracking utilities out there. With an add-on plugin and a script that is easy to run, the password hashes are extracted out of  packet captures, and cracked using John The Ripper.

The use of John The Ripper outside of the normal workstations and servers inside of ICS environments is very limited, as most devices you can’t get the information required to run the software against the password hashes.

With the rise of password complexity requirements inside of ICS environments, auditing the password complexity of PLC and like devices can be difficult and rely a lot of how much you trust the engineer. As an example there is nothing to say that the PLC configuration that you are looking at on the engineer workstation is the one that is truly pushed out to the PLC. With the ability to gather information from a packet capture and then verify the password complexity adds that much assurance to an assessment.

However, this utility can be used for nefarious purposes, take an example where on a support portal for a vendor some one uploads a packet capture to try to get help, now this packet capture is downloaded by an attacker. This utility makes it easier for an attacker to crack the passwords of S7 devices. This gives an upper hand on the ability to write custom malware to alter configurations and the likes. Thats not to be said this couldn’t and wasn’t a capability before, this just took their ability and made it easier for them.

Password cracking is dependent on the hardware in which you are running the password cracking software on. The only testing I was able to perform was on some packet captures that were given to me from Sergey Gordeychik of Positive Technologies, and the passwords were very simple passwords that cracked within a second or two. The more complex the password the more time it takes to crack via brute force techniques, with more and more password breaches happening the word lists are getting bigger which helps the dictionary attacks get that much more powerful. I expect to see more ICS devices fall to this type of attack in the future.

Photo from awaitingdawn

Last Call: Cyber Security Training in Chicago

Michael ToeckerThere are several seats still available for the upcoming Cyber Security for Power Generation training outside of Chicago.  The one-day course is specifically designed for those engineers and IT professionals responsible for securing a power plant DCS and balance of plant cyber systems. The entire course is taught in the context of model power plant.

I’m looking forward to conducting this training, as I’ve spent the past 8 or so years engaged in power generation work. There are nuances and concerns associated with Cyber Security for Power Generation, and I’m looking forward to sharing with attendees.  Cost is $495, which is a bargain for an 8 hour professional training.

For more details, please check out the event site.

S4x13 Video: Detecting 0-Day Attacks with Non-Signature IDS

Damiano Bolzoni’s of Security Matters presented Detecting 0-Day and Targeted Attacks on ICS with Non-Signature Based IDS. While the quantitative mode of anomaly detection, looking at the quantity of packets, has had some success, qualitative approach has had a lot of research with minimal practical results.

The session explains n-gram analysis and shows the results of four different n-gram approaches on finding anomalies in 30-days of Modbus/TCP traffic on a water ICS and 7-days of SMB traffic on a gas SCADA. This was great to see because using real world data in ICS research is actually still rare.

The results of the n-gram models were better for the Modbus/TCP, but even the best model had 10 false positives a day even with the highly repetitive Modbus traffic.

Damiano then proposes a new method that is an extension of what is done in Tenable’s Passive Security Scanner, INL’s Sofia and other products. Those products will identify “normal” communication on the network by source IP, destination IP and destination TCP/UDP port. The example demonstrated in the presentation includes other ICS protocol parameters.

For example, it will identify what function codes are used and alert on new function codes. It will identify data lengths and alert on new data lengths. Obviously the similar the protocol the easier it is to do this. An application layer protocol like Modbus/TCP is relatively simple. A protocol that includes its own data and transport layer, like DNP3, could cause more false positives unless packet fragmentation is dealt with. A complex protocol like EtherNet/IP would be much more difficult. It is similar to the issues that require a preprocessor in signature based network IDS.

In fact, some of our early Quickdraw IDS signatures detect similar anomalous behavior. However, the signature based approach requires selecting and modifying the signatures manually. Damiano’s approach generates these rules automatically.