We were thrilled to have some of the world’s top security researchers enter the ICS world and present at S4x14. In this case, S4 veteran Darren Highfill introduced langsec pioneers Sergey Bratus and Meredith Patterson to the world of ICS, and they worked together to give a novel talk to the ICS community.
After an introduction to langsec, they look at the DNP3 protocol. They actually created a DNP3 parser using the Hammer parser generator library. But you start to see the problems, or challenges, in a robust DNP3 protocol stack with the context dependency between the three DNP3 layers.
The money quote from the session was “Your parser two layers down from where you started parsing the packet has to be able to refer back up to its ancestor just to know how many bytes it is suppose to parse. This puts us way, way, way out in to heavily context sensitive territory.” After listening to this talk it is not surprising that DNP3 protocol stacks are bug filled.
The presenters actually worked with Adam Crain & Chris Sistrunk to analyze a specific Project Robus DNP3 protocol stack vulnerability from a langsec perspective. The DNP3 protocol requires a Transport Frame to have at least one valid APDU. Bad things happened when this was violated.
I found myself writing down a number of notes to think about more from this session.
- Context Dependency – Do you have to have additional information to determine the value or meaning.
- Weird Machines – Hidden functionality unintentionally built into a device.
- You save when you throw out bad input early.
- Computational power is a privilege; don’t expose it to an attacker too early.
- If you want fast, simplify your language.
I challenge S4x14 speakers to have so much technical meat that they leave 1/3 of the audience behind, Seth Bromberger of NCI Security took me up on this in a math heavy talk on incident response in a smart grid network. However he explains the graph theory with easy examples from typical smart grid deployments so loyal readers will understand the concepts even if they don’t want to do the math.
Many of you probably remember the IOActive videos showing a worm propagating through a smart grid network. Seth’s session looks at how to stop the worm from propagating through a variety of graph theory strategies such as Degree Distribution Cull, Betweenness Cull (the most promising strategy) and Shortest Paths. I found the Interdiction Problem to be interesting and something to consider for SCADA systems.
On a related note … the random mixing assumption in the epidemiological models doesn’t hold for the smart grid.
The court battle between Battelle/INL and Corey Thuen at Southfork Security is over. The settlement agreement gives Battelle all rights to Thuen’s Visdom product. While the case hinged on whether Visdom was a copy of Sophia and the Thuen employment agreement, the courts reaction to “you called yourself a hacker so you will break the law argument” and the lame national security impact contention were what made it worth watching. Now clear of entanglements with INL, Theun could start over and build a similar product, but neither Sophia or Visdom were hardly novel or even competitive with more full featured solutions.
Microsoft introduced a new version of their free threat modeling tool. We used their old tool in consulting projects, and look forward to trying out and writing about the new version. One immediate plus is it no longer requires Visio. Microsoft has included a drawing tool in the package.
Bloomberg reported “Electric, natural gas and major water companies and regional distribution systems in Connecticut have been penetrated by hackers and other cyber attackers, but defenses have prevented interruption”. We will be seeing this in slide decks.
WOW! NYISO unveiled their new $38M control center with a 2300 square foot video wall. There are a wide range of opinions on what makes a useful control room, but this one will certainly make an impression on visitors.
Wireshark added another ICS protocol dissector … Landis & Gyr (Telegyr) 8979.
And we probably need to put a note in about Heartbleed. There have been a few ICS-CERT advisories on the issue. Asset owners should look at SSL remote access to the ICS and SSL to security perimeter devices for management. Pre-Heartbleed, remote access to ICS should have been physically disconnected except for when emergency support is required.
Image by ChrisinPlymouth
Digital Bond recently released two Nmap Scripting Engine (NSE) scripts under our Project Redpoint. The second NSE was an attempt to convert S7 enumeration scripts written in Python by SCADA Strange Love into an Nmap NSE. Over the course of development of this script and the BACnet script a lot was learned about development in Lua, especially when it comes working with existing Python code.
Lua is a scripting language that I had not previously developed in, and the only places we knew it of it being widely used, outside of Nmap, was in the gaming industry (e.g. World of Warcraft). Based off some research into Lua and some sample code exercises, we learned that there are many reasons why Nmap had chosen Lua as its scripting language choice. According to the Lua Website, Lua is meant to be fast, portable and light weight. Lua has a great community around it with lots of documentation on how to develop inside the language.
Nmap has also developed its own Lua libraries that is included with the Nmap builds. These libraries help implement a lot of the missing functionality in Lua as compared to other languages such as Python and Ruby. Within the Nmap documentation there is a specific Nmap library that includes how to create sockets and also how to perform error handling within a try/catch for an example.
Nmap has 110 Libraries that were developed to help extend Lua for nmap for basic functions to protocol specific applications. These applications include as an example http, dns, eigrp and snmp. This helps when trying to interface with well known protocols and can make simple queries against the protocol to gather information. Below is an example of both creating a new socket, and using a try/catch to validate that if the socket was created successfully or not.
local result, socket, try, catch
result = “”
socket = nmap.new_socket()
catch = function()
try = nmap.new_try(catch)
result = try(socket:receive_lines(1))
Lua is a classless language, and it proved to be a challenge when trying to convert existing Python code into Lua. In some cases, classes had been defined and multiple functions were created within each class. One solution to this was to create a function name with the class name inside of it. As an example, for a class named Protocol and a function to parse that specific protocol, you could write a function named Protocol:parse() as a basic way to achieve some basics of classes. More advanced classes that require initialization as well as other actions upon setup can be achieved in more complex ways. We do have some working examples of NSE scripts that utilize multiple mock classes.
Another lesson learned was Lua does not have fully developed methods and functionality. In some cases it was necessary to write a basic function that would be found in most other languages. In the An example in Project Redpoint is a function to determine if a string starts with a specific character.
With the S7 Enumeration NSE it proved to be easier to start with a clean slate rather than port the Python script. However the Python script was very helpful in identifying desired output, generating packet captures, and testing to make sure the NSE was working correctly. The clean slate LUA approach also makes it more likely the code will be accepted by the Nmap project for distribution with the other NSE. The Nmap project team has been very helpful in getting the Redpoint NSE scripts ready for submission and has provided great feedback on writing scripts to meet the Nmap NSE code standards.
Image by Roblox
Redpoint is our internal project to develop NSE scripts for Nmap to identify and enumerate ICS devices. We are releasing some of the more helpful and less intrusive scripts on GitHub. The first was for BACnet devices, and now we have released a NSE script to identify and enumerate Siemens SIMATIC S7 PLCs.
Full credit for the idea and concept for this script goes to Positive Technologies and SCADA STRANGE LOVE. We ran PLCScan to generate the pcaps, copied what they found useful to enumerate, and used it as QA for our Redpoint script. This script is basically a way to get the PLCScan capabilities in Nmap. Since we use Nmap for enumeration in our assessments and have a number of Redpoint scripts we run in an Nmap category, it was worthwhile for us to port it over. Hopefully the large community that likes Nmap will find it useful.
Let’s go to the screen shots:
We pulled the information from the Module field to identify the model, eg S7 315, S7 317, S7 312. Beyond the value for inventory purposes, it is helpful to identify what are the high powered, most critical PLCs or how large and complex the process you found is.
The System Name and Plant Identification Fields could be very helpful if they were entered by integrator or asset owner. We have scanned devices where the System Name was descriptive and others where it was not changed from the default (and therefore not included in output from the NSE). We have yet to see the Plant Identification Field used, but a large organization with multiple plants may find this helpful, as would the person enumerating the S7 PLCs.
Image by Simon and Katrina McPherson
Rotem Bar of Limpox Advanced Solutions closed out S4x14 with a look at how integrators can introduce vulnerabilities into an ICS. This point was actually brought out as well by Sistrunk and Crain with the DNP3 vulns. In that case the TMW master station was not vulnerable to the Project Robus attack methods, but some vendors who had implemented the TMW stack in their master station fell over when fuzzed.
Rotem looks at an example API, from GE Cimplicity, and finds a lack of validation, control and unnecessary features. He then proposes an architecture to resolve many of these issues.
The Crain/Sistrunk disclosed vulnerabilities from fuzzing of master stations have all been related to DNP3 protocol stacks … until today. ICS-CERT announced the first Modbus protocol stack vulnerability from Project Robus. Welcome to the party Modbus.
We normally don’t bother commenting on ICS-CERT alerts or advisories, but since we broke that rule already … the latest advisory update on an Allen Bradley denial of service vuln is another reminder that vulns and patching matter little in this insecure by design world. Why worry about a vuln that can cause a denial of service when an adversary can send a legitimate EtherNet/IP Stop CPU command?
Siemens and McAfee announced they “are extending their partnership to enhance the security offerings for industrial customers to protect against rapidly evolving global cyber threats.” Hard to tell if this is marketing fodder or more. The ICS vendors are choosing partners to work with for application white listing, security monitoring and other solutions. Symantec and McAfee being the major players along with the newest Lockheed/Industrial Defender combo.
Innominate’s mGuard was one of the first industrial field firewalls. At Hannover Messe this week they announced support for OPC. The “OPC Inspector masters the complex connection tracking of OPC dialogues across their changing ports and connection directions, thus enabling an effective control and filtering of OPC based on the stateful inspection firewall principle”. They sell a virtual machine software version in addition to the physical, industrial rated module.
Let me give you a real world anecdote to provide a little context about my comment to Kelly Jackson Higgins over at Dark Reading that the Windows XP end of life was in many ways a positive experience for ICS organizations that care about security.
Last month I had a conversation with security conscious client who was forced by corporate policy to upgrade their engineering laptops from XP to Windows 7 prior to XP’s end of life. These were the laptops that had the PLC and other programming tools on them to troubleshoot and update logic/programs and other devices at field sites in a SCADA system. They followed the corporate edict and updated the OS. And some of these software tools stopped working.
The story continues, they had to go back to the software tool vendors and upgrade or load a newer version of the application tool that was supported on Windows 7. Now I would like to say the right lesson was learned, but we were not there yet. The wrong lesson the engineers initially took away was it was dangerous to update an OS on ICS workstations and servers, and it should not be done.
However, once we started to have a discussion on how they approached this upgrade and how they approached physical system upgrades at the field site they started to identify and internalize the right lessons. They consider the impact of other engineering changes to the field sites, and perform the appropriate research, testing, outage (if necessary), recovery, etc. In this case they did not check with the tool vendors, who had very clear documentation on their sites on the versions that were compatible with Windows 7. There is nothing ICS unique about verifying application and OS compatibility.
This was one of the many conversations I had with asset owners dealing with the move from XP to Windows 7. There were a number of other lessons learned in processes, vendor selection, budgeting and other areas.
Starting in the 90′s the ICS community bought into the myth that we could have mission critical IT networks that did not require any time or expense to maintain. XP end of life was a great example that asset owners need to plan for periodic upgrades of operating systems and applications.
I think the best treatment of ICS security through basic engineering practices is Ralph Langner’s Robust Control System Networks.
Image by Bruce Turner
The Great Debate topic for S4x14 was:
Are Risk Based Approaches Bound to Fail in Securing Critical Infrastructure ICS?
The idea for the topic was a Bound to Fail paper by Ralph Langner and Perry Pederson for the Brookings Institution. We had Jim Gilsinn of Kenexis and Marc Blackmeer of Cisco arguing that risk based approaches are helpful and necessary. Zach Tudor of SRI and Mike Ahmadi of Codenomicon making the case that risk based approaches are bound to fail.
After the four of them argue the issue for 25 minutes it is thrown out to the audience for the remaining 25 minutes. You will see the S4 attendees are not shy about giving their opinion and mixing it up.
Friendly reminder that there are a few seats still available for the CIPv5 Foundations course partnered with Digital Bond’s Cyber Security for Generation (click link for more details).
This two day course starts with the NERC CIPv5 Foundations course offered by EnergySec, and concludes with a deep dive by yours truly into applying technical NERC CIP and security principles and practices to the generation environment.
Those interested may register on the EnergySec website here. Cost for the paired training is $1,295.00 total, a savings of $100 is recognized when signing up for both courses. Seats are limited to the first 25 participants, and EnergySec partners are offered their customary training discount.