Switches Get Stitches (or: Switches Get DNA Helicased)


There is a ‘talk franchise’ that has started titled ‘Switches Get Stitches.’  Started by Eireann Leverett and Colin Cassidy, it showcases problems in industrial network switch hardware and firmware.  Digital Bond Labs offers a humble contribution to the cause: a demonstration of a firmware rootkit for an (admittedly somewhat dated) industrial switch.  If you are attending Defcon 23, be sure to check out the ‘official’ SGS talk there.

One of the components in this year’s ICS Village CTF is going to be pretty unique: we have modified a network switch firmware.  This ends up giving a lot of interesting leeway: we can now mangle packets, talk to a command and control server, and make a few other interesting flags for participants to find.

Most ICS equipment lacks any kind of firmware protection.  Scarier is the fact that some operators, including a very small subset of utility operators, purchase safety-critical equipment from dubious sources such as eBay.

So, let’s take apart a network switch and show just how easy it is to trojan a device!

Read More

ESCAR USA Thoughts and the ROI of Investing in Cybersecurity

ESCAR was an interesting event. There were about 150 in attendance from various parts of the auto cybersecurity community including OEMs, tier 1 vendors, and defense products. There were speakers on a variety of good topics, the full lineup is available at https://www.escar.info/escar-usa/program.html. Being an “outsider” to the auto security community I really enjoyed the opportunity to meet people and have some interesting discussions.

The opening keynote by Alex Halderman of U of Michigan was highly interesting regarding the all-too-common re-use of keys or predictable keys that are prevalent in embedded systems due to a common starting point. Relevant information that will hopefully sink in and be of use in the future as more devices are incorporating encryption, but as of now those devices are rare (in both the auto industry and ICS).

“Methods for Penetration Testing of Automotive Embedded Systems” presented by Argus was a good primer in security analysis for these types of systems. A few talks discussed handling the increasing connectivity in vehicles moving forward and how to best separate the concerns of interactive vehicle features affecting safety and operational control systems. I feel like this is the most important area to be considered at the moment for the industry.

password-704252_640Vendors at the conference seem to be focusing on IDS/IPS for vehicle systems. The idea being that installing extra equipment to monitor the CANBus to identify messages that occur when they should not and to prevent those messages from reaching the ECU. This requires keeping a knowledgeable running state of the vehicle to determine if the “steering wheel left park assist” action is correct at a given time. Some vendors seem to do this via software on existing vehicles and others via entirely separate sensor systems to be installed aftermarket. I think this is a *VERY* difficult problem to solve (having worked in the situational awareness space previously) and is a common issue that people attempt to band-aid over in cybersecurity.

Read More

Shodan for Rocket Scientists

shodan_pasukaru76Shodan is a really useful tool for, well, all sorts of research.  Not only can you quickly determine what the public-facing security impact of a new vulnerability is going to be, you can find all sorts of control systems attached to the Internet that shouldn’t be. Searching for random control-systems related terms sometimes even steers a researchers towards new and interesting equipment to test.

John Matherly, who runs Shodan, is constantly tweaking settings and adding features (and new scan types) to help the security community. [On a personal level I can’t thank him enough for teaching me all of the tricks that I’m writing about here].

Two of the recent changes made ended up being really helpful for finding some of the most vulnerable ICS systems: telnet options searching and bannerless telnet searching.  The latter of these is only available to folks who pay for API access, but it opens up some rather interesting critical infrastructure to locatability.

Way back in 2012 we did Project Basecamp.  The ‘Biggest Loser’ of Project Basecamp, purely on the number of red ‘X’ security failures, was General Electric’s D20ME RTU.  (I should mention that GE has made strides in improving the line with the release of their D20MX, but the D20ME line will remain forever vulnerable).  Back then, I really wanted to be able to search for the D20 on Shodan but couldn’t.  This was because the D20 only supports Telnet, and it supports it in a way that Shodan didn’t support.  Until now.

Read More

Unsolicited Response Podcast: Eric Byres after Tofino

After a long and successful struggle to bring an industrial firewall to market, Eric Byres is leaving Belden and Tofino behind. We shouldn’t call it retirement because I expect that Eric will be contributing in a number of different ways in the next ten years.

I gave Eric a few months to clear his head and then talked with him for this episode of the Unsolicited Response Podcast.

The first 16 minutes of the episode are a retrospective of Tofino. What features were surprisingly effective, what were the biggest challenges and dark times, when will we see Tofino on a chip and more.

After that we talk about bigger questions on the ICSsec community, Eric’s home automation and what he may do next.

ESCAR Presentation

I enjoyed last week in Detroit at ESCAR (Embedded Security in Cars). I went there to present on the topic of vehicle security and how remote access and third party devices impact the threat landscape. Many researchers have published about the security concerns of vehicle systems, namely the CANBus and it’s simple nature that lacks security controls entirely. It has been shown that if an attacker is able to send messages on the CAN, they are able to control the vehicle.

I performed a security assessment of a third party OBDII dongle. This dongle sits in the diagnostics port of the vehicle (which is on the CAN), collects information, and sends that information through the cellular networks to the third-party servers. I found that this device follows the pattern we see in embedded systems which is to say that it was designed and created without any security in mind whatsoever. Cleartext communication channels, no hardware separation of concerns, hardcoded database credentials…the list goes on.

What this means for auto manufacturers is that it cannot be assumed that the vehicle CAN will be isolated. An attacker may not need physical access to a vehicle to execute an attack. Attacks are no longer limited to a single vehicle and may affect entire fleets managed by these types of systems. Defending against these attack vectors requires a separation that prevents a compromised dongle from affecting the vehicle in any way whatsoever. I created a proof-of-concept “CAN Protector.” This device acts as a gateway that propagates information out of the vehicle network (one-way/read-only) for consumption by third party devices.

You can find the slides here: http://www.slideshare.net/dgpeters/thuen-escar-48872976

Website for the conference is: https://www.escar.info/escar-usa/program.html

ICS Security Research Newsletter: Issue 15-2


The team at Digital Bond Labs has published their ICS Security Research newsletter for the 2nd quarter. I suggest you subscribe to the newsletter, but if you want to view this issue directly, it is available at this link.

The issue includes:

  • the latest on Corey’s CAN tools and hacking
  • news on a joint research project with the State of Virginia
  • an interesting essay from Reid on Bug Bounties in ICS
  • call for S4x16 ICS Village volunteers
  • a review of CybatiWorks educational platform
  • and more …

S4xJapan Call for Presentations

HankoWe are pleased to announce a return to Tokyo for the S4xJapan event on Friday, November 6th.

S4xJapan will be held again at Academy Hills on the 49th Floor of the Roppongi Hills Mori Building. There will be a fun and novel social event (last year was the Kaspersky KIPS game for the first time in Japanese) with food and drink after the days sessions complete. And then you will be close to the Tokyo nightlife on a Friday night to have some fun with old and new friends in the ICS security community.

The event will be a very full one-day that will cover ICS security, Operations Technology (OT), ICS cyber weapons and related topics.

We are looking for sessions in both English and Japanese (simultaneous translation will be provided).

If you have a session you would like to present or know of a speaker or topic we should chase, send us an email at S4@digitalbond.com.

We will welcome some presentations from overseas experts with new information and techniques. If this is you, please note that PACSEC JP is the following Wednesday and Thursday so you can potentially speak at two events and enjoy some time in Tokyo.

Vendors Step Up & Step Down

While progress on adding basic security to PLC/RTU/Controllers, Level 1 of the Purdue Model, continues to be excruciatingly slow, there is much good news from vendors that make the applications that reside at Level 2.

Vendors offering HMI, Engineering Workstations, Historians, SCADA and DCS Servers and other Level 2 applications are adding security features to their software. Equally important is they are providing quality configuration and deployment advice to their customers in the form of white papers and videos. This is a growing trend for at least the past five years.

Check out Bryan Owen’s 2014 and 205 User Group presentations on securely deploying the OSIsoft PI Server. These are two of many videos and tools they provide to help customers secure their PI deployments.

OSIsoft 2014

OSIsoft PI Server Security

OSIsoft is not alone. We work with numerous vendors who provide quality secure deployment advice for the architecture, OS hardening and ICS application security settings. We have also seen this type of great information at S4 OTDay and other vendor User Group events.

The ICS Community should recognize and appreciate the vendors that are stepping up to this challenge, but … the unfortunate reality is this quality guidance seems to be rarely followed and new system deployments are not following even the key items stressed in the vendor deployment guidance.

In some cases this is an integrator, consultant or customer that is installing the ICS incorrectly. This is bad enough, but the real shame is when the ICS vendor installs their own solution without following their own security advice.

Many times a year we have a disagreement with the ICS vendor’s deployment team on how their product should be installed, and we have to pull out the ICS vendor’s own documentation to show them they are installing it wrong. Even then more often than not they fight back and tell us why it is impossible to install the ICS the way their own product team recommends.

It’s not hard to understand why this happens. The team that works on design and implementation has a difficult job, and many of these team members have been installing systems for decades. They have a way to make a complex deployment work and are not looking to make their job harder. If they know how to install a legacy configuration, they will install a legacy configuration even on a completely new system.

The ICS vendors need to step up even more and push this secure deployment information and processes down to the deployment teams. I’ll fully admit this is a big task that is unlikely to be 100% in the near term, but we are not seeing installs following ICS vendor guidance even 25% of the time.

Asset owners purchasing new ICS are going to have to step up as well. They will need to get the ICS vendor secure deployment recommendations and add secure deployment requirements to the acceptance tests.

This can be done.

S4x15 Video: Simulating Multiple Substation Failures

This is a great session for power engineers and those involved in substations to watch. It is an extremely technical session by Dr. Chee-Wooi Ten of Michigan Technological University.

The key point is actually easy to understand. The most critical substations to secure may not be the highest voltage substations, and this session provides a set of mathematical equations to perform an impact analysis to identify the most critical substations.

Dr. Ten gets into the modeling and mathematics in significant detail in the video.

Identifying Protocol Parsers and Cryptographic Algorithms in Firmwares and Binaries

no_reversing_rubygoesIn my talk way back at S4x15 I briefly mentioned a few techniques at identifying interesting parts of an application for reverse engineers.

A lot of times we as reverse engineers load up a firmware or DLL or executable.  We want to get hunting for bugs as quickly as possible.  Hunting for bugs usually means rapidly identifying the data entrypoints into an application.  In the case of networked equipment and networked software, data entrypoints mean network protocol parsers.

One of the techniques that I talked about was using CRC/hashing/crypto algorithm precomputed tables for identifying interesting functions.  I thought a little post about this technique was worth writing.

Protocol Parsers

Obviously there is an easy button for some application software: searching for OS functions like recv() is a sure way to find your protocol receiver (and perhaps your parser as well).  For deeply embedded systems this is a bit trickier though, as a disassembler will not necessarily know what function in the disassembly constitutes recv() or any other C library call.

Most protocols in the ICS space have a lot of legacy cruft hanging around, though, which can help us a lot.  The ‘cruft’ parts of protocols are those that are no longer necessary in the Ethernet and IP worlds, but are left in the protocol to simplify the lives of implementers.  Usually, the IP variants of protocols are just wrapped-up versions of the serial protocols.  Serial protocols usually included CRCs, since the serial physical layer provides for no error detection or correction.

One of the neat tricks that was invented long ago was to partially precompute CRCs using a table. The algorithm, known as the Rocksoft Model CRC Algorithm, is documented in a nice paper, which includes the source code needed to generate the precomputation table given any CRCs exponents.

Read More