Podcast: Play in new window
| Download (Duration: 42:32 — 38.9MB)
Billy Rios of Laconicly joined me on the Unsolicited Response Podcast to discuss two topics:
- WhiteScope – an online ICS/SCADA whitelist that is trying to solve the last mile supply chain problem until vendors start signing their code. The WhiteScope data repository is available to all, free of charge.
- Medical Device Security – an area that Billy is a pioneer on. We discuss progress, FDA involvement and how similar or different it is as compared to the classic SCADA/DCS/Process Control.
In part 1 we looked at what CAN is and what the difference between CAN and OBDII traffic is on a vehicle network. In this part we’re going to look at simple reverse engineering techniques to determine which CAN IDs are of interest to us. For this exercise, we’d like to see if we can confuse the tachometer display.
First, we need to identify which ID Toyota is using for the Engine RPM on the CAN. Manufacturer specific IDs are not published so we’re going to have to figure it out ourselves. For this work, I hooked up my beaglebone canbus tool (you can build your own by following instructions here: https://github.com/digitalbond/canbus-beaglebone). Then I started my truck and let it run for about 15 seconds before powering it off. Dumping all CAN messages shows us a lot of traffic and it can be difficult to make sense of what’s going on. We’re going to use the unique_ids tool (part of the canbus-utils repo available at https://github.com/digitalbond/canbus-utils) l to get a list of all of the IDs that were seen while the tool was running. My list was:
We need to do some filtering and looking at the data to identify which data bytes are of importance. We’re going to be using the watch_id tool that I wrote. Cheating, we’ll start by looking at ID 0x2C4. The bytes that change are color coded red for ease of display. Some fields may be a single byte, multiple bytes, or have multiple values in a bitmask.
$ node watch_id.js –id 0x2c4
The results from the tool were:
Looking at the results we can see that the first two bytes remained at zero until a point where they increase and remain steady at around 0x08CC (2252 decimal). The 5th byte goes from zero to 08 and then never changes. The last byte changes frequently. Later, the 4th byte changes from 0x10 to 0x0F.
We can figure that the first two bytes are a 16 bit integer, the 4th byte contains some kind of bitmask, the 5th likely contains a bitmask or flag, and the 4th is a single byte integer. Converting the first two bytes to a decimal integer we see that it appears to be a direct value of the current engine RPM which starts at 0, ramps up, and then settles lower as the engine comes to idle. We correlate this with the readout from the tachometer and can be pretty confident that we’ve found the RPM.
In this case, the RPM was a direct value. For other values (or even other manufacturers) there may be a static value or formula applied to get the proper result (e.g. RPM=bytes 1&2 + 1k). Now, we know the ID and packet formula for the RPM on the CAN. We can write a little tool that spams the bus with our “attack” messages in order to confuse the tachometer into displaying an incorrect value.
Save the date: S4x16 is January 12-16
S4x16 is moving to the Fillmore Miami Beach at Jackie Gleason Theater in the heart of South Beach. It’s literally 3 blocks from the beach, 1 block from Lincoln Road and right in the middle of all the SoBe restaurants, shops and night life.
This is a classic art deco venue where The Dick Clark Show, The Ed Sullivan Show and the Miss USA and Miss Universe Pageants were often filmed in the auditorium. In 1964, Jackie Gleason even moved his hugely popular tv show there, hence the name.
In 2007, the Jackie Gleason Theater underwent a major renovation and now has state of the art lighting, sound, video and just about any staging option we can think of — all without losing that cool art deco feel that is South Beach. In fact, the planning team is feeling the challenge of coming up with unique content, formats and staging worthy of this non-traditional ICSsec conference facility.
You often hear this security professional or that automation engineer is a “rock star”. At S4x16 we will see who the real rock stars are on a real stage that today is a major concert venue. Speakers will literally be in the spotlight on the big stage.
In addition to the main theater, the venue has so many interesting rooms for us to use. For example, the ICS Village will be held in the Red Star Lounge, a VIP lounge for concerts with coffee and cocktails and comfortable couches and tables to better get to know your fellow S4 attendees. Much like the staging, taking full advantage of all the rooms is a fun, creative challenge.
The Kimpton Surfcomber will again be an official S4x16 hotel, and it is only a 3 block walk to the Fillmore Miami Beach at Jackie Gleason Theater. Attendees raved about this hotel last year … except for the bus ride to Kovens. No buses for S4x16!
Note: We want to publicly thank the Kovens Conference Center that has been the home of S4 since the inaugural event back in 2007. They did a fantastic job for us each and every year. Unfortunately we outgrew the ballroom there and wanted to move somewhere that buses were not required. We highly recommend Kovens if you need a South Florida venue for an event.
I thought I’d take a step back after releasing tools and presenting on CAN to do a quick intro into what communications are going on inside a vehicle anyway. What is CANBus? What is OBDII? Is there a difference?
We’re going to skip all the electrical fun parts, the packet formats, and the theory and move right into using some tools to look at CAN traffic. This means my language is simplified for example purposes. For this exercise I’m going to be analyzing traffic from my 2013 Toyota Tundra.
This is CANBus (image courtesy Wikipedia):
As you can see, it is a bus. We have a bunch of systems on a 2 wire bus. Each system has a particular job. In the case of a vehicle that might be the Engine controller, instrument display, or the tire pressure sensor. CAN is also used in industrial control as well.
This is what CAN traffic looks like:
630  17 00 00 00 00 00 00 00
638  13 00 1E 00 00 00 00 00
440  42 02 00 00 00 00 00 00
The first number is the ID or address of the device sending the message (e.g. 0x630, 0x638). This is not unlike an ethernet MAC address or an IP address. The following numbers are the data bytes (up to 8 per message).
ANY device on the CAN is able to view and send a message to ANY ID. Therein lies the entire problem with vehicle security. There is zero authentication, verification, non-repudiation, or any other accountability whatsoever. Anything on the bus can pretend to be anything else per the spec. Further, you only have 8 bytes of data so adding in something like encryption or cryptoverification is particularly challenging. Basically in order to not suck we have to ditch the whole thing (sound familiar ICS people?).
Ok so that’s CAN in a nutshell. What is OBDII? OBDII is a standard SUBPROTOCOL to CANBus. It’s a set of ID’s that all the vehicle manufacturers must support.
Every manufacturer builds their cars differently. In a vehicle, for example, the RPM may show up on the CANBus on ID 0x2C4 as a 16-bit integer on bytes 1 and 2. Another manufacturer may decide to put the RPM on CAN at ID 0x310 bytes 7-8. OBDII is an attempt to create a standard method of finding out that information. Rather than standardize on IDs, however, a query/response system was created.
The method for obtaining the engine RPM via OBDII is:
- Diagnostic tool sends a message on ID 0x7DF, the standard OBDII query ID. The message contains the number of additional data bytes, the “Mode”, and the “PID” of the requested information.
For querying the RPM the Mode is 1 and the PID is 0x0C. The message would look like:
0x7DF  02 01 0C 00 00 00 00 00
- The modules in a vehicle respond to these diagnostics messages. The ID for the response depends on the module but the IDs start at 0x8 higher than the standard query ID. Thus, the ECU responding to an RPM request will respond on ID 0x7E8. The message contains the number of data bytes, the mode + 0x40, the PID, and the value.
For responding to RPM the Mode is 0x41, the PID is 0x0C. That message may look like:
0x7E8  03 41 0C 09 C4 00 00 00
In summary, OBDII is simply a sub-protocol that runs on top of CAN. It uses standardized CAN IDs and message formats to request vehicle information. The problem with plugging in third party dongles to the OBDII port is that that port still gives you direct CAN access. The OBDII port can be used to write arbitrary CAN IDs to the bus. In practice, this is used to do things like provide vehicle software updates (e.g. flashing new firmware to the ECU).
An attacker is not interested in the OBDII spec or most of the information it provides. An attacker is not interested in querying a vehicle for diagnostic information. Instead, an attacker wants to see what kind of control over vehicle operations is possible by writing to the CAN.
In part two of this series, we’ll do some CANBus reverse engineering to figure out what CAN messages we have an interest in if we want to cause the vehicle to behave erratically.
Belden buys Tofino, GE buys Wurldtech, Lockheed Martin buys Industrial Defender and now iSight Partners acquires Critical Intelligence. The trend continues of larger organizations buying ICS security expertise.
Bob Huber and Sean McBride left Idaho National Labs (INL), after being involved in setting up what became ICS-CERT, to form Critical Intelligence. Critical Intelligence in many ways competed, or augmented if you want to play nice, the information ICS-CERT provided. However, the depth and breadth of the Critical Intelligence product far exceeded what ICS-CERT provided. Whether this was due to the talent disparity, fewer restrictions on what could be written, or both is not known.
I spoke with Bob and Steve Ward of iSight yesterday to understand the motivation for partnering and what future ICS services, products and events will look like. It is too early to answer the later question, but the motivation was clear.
iSight is looking to improve their threat intelligence in the ICS area, basic and easily understood reason. From a Critical Intelligence standpoint it’s more interesting. iSight has 200+ analysts that speak 16 different languages.
- A lot of the important ICS threat info is written in Chinese, Russian and Arabic, not to mention the videos and podcasts that require the ability to understand the spoken language.
- A fair amount of the technical analysis of malware and other attack code is not ICS specific. Look at the work that Kyle Wilhoit is doing over at Trend Micro on Havex and Black Energy for an example.
- iSight has a methodology that will add rigor to the analysis and reporting process.
What iSight was missing was an understanding of what matters in an ICS, who are the players, important protocols and products, and the ability to task all those resources in a smart way that would lead to useful product. If the two companies can integrate the capabilities well the result should be more than the sum of the parts.
The biggest question then will be are the asset owners able to take in and act on this better threat intel?
Admittedly I’m a big fan of Critical Intelligence’s work. They helped with content on our site for a couple of years, were guests on the podcast, speakers at S4 and one of the people I talked to when I was trying to figure out who was doing what to whom.
Another thing this latest acquisition has in common with the other ICSsec acquisition is the price and terms were not disclosed.
Congratulations to Bob and Sean and the rest of the team at Critical Intelligence.
At S4x14 Adam Crain of Automatak, along with Chris Sistrunk, presented the results of their Project Robus that fuzzed DNP3 stacks and found most had problems with processing malformed or illegal responses. This year at S4x15 Adam talked about Avoiding Insecurity in ICS Protocols.
Adam compares Schweitzer’s Streaming Encryption Protocol (SEP) with DNP3 Secure Authentication Version 5 (SAv5).
Two of the main criteria he discusses and demonstrates with those two protocols are 1. have a clear trust boundary and 2. keep it simple. It is clear why there were so many bugs that led to vulnerabilities in the DNP3 protocol stacks.
This is a must watch for any group adding security to an ICS protocol or those that need to start this important and necessary ICS protocol feature.
Part 1 covered the need to pull and publish more useful information from the gathered ICS incident and vulnerability data. Part 2 covers “Are the numbers intentionally misleading?
245 Incidents Reported To ICS-CERT in 2014 Means What?
The big statistic picked up by the press from the front page of the latest ICS-CERT Monitor is in 2014 ICS-CERT “received and responded to 245 incidents reported by asset owners and industry partners.” This number is meaningless, but it gets the lead from DHS/ICS-CERT/INL.
In fact, this number is actually slightly less than the number of incidents reported in 2013. If the gross number has meaning, then we should saying we are succeeding in reducing ICS attacks.
Or perhaps it is evidence that asset owners, vendors and researchers had bad experiences and are choosing to no longer work with ICS-CERT.
The number of vulnerabilities handled by ICS-CERT was also down from 187 in 2013 to 159 in 2014. ICS-CERT’s method of counting “vulnerabilities” is so broken and wrong that the gross numbers have little value beyond PR for ICS-CERT. For example, some vulnerabilities in ICS common components are counted individually for each product that integrated the component. Others are counted as one vulnerability. And still others are completely ignored by ICS-CERT leading to 100’s of vulnerable products being counted as zero.
Looked at another way, when an ICS application distributed from an ICS vendor contains Havex, Conficker being found on an ICS computer, and an Internet scan of a building automation controller are each counted as one incident, the gross number of incidents has little value.
Everything ICS is “Sophisticated”
The most misleading number in the statistics from the ICS-CERT Monitor is 55%. “Of the total number of incidents reported to ICS-CERT, roughly 55 percent involved advanced persistent threats (APT) or sophisticated actors.” This is despite “In many cases, the threat actors were unknown due to a lack of attributional data.” So this means almost every incident reported to ICS-CERT is considered APT or sophisticated actor.
ICS-CERT needs to publish the criteria of what makes one a “sophisticated actor”, but based on earlier Alerts and Advisories it is a low bar.
The last set of numbers has Idaho National Labs and DHS/ICS-CERT increasingly competing with industry for some undefined reason. ICS-CERT performed 37 onsite assessments in the six months from Sept 14 to Feb 15. Why?
Why 37? Why not 10 or 100?
What is anyone to take from reporting this number and the sectors the assessments took place in?
Where are the statistics showing progress from this work? And what is the progress they were hoping to achieve?
Why is DHS/ICS-CERT doing this work? Why are they doing it for free for critical infrastructure companies?
There are a number of companies, including Digital Bond, that do onsite assessments. There are also companies that do quality ICSsec incident response and ICSsec training.
This work and related reported numbers feign useful activity and avoid the reality that DHS/ICS-CERT is not taking the leadership role in providing technical expertise to help in developing secure ICS protocols and standards, accurately informing government and industry, analyzing ICS attack code and performing the role that ICS-CERT was created to do.
“There are three kinds of lies: lies, damned lies, and statistics.” Mark Twain (purportedly quoting Benjamin Disraeli)
The latest edition of the ICS Monitor, last week’s USA Today articles and the reemergence of Joe Weiss’s secret database warrant a hard look at the numbers coming out. Specifically two questions:
- Are the numbers intentionally misleading?
- Are there additional insights that should be pulled from collected data?
The second question is more interesting and involves less speculation on the motives. Consider Joe Weiss’s database of 400 Cyber Incidents (NIST Definition). No doubt there was a lot of labor collecting and documenting those Cyber Incidents, and there are trust issues that probably limit the information that can be distributed. That said, the number 400 is of little or no help.
Start to break them down by reason for incident (attacker, mistake, malware, etc.), category of impact, qualitative measure of impacts across different categories (cost, loss of life, environmental, etc.), and other divisions and the information begins to be of value. Even then we have to be aware of collection bias. Joe’s background in the electric sector would likely lead to an overweighting of Cyber Incidents in that sector, but looking at each sector as a separate population could provide some helpful information.
ICS-CERT is the champion of intentionally misleading statistics, see ICS-CERT
Monthly Monitor for prime examples, but I’ll cover that in part 2 of this article. Let’s focus now on how ICS-CERT could provide useful numbers from the data they have.
- ICS-CERT provides data on the number of vulnerabilities reported. In 2014, they just provided a raw number of 159 and a trend line. Here would be some useful info, especially if they were tracking this over years.
- How many of the vendors had a process to deal with the reported vulnerability? This could be broken down further to include a PGP key, security@ email, CERT and other categories.
- What percentage of the vulnerabilities were verified fixed by ICS-CERT, 3rd party, or the disclosing party?
- A distribution of the timeline from disclosure to fix … especially now that Google has set the bar at 90 days.
- Severity distribution to US critical infrastructure (since it is US ICS-CERT)
- … you probably have already thought of more
- ICS-CERT provided data on “incidents”. It is very hard to hold off on misleading and just plain wrong, but here are some examples of useful data to pull from the 245 incidents in 2014.
- Impact, similar to the discussion on the Weiss database
- Identification and traits of what they called APT/Sophisticated Actors. Somehow they determined 55% fell in this category. What was the distribution of characteristics that led to the 55% being called APT?
- Discovery, a distribution of how the attackers were discovered, perhaps broken down by type of attack or APT/non-APT in their parlance.
- Most common attack trees.
- many, many more
If you really have hundreds of incidents that are not simply corporate network incident noise on big companies that run ICS, then there is a lot of useful information there.
Coming Soon: Part 2 – The Fine Print and Methodology
Andrew Ginter of Waterfall Security Solutions speaks on Embedding Malware in ICS Protocols. His conclusion is this is harder than one thinks. The easier solution might be to use the SQL server, web server, ftp server, or other commonly exploited protocols that ICS applications integrate.
Fair warning – the second half of the session gets a bit commercial on his/Waterfall’s view on why unidirectional security solves ICS security challenges.
Eireann Leverett of the University of Cambridge Centre for Risk Studies looks at control system related catastrophe scenarios and the economic impact of these scenarios with an eye towards how insurance and reinsurance policies will be written and priced.
Admittedly critical infrastructure cyber security is a new topic in an insurance industry that has been around hundreds of years. Eireann points out that insuring against malicious attacks is not new to the insurance company. They insured against piracy on the seas.
The session provides some relevant macro economics in easy to understand language and graphs, and Eireann admits “we’re inventing rough metrics in a land of no metrics”.
His initial efforts are related to an important cyber incident that could impact the US, UK and European bulk electric system. The % loss of GDP due to an incident sounds like a good measure if it can be credibly calculated.
The Q&A in this session was particularly good, which is understandable since there are more questions than answers at this time. It’s a fertile field for those looking for an important economic problem.
For what it’s worth … this was my 18-month old daughter’s favorite session.