Lies, Damned Lies and Statistics

“There are three kinds of lies: lies, damned lies, and statistics.” Mark Twain (purportedly quoting Benjamin Disraeli)

The latest edition of the ICS Monitor, last week’s USA Today articles and the reemergence of Joe Weiss’s secret database warrant a hard look at the numbers coming out. Specifically two questions:

  1. Are the numbers intentionally misleading?
  2. Are there additional insights that should be pulled from collected data?

The second question is more interesting and involves less speculation on the motives. Consider Joe Weiss’s database of 400 Cyber Incidents (NIST Definition). No doubt there was a lot of labor collecting and documenting those Cyber Incidents, and there are trust issues that probably limit the information that can be distributed. That said, the number 400 is of little or no help.

Start to break them down by reason for incident (attacker, mistake, malware, etc.), category of impact, qualitative measure of impacts across different categories (cost, loss of life, environmental, etc.), and other divisions and the information begins to be of value. Even then we have to be aware of collection bias. Joe’s background in the electric sector would likely lead to an overweighting of Cyber Incidents in that sector, but looking at each sector as a separate population could provide some helpful information.

ICS-CERT is the champion of intentionally misleading statistics, see ICS-CERT Monthly Monitor for prime examples, but I’ll cover that in part 2 of this article. Let’s focus now on how ICS-CERT could provide useful numbers from the data they have.

  • ICS-CERT provides data on the number of vulnerabilities reported. In 2014, they just provided a raw number of 159 and a trend line. Here would be some useful info, especially if they were tracking this over years.
    • How many of the vendors had a process to deal with the reported vulnerability? This could be broken down further to include a PGP key, security@ email, CERT and other categories.
    • What percentage of the vulnerabilities were verified fixed by ICS-CERT, 3rd party, or the disclosing party?
    • A distribution of the timeline from disclosure to fix … especially now that Google has set the bar at 90 days.
    • Severity distribution to US critical infrastructure (since it is US ICS-CERT)
    • … you probably have already thought of more
  • ICS-CERT provided data on “incidents”. It is very hard to hold off on misleading and just plain wrong, but here are some examples of useful data to pull from the 245 incidents in 2014.
    • Impact, similar to the discussion on the Weiss database
    • Identification and traits of what they called APT/Sophisticated Actors. Somehow they determined 55% fell in this category. What was the distribution of characteristics that led to the 55% being called APT?
    • Discovery, a distribution of how the attackers were discovered, perhaps broken down by type of attack or APT/non-APT in their parlance.
    • Most common attack trees.
    • many, many more

If you really have hundreds of incidents that are not simply corporate network incident noise on big companies that run ICS, then there is a lot of useful information there.

Coming Soon: Part 2 – The Fine Print and Methodology

S4x15 Video – Ginter on Embedding Malware in ICS Protocols

Andrew Ginter of Waterfall Security Solutions speaks on Embedding Malware in ICS Protocols. His conclusion is this is harder than one thinks. The easier solution might be to use the SQL server, web server, ftp server, or other commonly exploited protocols that ICS applications integrate.

Fair warning – the second half of the session gets a bit commercial on his/Waterfall’s view on why unidirectional security solves ICS security challenges.

ICSage Video: Eireann Leverett on Catastronomics

Eireann Leverett of the University of Cambridge Centre for Risk Studies looks at control system related catastrophe scenarios and the economic impact of these scenarios with an eye towards how insurance and reinsurance policies will be written and priced.

Admittedly critical infrastructure cyber security is a new topic in an insurance industry that has been around hundreds of years. Eireann points out that insuring against malicious attacks is not new to the insurance company. They insured against piracy on the seas.

The session provides some relevant macro economics in easy to understand language and graphs, and Eireann admits “we’re inventing rough metrics in a land of no metrics”.

His initial efforts are related to an important cyber incident that could impact the US, UK and European bulk electric system. The % loss of GDP due to an incident sounds like a good measure if it can be credibly calculated.

The Q&A in this session was particularly good, which is understandable since there are more questions than answers at this time. It’s a fertile field for those looking for an important economic problem.

For what it’s worth … this was my 18-month old daughter’s favorite session.

Unsolicited Response Podcast: SANS ICS 410 Course & GICSP

Episode 2015:2 SANS ICS Security Training and Certification

SANS provided four individuals for our Unsolicited Response podcast on the 5-day ICS 410: ICS/SCADA Security Essentials training course and the related Global Industrial Cyber Security Professional (GICSP) certification.

  • Scott Cassity, Managing Director of GIAC
  • Mike Assante, SANS Lead for ICS/SCADA security training
  • Justin Searle, SANS Instructor and major course content creator
  • Graham Speake, SANS Instructor and participant in GICSP creation

In the hour long discussion we cover:

  • Why SANS developed an ICS certification and training course
  • The difficulties and benefits of having a mix of IT Security and OT students in the class
  • How the course and certification were created and how they are structured
  • Early feedback on the course and certification along with likely changes and possible future courses (2-day course) and certifications
  • The number of students trained and the number of certified GICSP
  • Why SANS courses are so expensive relative to other courses
  • How should a potential employer view an individual who has the GICSP certification

I also gave the SANS team a chance to answer the criticism that this is an IT Security course from an IT security organization.

I appreciate SANS providing so much time and resources to the podcast. I think there is a fair argument on how the SANS course rates in comparison to the competition, and it might depend on the attendee profile and goal of taking the course. The one thing that SANS has going for it is they know how to scale up to train thousands of students, and this is needed in the ICS security space.

Related Links:

I’m committed to a minimum of 20 podcasts in 2015; this is episode 2. We will wait until five episodes are recorded before bringing on podcast sponsors, but let us know if you are interested in sponsoring Unsolicited Response.

Subscribe to the Unsolicited Response Podcast in iTunes.

Save The Date: S4xJapan is Nov 5-6 in Tokyo

HankoDigital Bond is pleased to announce the 2nd edition of S4xJapan will be held on November 5 – 6 in Tokyo.

The event will be in the Mori Building, Roppongi Hills. The Academy Hills facilities on the 49th floor were perfect for the event last year. The room where the sessions are held have desks/power/Internet for all with a tiered seating so there isn’t a bad seat in the venue.

Thursday, November 5th will be Operations Technology Day (OTDay) where we will focus on real world examples of operating and securing critical control systems implementations.

Friday, November 6th will be the S4 Technical Sessions. You will hear the latest offensive and defensive ICS security research and technicals in technical detail.

Our goal is to have the sessions in English and half in Japan, with simultaneous translation as appropriate. You can see some of last years sessions our video site, and we will be posting the remainder over the next two months.

We also will have a social event after the Thursday sessions in the Academy Hills library. Last year it was food, drink and the Kaspersky ICS Simulation Game. We are planning something fun for this year as well.

The Call For Presentations will be released just after Golden Week, so be thinking about what research you will want to submit to be a part of the event.

Tool Release – CANBus Protector

beagle-canbusContinuing in the line of CANBus research and tools release I’d like to announce some quick work on a proof-of-concept CANBus IPS called, unoriginally, the CANBus Protector. I took some time to work on defense of CAN after conducting a lot of vulnerability research in recent weeks.

The trouble with trying to defend insecure by design protocols is that you can’t. CANBus is a protocol that cannot be done correctly….ever. I don’t think I’m being unreasonable in stating so. When considering defensive security methods there really is only one design that makes any sense.

CANBus protector is a system built on two separate pieces of hardware that use one-way communication to get information out of the “trusted” vehicle network. The only way I could see protecting the bus was to create a “server” which would sit on the actual CAN and perform the queries for vehicle information. This is done through standard set of OBDII queries made by the server (e.g. Vehicle Speed, RPM, VIN, etc). Future expansion of the project would include more queries and manufacturer specific information. The server then publishes that information via one-way communication to a “client”. The client sets up an entirely separate CANBus where any third party systems would sit. These third party systems requiring vehicle information would not be aware they are not speaking to the actual vehicle network.

This limits the attack surface by removing the risk of third party dongles plugged into a vehicle OBDII port. This device does not attempt to address the vehicle manufacturer itself attaching a network-enabled system directly to the CAN (which is happening). That particular action cannot be protected against except by vehicle manufacturers. The best one could hope for out of a third party solution is alert the user if a certain type of message is seen on the bus. If that message is malicious, however, it may be too late. It could certainly work as a “black box” of sorts though. Hmm…perhaps vehicle CAN forensics will be the next project.

If you are so inclined you can download the source and create your own CANBus Protector. The repo can be found at https://github.com/digitalbond/canbus-protector

Happy Hacking,
Corey

S4x15 CTF ICS Village Page

The Capture The Flag (CTF) contest in the ICS Village at S4x15 was a big hit. We have had numerous requests from attendees and those that heard about it for more information and data. So Stephen has put together a page of information. The page includes:

  • Examples of flags in each of the five categories
  • Packet captures with ICS protocol and attack data (the most requested item)
  • Screenshots of detected data and the scoreboard
  • Pictures from the ICS Village
  • An explanation of the event

You may also want to watch an interview with the team that won the CTF.

Great job by Stephen and the team of volunteers who put the CTF together and kept it running under three days of attacks. It puts a lot of pressure on the team to make it bigger and better for S4x16.

S4x15 Video: Ralph Langner or Cyber-Physical Attack Engineering

Ralph Langner presented at ICSage: ICS Cyber Weapons during S4x15 Week. As always Ralph is introducing new thoughts to push the industry forward, but this session is more on how to orient and organize the ICS communities’ thinking on attack / defense on ICS.

There is entirely too much attention paid to 0days and compromising an ICS computer or application. This is still trivial to do based on code quality and is almost always unnecessary. A more useful line of thinking is what would or could an attacker do with this access, what would be the intended result, and what can we do to defend against it.

  • At the 9 minute mark, Ralph discusses different types of ICS cyber-physical attacks.
  • At the 22 minute mark, he breaks down impact categories of cyber-physical attacks.
  • At the 29 minute mark, he discusses examples of how to identify the defensive controls to prevent catastrophic results.

The pull quote, in my opinion, was “is there any combination of bits and bytes that if I throw that at this plant will result in harmful physical effects? This is a question that can be answered through engineering methodology”.

Tool Release – Digital Bond CANBus-Utils

beagle-canbusI’d like to make a quick post with the release of some CANBus analysis tools I wrote.

The tools are written in javascript using nodejs, which comes preinstalled on the Beaglebone black — my hardware of choice when doing CAN analysis. I wrote up a brief README on setting up a Beaglebone for use as a CAN analyzer which you can find at https://github.com/digitalbond/canbus-beaglebone. That README provides links to the hardware setup we used at the S4 CANBus Hacking Class along with some instructions for getting an environment set up to start doing vehicle work in javascript or python.

For Version 0.1.0 we are releasing four tools:

  1. unique_ids: Watches the CAN and logs all IDs on which data is sent
  2. watch_id: Prints out every data packet for a given ID or IDs. Bytes that change from one packet to another are colorized for easy viewing.
  3. decode_obdii: Decodes any OBDII standard request/reply messages sent over the CANBus. Includes a description of the message type according to the spec.
  4. canbus_IDS: A very simple Intrusion Detection System for CANBus that will take configuration via JSON and alert on messages of a given ID. TODOs are extending alerts to use beaglebone GPIO and allowing ctype configuration of message data to alert on things like a bit flag being set.

The code is licensed under MIT. Contributions, questions, complaints are welcome. Check out the README file for more details on using the tools.

The repository is available at https://github.com/digitalbond/canbus-utils.

Happy Vehicle Hacking,
Corey

ISA99 – Safety and Security

book

ISA99 Working Group 7 has a draft document out entitled “Recommendations to align safety and security for industrial automation control systems“.

The document begins by noting the failed efforts to find a “mathematical coupling” between Safety Integrity Level (SIL) calculations and the Security Levels being developed by ISA99. This failure was not due to lack of effort. ISA99 struggled with this for years because the idea is so appealing.

The key part of the document is Section 1.2.

TG1 adopted Leveson’s technical approach[8] which uses the mitigation potential of the hazard as an estimator of, or surrogate for, likelihood for two reasons:

1) The potential for eliminating or controlling the hazard in the design or operations has a direct bearing on the likelihood of the hazard occurring.

2) Mitigation potential of the hazard can be determined before architecture or design is defined.

This is very similar to the Bryan Singer session at S4x15 and related to the Ralph Langner ICSage session (video will be up next week).  The basic concept is to identify the really bad things that can happen in a factory, pipeline, process, and then put in controls that cannot be hacked. These controls are not additional firewalls, application white listing, or other security products. They could be something as simple as a visual inspection before an action is taken or a safety control that cannot be altered via a network and doesn’t rely on data that can be maliciously altered via a cyber attack.

Safety engineering has done this for years with various forms of hazards analysis, but it did not take into consideration a malicious attacker. The good news is the number of really bad things that can happen in a process is smaller than you might think.

The document touches on the LOGIIC study of Safety Integrated Systems (SIS) with Control Systems. Some of the big vendors are pushing this integration, and the report lacked the courage to recommend against this integration. The report did admit that “greater integration may introduce greater risk“.

If you are interested in the Safety/Security approach moving forward in ISA99/IEC62443 it would be worthwhile to spend some time with the Leveson’s: Engineering a Safer World: Systems Thinking Applied to Safety (Engineering Systems).