Duplicity, Ineffectiveness & Challenge Pass/Fail

CoDeSysReid Wightman of Digital Bond Labs presented Vulnerability Inheritance in ICS at S4xJapan, and he posted the video and a technical article yesterday. I’d like to weigh in on the duplicity of 3S, the ineffectiveness of ICS-CERT, and the challenge passed and failed by integrators.

What Reid showed clearly in his presentation, and in the tools he released, is that the six categories of Version 2 vulnerabilities had not been fixed in Version 3. All that CoDeSys did was modify the software slightly so the previous tools did not work.

Here is a simple analogy. Imagine Version 2 was a door that had no lock. All a burglar had to do was turn the doorknob clockwise and the door opened. Rather than putting a lock on the door in Version 3, CoDeSys simply made it so the door would not open if you turned the doorknob clockwise. But if you turned the doorknob counterclockwise, the door opens. What was needed was security, a lock on the door, rather than some trick.

I don’t know what else to say about 3S/CoDeSys except they have done their vendor customers and the end users a major disservice by saying Version 3 fixes the security problems. At least Festo was honest when they said they were not going to fix the vulns.

————

ICS-CERT is a vendor megaphone, little more. I know they should expect a forthright answer from the vendor, but is it too much for them to ask a couple of questions on how the vulnerabilities were fixed? These are vulnerabilities that affect 100′s of vendor products.

DHS touts the number of vulnerabilities they have handled as a measure of their value and effectiveness. They add little or no useful information to other disclosures, and they don’t perform even basic evaluation of the information.

It is hard to see any benefit to the DHS/ICS-CERT role in disclosure. Close down shop and move the resources to something more useful. My recommendations for years now is for ICS-CERT to ignore 95% of the vulns and to a great job providing value information on the 5% they deem important.

————

Which leaves the vendors that integrated the CoDeSys software. We are aware of two vendors that looked at the 3S fixes in Version 3, realized they didn’t address the security problems, and built their own protection into the integrated product. A great example of internal Red Teams and SDL doing its job.

The examples of Hitachi and Sanyo-Denki that Reid used at S4xJapan are the case where the vendor did not adequately test third party software that is integrated into their product. Hopefully this will be a learning experience for the CoDeSys customer base.

All ICS vendors are going to have security issues. The important point in evaluating ICS vendor security is how they fix identified problems, and the root cause of the problem in the development lifecycle.

Image by Lyn Matthews

S4x15 Theme & Other ICS Security Events

ICS Security Event

Registration for S4x15 Week will open this Thursday, and be ready if you want to get one of the 50 lowest cost tickets to the event.

We are still working on the one word theme for the event. Some of the leading contenders are Advance, Beyond, and Push. I’ve seen the session abstracts and it is going to be a novel and exciting event, a significant leap forward in the ICS security research community. The gap between S4 and other ICS security events has grown significantly over the last three years and S4x15 will extend that even further. In fact, the technical research and discussions at S4 are going so far beyond the standard ICS security event that it is almost unrecognizable that they are all in the same general category ICS security events

This is not a negative comment on SANS, ICSJWG, WeissCon and the international events. There is still a need to provide basic ICS security education and awareness to a huge portion of the ICS community. In fact, the number of people who need one of these traditional and excellent events is 100x or even 1000x the number of people who need an event like S4.

The problem is the top researchers and thought leaders in this space need to continue to push forward. I guess we could worry about getting too far ahead, outrunning the supply lines. However if we have an event that is accessible and understandable to the newcomer to ICS security, or even an advanced beginner or intermediate, it is worthless to the leaders in the ICSsec space. The S4 target attendee is the type that has long outgrown the other ICSsec events.

A very brief history of recent S4 conferences:

  • S4x12 was Project Basecamp (Insecure By Design), Stuxnet Deep Dive (Detailed discussion of first ICS cyber weapon) and the first session on Internet connected ICS. It opened a lot of fronts and took off the gloves.
  • S4x13 was ICS Exploitapolooza. There was session after session showing a pathetically insecure ICS application or device and watch the speaker exploit it. We had over 50 0days at the event. It brought a number of new researchers into the space, but the point was beaten to death for the S4 audience. This was a turning point.
  • S4x14 was a big step forward. ICS low-hanging fruit exploits were banned. Novel attack techniques for ICS and a greater exploration of what an attacker would do post exploit were the highlights. Some big names in security research stepped into the ICS realm. Plus we moved up to the ballroom, added OTDay, ICS Village, and ICSage: ICS Cyber Weapons as well as a lot more fun at the social events.

So what is in store for the main two days of S4x15? It is a continuation of what was hinted at and started at S4x14. The focus is on the engineering and automation aspects of attacking and defending ICS. We have some great session on simulation for analysis and defense, some novel attack techniques, basically things that you will not see anywhere else. … and there will be triangles.

We have said from the first S4x07 that this event is not for everyone. If you want to discuss OT vs IT or information sharing or what some government agency is doing, go to one of the other great events. If you want a lot of technical meat, new concepts and to mingle with best minds in the ICS security space you should grab a ticket for S4x15.

Friday News & Notes

SCADA Security NewsThe biggest story of the week … we may have the 3rd example of malware targeting ICS. Kyle Wilhoit and Jim Gogolinski of Trend Micro write about Sandworm attacking GE Cimplicity HMI. Interesting pull quote, “As further proof of the malware targeting CIMPILICITY, it drops files into the CIMPLICITY installation directory using the %CIMPATH% environment variable on the victim machines.” These directories are likely excluded in anti-virus deployments.

Digital Bond held the first S4xJapan in Tokyo this week. We will be posting the presentations on Monday and the video over the next two weeks. It was great to see some strong sessions from Japanese researchers, and we were particularly impressed by the graduate students at the Nagoya Institute of Technology. The Dynamic Zoning sessions could be one of the best defensive ideas to come to ICS in a while.

ISA acquired the Automation.com site. The terms of the acquisition were not disclosed in the press release. Walt Boyes, a veteran of the automation press and all things ISA, thinks this is a great move. I’m hesitant to disagree with Walt, but I’m not sure what this says about ISA. Automation.com publishes thinly veiled, if not blatant, vendor advertising disguised as articles and newsletters. At least they are honest about the advertorial. “As you know the most successful marketing campaigns include a combination of editorial, brand recognition and lead generation components. We look forward to working with you and your team on compelling editorial features, as well as integrated marketing campaigns.” My favorite example was when Automation.com insisted that Siemens responded well to Stuxnet even though they lied about fixing the problem. ISA will now be even more motivated to curry favor with vendors rather than provide honest information for the user community.

Billy Rios has started Laconicly, a team of “Building Automation Systems and Internet of Things Risk Management Professionals”. They are also selling a building automation system enumeration product or service called Soteria. Good luck in the new venture Billy.

Protocol Differential Analysis

helix_nigel_brownThe term Protocol Differential Analysis needs to make Google as an infosec technique.  I first heard the term from esSOBi at Indianapolis’ Circle City Con.  I first encountered the trick, though, in a research lab a few years before: a quick and dirty tool was written by a colleague there to help analyze a very, very bizarre serial protocol.

The problem, briefly explained, is this: as an attacker, we want to find out what interesting packets are in a conversation between a controller and its engineering software. For example, we want to find out what packet represents the ‘Stop CPU’ command in a proprietary protocol.  Since the protocol is undocumented, we are left either reverse engineering the master application, which can be extremely time-consuming, or analyzing the protocol stream itself to find the interesting packets.

Protocol analysis is often the easier path.  Unfortunately, industrial proprietary protocols are extremely ‘chatty.’ Based upon the classic industrial poll-response model, the protocols may be sending tens or even hundreds of packets per second back-and-forth between the PC software and the industrial controller. By the time we interact with the software and click the ‘Stop CPU,’ button on a graphical interface, we may have thousands of packets to dig through.  We want to find the packets that are interesting, but end up wading in a river, looking for the raindrop that holds the key to an attack.
Read More

Friday News & Notes

Letter FWurldtech announced the Achilles Industrial Firewall. It was hard to understand why GE purchased Wurldtech for their protocol testing, but if they were purchasing this product it begins to makes sense. The pricing for the perimeter model starts at $30K and the field model starts at $6K. This is significantly more than competitor products, not to mention non-industrial firewalls that are about 1/10 this price. The first release has some deep packet inspection for Modbus, DNP3 and OPC Classic, awaiting more details on this.

Mandiant announced an ICS Gap Assessment service. Not a lot of detail and not a big surprise given they had hired a handful of experts. Still my guess is this is a sidelight to the main goal of adding ICS expertise to the incident response service that Mandiant is known for. Many of the largest companies in the US and world own and operate ICS.

This week was the semi-annual, fall meeting of DHS’s ICSJWG in Idaho Falls, ID. There were between 140 and 160 attendees with half attending for the first time. Spy reports say the agenda was solid, but not much new from past events. It’s a reasonable free event for newcomers to ICSsec to attend, and there is probably a place for that.

S4x15 Registration Info

october23

S4x15 registration will open at noon EDT on October 23rd. Registering early will not only guarantee you a spot at the event, it will also save you some money.

We have kept the price for the two-day S4 event at $995 since the first S4 in 2007. We even added a third day, Operations Technology Day (OTDay), last year and kept the $995 price. This year there is a small price increase … unless you are in the first group of registrants.

We will be selling tickets for the main two-day S4x15 sessions in blocks of 50:

  • First 50 tickets will cost $995
  • Tickets 51-100 will cost $1,095
  • Tickets 101 -150 will cost $1,195
  • Tickets 151-capacity will cost $1,295

All tickets will include OTDay on Tuesday at no extra charge. The Friday events for S4 Week (ICSage: ICS Cyber Weapons conference or advanced ICS security training) will cost $600 as in past years.

All this may be a bit confusing, but it will be clear on the registration site. The key is if you are a humble researcher, independent consultant, student or work for a company that is hard to get training funds, register early. Capacity is 190 attendees, but that includes about 20 speakers who get in for free.

The agenda is both great and novel this year. I’ll write up a blog next Monday on the theme and characterize the talks. If you are tired of the same old talks at other ICS security events, you will love this agenda.

We are full up on the 30 and 45 minute sessions on the S4x15 agenda, except for saving a space for some late breaking, amazing research. We are still looking for two or three 15 minute sessions with very fresh content.

OTDay will have full two full tracks this year. One track is already full with a number of potential sessions being worked on / recruited for the second track. The challenge is we are not allowing vendor sessions at OTDay. Instead we are getting owner/operators to discuss what worked, lessons learned and practical applications of OT. OT is more than security. It is how to deploy and maintain a robust, mission critical system. So vendors find a good customer and asset owners, here is a way to attend S4 Week for free.

The agenda for ICSage – ICS Cyber Weapon is about 75% complete. I’m really pleased with how this event has matured in it’s second year. I actually believe that ICSage sessions will generate the most news in S4 Week. For the last 25% we are hunting for historical, economic, political theorist and other non-technical sessions.

Friday News & Notes

SCADA Security NewsThe US Food and Drug Administration (FDA) published Content of Premarket Submissions for Management of Cybersecurity in Medical Devices. We haven’t had time to read it yet, but take a look at Patrick Coyle’s analysis. Pull quote, “Interestingly, in this section the FDA specifically abdicates responsibility for cybersecurity system updates, noting that: ‘The FDA typically will not need to review or approve medical device software changes made solely to strengthen cybersecurity.’”

Oops. Bloomberg reporter Jordan Robertson, who has written good articles on ICSsec, was led astray on ICS honeypot data by ThreatStream. Chattanooga appearing so high on the list should have been a red flag. This is a great cautionary tale with CSO covering the analysis flaws. ThreatStream made matters worse with “The scans were on tcp port 102 and the requests were mostly protocol compliant. Siemens utilizes port 102 … We are not familiar with other services that use this port.”  ICCP, other iso-tsap …

Bob Radvanovsky and the Project Shine team have posted a paper showing the results of their search for Internet connected ICS devices. Great work by this volunteer team. It raised awareness for a lot of asset owners to look and pull these connections. It may also have encouraged John Matherly to add ICS scanning capabilities to Shodan. It is now so fast that Shodan has integrated and scanned for ICS devices within days of a Project Redpoint release.

If you want more on Internet connected PLC’s, read Distinguishing Internet-Facing ICS Devices Using PLC Programming Information by Paul Williams at AFIT.

Stephen Hilt’s presentation from DerbyCon on Project Redpoint is up on YouTube.

On October 11th Altamira is running a CTF called Scram Hackathon 2.0. The goal is to cause a nuclear power plant scram, emergency shutdown. (ht: Paul Asadoorian’s Security Weekly)

A near complete agenda is now up for the ICS Cyber Security Conference, Oct 20-23 in Atlanta, GA. Can we call it WeissCon for one more year even though Joe sold the event?

ISA99 Co-Chair Eric Cosman put together all of the work the committee has done on ICS cyber security. Eric wrote “the sum-total of our work to date, weighs in at slightly less than 900 letter sized pages, with a file size of just over 20MB.”

SSI Software and Technology acquired 60% of S21sec. S21sec is one of the largest ICS security consultancies in Spain, and perhaps in Europe. Schneider Electric is also a minority shareholder.

ARC Advisory Group continues to promote anytime, anywhere, any device control of an ICS. The latest is in their work with/for ICONICS mobile app. “While this is largely driven by the new Millennial generation of workers, most stakeholders are beginning to embrace smartphones, tablets, ‘phablets,’ and other mobile devices to access manufacturing processes, information and intelligence at any time from any location with wireless or cellular access.”

S4x15 CFP Ends Oct 1

OLYMPUS DIGITAL CAMERAThe clock is ticking to get your session proposal in for S4x15 Week. Take a look at the full CFP and get it in by October 1.

We don’t just wait for the CFP responses. We actively chase down researchers and topics. So if you see something that is S4-worthy please send us an email.

I’ll take it a step further. If you have any idea for a S4 session, a Great Debate topic, onstage interviewee or proven good practice for OTDay, send us the idea and we will find the right speaker.

One other S4x15 Week note … we will have a slight increase in prices, our first since 2007. So the best way to get in for free is to present a great session. We also will have group pricing and the first 50 registrants will see no price increase. More on registration on October 1 after we finalize the agenda.

SCADA & Me in Japanese at S4xJapan

We have been working with author Rob Lee and the very helpful Richard Stiennon to translate SCADA and Me – a book for children and management into Japanese. Attendees at our S4xJapan, Oct 14-15 in Tokyo, will receive a free copy of this fun book. It’s being printed now, so enjoy a few of the galleys below (click on a picture to see full page in more detail).

We also have the full agenda translated into Japanese now with the very kind help of Mai Kiuchi of the Cyber Defense Institute. Kiuchi-san will be assisting with the Q&A portion of S4xJapan as she is fluent in English, Japanese and ICS security.

S4xJapan Registration Open

Hanko

The agenda is up and registration is open for the first S4xJapan, Oct 14-15 in Tokyo. There is space for 100 people so register now to get your spot.

Tuesday, October 14th is Operations Technology day (OTDay). Attendees will learn proven techniques to run a reliable and secure ICS. There will be sessions on virtualization in ICS, unidirectional gateways, wireless on the plant floor and more. I will have a session that shows how to use assessment tools on an ICS in production without causing an operational impact and obtaining maximum information.

We are proud to announce that the Kaspersky Industrial Protection Simulation (KIPS) will take place as the last session of OTDay. Kaspersky graciously has translated KIPS into Japanese and will have a team of native Japanese speakers to lead everyone through the simulation. KIPS has received great reviews at ICS events, and we are pleased to bring it to Japan for the first time.

After KIPS is open stay and enjoy food and drink and connect with fellow ICS security professionals at the S4xJapan social event. Also enjoy a great view of Tokyo at night from the 49th Floor of the Mori Building in Roppongi where S4xJapan is being held.

Wednesday is the main day of S4xJapan where we move from good security practice to leading and bleeding edge ICS security research. They include a variety of perspectives:

  • Offensive – Reid Wightman’s session on Vulnerability Inheritance where he will show examples of third party software integrations leading to compromise in Japanese ICS
  • Defensive - Wataru Machii’s session on dynamic zoning based on operational mode
  • Intelligence – Chris Sistrunk and Kyle Wilhoit showing new data from observed ICS attack techniques
  • Education – Learn in detail what Havex does to ICS applications and devices
  • Tragedy – See the survey of Internet accessible ICS applications and devices in Japan

We will be previewing many of the sessions and other S4xJapan information on the digitalbond.com and digitalbond.jp sites.

Some of the delay in opening the registration is we have been working hard to make this a Japanese event. Of course there will be simultaneous translation English/Japanese or Japanese/English as necessary, and we searched hard to find the best individual translators with security and technical knowledge. But is more than just that:

  • approximately half of the sessions are in Japanese, and the other half are in English.
  • the presentation slides will have key content translated. I saw this at one JPCERT session, and it was even more helpful than the simultaneous translations.
  • an ICS security expert fluent in Japanese and English will handle the Q&A. Q&A translation of technical questions is a common failure.
  • we have an Internet based Q&A engine so attendees can ask questions anonymously if desired.
  • The Kaspersky KIPS is in Japanese.
  • more surprises to come.

If you have any questions or difficulties registering contact us at s4@digitalbond.com.