I am very happy to announce that Corey Thuen will be joining Digital Bond Labs as a researcher and consultant. Long-time followers of Digital Bond and the S4 conference will know Corey as co-creator of, ”SCADA from Scratch,” a project he started with Ken Shaw using off-the-shelf embedded tools to create a secure-by-design control system (this tool is the subject of a recent article in Forbes, as well as an upcoming Kickstarter). He has also proctored Idaho National Labs’ much-lauded Red/Blue Training, and co-taught a protocol fuzz-testing class with Adam Crain at S4x14.
Corey brings us a ton of experience in protocol analysis, fuzz-testing, and building and testing control systems. We could not be happier to have him on our team.
Kaspersky issued a research report on Havex they called Energetic Bear – Crouching Yeti after the threat actor. It’s probably worth it’s own post and worth reading but here are three highlights.
On page 15 (HT: Damiano Bolzoni) they describe the Network Scanning Module that looks for much more than OPC servers. It is scanning for Modbus (502), Siemens S7 (102), EtherNet/IP (44818) and ports for two proprietary ICS vendor protocols. Much like Stuxnet, I expect we are just starting to learn what Havex’s ICS capabilities are. Is it asking too much for DHS/INL to actually perform research and inform the community? It’s understandable, after the fact, why they didn’t research Stuxnet, but this is only the second piece of public ICS malware. Stop sending fly away teams for telnet password cracking attacks and other corporate network exploits and use that pricey ICSsec expertise developed over the last decade.
Kaspersky identifies the Swiss company, Mesa Imaging. This is what we were told and is very helpful for identifying the target. Mesa Imaging is not an ICS vendor. So what company or country sector is using eWon, MB Connect and Mesa Imaging products? That is the best clue so far for who the threat actor was targeting with that phase of the attack.
Kaspersky states there is not enough data to identify the Crouching Yeti threat actor. Some have pointed the finger at Russia, but I’d agree that there is not dispositive evidence in the public at this time.
Belden announced Tofino 2.0 this week. Lot’s of good technical info surrounding the obligatory marketing hype in Eric’s blog entry. I want to dig into the technical details more in a future article and perhaps podcast. The EtherNet/IP Deep Packet Inspection had to be a bear to write, and I’m looking forward to running it through some use cases. When are we going to see this technology integrated into a PLC Ethernet module?
Take note of the latest ICS-CERT advisory from the Crain/Sistrunk
DNP3 Telegyr 8979 master fuzzing. This one is related to a SUBNET Solutions product. Most important is a 13-word sentence: “SUBNET had also determined the root issue was in the GPT software library“. The GPT software library was sold by ASE to a number of ICS vendors that now have a latent, remotely exploitable vuln that is available to all. Shouldn’t ICS-CERT be disclosing these vendor names so affected electric utilities can take action? This could be a difficult fix because ASE is no longer selling the GPT/Protocol Pak, but SUBNET found a way.
All software needs to be part of your security patching program … including your security software. Latest example is a new 0day in Symantec Endpoint Protection (SEP). It’s also why you keep your attack surface as small as possible.
Cisco takes your IoT and raises it to Internet of Everything (IoE). They announced partnerships with Rockwell Automation and Yokogawa to support “risk management and compliance for industrial control environments.” It’s one of those press releases that is hard to digest. “It addresses risks using a combination of people, process and technology.” The best I can tell is Cisco will be helping Rockwell Automation and Yokogawa design Cisco products into their ICS.
Graham Speake made the move from Yokogawa to NexDefense. NexDefense is another Mike Assante venture that is trying to commercialize the INL Sophia product. Mike also heads up the SANS ICS security program, and Graham is a SANS ICS security instructor as well as longtime active participant in ISA99. Good luck Graham.
EMET 5.0 is now available from Microsoft. This latest version adds Attack Surface Reduction and Export Address Table Filtering Plus. EMET has gotten some traction in the ICS space, especially for vendors that seem to have little concern for security or fixing identified vulns.
You can get some great tools at reasonable prices by backing the right Kickstarter campaigns. We will be taking the RFIDIer out to an assessment next week, and we should be soon getting our HackRF’s. Look for soon to follow reviews so you can consider them for your toolkit.
Image by ChrisInPlymouth
I was talking a while ago to Justin Engler, a friend who also happens to be a really talented web app and mobile app security researcher, about the popping-up of ICS management software for mobile devices. He theorized that mobile apps for ICS would be an interesting place to watch for bugs nearly three years ago. Dale’s recent ICSJWG Q&A over mobile device security gave me a little motivation to dig into some sample apps and see how the field actually looks. The results highlight some of the issues that your organization will run into if and when you decide to adopt mobile.
The focus of this post is not just application security. While there are a few specific vulnerable applications mentioned, I think that the big lessons should be ones of architecture and integration challenges. The current lot of ICS management apps pay little mind to securing access or preventing bad operation. Even an app with ‘secure’ on its product homepage may leave you wide open.
I decided to pick on Android simply because my only jailbroken iOS device at the moment is so terribly destroyed from years of abuse that installing new apps is a nonstarter. There also seems to be more interesting control systems apps for Android at the moment.
A quick survey of Google’s Play store for terms such as ‘SCADA’’, ‘PLC’, and ‘OPC’ turns up a few applications worth checking out. Unfortunately there are no apps that I could find which do what Dale prescribes: obtain safe, accurate, remote, ‘read-only’ access to control system data. Doing so will require a lot of backend work on your part.
Let’s take a look at two interesting vulnerable applications.
You are pounded with the message: ICS security is different than IT security. The fact is the Operations Technology (OT) in an ICS is a mission critical / high value IT system and needs to be treated like one. Don’t let the ICS is different argument allow you to accept fragility and insecurity in your OT.
The trigger for this entry is Bob Huba’s article in ISA’s Intech: Top Ten Differences Between ICS and IT Cybersecurity. The article is well written and accurate in comparing ICS and IT Desktop management, but that is the wrong comparison. ICS should be compared to mission critical / high value IT where companies can tell you how much money they lose, and it’s often a big number, for every minute of downtime.
There are plenty of mission critical IT systems that have availability requirements that rival ICS (1: Security Objectives). They have a better availability strategy than the too often ICS approach of make it redundant and don’t touch it if it is working. This is software and hardware that must be managed and supported to meet the availability requirements and not devolve into fragility.
Bob’s 8: Untested Software applies to both high value IT and OT. The change management around both includes rigorous testing, phased deployment, rollback strategy, etc. If you load up a patch or upgrade in high value IT and bring something down, you will see a similar management reaction to an ICS outage. Clearly the physical danger in a lack of integrity or availability is a factor in ICS, but do you honestly believe that a large financial impact is not treated with a similar level of concern to a business?
#9: Patching and #10: Security Inconveniences are ICS excuses and actually make the ICS more insecure and fragile than the high value IT. An ICS needs to be designed with a plan to support and maintain the mission critical IT system that it is. Would you put in physical equipment and not consider what maintenance will be required to the physical equipment to support reliable operation? I always point to Langner’s Robust Control System Networks as the book to hand an engineer who needs to understand why a fragile ICS is a problem.
The #3: Network Topology for ICS actually sounds like high value IT systems. #2 Network Segmentation is a partially valid difference. Some high value IT systems are Internet accessible, others are not. The high value IT systems will use DMZ’s and least privilege rulesets that quite frankly are much more restrictive than the average ICS firewall ruleset.
#6: User Accounts really is not a valid difference between IT and ICS, except that ICS too often accepts poor identification. Role based authorization is common in both as are other authorization methods and principles. Authorization has traditionally been an area of strength for ICS, and increasingly this is being performed in Active Directory.
There are a few areas of genuine ICS and High Value IT difference that Bob points out:
- High value IT does not typically have Safety Instrumented Systems (#7: SIS)
- The Purdue model does not apply to IT (#4: Functional Modeling), but you could argue that they have similar data flow and responsibility models with different names.
- Many of the #5: Physical Components are unique to ICS … although you are seeing some of these physical components in a Data Center.
I agree with Bob that if you treat ICS HMI, EWS, Historians, Real Time Servers, PLC’s, RTU’s … like desktop user PC’s you will cause some serious problems, but this is a flawed comparison.
There is a lot the ICS community can learn and emulate from Mission Critical / High Value IT in designing, deploying and maintaining a robust OT environment. We should be leveraging their knowledge and processes rather than pushing them away. And certainly we shouldn’t continue down the path of install and don’t maintain because we didn’t plan to support the OT and don’t know how.
This is not to say that High Value IT is perfect. There are systems running with uncertainty and fragility and just hoping things won’t break … a similarity with many critical infrastructure ICS.
Image by Dan McKay
After the PG&E substation shooting, FERC had ordered NERC, as the ERO, to develop and submit a Physical Security Reliability Standard within a very short time frame for this type of work. NERC complied and now FERC says they will approve the standard with two changes. FERC wants the ability to add or remove facilities from the critical facilities list. While they say this would be “exercised only rarely”, this is a crack in the door or slippery slope that is likely to give utilities heartburn. FERC also wants to replace “widespread instability” with “instability”. There needs to be an adjective in front of instability.
Critical Intelligence is holding a one day conference and two days of training called CounterIntel, Sept 16-18 in Park City, UT. The two day training is to help you be a more effective Cyber Intelligence Analyst, and the whole event is limited to owner/operators. Living in the Park City area, I can tell you it is a great time to hold a conference here.
Read the Kyle Wilhoit of FireEye article on how Havex enumerates OPC Servers. Great work.
The automobile sector has started the Auto Information Sharing and Analysis Center (Auto-ISAC). ISAC’s have a very mixed record based, but it seems every sector will have one.
Image by ChrisinPlymouth (the F king)
The S4x15 Week Call for Papers/Presentations is now out.
Send us your session ideas asap to have the best chance of getting on the agenda. All we need is a short description and time requirement mailed to firstname.lastname@example.org.
We are calling it S4x15 Week now because it goes Tuesday – Friday (Jan 13-16 in Miami Beach):
- Tuesday – OTDay and ICS Village Opens
- Wednesday – Day 1 of S4x15
- Thursday – Day 2 of S4x15
- Friday – ICSage:ICS Cyberweapons and Advanced Topics ICS Security Training
The CFP gives more detail on each day and the type of sessions we are looking for.
I wish that we could sit back and wait for all the great sessions to come in, but history has shown that we need to hunt for this great work and unknown talent. If you see or hear about anything that we should chase for S4x15 week, please let us know.
Last year was a big step forward for the ICS security community. We moved past low hanging fruit; we brought in some top security researchers from outside the ICS space; and there was a new focus on what an attacker would do after successful exploit.
We are looking forward to seeing some new and amazing work for S4x15.
Way back at the Spring 2014 ICSJWG meeting, Dale announced that Digital Bond is opening a new division — Digital Bond Labs. This week, we are officially opened for business…and we are hiring.
Digital Bond has a long reputation for building the tools that other ICS consultants use ten years down the road. It seems that every other talk given in the ICS security community lately make reference to Digital Bond’s intrusion detection signatures, Nessus audit files, or Project Basecamp exploit demos.
At Digital Bond Labs, we aim to provide the best in the business at breaking control systems software, security add-ons, and access control systems for both end-users and vendors. Our goal is simple: break all the things that make all the things so that we can rebuild them to be more robust and more secure. In the Digital Bond tradition, we will also continue to focus on valuable research to share with the ICS community.
We are working with Robert M. Lee and his publisher to get SCADA and Me in Japanese for a giveaway on OTDay of S4xJapan (agenda and registration open on Aug 4th). I wish I had the page above as a hidden slide to pull out at ICSJWG last month.
While most of my presentation involved the secure and insecure way to use the cloud in the future for analysis of process data, the most contentious point was on remote access. The easiest way to get into an ICS with a good security perimeter is to compromise an administrator, engineer or technician that has remote access to the ICS. The ICS Spear Phishing session at S4x13 showed how something as simple as a fake maintenance bulletin would have led to compromise of over 25% of the targeted users with remote access to the ICS.
Here’s the basic solution. Push the data out so the right people can view it without jeopardising the integrity and availability of the ICS. Have a physical disconnect for the remote access, and close the connection only in emergency situations following a defined process. Use your automation skills to put this capability on a display in the control room with the appropriate alarms and logging, and auto open after a time limit. If you are having multiple emergencies a week that require remote access your system is not under control or you are understaffed.
Someone in the audience, who actually is in the business of advising industry, pushed back hard at these limitations on remote access. Paraphrasing he said “c’mon we all know that this generation is going to demand and have remote access with a control and admin capability from their smart phone in their basement. What do you recommend to secure this?” This is when I needed the SCADA and Me page. “If you can control it from a phone — so can Bad People.”
Lior Frenkel of Waterfall said something after my session that I told him I’ll be stealing from now on. “You’re part of the critical infrastructure. Act like it!”
Give eWON some credit. They released information that their website was compromised for a short time in January, and issued an updated notice late last week on their home page. Still nothing on the MB Connect or Swiss vendor site to tell customers they may be compromised by Havex if they downloaded and ran their software. Companies are going to have security incidents; customers should be looking and considering how they respond.
Alstom Grid has a new product coming out in response to the PG&E substation shootings called e-terrasheriff. It will detect and report gunshots at unmanned substations, and presumedly integrate this into the e-terra SCADA displays.
DHS has opened the CFP for the ICSJWG Fall Meeting. After attending and speaking at the Spring Meeting I was going to pass on this one, but holding it in Idaho Falls will dampen attendance.
The first release of Automotive Grade Linux is out. “Each component includes a detailed Design Requirements Document (DRD) with descriptions, use cases, HMI flows, graphical assets, architecture diagrams and more.”
We have always appreciated the Swedish contingent that has supported S4 since the start. Now they have created their own event, 4SICS, Oct 22-23 in Stockholm. They are working on the agenda, but they already have some great technical ICS talent in Europe lined up. Lueders, Santamarta, Hjelmvik, …
DHS is looking for a lead and partners for their Critical Infrastructure Resilience Center of Excellence (CIRC). “Each COE is led by a U.S. college or university and has multiple partners. COE partners include other academic institutions, private industry, DHS components, DOE National Laboratories and other Federally-Funded Research and Development Centers (FFRDCs), other federal agencies that have homeland security-relevant missions, state/local/tribal governments, and first responder organizations.”
A Honeywell help wanted ad is illustrative of how ICS vendors are trying to generate revenue from cyber security. “This position will be responsible for leading, managing and growing the Honeywell Process Solutions (HPS) industrial cyber security global remote managed services business.”
IETF has an initial draft standard out for “Two Way Authentication for IoT“.
Thanks Dan for the tip.
First a reading tip to save you time. Most of the 840 pages are weekly reports from the DHS Control System Security Program (CSSP). There is a ton of repetition as each week’s report carries forward all of the previous week’s items. So go straight to page 750 and you will see the reports going backwards from 19-23 Nov 2007 to 22-26 Jan 2007.
The most interesting excerpt is from the 12-16 March 2007 report:
The CSSP large scale validation test of a significant control systems vulnerability (Pandora) was successfully completed at the Idaho National Laboratory on March 4, 2007. Results and findings from the test are being documented and significant follow-on activities are anticipated. The Tiger Team formed to coordinate activities for this vulnerability will meet on March 13. to U/S Foresman is scheduled to be briefed the afternoon of March 13. Briefings to Secretary Chertoff, House Homeland Security Committee and White House Homeland Security Council are anticipated.
After that Pandora entry there is no other mention of Pandora in the weekly reports. It evidently was classified and changed its name to Aurora. A meeting to discuss the technical details of the Aurora vulnerability appears next in the 19-23 Nov 2007 weekly report on page 751.
There were mentions of this “large scale validation test of a significant control system vulnerability (Pandora)” in the weekly reports prior to the test. A few other tidbits:
- It was a large scale test with an estimated cost of $2.8M (page 57).
- There are some good pictures of the physical site beginning on page 100.
- Pages 70 and 71 have some good examples of specific systems that could be affected by Aurora.
- There is mention of a Control System Malware Identification Team being formed by the CSSP back in Jan 2007 (page 233). Let’s put this team on Havex.
- A Firmware Upgrade Vulnerability report is discussed on page 165. I don’t remember this being issued, but it was seven years ago and DHS was calling these insecure by design features as vulnerabilities back then.
- The mention of CSSP working with JASON – an independent group of scientists that advises the USG and particularly intelligence is interesting, especially back in 2007. Stu….
- The mitigation strategy memos start on page 36. The early briefing milestones were met, but little else after that seems to have been accomplished and much of the detail on what was to be done is redacted. They do show a plan for software and hardware fixes being developed and deployed within two to three years.
- The technical team memo on page 821 is worth a read.
It’s been seven years since that turbine shook and the smoke came out, yet I always thought Aurora was a lost opportunity.
The real beauty of the Aurora demonstration was it clearly showed that a cyber attack could affect a physical process. The specific vulnerability they chose to achieve this, while not unimportant, was not the main point to take from Aurora. It was an effective and dramatic demonstration.
Aurora should have led a massive DHS and US Government push to address the insecure by design ICS that run the critical infrastructure. Instead of taking this and leading a massive PR and bully pulpit campaign building off of this expensive but effective demonstration, people lost their jobs because the video and secret got out.
Perhaps the idea of physical damage through a cyber attack struck too close to Stuxnet, or maybe it didn’t have the internal support and program to leverage the successful demo. Whatever the reason it was a lost opportunity.
I knew it was lost during the Congressional Hearings. Senators and Representatives asked the august panel from DHS, NERC, utilities, etc. if the Aurora problem had been fixed. Rather than use the question to pivot and highlight Aurora is a small symptom of the larger problem, the experts would go into the plan in place to address Aurora.
I can’t end this long post without a nod to my friend Joe Weiss. He has beating the Aurora drum harder and longer than anyone else. Perhaps this will give him more ammunition for his cause. It is difficult to reconcile Pandora being called “a significant control system vulnerability”, being classified, resulting in all those briefings, tiger teams, remediation plans, … and the relatively small expenditure and effort to address the “Aurora vulnerability”.
and just in case you want to see the video again: