Digital Bond

For Secure & Robust ICS

ICSJWG Needs A Refresh

by 1 Comment

icsjwg-generic-bannerI attended my first ICSJWG since 2011 last week in Indianapolis. It was an ok event with some interesting talks and a chance to reconnect with familiar faces in the ICS industry. It is however a far cry from the must attend DHS event back when it was called PCSF. I rate a few other similar events, such as WeissCon and the SANS Summits, as much better. The main thing ICSJWG has going for it in its current form is the price — it’s free.

This is disappointing because there is a place for a premier US government event in ICSsec. Below are recommendations for the next ICSJWG.

1. Have the best, can’t miss government sessions

a) ICSJWG should be the event where DHS and other USG organizations make the most important, can’t miss ICSsec announcements of the year. I don’t believe there was any news at ICSJWG … and little or no press.

b) Throw in a big government name speaker each day. The Indy ICSJWG had Governor Mike Pence, DHS Assistant Director Touhill and NCCIC’s Larry Zelvin. This hit the mark and should be continued. These presentations often lack new information, but the audience likes to see the names.

c) It also should be the event where the government explains in detail the most important programs. The Indy ICSJWG had a big miss on the NIST Cybersecurity Framework, arguably the most important USG ICS initiative. A NIST representative read a dry speech to the audience that included little new or helpful information. The DHS speaker on the subject was a no show so she read his speech as well. An attendee could reasonably draw the incorrect conclusion that now that the document is out the effort is over.

ICS vendors, asset owners, consultants and other ICS security professionals should feel ICSJWG is where important USG information will be revealed and explained in detail. This is the most important and easiest improvement for DHS to make. (And just to be clear, this does not mean more presentations explaining the bureaucratic organization structures in the USG)

2. Hold a professional event

This is a hard criticism, because I know some of the organizers worked hard on ICSJWG Indy. If that truly is the best that can be done due to USG limitations then don’t hold ICSJWG.

  • Publish the agenda earlier, two months before the event
  • Have some basic refreshments at the breaks. There is no coffee or drinks or snacks or even tap water at the breaks. This may seem petty, and a fancy lunch or party is not necessary, but it’s common courtesy and a bit embarrassing that there are not basic refreshments.
  • Find a quality venue. The main auditorium had a very poor projector, strangely inconsistent air con, and no power for laptops. The break out rooms had problems as well. I know they like to move ICSJWG around, but perhaps they should stick with a quality government venue in DC. The possibility of holding the next ICSJWG in Idaho Falls would be another big mistake. (The Indy area was great and well received; walk to everything)

3. Only one ICSJWG event each year

Based on the agenda, there is not enough content to hold two events a year, and they would be better served placing all the effort on one quality event each year. It also would draw a bigger crowd and more buzz.

4. Something special

There should be something new each year. The classified briefing may have qualified this year. I don’t have a clearance so I’m not sure if it provided helpful and new information, but it is something that other events could not offer.

I’m rooting for ICSJWG. With all the advantages they have, and admittedly some bureaucratic challenges, it should be a great event and an important way to move forward the public / private partnership that is often touted as being so important. If it is no longer a priority and can’t be significantly improved it should end.

My ICSJWG Prezi

by 8 Comments

Cloud Computing

I had finished my presentation on a wide variety of topics Big Data / Cloud Computing / Internet of Things / ICS remote access, and the Q&A had started. After stressing in the presentation that ICS data can be shared anywhere without jeopardizing the integrity and availability of the ICS, but non-emergency remote access to critical infrastructure ICS must not be allowed, I got the question that illustrates the challenge in making progress in ICS security.

Paraphrasing the question … “What you recommend is impossible, especially for the next generation of workers that expect to be able to make changes to the plant from their basement on their iPhone. Given that prohibiting remote access is impossible, what is your recommendation to secure it?”

IT’S A TRAP!!! and one that I refuse to play along with. The depressing thing was looking out at the audience I could tell that a large portion, a majority?, agreed with the questioner. An audience of vendors, asset owners, consultants, government officials et al that are looked at to define ICS security thought that it is inevitable and acceptable that critical infrastructure will be controlled from phones, tablets and laptops anywhere, anytime as a regular occurrence.

This is one of the reasons I have significantly reduced the number of ICS security events I attend and speak at. If the ICS security community was going to force change and solve this problem it would have happened by now. Change is going to come from outside the ICS security community or not occur until a very sad and tragic event or two happen. And this is not something I’m willing to wait for.

There were a number of supportive attendees who came up after the presentation. And please don’t misunderstand, I welcome disagreement on a presentation or solution (see Darren Highfill’s S4x14 Unsolicited Response), but not surrender. It is also important to note that there are a number of critical infrastructure asset owners that are doing, and are committed to continuing, what the questioner said was impossible.

This is one of many areas that the US Government and DHS could take leadership if they choose to. The DHS response to the insecure by design problem was not to focus on this as an issue that must be fixed. Instead DHS took the position that insecure by design would not be considered a vulnerability worthy of an ICS alert or advisory. It would have been surprising, but refreshing, to have someone from DHS push back hard on the inevitability of anytime/anywhere critical infrastructure remote control comment and say this should not be an option in critical infrastructure.

Attendees and others interested can see my Prezi online at this link. Admittedly, the picture based Prezi is a bit harder to understand unless you were there or the entire script is included.

Given this was a DHS event, I thought it only appropriate to focus on ICS that monitor and control the critical infrastructure. So after quickly dismissing the Internet of Things, with an interesting WEIS statistic, the bulk of the presentation used the GE On Site Monitoring / Atlanta Data Highway as an example.

Monitoring 1800 power generation systems in 60 countries is a great example of the promise and benefits of Big Data / Cloud Computing. It also is a big, fat, high value target. Does this mean that critical infrastructure ICS should avoid these types of services? Absolutely not. Just push the data to them so the integrity and availability of the ICS is not at risk.

Does Software as a Service (SaaS), e.g. an HMI in the cloud, have a place in ICS? While SaaS has no place in a critical infrastructure ICS, you can make an argument that an HMI in the cloud might be lower risk for a small municipal water utility than a completed neglected ICS with a weak security perimeter.

Tomorrow I’ll write about the rest of the ICSJWG event.

ICSJWG in Review

by Leave a Comment

Here me, Hear me.The ICSJWG meeting was this past week in Denver, and the schedule was packed with great presentations, and speakers with a wealth of experience to share with the ICS community.  There was a significant bump in attendance this time around. Attendees were from a mixed bag of industries and sectors, we had vendors and owners, oil, gas, electric, manufacturing, and the usual faces from the ICS community. This was my first time attending, and enjoyed myself thoroughly.

This year’s keynote was given by Billy Rios, of Spearpoint Security. Billy is known for a take no prisoners approach to security, demonstrated by the 1,000s of bugs he and business partner Terry McCorkle have reported over the past year to ICS-CERT. The keynote was an exploration into the mind and motivations of security researchers, why they spends hours of personal time meticulously analyzing commercial products for vulnerabilities, why many make use of public disclosure as a tool to get issues fixed, and touching on the controversial exploit market as well . The major point the community debated on during week was that vulnerability research is both an established business and a means to promote a brand. In this business is market pressure to sell vulnerabilities to other ‘interested parties’, and often these transactions won’t reach the light of day, meaning the product stays vulnerable. This contrasts with the promotion of a brand, where researchers can establish their credentials and capabilities by publicly disclosing issues, and get issues fixed. A good keynote generates controversy and discussion over the entire conference, and Billy’s had that effect.

With three separate tracks each presentation hour, I didn’t have the opportunity to see every presentation. However, one of the more interesting presentations I was able to attend was “ICS Challenges in Naval Surface Combatants”, given by two US Navy professionals. The presentation talked about improvements in new classes of warship, and how automation was providing a significant benefit in naval control systems. Acquisition and certification of the naval ICS is a challenge the US Navy will face in it’s new warships, and cyber security is a large part of those challenges.

Panel discussion for Day 1 discussed interoperable standards for ICS security. Each panelist was given time to present components of their research, and then opened to the floor for questions. I got involved in the discussion at one point, regarding a developing proposal for interconnecting new and existing control systems over common carrier. I’m all in favor, as this could provide a significant cost savings for many types of critical infrastructure. It obviously has security implications as well, but my main focus was on ensuring that connected control systems would have a guaranteed level of service, so that there would be a reasonable expectation of appropriate latency, message delivery, and timeliness. Different control systems have different requirements, and it would be detrimental to use a common carrier without ensuring communication requirements could be met to a certain standards. The communication standards for different types of control systems would need to be addressed, otherwise owners could consider the approach too great a risk. Cost savings alone merit further study here.

Day 2 started out with a lively presentation by Mark Fabro, where he discussed how our normal use of probabilistic risk was not appropriate to the cyber domain, and how using a capabilities model would be more representative of the threat environment. Mark also advocated the use of attack trees when modeling how attackers would interrupt your process. The attack tree discussion was a fascinating dive into the mind of an attacker, and even had some references to the much maligned “Live Free or Die Hard”. Joel Langill, the SCADAhacker, followed Mark with a solid 101 networking presentation for the more automation minded, the main theme being how basic networking practices can give security benefits.

Day 2’s panel was “Separation or Unification”, which discussed the implications, and fallacies, around the separation of control systems from the rest of the world. I would discuss in more detail, but most of my attendance was towards the tail end, which consisted of a lot of binary discussion. Binary, in the sense that something was either wrong and should be removed completely from ICS, or right, and should be used everywhere. I would welcome comments from other attendees regarding this panel.

There was also this guy named Michael Toecker that gave a talk on the Microsoft Attack Surface Analyzer tool.

Ed: Fixed incomplete statement regarding exploit market 10/19 @ 10:00

 

Pick Your Fall ICS Security Conference

by Leave a Comment

ICS Security ConferenceRemember S4 Call For Papers/Presentations Closes This Friday

September / October is a busy week for ICS security events. Joe Weiss just posted the full agenda for ICS Cyber-Security Conference the week of October 22nd in Norfolk, VA (called WEIScon by many). The week prior to this DHS holds their semi-annual ICSJWG Event in Denver. These are both quality events, especially for those new to ICS security, that draw a good crowd. The later point is important because the people you meet and hallway discussions are a big part of an event.

So if you have to pick one, which should it be? First, have you been to either event recently. If yes, pick the other event. While there is new content all the time, the approach and culture of the event changes little year over year. Also you will meet new people at the other event. There is some crowd overlap, but not that much.

ICSJWG is much more government and organizational structure focused, as you would expect from a DHS event, especially in the plenary sessions. If you want to talk to or connect with DHS, ICS-CERT, National Labs, and some other government officials this is the place to go.

WEIScon is the most traditional and conservative of the ICS security events. There is an abundance of caution about security affecting the process, and Joe is adamant on focusing on anything cyber that could affect the process not just a cyber attack. I always recommend WEIScon to those coming from the IT security world who want to learn about the ICS culture.

[Read more…]

3 More ICS Vulnerability Handling Success Stories

by 4 Comments

SCADA VulnerabilitiesA lot’s happening this week in ICS vulnerability handling and a lot of it is positive.

1. ICS-CERT Takes Control

I have been critical in the past of ICS-CERT’s letting vendors determine when a vulnerability is disclosed. They have changed their policy.

UPDATE!  In cases where a vendor is unresponsive, or will not establish a reasonable timeframe for remediation, ICS-CERT may disclose vulnerabilities 45 days after the initial contact is made, regardless of the existence or availability of patches or workarounds from affected vendors.

This new policy gives ICS-CERT the flexibility that the myriad of disclosure cases require. They don’t have to disclose in 45 days, and there are many cases where it wouldn’t make sense. For example if there is no exploit in the wild, it is not an easily identified vulnerability, there are not could compensating controls and the vendor is diligently working on a patch or other fix.

Good job ICS-CERT. Now use this discretion if the vendors aren’t doing the right thing.

2. The ICSJWG Vendor Subgroup Releases Common ICS Vulnerability Disclosure Framework

Rob McComber, Ernie Rakaczky, Bryan Owen and a bunch of other respected vendor security representatives have been working on this for a while, and it’s great to see it out there to educate other ICS vendors. While there are a few items I disagree with, especially in the customer discovery section, the vendors didn’t evade responsibility for disclosing embarrassing info. The best example is this quote from Section 4.2.2:

For exceptionally high risk vulnerabilities which expose customers to significant threats, disclosure of an internally discovered vulnerability is highly recommended even in situations where a resolution is not available.

ICS vendors should definitely check out Section 6 which describes the elements in a vendor vulnerability disclosure policy.

Now that this is out it should get some publicity at the ICSJWG fall meeting, recorded webinar and provided to every ICS vendor who gets drawn into the vulnerability disclosure process by ICS-CERT.

3. More Examples of Vendors Self Reporting

[Read more…]

ICSJWG Agenda Is Out

by Leave a Comment

SCADA Security ConferenceThe ICSJWG Spring Meeting Agenda for May 8-9 in Savannah, Georgia is now posted. Only fair that we give this a prominent mention after tweaking them on the lack of an agenda last Friday.

The highlight to me is the Security Responsibilities of the Control System Vendor Panel that closes the event. Mike Assante is the moderator and Marcus Braendle, Rob McComber, Ernie Rakaczky, and Graham Speake represent the vendors. These are vendor reps that have driven significant security improvement in their companies. Yes, that actually has happened in some companies, especially in the server and workstation ICS components.

Also of selfish interest is the Key Take-Aways from Digital Bond’s S4 and Project Basecamp Panel. A bit funny since Reid’s PLC mitigation talk was not selected, but no complaining. Conference organizers should have full latitude to put forward the agenda they want, and they obviously felt having others discuss the impact was the better way to go.

A panel on Information Sharing and Analysis Centers (ISACs) got the prime spot in the first morning. So far ISAC organizations are forming the panel in this preliminary agenda. DHS should add some contrary opinions to the panel, or it could devolve into explanations of administrative actions. Personally I’ve given up on these information sharing efforts because the ROI on time spent has been extremely low, but it still is of interest in the ICS security space so the right panelists could be a popular session.

Day 2 kicks off with a case study of the Springfield, Illinois water system non-hack hysteria. Great move by DHS if it highlights what worked, what went wrong and what has changed.

[Read more…]

DHS ICSJWG Creates Roadmap of Roadmaps

by 1 Comment

ICS Security RoadmapThe Energy Sector Cyber Security Roadmap developed by the US Dept of Energy was well received when it first came out in 2006 and was recently revised. Other sectors saw this and it has led to a Water Sector Roadmap, Chemical Sector Roadmap and various other sector roadmap efforts. It is yet to be seen if these other roadmaps will have an impact. A key factor will be the funding and programs the government and sector organizations but behind the goals highlighted in the roadmap as the Dept of Energy did. (Full disclosure, Digital Bond’s Bandolier and Portaledge were funded by Dept of Energy to help address some of the roadmap goals.)

With all this roadmap mania, DHS decided a roadmap of roadmaps was needed. Eventually it became known as the Cross Sector Roadmap for Cybersecurity of Control Systems. When I first heard this at a 2010 ICSJWG meeting I must admit I tweeted away with a bit of derision. So going into the reading I was skeptical of the value. The document exceeded my expectations.

The document is definitely more accurately titled as a “Cross Sector Roadmap” than a roadmap of roadmap. There are three goals listed in the 47-page document:

  1. Measure and assess security posture
  2. Develop and integrate protective solutions
  3. Detect intrusion and implement response strategies

It doesn’t get much more vanilla than that, and if the document stopped there it would be a wasted effort. But it starts to get interesting when they list 2, 5 and 10 year milestones for each of these three goals. Some of the milestones are general such as “implementation of new protective tools and appropriate training”. Some of the milestones are more specific such as “development of training for control room operators in identifying and reporting unusual events, breaches, and anomalies from a cyber event”.

The real payoff is in Table 1 beginning on page 4-2. This table defines specifically how success will be measured on a scale of 1-5 for seven specific measurements.

  1. CSET adoption and use
  2. ISAC or CERT connection
  3. Employed certified professionals or accredited systems
  4. Use of the procurement language
  5. Mandatory security awareness training
  6. Implemented security standards
  7. Implemented incident response planning

For each of these measures 25% or less = 5, 25% – 50% = 4, 50%-75% = 3, … And the idea is there would be a periodic survey of each sector to measure progress. The likelihood and practicality of this effort can be debated, but credit is due for actually coming out with a clear way to measure progress.

[Read more…]

ICSJWG Day 2 and Summary

by 1 Comment

ICSJWGPrevious blog entries have covered Day 1 and the Vulnerability Disclosure Panel. Here is a bit of news from Day 2 and summary thoughts.

Summary Thoughts

  • DHS puts on a quality event both in the organization and agenda. It’s definitely worth attending if you haven’t before, and the price is right (free).
  • I’m not sure what ICSJWG is now. What is the theme or purpose of the event? I think it is an opportunity for the government to explain to industry what they are doing in ICS security and to foster the “public/private partnership”. Personally I find the government process presentations to be dull and not particularly helpful to the community, and I confirmed this general impression with some of the usual suspects.The best part of ICSJWG is always the break out sessions, but these are not in line with the theme. PCSF struggled with this same problem and finally just gave in and tried to make PCSF the harmonic convergence where every group involved in ICS security came. You went there to get updated on what had happened over the last year and see everybody.
  • The audience is very much the usual suspects. Mostly government types, niche consultants and vendor reps. Quality people who know the topic well. Some owner/operators but mainly the people that attend a lot of these events. It would be interesting to find out the percentage of owner/operators and percentage of new owner/operator attendees.
  • This year I attended both the spring and fall ICSJWG; one a year is enough mainly because of the previous bullet. It’s great to see and catch up with everyone, but the limited new attendees and new information doesn’t warrant twice a year attendance.
  • There was a very large Japanese contingent at ICSJWG this time. Even a film crew from NHK (think BBC of Japan). I’ll check with my NHK friends to see when it will be available online.
  • I get a lot of blog ideas and future stories from ICSJWG.

Day 1

I attended three talks besides the vuln disclosure panel on Day 1.

[Read more…]

Disclosure Panel at ICSJWG

by 1 Comment

The reason I attended ICSJWG was I had the surprising opportunity to participate in a vulnerability disclosure panel. Surprising because DHS knew I was likely to be quite critical of certain vendors and ICS-CERT.

The panelists had ten minutes for a presentation then it was open discussions. The major points in my presentation [pdf] were:

  • Talking about responsible disclosure is a waste of time. Stop doing it. The person who finds the vulnerability will do whatever they want, and every person has unique experience, motivations and self-interest. I gave Project Basecamp as the example. Everyone in the room could agree on the responsible disclosure process, but it would not matter. Digital Bond and the Basecamp team, as the finder of the vulns, decides how we will disclose the vulns. (Not to ICS-CERT in advance because no need for coordination help, advance notice to some consulting clients, a small portion coordinated with vendors, and most disclosed at S4 along with enumeration tools and exploit modules).
  • ALL EFFORT should be on “Effective Disclosure” – giving owner/operators of ICS the information to understand the details and impact of the vulnerability, mitigations (if available) and compensating controls so the ICS customer can determine how to address the change in risk.
  • ICS vendors have the primary responsibility for providing honest, forthright and clear information to their customers. ICS-CERT has two roles in Effective Disclosure. 1) Use their bigger megaphone to get the information out to those ICS users and support organizations the vendor may not reach. And 2) provide honest, forthright and clear information about the vuln when the vendor does not. (ICS-CERT has met #1 and failed on #2)

Check out the presentation (pdf) to see examples of negative examples from Siemens, positive examples from Rockwell Automation, and both positive and negative examples of ICS-CERT. The key is if the vendor is providing the owner/operator with the information needed and often hinge on being forthright and clear. I will try to record the voice track for this presentation.

[Read more…]

ICSJWG Day One

by 2 Comments

ICS SecurityI arrived at ICSJWG after the DHS intro and keynote, but attended all the rest and a fair amount of hallway chatter. Here are the highlights.

FBI Unclasssified Cyber Threat Briefing

The FBI spoke in very vague terms, but had two important statements:

  1. “Protect the US against cyber based attacks” is the FBI’s 3rd item on the mission statement. “Protection of critical infrastructure” is the top priority in the Cyber Division.
  2. “Reporting indicates shift to ICS”. FBI is seeing questions being asked about control systems post Stuxnet. The speaker also stated that terrorism groups are recruiting for control system expertise. Not surprisingly there were no details.

The FBI talked about two tracks they pursue, the criminal track and the national security track. The goal on the criminal track is to successfully prosecute the criminal. There will be discoverable evidence; court documents will eventually be unsealed. This is not in line with the goal of most owner/operators. The National Security track avoids most of these problems and with DHS/ICS-CERT’s assistance can help the owner operator. What I need to find out is how an owner/operator can say they the criminal track is out of bounds when they call FBI.

FBI really pushed the partnership between FBI and DHS/ICS-CERT. They have done multiple responses to asset owner security incidents in conjunction with ICS-CERT.

Behavior Based IDS for ICS

Joel Langill, @scadahacker gave a demo and presentation on mapping out current communications, actually creating a truth table by monitoring a switch span port with Wireshark. He used SMART then to go through the Wireshark data and map out the communications.

[Read more…]

About Us

Digital Bond was founded in 1998 and performed our first control system security assessment in the year 2000. Over the last sixteen years we have helped many asset owners and vendors improve the security and reliability of their ICS, and our S4 events are an opportunity for technical experts and thought leaders to connect and move the ICS community forward.

Recent Comments