Bandolier - - Take 3
There has been some talk on Bandolier on mailing lists and blogs, and it is clear that we have not done a good enough job describing what Bandolier will do and what Bandolier will not do. Actually, a number of these discussions have been helpful in understanding the best way to describe Bandolier to a broad audience. So let me try again.
Bandolier audit templates, in conjunction with a scanner such as Nessus, allow asset owners to determine if their HMI, Historians, or other control system applications on workstations and servers are in their optimal security configuration.
How do we determine the optimal security configuration for Vendor A’s Historian Version 2.3 running on Windows 2003 Server?
Digital Bond security researchers work with the engineers or development team at Vendor A and one or more asset owners that are using that Historian to determine the most secure configuration of the operating system [2003 Server in this example], supporting applications on the server [perhaps a database in this example], and the actual control system application [the Historian app in this example].
We typically start with NIST and CIS best security practice recommendations for the OS and IT apps and modify these hundreds of settings if necessary for the control system app to work properly. We then identify all possible security parameters in the control system application and the optimal security settings for these parameters. We have been calling this optimal security configuration the “Gold Standard”.
Not all “Gold Standards” are equally secure or hardened. Some vendors have done more in terms of integrating their app with a hardened version of the OS. Some vendors have done more in terms of adding security features to their control system app. Each audit template for a particular product is simply the best you can do for that product.
Once the team identifies the Gold Standard, Digital Bond has scripts and inspection methods to extract all of the security settings. We create an audit template that will work with Nessus. Nessus will create a secure admin connection to the computer under test and identify any variances with the Gold Standard.
We believe asset owners will use Bandolier audit templates and scanners to:
- Verify the vendor or integrator has installed the applications in a secure manner for new systems. In fact, we believe vendors and integrators will use the Bandolier templates as part of their deployment process.
- Periodically test whether the security settings have degraded.
On Tuesday Jason will post the initial 12 systems applications that have audit templates in process. The first of these templates will be available in July. You will see a focus on newer systems that can be secured. There is little value in auditing an older system that can not be configured securely - - why would we want to verify an insecure config?
Pricing is part of the good news here. The audit templates will be available for site subscribers, only $100 per year. We are also allowing vendors that participate in the process to distribute the audit files applicable to their system through their support channels, so it is likely to be free to customers with a support contract.
Digital Bond’s approach to research is to identify projects that have an extremely high probability of technical success in the near term and will have a significant impact on the control system community. If we can’t see a solution from start to finish, we typically will not propose or work on the project. Digital Bond is more of a high OBP singles or doubles hitter rather than a home run slugger.
There have been some comments that Bandolier will not identify the risk of a misconfiguration to an overall control system since it is only testing one component or even more broadly identify and prioritize risks to the control system. Exactly correct; we never purported that it would. It will simply allow asset owners to determine if an individual HMI, Historian, Control Server, OPC server, … is in an optimal security configuration.
Addressing the overall system issues is a dramatically harder problem. We have proven successful methodologies to do that on the consulting side, as do Industrial Defender, Byres, Lofty Perch, Wurldtech, and others. There are so many variables in systems, process, protocols, architecture, OS, apps, industries, individual business drivers, culture, … that we do not see a path to success to turn this into a tool. We have not identified a way that meets our criteria of high probability of technical success in the near term.
It is an exceedingly hard problem that some are trying to address. CS2SAT tried to walk asset owners through some system analysis. Clint Bodungen at CIDG has a project that has a system approach and includes considerations of regulatory and compliance issues. I’m sure others are working on it, but it is nice, juicy problem for someone else to tackle. If any readers or researchers are tackling this, we would like to track their efforts so get in touch with me.
Lastly my friend Joe decided to blog in detail about a private phone call we had last week [I'll forgive him since he got the facts mostly right :-)]. He is right we don’t see Bandolier being a good platform to extend to a system tool. It would be like saying I have a row of seats now let’s build an airplane around it.
The one correction I want to make is we do not see any system risk assessment or system analysis tool plugging into Bandolier. What I suggested is that if Joe or anyone else developed a system risk assessment tool we would gladly provide an interface to send the Bandolier results into that system tool. Bandolier results would be one of many data points a system tool could consider in its analysis.
Author: Dale Peterson
Posted: June 8th, 2008 under Assessment Tools, Bandolier, Calculating Risk.
Comments: 4
Comments
Comment from Ralph Langner
Time: June 8, 2008, 2:42 pm
Years ago, when we were pushing the risk assessment business, we started to work on an automated software tool to do the job. After all, risk assessment is a combination of network traffic + firewall analysis along with piles of structured interviews, so why not write a neat little tool to do the job in less time and with 100% performance all the time.
The more experienced we got, the more we recognized how limited such a tool would be. You do your job in a complex facility, and suddenly there are vulnerabilities popping up that strike you — but which wouldn’t have been caught by a software tool, as they are somewhat out of the ordinary, but still easily recognizable for the human expert.
So we quitted working on the software tool. We still brush up our methodology, but we believe that the good old eyeball and reasoning can’t be beaten in this business for quite some time.
Nevertheless, I think any solid basic component testing and rating, what Bandolier seems to do, is a good thing, and should be utilized. We can use any help we can get. However nobody should expect any wonders. What puzzles me is when I see organizations boasting about being certified after security standard XYZ with security tool ABC, and if you know the internals you know it’s mostly marketing with no hard facts about the actual security level in the plant. I hope that Bandolier and similar tools won’t be abused in such a fashion.
Besides… I still think it’s sad that Joe chooses to post his comments somewhere else. Let’s have some lively and controversial discussion at one place — we’re all working towards the same goal.
Comment from Matthew Franz
Time: June 8, 2008, 4:53 pm
Dale,
As usual your blog responses are far too charitable. One of my pet peeves is the sloppy use of the term “assessment tool” or and “system.” Both of these have can be used to mean a half-dozen things leading to folks talking past each other. So it even worse when folks use them together — especially when they don’t really understand the fundamentals of how vuln scanning tools (let alone firewalls) work.
Ralph,
My attempts to work on “risk assessment tools” were more in terms of automated the process of doing threat modeling and these, too, ended in failure 
If you are talking about a “plant” or “control center” I agree with you — but saying that a given host conforms to a configuration standard (either from a vendor or a reputable standards organization) I don’t really see that as marketing.
Of course the normal caveats apply: the best practices should have been validated and as with any technical assessment (meaning scan) it is obsolete by the time the assessment is complete.
I don’t there is a common goal, hehe 
= mdf
Comment from Jake Brodsky
Time: June 9, 2008, 9:39 am
This is where we get tripped up by language. When doing a risk assessment from an IT perspective, we tend to focus on the IT-ish things. We’re talking about the Computers, the Networks, the software, and so forth.
When we talk about risk assessment to an engineer, we’re looking at the process, the failure modes, the critical elements, the consequences, and so forth.
The problem is that both professions have overloaded the same term. When one says it, the other is thinking about something completely different. Perhaps we need to develop a common vocabulary for discussing this stuff and avoid such overloaded terms.
Comment from Marc Tritschler
Time: June 10, 2008, 9:00 am
A common vocabulary would be a good start. As I have commented before, we (the control systems security community) needs to speak a language which is better understood at corporate level, and has a clear association with corporate or enterprise risk management. If we don’t achieve this then control systems security risks are unlikely to become serious topics on the corporate agenda.
Write a comment