A lot to cover here so I’ll break this into four blog entries:

Part 1 - Why Protocol Stack Testing?

Part 2 - Testing Methodology and Coverage

Part 3 - Achilles Certification Levels

Part 4 - Early Feedback and Questions for You

Part 1 - Why Protocol Stack Testing

Achilles is a black box testing platform. For those new to testing, the term black box means the tester and tools have no internal knowledge of the device being tested. Achilles sends data to the device under test’s interface, evaluates the response, and verifies proper operation is maintained.

The initial Achilles Certification focuses on controllers - - PLC’s, RTU’s, IED’s and other field devices. Controllers have a bad history of falling over when any unexpected traffic is sent their way. We have seen this numerous times in assessments. In fact, many clients tell us to not even bother testing the controller because they know they will fail and have horror stories of broadcast or some other abnormal traffic causing problems in the past.

Simply stated - - the bar is set pretty low for controller security, and we see Achilles Certification as a near term way to significantly raise the bar.

Achilles Certification tests the protocol stack, such as Ethernet, TCP, HTTP, or Modbus TCP. A secure and reliable protocol stack is only one part of a secure implementation. The controller must be deployed with an appropriate security perimeter, support necessary security functions, be configured correctly, implement a least privilege methodology enforced by authentication and authorization, practice physical security and change control, and much more.

The factors in the previous sentence are what most of the security guideline and standards documents are attempting to specify. Look at NERC CIP, SP99 Part 4, IEEE P1686 and many others for examples. Once one or more standards are developed, with enough technical detail, it may be possible for a standards body to develop a certification program around these efforts. Developing these standards is not as easy as it seems as evidenced by the difficulty the IEEE P1686 team is having agreeing on a minimal set of security functions for IED’s. Probably the most promising technical standard / certification effort is ISA’s SP99 Part 4 and Automation Standards Compliance Institute, although both are in the early stages.

Even with proper configuration, change control, policy, … an attacker that can send a malformed packet that busts the protocol stack will be able to either crash or completely control of the device. This is why in the drawing below I have a secure and reliable protocol stack as the foundation in the many factors in device security.

Contributing Factors To Device Security

Developing a secure and reliable protocol stack is difficult, and performing quality assurance (QA) testing on the stack is very difficult. This is why controller vendors have been sending their products for Achilles testing for years now. Automated tools are needed to perform this QA, and there are a group of products going after this market.

Now put yourself in the asset owner’s shoes. How is the asset owner going to evaluate the security and reliability of the protocol stack? They could purchase expensive test tools and take man weeks to evaluate each the protocol stack in each potential product - - unlikely. They could evaluate each vendor’s QA program in this area. This is more likely, but still happens rarely and it is easy for a vendor to finesse these discussions if the asset owner is unwilling to spend many days reviewing QA records and results.

So an independent, third party certification of the protocol stack seems like a great first step to Digital Bond. It addresses an area with a history of serious security problems in controllers and other products, and it allows asset owners to focus on issues more under their control such as security features, configuration settings, policy and architecture.

I will go into more detail on the test cases for the various Achilles Controller Certification Levels in Part 3 as well as how an asset owner and vendor would use and benefit from the protocol stack Certification effort.

Digital Bond is a Wurldtech Partner