Part 3 - Achilles Certification Levels

With Part 1 and Part 2 as background, we are finally able to describe the various Achilles Certification levels, timetables and publishing of the Achilles Certified Controllers.

Level 1

Level 1 Controller Certification is the base level certification and covers the common protocols in layers 2 to 4 in the OSI stack. These include Ethernet, ARP, IP, ICMP, TCP and UDP. There are over 30 million tests in level 1, and each test can consist of multiple packets. A failure of a single test causes the controller to fail the certification test. So a controller that passes Level 1 testing has proven a significant degree of security and reliability that is not found in the typical controller today.

In addition to the layer 2 to 4 Achilles testing, Level 1 Certification also includes a Nessus and nmap scan. Nessus and nmap are two of the most frequently used scanning tools by both IT Departments and hackers. The purpose of these scans is to determine if the controller under test can maintain proper operation while being scanned by these tools. There have been many examples where a controller has crashed during a well meaning scan by IT Department staff.

+ Control Protocols

You may have noticed that no control system specific protocols were included in Level 1. They were not included in Level 1 because controllers support a variety of control protocols. It would not be fair to compare a DNP3 implementation to an Ethernet/IP implementation. However testing these control protocols is essential because they are much less likely to have undergone the same level of testing as an Ethernet or IP stack.

Achilles currently offers three Controller + certifications: standard Modbus/TCP, standard DNP3/IP, and the proprietary Vnet/IP. Test cases for additional control system protocols are under development and will be released in stages throughout 2007 and 2008.

A controller that has passed the Modbus/TCP test case family will be certified as Controller Level 1 + Modbus/TCP. An asset owner considering a new controller should look for a model that passed the core certification + the control system protocols they will be using.

Schedule

The Level 1 Certification test cases and procedure were set in January 2007 and Achilles Controller Certification testing began on February 1. There are a number of products that have already achieved Level 1 Certification, but Certified Controllers will not be named until May 2007. This delay is to prevent any one company from having the only Achilles Certified Controller, and to give the early adopter vendors an opportunity to be in the first set of Certified Controllers. That said, I’m thrilled by the May date and see this as a big step forward for the community.

Work has begun on a Level 2 Controller Certification with a goal of specifying the test cases in 2007 and certification in early 2008. We are looking to include more complex storms (denial of service) testing at layers 2 to 4 and to cover the protocols commonly used to manage controllers such as ftp, telnet, http(s), snmp, and ssh. We would like feedback on what protocols should be covered in Level 2 so please either comment on the blog or send me an email.

In talking with some of the asset owners who have been long time supporters of Achilles and have required Achilles testing as part of their procurement process, it was interesting to hear that they would require Level 1 certification for most installations and perhaps Level 2 certification for their more critical installations.

Publishing and Publicizing Achilles Certifications

A list of Controllers passing the various Achilles Certifications Levels and + control protocols will be published on the Wurldtech site. The vendor, model, and firmware version tested will be included on the site. A MD5 hash of any modified configuration file required to achieve certification will also be included on the site. An example of the level of detail of the certification information that will be made public is shown in the screen shot below.

Results will be published only for Controllers that have passed the Achilles Certification test. Failed test results will remain strictly between Wurldtech and the vendor or entity that submitted the Controller for testing.

An awareness and recognition program is being developed so asset owners, vendors and others in the community will understand what Achilles Certification means and what controllers have achieved Achilles Certification.

Next: Early Feedback and Questions For You

Digital Bond is a Wurldtech Partner