Achilles Controller Certification - Part 4 of 4
Early Feedback and Questions For You
We had a great opportunity to get some feedback on the Achilles Certification Program at PCSF. Here are a couple of points:
How to handle the situation where a protocol is not present or disabled?
For example, some controllers may not support ARP which is in the Level 1 Certification. Our initial plan was to put a N/A with an explanation that N/A means the protocol was not present. The ARP tests still would be run, but there is no protocol to test.
A more interesting case is in the management protocols. As an example, let’s say a controller supported ftp, http and telnet. However the vendor decided their http implementation was weak or turned off by default. This is a case where N/A could be misleading. So we are looking to differentiate between protocol not present in the controller and protocol present in the controller but disabled in the test configuration.
Many of the asset owners requested that all protocols in the box be tested for certification, and this would be my recommendation to a vendor. In the end, it is the vendor’s decision on what is submitted for certification testing.
Possible Confusion with the + Control Protocol Certifications
A number of the attendees did not like the term Level 1 + Control Protocol such as Level 1 + Modbus TCP. They felt it might be misleading. A controller with Level 1 + Modbus/TCP and Vnet/IP (a proprietary protocol) would not necessarily be better than a controller certified as Level 1 + Modbus/TCP that did not support Vnet/IP.
From a pure linguistic viewpoint the + is accurate because it is Level 1 and additional tests. And we do want asset owners to look at the Achilles Certification and say we require Level 1 + the control protocols we use. That said, enough people have had the same comment so we are looking for alternative terms or presentation structure for the +.
Neither of the two issues above affect the testing. They affect the public presentation of the certification results, which is extremely important.
There are a number of areas we would like to get your feedback.
- Any suggestions on the two issues above.
- What protocols or other tests should be included in Level 2?
- Where should we focus the Achilles Certification awareness plan? What trade publications, industry or vendor events, etc.
- What would it take for you, as an asset owner, to include Achilles Certification in your next RFI or RFP?
You can comment on the blog for all to read or send comments directly to me at peterson@digitalbond.com.
The first Achilles Certified Controllers will be announced in May, and there will be controllers from multiple vendors in this first announcement.
Digital Bond is a Wurldtech Partner
Author: Dale Peterson
Posted: March 16th, 2007 under Assessment Tools.
Comments: 4
Comments
Comment from Ron Southworth
Time: March 18, 2007, 9:02 am
Hi Dale,
The question of how you describe certification is a difficult one to answer in my opinion and would be a difficult one to put down into a benchmark system and to then place some level of certification. Given that the testing of devices is not going to be published until the device has passed as a certificate completing level 1 or completing level 2 should be about the amount of rigorthe device has passed more so than what
Comment from Ron Southworth
Time: March 18, 2007, 9:27 am
protocols were or were not present as a certificate title. I think that the certificate documentation is going to have to contain the details of how the device was configured and what protocols the device supported in that configuration. It is conceivable that the device will behave differently when fully optioned to when it has a base level of configuration. As the testbed to me is about a very important aspect of quality assurance testing the mere fact that a vendor uses the system as a means to validate and improve their product is going to make the product more attractive than another one that has not been through the process.
I would target the likes of the ISA and similar peak industry organizations. Magazines like control global and similar but I would also be targeting end user conferences to raise knowledge of the environment to end users and let the users of the device product market drive the demand.
As far as RFP and RFI I have been asking vendors if they have sought out testing of their product by this system for quite some time now. It will certainly be a question that would be raised at some point in a any selection process I am to be involved with.
I suspect however that as this is not too far away in my case that most if not all the likely candidates will have yet to have completed the process so it would not be an avenue to pursue in the short term but the long term it could be a very effective differentiator.
Hope this is of some Help Dale.
Comment from Terry Martin
Time: April 25, 2007, 4:15 pm
Certification implies conformance, and conformance implies a standard. The IEC 61508 standard for functional safety directly addresses the deficiency left by the “non-network” exception of the UL ANSI 1998 standard for software in programmable components, and in fact UL appears to have adopted the IEC direction.
Organizations that comply to UL 1998, either voluntarily or compelled by regulation should have a great interest in Achilles as a tool in performing a functional safety assessment, and in Wurldtech as an independent entity qualified to assist in the performance of such an assessment.
Perhaps a news release from UL? Why should TUV Rheinland have all the fun?…
On second thought, maybe TUV could use an Achilles box…
As could contractors like Booz Allen who operate Common Criteria certification labs…
http://www.niap-ccevs.org/cc-scheme/testing_labs.cfm
Comment from Terry Martin
Time: April 26, 2007, 9:40 am
Safety.
It’s the dominant consideration that drives everything in security. A “marketing through education” approach directed at the relationship of certification to safety is the most leveraged entry point, in my mind.
If we follow that line of thought we get to the safety of safety sytems themselves. Programmable controllers are being deployed to automate the operation of safety systems. In a production environment the threat from the environment to human safety is now complicated by the threat from the safety systems themselves, introduced by networked controllers.
They should probably be known to do what they are supposed to, and only what they are supposed to, when they are supposed to. That requires verification. Verification implies certification.
That’s about as targeted as it gets.
Write a comment