In our previous post, we provided a few comments on Brian Snow’s insightful paper discussing information assurance. We wanted to address one more part. Mr. Snow makes the following comment regarding third party testing:

NIST (and NSA) provide third-party testing in the National Information Assurance Partnership Laboratories (NIAP labs), but Government certification programs will only be successful if users see the need for something other than vendor claims of adequacy or what I call “proof by emphatic assertion – Buy me, I’m good.”

This point is accurate, and an entire whitepaper could be written on this alone. FIPS 140 and Common Criteria (the latter of which is tested via NIAP labs referenced by Mr. Snow) provide a third-party validation methodology to support the vendor’s claims of adequacy/goodness. These programs scrutinize a vendor’s configuration management procedures, test plans, cryptographic algorithms, and other facets of product design and implementation against a specified set of functional and assurance requirements.

These certifications are required for security products deployed in Federal systems, but very rarely are certified products deployed and run in the evaluated configuration. If the end users don’t see beyond the fact that a security product either has or doesn’t have a certificate and they don’t use certified products as they should, then there is one rather poignant result: FIPS 140 and Common Criteria will continue to be substantially (and sadly) a procurement checkmark.*

So why does a product vendor pursue certifications?

1. To sell products to the Federal Government
2. To promote a commitment to process and to security assurance
3. To remain competitive in the marketplace

Incidentally, these same three reasons provide context to a product vendor’s advertising campaign. Product vendors should advertise their certifications (as Microsoft recently did). After all, the rigor of these programs requires a tremendous amount of monetary and resource commitment, and certifications help sell products.

But what is the net benefit to all stakeholders if end users only consider certifications at procurement and do not deploy the product in its evaluated configuration? This is challenging to answer in a simple blog post. From the end user perspective, purchasing certified products not only meets government procurement requirements but also provides some mitigation of risk.** From the vendor perspective, a sale will not (typically) be lost because of a lack of certification. They’ve met the ante, and now the features/performance/scalability/interoperability must meet the end user requirements to get the purchase order. Also, a vendor does receive a very tangible benefit of third-party review of design documentation and security function implementation, which can often result in advancing development processes and fixing bugs (or more drastic flaws such as poor cryptographic algorithm and key management implementations). So a vendor qualifies to sell a product, and hopefully the address shortcomings in their development lifecycle. Those are good things.

Paraphrasing Snow’s comment above, FIPS 140 and Common Criteria will be successful if end users understand the purpose and value of these certification programs. Common Criteria and FIPS 140 are complicated, and we are not suggesting that end users become proficient in the specific processes and requirements of these programs. We do feel that a better understanding of the meaning and value of these programs in the end user community will push us past the “procurement checkmark” stigma tied to these programs.

To help foster this success, Apex Assurance Group has services to advance the mission and understanding of Information Assurance in the end-user community.

*Disclaimer - Not all end users view FIPS 140 and Common Criteria as a procurement checkmark, and some do run the evaluated configurations. These generalizations are derived from years of talking directly with procurement officials and end users in DoD, Intelligence, and Civilian operations.

** More to be written on this subject in a future post.