The Institute for Defense Analyses led a review of the National Information Assurance Partnership (NIAP) to uncover issues in the Common Criteria evaluation program and to provide recommendations for resolution. The IDA led a series of interviews and solicitations for feedback to the Common Criteria evaluation process, and at the present time the report is not available for public review (perhaps because it’s not complete, or perhaps because the project sponsors do not want the information released).

Here are a few things to keep in mind:

• Common Criteria is a methodology which is recognized globally. Therefore, issues found in the US program may also exist in other certificate producing nations because the process is meant to be similar across different schemes (hence Mutual Recognition).
• The two most prolific criticisms of the Common Criteria are:
  1. Evaluations take too long.
  2. Evaluations are too expensive.

  The Common Criteria is a complex, thorough evaluation methodology that is not part of most vendors’ development life cycles. While a product vendor can take steps to alleviate the time and cost of evaluations, AAG feels that these two points will not be easily mitigated unless the CC evaluation methodology changes significantly. And hopefully the IDA report will go beyond these two criticisms.

• The Common Criteria Community should realize that implementing changes recommended by IDA will not happen overnight. Agencies will continue to purchase and use evaluated products, and it’s doubtful that the IDA report will have any effect on that. But hopefully some gaps in the evaluation process can be tightened and the Community will have more awareness of Common Criteria once this report is releases and digested.

IDA certainly did its job in talking to a broad range of communities (including test labs, product vendors, and end users. AAG’s Managing Director also provided perceptions and insights to support the effort.).

Hopefully this report will be available soon for public consumption, and hopefully it will deliver the value that it has the potential to deliver.

Part of our mission is to help the IT industry (including the product vendor, systems integrator, and end-user communities) understand the meaning and value of Common Criteria, FIPS 140, and other government assurance vehicles. After reading many posts on TaoSecurity and Schneier’s blog regarding Microsoft’s latest Common Criteria evaluation, we have reached a very simple conclusion: the IT industry lacks awareness and understanding of the Common Criteria. We don’t mean this in a condescending manner- the CC is not an easy topic to digest. There are many misconceptions of what the Common Criteria actually is and what problems it attempts to solve. In this post, we are going to start addressing these misconceptions at the highest level:

What is the Common Criteria?

Here is our one-sentence answer: The Common Criteria is an evaluation methodology that provides assurance for security functions of an IT product.

A Target of Evaluation (e.g., what is being evaluated) is a set of product components that perform security functions and is defined by a set of boundaries and interfaces. The Target of Evaluation’s set of security functional requirements counter a series of threats and meet certain objectives. The threats, objectives, and security functional requirements may be specified in a Protection Profile (which can be considered as a requirements specification for a certain technology type) or may be addressed in a standalone Security Target. The Security Target is the cornerstone of the evaluation, as it delivers the threats, assumptions, objectives, and requirements for the Target of Evaluation.

A Common Criteria evaluation requires adherence to certain security assurance requirements, which become more stringent as the Evaluation Assurance Level increases. These assurance requirements address only how the TOE implements security functions and how it meets the security objectives. Moreover, a Common Criteria evaluation addresses only the security objectives and requirements listed in its respective Security Target; it does not implicitly test/evaluate all security functions of a product.

Clear as water? Or clear as mud?

We will address other common questions and issues related to Common Criteria and FIPS 140 in future posts.

See another comment from Apex Assurance Group on Schneier’s blog regarding the EAL4+ evaluation of Windows. Here is a quote of our comment with some additional comments following:

A Common Criteria evaluation (against CAPP or not) will not deliver a secure product. It will deliver a measure of confidence that the Target of Evaluation’s set of security functional requirements implemented counter a series of threats and complete certain objectives.

The Protection Profile is the embodiment of threats, assumptions, objectives, and requirements defined by an end user. And one thing people need to remember about the CAPP … Many government customers require compliance to that Protection Profile. Microsoft should not be blamed for pursuing a PP that may not be the end-all and be-all of operating system security. They are meeting (and exceeding, btw) the needs of their *many* end users who require a Common Criteria evaluated OS against the CAPP. The procurement policies stipulating adherence to the CAPP may not be the best solution, and one can make a safe bet that end users aren’t running in evaluated configuration anyway. But that’s an entirely different issue.

The CC can help a vendor create more secure software by defining a set of assurance measures (e.g., creating LLDs, detailed test plans, etc.) against security functions. Those who have been through CC @ EAL4+ know that it is not to be taken lightly. It is a very big deal, and MSFT should be proud to market their activity.

I wonder if similar dubious comments will be made when *nix platforms pursue this PP.

It’s important to note that Red Hat is being evaluated against the CAPP. There is another ongoing evaluation @ EAL4+ against the Labeled Security Protection Profile (LSPP) and Role Based Access Control Protection Profile (RBACPP). We truly wonder what the Internet community’s response will be once these evaluations complete.

Apex Assurance Group offered a few comments on the TaoSecurity post about Microsoft’s recent Common Criteria evaluations. See our comments here.

In our previous post, we provided a few comments on Brian Snow’s insightful paper discussing information assurance. We wanted to address one more part. Mr. Snow makes the following comment regarding third party testing:

NIST (and NSA) provide third-party testing in the National Information Assurance Partnership Laboratories (NIAP labs), but Government certification programs will only be successful if users see the need for something other than vendor claims of adequacy or what I call “proof by emphatic assertion – Buy me, I’m good.”

This point is accurate, and an entire whitepaper could be written on this alone. FIPS 140 and Common Criteria (the latter of which is tested via NIAP labs referenced by Mr. Snow) provide a third-party validation methodology to support the vendor’s claims of adequacy/goodness. These programs scrutinize a vendor’s configuration management procedures, test plans, cryptographic algorithms, and other facets of product design and implementation against a specified set of functional and assurance requirements.

These certifications are required for security products deployed in Federal systems, but very rarely are certified products deployed and run in the evaluated configuration. If the end users don’t see beyond the fact that a security product either has or doesn’t have a certificate and they don’t use certified products as they should, then there is one rather poignant result: FIPS 140 and Common Criteria will continue to be substantially (and sadly) a procurement checkmark.*

So why does a product vendor pursue certifications?

1. To sell products to the Federal Government
2. To promote a commitment to process and to security assurance
3. To remain competitive in the marketplace

Incidentally, these same three reasons provide context to a product vendor’s advertising campaign. Product vendors should advertise their certifications (as Microsoft recently did). After all, the rigor of these programs requires a tremendous amount of monetary and resource commitment, and certifications help sell products.

But what is the net benefit to all stakeholders if end users only consider certifications at procurement and do not deploy the product in its evaluated configuration? This is challenging to answer in a simple blog post. From the end user perspective, purchasing certified products not only meets government procurement requirements but also provides some mitigation of risk.** From the vendor perspective, a sale will not (typically) be lost because of a lack of certification. They’ve met the ante, and now the features/performance/scalability/interoperability must meet the end user requirements to get the purchase order. Also, a vendor does receive a very tangible benefit of third-party review of design documentation and security function implementation, which can often result in advancing development processes and fixing bugs (or more drastic flaws such as poor cryptographic algorithm and key management implementations). So a vendor qualifies to sell a product, and hopefully the address shortcomings in their development lifecycle. Those are good things.

Paraphrasing Snow’s comment above, FIPS 140 and Common Criteria will be successful if end users understand the purpose and value of these certification programs. Common Criteria and FIPS 140 are complicated, and we are not suggesting that end users become proficient in the specific processes and requirements of these programs. We do feel that a better understanding of the meaning and value of these programs in the end user community will push us past the “procurement checkmark” stigma tied to these programs.

To help foster this success, Apex Assurance Group has services to advance the mission and understanding of Information Assurance in the end-user community.

*Disclaimer - Not all end users view FIPS 140 and Common Criteria as a procurement checkmark, and some do run the evaluated configurations. These generalizations are derived from years of talking directly with procurement officials and end users in DoD, Intelligence, and Civilian operations.

** More to be written on this subject in a future post.

The definition we use in our presentations is as follows:

A measure of confidence and trust (usually via a third party) that a product or system meets its claims or meets a specified set of requirements (even when under attack)

Brian Snow wrote an interesting paper on Assurance (thanks to Schneier’s blog and D-Kriptik for pointing this out). Here is his high-level definition of Assurance:

[Assurance] makes a user (or accreditor) more confident that the system works as intended, without flaws or surprises, even in the presence of malice.

It looks as if we agree on the basics of what Assurance is. That’s a good thing. Mr. Snow more specifically defines assurance later in the paper, and AAG offers the following comments:

Assurances are confidence-building activities demonstrating that
1. The system’s security policy is internally consistent and reflects the requirements of the organization,

This point is certainly targeted to systems-level assurance (e.g., Certification & Accreditation). A Common Criteria evaluation may reference an organization’s security policies, and the Common Criteria can even be positioned at a systems level (though currently this is very rare). End-users in an organization may define requirements for point product functionality and assurance level through a Protection Profile, which specifies threats, assumptions, objectives, and Security Functional Requirements for a given technology type (e.g., firewalls, IDS, etc.). One look at NIAP’s list of Protection Profiles in Development makes a strong case that end users are perhaps beginning to embrace the CC and defining requirements for point products deployed in their systems.

2. There are sufficient security functions to support the security policy,

This is certainly an important point. Even from a product assurance perspective- if a Common Criteria evaluation is conducted on a feature-rich COTS security product at EAL1 (or even EAL2) with 3 Security Functional Requirements, then the evaluation is likely not of much benefit to the end-user- unless, of course, that level of product assurance meets their overall requirements for systems assurance. Possible, but not likely.

3. The system functions meet a desired set of properties and only those properties,

Again, this more applies to systems-level assurance. The point is valid. Assurance should provide answers to the following questions: does the system do what is supposed to do, and does it do something it’s not supposed to do?

4. The functions are implemented correctly, and

A “correct” implementation is one which is tested and validated (usually via third-party) against a set of industry-wide best practices or standards. For example, in FIPS 140, product vendors are required to test their algorithm implementations against a suite of tests defined by the Cryptographic Algorithm Validation Program (CAVP). These tests provide a level of assurance that the tested algorithm conforms to its respective implementation standard (e.g., FIPS 197 for AES).

5. The assurances hold up through the manufacturing, delivery, and life cycle of the system.

This is a very important point that is (almost painfully) addressed in the Common Criteria and addressed to a lesser extent in FIPS 140.

Mr. Snow’s paper is quite excellent and does a great job of addressing assurance and assurance techniques for different technology types.

AAG provides services to support FIPS 140 and Common Criteria, which are two examples of programs/methodologies that provide some level of assurance for security functions in products. While similar in concept (and even somewhat symbiotic as each one contains references to the other), the two programs provide different evaluation methodologies and different levels of value to the end-user (watch this weblog for more thoughts and insights). In addition to FIPS 140 and Common Criteria support, AAG also provides Security Assurance Strategy services to help clients understand the need for and value of assurance and to help build a Security Assurance Program to address assurance horizontally and throughout the organization.

Apex Assurance Group provides risk analysis and assessment services to help companies identify and quantify areas of vulnerability in their business processes and system architectures. *

Risk assessments are valuable for several reasons:

1. The company formally recognizes (and possibly even learns) its current business processes and/or systems and network infrastructure.
2. Stakeholders for business processes and systems obtain an idea of threats to their respective assets and the potential costs associated with those threats.
3. The company can issue policies and procedures for risk reduction or risk recovery in the event that an identified threat occurs.

A risk analysis/assessment in and of itself is not a panacea for providing assurance of sound business processes and systems architectures. Furthermore, there is a seemingly obvious threat/risk that may not be covered in the risk analysis/assessment deliverable: what is the risk and implication of not implementing internal policies or business process reengineering practices to address the risk mitigation plan?

The organization’s stakeholders (e.g., business process owners and systems owners) must recognize, adopt, and address the threats and mitigation plans detailed in the risk assessment. If they don’t, then what was the point of the risk assessment?

*Stay tuned to this weblog and to our website for more information on our perspectives on and services for risk assessments.

A colleague posted an interesting view on the commoditization of traditional software “products” and the proliferation of services to support deployment of open software. The comment from Apex Assurance Group’s managing director on this always-interesting blog spawned a question about AAG’s business model for software development and deployment.

Does AAG have plans release your software as an “open” solution?

Apex Assurance Group offers highly-specialized security software development solutions. Our software is not for general deployment and typically requires extensive customization based on customer requirements (hence, the “solutions”). We support the concept of open systems/solutions in general. However, due to the nature of our services and solutions, we do not feel that a traditional open model for our software fits our business model, nor does it meet the needs of our customers. Thus, we do not plan to offer free downloads of our software, nor do we plan to publicly post source code on our website.

Earlier this year, Apex Assurance Group gave a presentation in Las Vegas. The night before I was scheduled to talk , I tweaked the presentation slightly and went to the business center of a major hotel (whose name will not be disclosed here) to print some handouts. The folks working there requested that I copy the file to a USB flash drive, and they would take care of it. Fair enough.

As I prepared to copy the PDF file to the flash drive, I noticed something very interesting- a document called “passwords.txt” … Wow. Of course, it’s common for some people (especially home users) to leave usernames and passwords in the open, whether taped to the underside of their keyboard or just casually jotted on a notepad on the desk. But this is a major, high-profile hotel/casino that manages millions of dollars per day. Aren’t they supposed to have a strong security posture across the entire organization?

Assuming the file contained anything useful, imagine the damage that could have been done if it ended up in the wrong hands. Perhaps an attacker could have gained access to personal information of guests (including names, SSN numbers, net worth, account statements/balances, credit card numbers). Not good.

Hopefully they will change their internal security policies and auditing policies. It should be noted that I did not view the file, and I checked to ensure that no copies were resident in any temporary location on my hard-drive.

And you thought hotel/casino security was unbeatable. This would have made one very quick Alias episode. Or maybe this will set the plot for Ocean’s 13.

Apex Assurance Group was invited to speak at the Infosec Conference in New York City this week. The conference was well attended, and the sessions were excellent as usual. Ray Potter, Managing Director of AAG, presented an overview of product assurance programs such as FIPS 140, Common Criteria, and ICSA. The presenter addressed the typical costs, timeframes, and processes with each certification program. The presentation concluded with a discussion on the value of product assurance programs in overall systems assurance and common issues/best practices for pursuing these security certifications. A copy of the presentation can be downloaded from AAG’s library of presentations and whitepapers.

The conference sessions had high attendance, which attested to the strength of the curriculum and breadth of tutorial offerings. Unlike other conferences, the session tracks were not disguised as advertisements for sponsors or speakers. Of the attendees polled in AAG’s session, most felt that the program was strong and would recommend it to a colleague.

At the tradeshow floor, most of the usual security vendors were present, and some new vendors were in attendence as well. Network management and monitoring solutions were the most prevalent, followed by secure email/messaging solutions and proxy solutions such as URL filtering, spam filtering, and virus detection.