Product selection

Common Criteria

The Common Criteria for Information Technology Security Evaluation (usually abbreviated to Common Criteria or CC) is an ISO standard (ISO/IEC 15408) and includes a mechanism to provide assurance that the process of specification, implementation and evaluation of a computer security product has been conducted in a rigorous and standard and repeatable manner measured against a defined target environment or security target.

Measurement Standards

There are two measurement approaches in use within this standard, depending on the certification authority in each country.  The original measures were the Evaluation Assurance Level (EAL) which dates from the inception of the CC standard in 1999.

More recently the Protection Profile (PP) mechanism was developed to address concerns on the length of time evaluations were taking and the cost of these evaluations, while maintaining assurance levels. 

While the two mechanisms both provide assurance, they are different evaluation mechanisms and it is not possible to always directly compare an EAL result with a PP result.

Common Criteria Recognition Arrangement

The Common Criteria Recognition Arrangement (CCRA) is mutual recognition of evaluations undertaken by other members.  It is intended to avoid duplicating evaluations of IT products and protection profiles already undertaken by another participating member.  AT present there are 30 participating nations in the CCRA.

EAL

Evaluation Assurance Level (EAL) certifications range from 1 to 7, with EAL1 being the most basic and EAL7 the most comprehensive and costly.  Although basic assurance requirements for each product and system were the same, functional requirements were different, and each product could have different levels within the same evaluation profile. This made comparisons between products very difficult, even product from the same manufacturer.

At present, only assurance levels up to EAL2 have been incorporated within the international Common Criteria Recognition Arrangement (CCRA) and are mutually recognised.  There is a move by CCRA from EAL to PP based assurance for wider acceptability.

Protection Profiles

A Protection Profile (PP) describes the complete set of a product’s security functionality.  A PP specifies both functional and assurance requirements.  Any evaluations are then undertaken against the PP.  This provides comprehensive assurance of the security of evaluated products.  

Protection Profiles are generally published by national authorities for a specific technology type, for example, a firewall, switch or router.  A given product may conform to multiple PPs.   

A centralised repository of PPs is published by the Common Criteria.

While PPs are generally nation specific, there is work underway to develop a set of Collaborative Protection Profiles (cPP).  These are being developed by international technical communities and approved by multiple national schemes.  

PPs also introduced the concept of equivalence in PP-based evaluations in order to find a balance between evaluation rigour and commercial practicality.  This helps in ensuring that assurance is achieved across differences in product model and platform, while recognising that there might be little to be gained from requiring that every minor variation in a product or platform be fully tested. 

System Software and Hardware Platforms can be equivalent, partially equivalent, or non-equivalent.  Non-equivalent software or hardware must be independently fully tested.

Comparison Chart

Protection Profile

EAL certification

All vendors within the same product type must adhere to the same security requirements.

Vendor individually chooses which security requirements to claim, causing inconsistencies across similar products lines.  Testing can be extensive for higher EAL levels.

Evaluation methods approved by the Common Criteria Recognition Arrangement.

Limited mutual recognition under the Common Criteria Recognition Arrangement, only up to EAL2.

An objective approach in evaluation methods.

A subjective approach from the vendor to identify product functional requirements.

Relevant, achievable, repeatable results with standard threat models and security functional   requirements captured in a Protection Profile.

Results not always repeatable across different products and vendors.

Protection Profiles developed by technical communities through the Common Criteria community.

Generic requirements developed by individual vendors.

Threats identified and mandated by security agencies; hardware requirements usually based on threat assessments.

Threats identified after vendor maps product functionality to Common Criteria.  Hardware differences and requirements selections often complicate assurance comparisons.

 

Many national authorities are moving to Protection Profiles and are retiring EAL testing.  PP evaluated products are the preferred option. EAL certifications now tend to reflect older products.

Where a PP does not exist or a selected product has not been submitted for PP evaluation, then an EAL evaluation may be acceptable, depending on the EAL result level (e.g., EAL 4 for firewalls).

In many cases, however, no EAL 4 firewalls are now manufactured, and newer models may not have been submitted for PP evaluation. 

In such cases the NZISM provides a block chart summary of the selection process in section 12.1 – Product selection and acquisition.  

Product selection is covered in the NZISM chapter 12 – Product security.  This includes guidance and controls over the entire lifecycle of product identification, selection, acquisition, installation, and disposal. System decommissioning is covered in section 13.1. 

The use of approved products is encouraged as this provides higher levels of assurance over the security functionality of particular products. 

The process and useful references are provided in section 12.1 – Product selection and acquisition. 

The process to follow where no evaluated product with required functionality is available is also described.

The GCSB does not evaluate or approve machines directly, rather, and as a first preference, we utilise equipment and machinery approved through recognised sources such as the AISEP (Australasian Information Security Evaluation Program) refer to NZISM chapter 12 – Product security, along with other EPLs (Evaluated Products Lists). 

In some cases, equipment or other products may not have been tested under these programmes but may have the required technical specifications.

Preference is given to equipment and other products listed on the EPLs.  Where equipment or functionality required is not available through an EPL’s refer to the selection process described in the diagram Product selection 12.1.27. 

The GCSB will always conduct limited testing, checking machine integrity, completeness, shred size etc. Prospective service providers are required to undertake their own research.