Objectives
One objective is to demonstrate that the security functions perform as specified.
An additional objective is to counter the risk of an incorrect assessment of the test outcomes on the part of the developer that results in the incorrect implementation of the specifications, or overlooks code that is non-compliant with the specifications.
Component levelling
Levelling is based upon the amount of test documentation, test support and the amount of evaluator testing.
Application notes
The testing specified in this family can be supported by a party with specialised knowledge other than the evaluator (e.g. an independent laboratory, an objective consumer organisation). Testing requires an understanding of the TOE consistent with the performance of other assurance activities, and the evaluator retains responsibility for ensuring that the requirements of this family are properly addressed when such support is used.
This family deals with the degree to which there is independent functional testing of the TSF. Independent functional testing may take the form of repeating the developer's functional tests, in whole or in part. It may also take the form of the augmentation of the developer's functional tests, either to extend the scope or the depth of the developer's tests, or to test for obvious public domain security weaknesses that could be applicable to the TOE. These activities are complementary, and an appropriate mix must be planned for each TOE, which takes into account the availability and coverage of test results, and the functional complexity of the TSF. A test plan should be developed that is consistent with the level of other assurance activities, and which, as greater assurance is required, includes larger samples of repeated tests, and more independent positive and negative functional tests by the evaluator.
Sampling of developer tests is intended to provide confirmation that the developer has carried out his planned test programme on the TSF, and has correctly recorded the results. The size of sample selected will be influenced by the detail and quality of the developer's functional test results. The evaluator will also need to consider the scope for devising additional tests, and the relative benefit that may be gained from effort in these two areas. It is recognised that repetition of all developer tests may be feasible and desirable in some cases, but may be very arduous and less productive in others. The highest component in this family should therefore be used with caution. Sampling will address the whole range of test results available, including those supplied to meet the requirements of both ATE_COV and ATE_DPT.
There is also a need to consider the different configurations of the TOE that are included within the evaluation. The evaluator will need to assess the applicability of the results provided, and to plan his own testing accordingly.
Independent functional testing is distinct from penetration testing, the latter being based on an informed and systematic search for vulnerabilities in the design and/or implementation. Penetration testing is specified using the family AVA_VLA.
The suitability of the TOE for testing is based on the access to the TOE, and the supporting documentation and information required (including any test software or tools) to run tests. The need for such support is addressed by the dependencies to other assurance families.
Additionally, suitability of the TOE for testing may be based on other considerations. For example, the version of the TOE submitted by the developer may not be the final version.
References to a subset of the TSF are intended to allow the evaluator to design an appropriate set of tests which is consistent with the objectives of the evaluation being conducted.