Cleanroom Testing - Statistical Use Testing and Certfication

Clean room testing is different from conventional testing approaches. The goal of clean room testing is to validate software requirements by demonstrating that a statistical sample of use-cases have been executed successfully.

Statistical use testing tests the software in a way the users intend to use it. the clean room testing teams determine the usage probability distribution or the software. Each increment's specification is analyzed to define a set of inputs or events that cause the software to change its behavior. A probability of use is assigned to each input or event based on the interviews with users, the creation of usage scenarios and a general understanding o application domain. The testing team executes these use cases and verifies software behavior against the specification for the system. Using the recorded interval times, the certification team can compute mean time to failure.

Within the clean room software engineering approach, certification implies that the reliability can be specified for each component. Reusable software components can be stored along with their usage scenarios, program stimuli, and probability distributions. The certification approach involves :
- usage scenarios must be created.
- usage profile is specified.
- test cases are generated from profile.
- tests are executed and failure data are recorded and analyzed.
- reliability is computed and certified.

Certification for clean room software engineering requires creation of three models:
- Sampling Model : Certification is given if no failure or a specified number of failures occur after executing m random test cases. The value of m is derived mathematically.
- Component Model : This model enables the analyst to determine the probability that component i will fail prior to completion.
- Certification Model : The overall reliability of the system is projected and certified.