What kinds of functions are used by Cleanroom Software Engineering approach?

Harlan Mills and his colleagues namely Linger, Poore, Dyer in the year of 1980 developed a software process that could promise building zero error software at IBM. This process is now popularly known as the Cleanroom software engineering. The process was named in accordance with an analogy with the manufacturing process of the semiconductors. 

The Clean room software engineering process makes use of the statistical process and its control features. The software systems and applications thus produced have certified software reliability. The productivity is also increased as the software has no defects at delivery. 

Below mentioned are some key features of the Cleanroom software engineering process:

1. Usage scenarios
2. Incremental development
3. Incremental release
4. Statistical modeling
5. Separate development
6. Acceptance testing
7. No unit testing
8. No debugging
9. Formal reviews with verification conditions

Basic technologies used by the CSE approach are:

1. Incremental development
2. Box structured specifications
3. Statistical usage testing
4. Function theoretic verification

- The incremental development phase of the CSE involves overlapping of the incremental development and from beginning of specification to the end of the test execution it takes around 12 – 18 weeks.

- Partitioning of the increments is critical as well as difficult. 

- Formal specification of the CSE process involves the following:

1. Box structured Designing: Three types of boxes are identified namely black box, state box and clear box.
2. Verification properties of the structures and
3. Program functions: These are one kind of functions that are used by the clean room approach.

- State boxes are the description of the state of the system in terms of data structures such as sequences, sets, lists, records, relations and maps. 

- Further, they include specification of operations and state in-variants.

- Each and every operation that is carried out needs to take care of the invariant. 

- The syntax errors present in a constructed program in clean-room are checked by a parser but is not run by the developer.

- A team review is responsible for performing verification which is driven by a number of verification conditions. 

- Productivity is increased by 3–5 times in the verification process as compared to the debugging process. 

- Proving the program is always an option with the developers but it calls for a lot of math intensive work.

- As an alternate to this, clean room software engineering approach prefers to use a team code inspection in terms of two things namely:

1. Program functions and
2. Verification conditions

- After this, an informal review is carried out which confirms whether all conditions have been satisfied or not. 

- Program functions are nothing but functions describing the prime program’s function.

- Functional verification steps are:

1.    Specifying the program by post and pre-conditions.

2.    Parsing the program in to prime numbers.

3.    Determining the program functions for SESE’s.

4.    Defining verification conditions.

5.    Inspection of all the verification conditions.

- Program functions also define the conditions under which a program can be executed legally. Such program functions are called pre-conditions.

- Program functions can even express the effect the program execution is having up on the state of the system. Such program functions are called the post conditions.

- Programs are mostly expressed on terms of the input arguments, instance variables and return values of the program. 

- However, they cannot be expressed by local program variables. 

- The concept of nested blocks is supported by a number of modern programming languages and structured programs always require well nesting. 

- The process determining SESE’s also involves parsing rather than just program functions.

What is a Cleanroom approach?

In this article we discuss the cleanroom approach in detail. The size of the team is usually small and is divided in to following three sub – teams:

1. Specification team: This team is responsible for the development and maintenance of the specifications.
2. Development team: This team is responsible for the development and verification of the software.
3. Certification team: This team is responsible for the development of statistical tests and reliability growth models. 

The incremental development is always carried out under statistical quality control so that the performance can be assessed at the end of every iteration using the following measures:

1. Errors per KLOC
2. Rate of growth in MTTF
3. Number of sequential error free tests.

The software development in cleanroom approach is purely based up on the mathematical principles whereas the testing is based up on the statistical principles. 

- Firstly, the system to be developed is formally specified and an operational profile is created. This profile and the formal specifications are then used to define the software increments which are then used for the two purposes namely:

1. Construction of a structured program
2. Designing of statistical tests: These tests also contribute to the first purpose.

- The constructed program is then formally verified and integrated with the increment.

Below mentioned is the flow of cleanroom approach:

1. Software requirements specification
2. Software design and development
3. Incremental software delivery
4. Incremental statistical testing
5. Regression testing
6. Software reliability measurement
7. Process error diagnosis and correction

- The incremental development planning is divided in to two parts namely:

1. Functional specification: It involves formal design correctness verification.
2. Usage specification: It involves statistical test case generation.

- Both these processes then merge down to statistical testing which then follows quality certification model and MTTF estimates.

- The whole cleanroom project develops around the incremental strategy. 

- Requirements are gathered from the customers and elicited and refined via the traditional methods.

- The definition of the data, its behavior and procedures are isolated and separated by the box structures at every level of refinement. 

- Specifications or the black boxes when iteratively refined become state boxes i.e., architectural designs and clear boxes i.e., the component–level designs.

- Formal inspections are carried out to make sure that the code confirms to standards, it is syntactically correct and its correctness has been verified. 

- Statistical usage planning involves creation of tests cases that match with the probability distribution of the usage pattern

- In place of the exhaustive testing, a sample of all the test cases is employed. 

- Once the programmers are done with all 3 activities (i.e., verification, inspection, usage testing, and defect removal) the increment is considered to be certified and ready to be integrated. 

- For developing a right system, customer feedback and involvement are 2 necessary elements throughout the process. 

- Increment planning is required so that the customer’s system requirements can be clarified. 

- There is a requirement of management of resources and control of complexity which is also achieved through incremental planning.

- In order to develop a quality product a control over the software development cycle and process measurement is very much required.

- Following are the benefits of concurrent planning:

1. Concurrent engineering
2. Step wise integration
3. Continuous quality feedback
4. Continuous customer feedback
5. Risk management
6. Change management

- All of the above benefits are achieved respectively by:

1. Certification and scheduling parallel development
2. Testing cumulative increments
3. Statistical process control
4. Through actual use
5. Treatment of the high risk elements in early phases
6. Systematic accommodation of the changes

Design verification advantage allows the cleanroom teams to verify each and every line of code. 

What are requirements for Cleanroom Software Engineering? What is the need for CSE?

The process got its name from the term cleanroom which is part of the process that fabricates the semiconductors. The basic ideology behind the cleanroom software engineering process is that it is more focused up on avoiding the defects rather than removing them.  It makes use of a combination software quality methods and formal methods. Cleanroom software engineering has shifted the individual craftsmanship to peer reviewed process, from sequential development to incremental one, from informal designing to discipline specification and designing, from informal coverage to statistical usage testing and so on.

Cleanroom software engineering is actually an integration of the following three practices:

1. Program verification
2. Software engineering modeling
3. Statistical software quality assurance

The correctness of the design specifications is verified via mathematically based proof solutions. The role of the statistical usage testing here is to dig out the high impact errors. 

The following the stages through the incremental development:

1.   Establishing the requirements

2.   Formal specifications

3.   Development of the increment

4.   Delivery of the software

5.   Requirements change request

Why Cleanroom Software Enginnering is required?

- Cleanroom software engineering is required to develop software systems and application with zero defects. 

- Though it takes a lot of time to be implemented but the payoff is quite high and both quality and productivity are increased. 

- The actual cleanroom process begins after the formal specification has been done. 

- The focus then shifts towards building a more explicit design. 

- Next, the design is verified as per the specifications. 

- Two things namely statistical and mathematical reasoning are combined and used together by the cleanroom approach for test generation and testing as well. 

- Cleanroom software engineering has proven to be the first practical attempt for developing the software with statistical quality control and delivering the software product with a known MTTF.

Requirements of Cleanroom Software Engineering

- The key requirements of cleanroom approach are nothing but specifications, requirements and verification methods.

- These formal specifications and verifications are used for developing software of high quality and with a minimum of errors. 

- Statistical usage testing is also required for the evaluation of the reliability of the software product. 

Cleanroom software engineering is followed by a number of benefits:

1. Zero failures: This is the objective itself and the software is developed with a minimum number of errors if zero not possible.
2. Short development cycles: These are the resultant of the incremental process. The rework is avoided and therefore new teams get a chance to experience productivity increase that has increased by two folds.
3. Longer product life: The product life is kept longer with the help of a usage model and investments detailed specifications.

- It is also a fact that the cleanroom techniques are not much used since some people believe them to be too mathematical, theoretical, radical etc. to be used in the real development processes. 

- Also, these techniques rely heavily on the statistical quality control and correctness verification rather than relying on unit testing. 

- This means that there is a large deviation from the traditional software development approach. 

- Proving the program is always there in the option list but it requires a lot of intensive sophisticate mathematical work. 

- Therefore, in place of this, cleanroom uses an alternative.

- A team code inspection is structured in terms of verification conditions and program functions.

-  Then an informal review is carried out which confirms whether or not all the verification conditions have been satisfied. 

What is Cleanroom Software Engineering?

Cleanroom software engineering is one of the fastest emerging software development processes and has been designed for the production of the software systems and applications with a reliability of certificate level. The credit for the development of this process goes to Harlan Mills and a couple of his colleagues among which one was Alan Hevner at the IBM Corporation. 

What is Cleanroom Software Engineering?

- The cleanroom software engineering is focused on defect prevention rather than their effective removal. 

- The process was named so since the word cleanroom evoked the sense of cleanrooms that are used by the electronics industry for preventing the defects from entering the semiconductors during the fabrication process. 

- The first time when the cleanroom process was used was in the late 80s. 

- This process began to be used for military demonstration process in the early 1990s. 

Principles of Cleanroom Approach

Cleanroom process has its own principles which we have discussed below:

1. Development of software systems and applications based up on formal methods: Box structure method is what that is used by the cleanroom development for specifying and designing a software product. Later, team review is used for carrying out verification of the design i.e., whether it has been correctly implemented or not.
2. Statistical quality control through incremental implementation: An iterative approach is followed in the cleanroom software engineering process i.e., the software system is evolved through increments in which the implemented functionality gradually increases. Pre–established standards are used for measuring the quality of all the increments for making verification that the process is making acceptable process. In case the process fails to meet the quality standards, testing of the current increment is stopped and the process is returned to the designing phase.
3. Statistically sound testing: Software testing in cleanroom development process is carried as a disguise of a statistical experiment. A subset that represents software’s i/p and o/p trajectories is selected and then subjected to testing. The sample so obtained is  then considered for statistical analyzation so as to get an estimation  of the software’s reliability and level of confidence.

Features of Cleanroom Software Engineering

Software products developed using the cleanroom software engineering process have zero defects at the delivery time. Below mentioned are some of the characteristics features of the cleanroom software engineering:

1. Statistical modeling
2. Usage scenarios
3. Incremental development and release
4. Separate acceptance testing
5. No requirement of unit testing and debugging
6. Formal reviews with verification conditions

The defects rate was recorded as follows:

1. <3 .5=".5" and="and" delivered="delivered" kloc="kloc" o:p="o:p" per="per">

2.7 Per KLOC between first execution and first delivery.

Basic technologies thus used can be listed as:

1. Incremental development: Each increment is carried out from end – to – end and in some cases there is overlapping development of the increments. This whole process takes around 12 – 18 weeks and partitioning though being critical proves to be difficult.
2. Function – theoretical verification: A parser may check the constructed program for syntax errors but it cannot be executed by the program developer. Verification conditions drive the team review for verification. Verification is improved by 3- 5 times than debugging. Formal inspections also fall under this category.
3. Formal specifications: This further includes:

a)    Box structured specification: It includes 3 types of boxes namely:

Ø  Black box

Ø  State box

Ø  Clear box

b)    Verification properties

c)    Program functions

4. Statistical usage testing: It helps in implementing cost effective orientation and process control. It provides a stratification mechanism to deal with situations that are critical. 

Cleanroom Software Engineering - Advantages, Principles and Process Teams

Cleanroom software engineering involves the integrated use of software engineering modeling, program verification and statistical software quality assurance.
- Cleanroom software engineering verifies design specification using mathematically-based proof of correctness.
- Cleanroom software engineering relies heavily on statistical use testing to uncover high impact errors.
- Cleanroom software engineering generally follows an incremental development process.

CLEANROOM PRINCIPLES

- Small Teams include independent specification, development, and certification sub-teams.
- Incremental development under statistical quality control.
- Software development is based on mathematical principles. The box principle is used for specification and design. The formal verification is used to confirm correctness of implementation of specification. The program correctness is verified by team reviews using questionnaires.
- Testing is based on statistical principles.

CLEANROOM PROCESS TEAMS

- Specification team develops and maintains the system specification.
- Development team develops and verifies software. The software is not compiled or executes during verification.
- Certification team develops set of statistical test to exercise software after development. Reliability growth models are used to assess reliability.

BENEFITS OF CLEANROOM SOFTWARE ENGINEERING

- Zero failures in the field which is a goal but a realistic expectation is<5 failures per KLOC on first program execution in the first team project.
- Short development cycles
- Longer product life.

Cleanroom Testing - Statistical Use Testing and Certfication

Clean room testing is different from conventional testing approaches. The goal of clean room testing is to validate software requirements by demonstrating that a statistical sample of use-cases have been executed successfully.

Statistical use testing tests the software in a way the users intend to use it. the clean room testing teams determine the usage probability distribution or the software. Each increment's specification is analyzed to define a set of inputs or events that cause the software to change its behavior. A probability of use is assigned to each input or event based on the interviews with users, the creation of usage scenarios and a general understanding o application domain. The testing team executes these use cases and verifies software behavior against the specification for the system. Using the recorded interval times, the certification team can compute mean time to failure.

Within the clean room software engineering approach, certification implies that the reliability can be specified for each component. Reusable software components can be stored along with their usage scenarios, program stimuli, and probability distributions. The certification approach involves :
- usage scenarios must be created.
- usage profile is specified.
- test cases are generated from profile.
- tests are executed and failure data are recorded and analyzed.
- reliability is computed and certified.

Certification for clean room software engineering requires creation of three models:
- Sampling Model : Certification is given if no failure or a specified number of failures occur after executing m random test cases. The value of m is derived mathematically.
- Component Model : This model enables the analyst to determine the probability that component i will fail prior to completion.
- Certification Model : The overall reliability of the system is projected and certified.

Introduction To Scrum - Type of Agile Software Development

Scrum is an agile method for project management developed by Ken Schwaber. Its goal is to dramatically improve productivity in teams previously paralyzed by heavier, process-laden methodologies. Its intended use is for management of software development projects as well as a wrapper to other software development methodologies such as Extreme Programming.Scrum is a lightweight management framework with broad applicability for managing and controlling iterative and incremental projects of all types. With Scrum, projects progress via a series of iterations called sprints. Each sprint is typically 2-4 weeks long. Scrum is ideally suited for projects with rapidly changing or highly emergent requirements.

A scrum team is typically made up of between five and nine people, but Scrum projects can easily scale into the hundreds. The team does not include any of the traditional software engineering roles such as programmer, designer, tester, or architect.
- The product owner is the project’s key stakeholder and represents users, customers and others in the process.
- The ScrumMaster is responsible for making sure the team is as productive as possible.
- The product backlog is a prioritized features list containing every desired feature or change to the product.
- At the start of each sprint, a sprint planning meeting is held during which the product owner prioritizes the product backlog, and the scrum team selects the work they can complete during the coming sprint. That work is then moved from the product backlog to the sprint backlog, which is the list of tasks needed to complete the product backlog items the team has committed to complete in the sprint.
- Each day during the sprint, a brief meeting called the daily scrum is conducted. This meeting helps set the context for each day’s work and helps the team stay on track.
- At the end of each sprint, the team demonstrates the completed functionality at a sprint review meeting, during which, the team shows what they accomplished during the sprint.

Scrum enables the creation of self-organizing teams by encouraging verbal communication across all team members and across all disciplines that are involved in the project. A key principle of scrum is its recognition that fundamentally empirical challenges cannot be addressed successfully in a traditional "process control" manner.

Cleanroom Software Engineering - Certification

The verification and testing techniques leads to software components that can be certified. Certification implies that the reliability can be specified for each component. The potential impact of certifiable software components goes far beyond a single cleanroom project. Reusable software components can be stored along with their usage scenarios, program stimuli, and probability distributions.

The certification approach involves five steps :
1. Usage scenarios must be created.
2. A usage profile is specified.
3. Test cases are generated from the profile.
4. Tests are executed and failure data are recorded and analyzed.
5. Reliability is computed and certified.

Cleanroom Certification Models :
- Sampling model : Software testing executes m random test cases and is certified if no failures or a specified numbers of failures occur. The value of m is derived mathematically to ensure that required reliability is achieved.
- Component model : A system composed of n components is to be certified. The component model enables the analyst to determine the probability that component i will fail prior to completion.
- Certification model : The overall reliability of the system is projected and certified.

Cleanroom Software Engineering - Design Refinement and Cleanroom Testing

Design Refinement & Verification
- If a function f is expanded into a sequence g and h, the correctness condition for all input to f is:
• Does g followed by h do f?
When a function f is refined into a conditional (if-then-else), the correctness condition for all input to f is:
• Whenever condition is true does g do f and whenever is false, does h do f?
- When function f is refined as a loop, the correctness conditions for all inputs to f are:
• Is termination guaranteed?
• Whenever is true does g followed by f do f, and whenever is false, does skipping the loop still do f?

Advantages of Design Verification :
- It reduces verification to a finite process.
- It lets cleanroom teams verify every line of design and code.
- It results in a near zero defect level.
- It scales up.
- It produces better code than unit testing.

CLEANROOM TESTING :
The strategy and tactics of cleanroom testing are fundamentally different from conventional testing approaches.
Statistical Testing
- Generation of test cases
* each test case begins in a start state and represents a random walk through the usage model ending at a designated end state.
- Control of statistical testing
* a well-defined procedure is performed under specified conditions.
* each performance is a trial and can be used as part of an empirical probability computation.
- Stopping criteria for testing
* when testing goals or quality standards are achieved.
* when the difference between the predicted usage chain and the actual testing chain becomes very small.

The Cleanroom Strategy and Functional Specifications

The cleanroom approach makes use of a specialized version of the incremental process model. A pipeline of software increments is developed by small independent software teams. As each increment is certified, it is integrated into the whole. Hence, functionality of the system grows with time.

- Increment Planning : adopts the incremental strategy.
- Requirements Gathering : defines a description of customer level requirements (for each increment).
- Box Structure Specification : describes the functional specification.
- Formal Design : specifications (called “black boxes”) are iteratively refined (with an increment) to become analogous to architectural and procedural designs (called “state boxes” and “clear boxes,” respectively).
- Correctness Verification : verification begins with the highest level box structure (specification) and moves toward design detail and code using a set of “correctness questions.” If these do not demonstrate that the specification is correct, more formal (mathematical) methods for verification are used.
- Code Generation, Inspection and Verification : the box structure specifications, represented in a specialized language, are transmitted into the appropriate programming language.
- Statistical Test Planning : a suite of test cases that exercise of “probability distribution” of usage are planned and designed.
- Statistical Usage Testing : execute a series of tests derived from a statistical sample (the probability distribution noted above) of all possible program executions by all users from a targeted population.
- Certification : once verification, inspection and usage testing have been completed (and all errors are corrected) the increment is certified as ready for integration.

Functional Specifications :
Cleanroom Software engineering compiles with the operational analysis principles by using a method called box structure specification. A "box" encapsulates the system at some level of detail. Three types of boxes are used :
- Black box : It specifies a set of transition rules that describe the behavior of system components as responses to specific stimuli, makes use of inheritance in a manner similar to classes. It also specifies system function by mapping all possible stimulus histories to all possible responses.
S*->R i.e. stimulus history->responses
- State box : It is a generalization of a state machine, encapsulates the data and operations similar to an object, the inputs (stimuli) and outputs (responses) are represented, data that must be retained between transitions is encapsulated. The state is the encapsulation of the stimulus history. State variables are invented to save any stimuli that need to retained.
S x T -> R x T i.e. stimuli X state data -> responses X state data
- Clear box : It contains the procedural design of the state box, in a manner similar to structured programming. It specifies both data flow and control flow.
S x T -> R x T i.e. stimuli X state data -> responses X state data

BOX PRINCIPLES :
- Transaction closure of stimuli and responses
* Users and uses are considered including security and error recovery.
- State migration within box hierarchy
* Downward migration of state data is possible whenever new black boxes are created inside a clear box.
* Upward migration of state date is desirable when duplicate data is updated in several places in the tree.
- Common services
* Reusable boxes from library.

Cleanroom Software Engineering Cont...

The Cleanroom Software Engineering process is a software development process intended to produce software with a certifiable level of reliability. The focus of the Cleanroom process is on defect prevention, rather than defect removal. The name Cleanroom was chosen to evoke the cleanrooms used in the electronics industry to prevent the introduction of defects during the fabrication of semiconductors.

Central principles of Cleanroom Software Engineering :
- Software development based on formal methods
Cleanroom development makes use of the Box Structure Method to specify and design a software product. Verification that the design correctly implements the specification is performed through team review.
- Incremental implementation under statistical quality control
Cleanroom development uses an iterative approach, in which the product is developed in increments that gradually increase the implemented functionality. The quality of each increment is measured against pre-established standards to verify that the development process is proceeding acceptably. A failure to meet quality standards results in the cessation of testing for the current increment, and a return to the design phase.
- Statistically sound testing
Software testing in the Cleanroom process is carried out as a statistical experiment. Based on the formal specification, a representative subset of software input/output trajectories is selected and tested. This sample is then statistically analyzed to produce an estimate of the reliability of the software, and a level of confidence in that estimate.

Advantages of Cleanroom Approach :
- Proven Practice : Several major products have been developed using Cleanroom and delivered to customers.
- Low Error Rate, highly reliable code.
- Increased Productivity.
- Can Be Incorporated Within A Quality Management Program Such As CMM Or ISO 9000.

Why are Cleanroom Techniques Not Widely Used ?
- Some people believe cleanroom techniques are too theoretical, too mathematical, and too radical for use in real software development.
- Relies on correctness verification and statistical quality control rather than unit testing (a major departure from traditional software development).
- Organizations operating at the ad hoc level of the Capability Maturity Model, do not make rigorous use of the defined processes needed in all phases of the software life cycle.

Cleanroom software engineering

Cleanroom software engineering is a formal approach to software development that can lead to software that has remarkably high quality. It uses box structure specification (or formal methods) for analysis and design modeling and emphasizes correctness verification, rather than testing, as the primary mechanism for finding and removing errors. Statistical use testing is applied to develop the failure rate information necessary to certify the reliability of delivered software.

The cleanroom approach begins with analysis and design models that use a box structure representation. A "box" encapsulates the system at a specific level of abstraction. Black boxes are used to represent the externally observable behavior of a system. State boxes encapsulate state data and operations. A clear box is used to model the procedural design that is implied by the data and operations of a state box.

Correctness verification is applied once the box structure design is complete. The procedural design for a software component is partitioned into a series of sub-functions. To prove the correctness of the sub-functions, exit conditions are defined for each sub-function and a set of sub-proofs is applied. If each exit is condition is satisfied, the design must be correct.

Once correctness verification is complete, statistical use testing commences. Unlike conventional testing, cleanroom software engineering does not emphasize unit or integration testing. Rather, the software is tested by defining a set of usage scenarios, determining the probability of use for each scenario, and then defining random tests that conform to the probabilities. The error records that result are combined with sampling, component, and certification models to enable mathematical computation of projected reliability for the software component.

The cleanroom philosophy is a rigorous approach to software engineering. It is a software process model that emphasizes mathematical verification of correctness and certification of software reliability. The bottom line is extremely low failure rates that would be difficult or impossible to achieve using less formal methods.