What Tools are used for code coverage analysis?

Code coverage analysis is quite an essential process that makes up the complete and efficient software testing process. 

This analyzation consists of the following three basic activities:

1. Checking out for the areas of the software system or application that have not been exercised by the set of tests that have been performed so far.
2. Creation of the additional test cases so that the code coverage can be increased.
3. Determination of the quantitative measure for the code coverage which some what provides an indirect measure of the quality of the software system and application.

Apart from this, there is one more optional aspect of the code coverage analysis which is that it helps in the identification of the redundant test cases that add to the measure of the code coverage but do not merely increase it.

In this article we have discussed about the tools that make this whole process of code coverage analyzation quite easy.

Tools Used for Code Coverage Analysis

- The code coverage analyzation is quite an effort and time consuming process and therefore is nowadays automated using tools like code coverage analyzer. 

- But a code coverage analyzer cannot be used always like in situations when the tests have to be run through the release candidate.

- For different languages, there are many different and vivid tools are available for code coverage analysis.

1. For C++ and C programming languages:

a)  Tcov

b)  Bulls eye coverage

c)  Gcov

d)  LDRA test bed

e)  NuMega True Coverage

f)   Tessy

g)  Trucov

h)  Froglogic’s squish coco

i)   Parasoft C++ soft

j)   Test well CTC++

k)  McCabe IQ

l)   Insure++

m)Cantata

2. Tools for C#:

a)  Mc Cabe IQ

b)  Jet brains dot cover

c)  Ncover

d)  Visual studio 2010

e)  Parasoft Dottest

f)   Test driven.NET

g)  Kalistick

h)  Dev partner

3. Tools for Java:

a)  McCabe IQ

b)  Clover

c)  EMMA

d)  Kalistick

e)  JaCoCo

f)   JMockit coverage

g)  Code coverage

h)  LDRA test bed

i)    Jtest

j)   Den partner

k)  Cobertura

4. Tools for Java Script:

a) Mc Cabe IQ

b) JS coverage

c) Code coverage

d) Script cover

e) Coveraje

5. Tools for Perl:

a) Mc Cabe IQ

b)  Devel cover

6. Tools for Haskell:

a) HPC (Haskell program coverage) tool kit

7. Tools for Python:

a) Mc Cabe IQ

b) Fig leaf

c) Pester

d) Coverage.py

8. Tools for PHP:

a) Mc Cabe IQ

b) PHP unit

9. Tools for Ruby:

a) Rcov

b) Mc Cabe IQ

c) Simple cov

d) Cover Me

10. Tools for Ada:

a) GNAT coverage

b) Mc Cabe IQ

c) Rapi Cover

Out of all the above mentioned tools for C and C++, the bulls eye coverage has proven to be the best code coverage analyzer in terms of reliability, usability and platform support etc. 

This coverage analyzer is different from the other analyzers in the following ways:

1. Better coverage measurement
2. Wide platform support
3. Rigorously tested
4. Efficient technical support
5. Quite easy to use.

- Using this tool it can be determined that how much of the software system’s or application’s code was tested and this information later can be employed to focus your testing efforts and areas that require some improvement.

- With the bullseye coverage a more reliable code can be created and time can be saved. 

- The function coverage provided by the bulls eye coverage gives you a very high precision.

You can include or exclude the parts of the code of your choice. And what more? You can even merge the results you obtained from the distributed testing plus the run time code can also be included from custom environments. 

What is meant by code coverage? What are different types of code coverage?

Code coverage is one of the important measures that comes across while carrying out the software testing. 

What is Code Coverage?

- Code coverage is the measure that gives the degree up to which the source of a software system or application has been tested. 

- Code coverage earlier was recognized as one of the methods that were developed for carrying out a systematic software testing. 

- The term code coverage was first referenced under the “communications of the ACM” in the year of 1963 by Maloney and miller.

- Code coverage is measured using a specialized tool called code coverage analyzer for instrumenting the binaries in order to add all the tracing calls and run all the automated test cases against the software system or application under question. 

- A good code coverage analyzer will not only give you the degree of the coverage but will also allow you to dig deeper in to the data and check out for the lines of the code that are executed during a particular test. 

- Today we have a large number of code coverage criteria that amount to the total code coverage. 

Types of Code Coverage

- The software system or application that is under testing is adorned with certain options that run under a particular environment. 

- Each and every function of the program that is executed is traced back to the function calls in the source code which allows the easy checking of the rarely accessed, never accessed parts of the program under normal conditions. 

- This gives the software testers an assurance that the most important conditions of the source code have been tested. 

- The outcome is then subjected to analyzation for sorting out of the code that has been analyzed. 

- The process is a combination of several code coverage methods that aim to provide a manageable and rigorous set of regression tests. 

- The output of the code coverage analyzation can be used by the engineers for devising the test cases and configuration sets that in turn will increase the code coverage relating to the vital functions. 

- Mostly two common forms of the code coverage are used by the software testers namely:

1. Line coverage or statement coverage: 

- This type of the code coverage reports regarding the execution foot print of the software testing in terms of the lines of code which are executed for the completion of the test.

- The statement coverage provides you with the benefit of the identification of the bugs that arise in the constructs of the control flow of the source code like consecutive switch labels and compound conditions. 

- The major setback of this code coverage type is that even though it seems like you have achieved 100 percent of the code coverage, there might be 100s of wicked and uncaught bugs.

- This disadvantage of the statement or line coverage is over-comed by a more advance form of code coverage called the branch coverage.

2. Edge or branch coverage: 

This type of code coverage reports regarding the branches or the code decisions that like statements are executed for the completion of the tests.

- Both of the above mentioned types of the code coverage provide the result in the form of code coverage metric in terms of percentages. 

- Out of the two, the branch code coverage is considered to be quite comprehensive that the equivalent percentage of the statement code coverage.

- It is also very easy to determine the number of branches that have to be tested since the Boolean decisions have only two outcomes i.e., either true or false. 

Multiple Condition Coverage is a white box testing technique - Explain?

There are so many types of the code coverage and the multiple condition coverage is also one of them and that is what has been discussed all about in this article.

What is code Coverage?

Code coverage as we all know is a process that aims at the following purposes:

- Creation of the additional test cases for increasing the coverage.
- Looking up for such areas of the software system or application code that have not been exercised by any of the test cases.
- Determination of the quantitative measure of the extent of the code coverage which can be thought of as an indirect measure of the software quality.
- Identification of the redundant test cases which does not affect the code coverage.

The whole process is a time consuming one and thus is helped by an automating tool called the code coverage analyzer.

What is Multiple Condition Coverage?

- Multiple condition coverage is a modification of the condition coverage type of the code coverage.

- As the term “multiple condition coverage” itself suggests that it implements almost all the possible combinations of the conditions in order to check whether those possible combinations do occur in the execution of the test cases or not.

- A large number of the test cases are required for carrying out a full multiple condition coverage of a decision and these are supplied by the truth tables of the logical operators obtained for that particular decision.

- The multiple condition coverage offers an added advantage in the languages such as C, C++, Java and so on which have short circuit operators which is that it helps in implementing a thorough testing.

- It in turn makes the multiple condition coverage in such languages very similar to the condition coverage.

Disadvantage of Multiple Condition Coverage

But, one should not think that the multiple condition coverage is void of any disadvantages.

- It does have one which is that the deciding up on the minimum number of test cases required for the coverage proves to be very difficult job.

- The problem worsens as the complexity of the boolean expressions implemented in the decisions increases.

- Another disadvantage adds to the problem of the test case designer which is that the minimum number of the test cases required for the coverage varies substantially from condition to condition having the similar level of complexity.

Lets take an example:

For example, in a decision, first condition may require 5 test cases while the second one may require 11 test cases and both the conditions are having the same number of operators and operands.

In contrast to the condition coverage that includes the decision coverage also, the multiple condition coverage does not include any decision coverage.

Now what about the languages like Pascal and visual basic that do not have short circuit operators? For such languages the multiple condition coverage is implemented in the form of the path coverage for logical expressions.

One point to be noted here is that the advantages and disadvantages remain the same in the path coverage form also since in the path coverage the number of each additional logical operator is doubled the number of the required test cases.

Multiple condition coverage takes in to consideration two or more conditions and then makes them in to a group and then they are tested. "Since the multiple condition coverage requires a very detailed knowledge of the software system or application’s code in order to understand all the decision aiming statements in that and in turn decide for the minimum requirement of the test cases, therefore it has been categorized under the white box testing techniques."

What is meant by code coverage? What are different types of code coverage?

- Code coverage can be defined as a measure to measure the extent to which the source code of a software system has been tested.
- Code coverage is categorized under white box testing techniques since the inspection of the code is carried out directly.
- Code coverage methodology was initially developed for systematic testing of software system. It was developed by two researchers:Miller and Maloney in year of 1963.
- Code coverage is regarded as one of the important considerations concerning the safety certification in the field of avionics equipment.
- Code coverage is not centralized on one idea. There are several different criteria or types to choose from. The criteria or type is chosen as per the requirement of the extent of testing. At a time one or more coverage criteria (types) can be used.

Different Coverage types have been discussed in details below:

Basic coverage type:
This category again has following criteria:
- Statement coverage: It determines whether or not each node in the program has been executed.
- Function coverage: It determines whether or not each function or sub routine in the program has been called.
- Decision coverage: It determines whether or not each edge in the program has been executed and it also ensures that requirements of every branch have been met.
- Predicate coverage: It is also known as condition coverage. It determines whether or each and every Boolean expression in the program has been evaluated to either true value or false value.
- Predicate/ decision coverage: It is combination of both predicate and decision coverage and determines whether or not both the types of coverage are satisfied. Fault injection technique becomes necessary here. It is done to ensure each and every part of the software system has got sufficient coverage.

Modified predicate/ decision coverage: It is usually abbreviated as MC/ DC. Some applications like avionics software applications are safety critical and they require satisfying the modified form of predicate/decisions coverage. The modification here is that each and every individual criterion for condition and decision should have a distinct affect on the outcome and it should be separately visible.

Multiple condition coverage: \
As the name suggests, this type of coverage has two or more conditions. The group of conditions is then tested. Multiple condition coverage determines whether or not all the combinations of the conditions in each decision are tested.

The above mentioned 3 code coverage types are frequently and mostly used coverage types. There are other types which are not so in use. They have also been discussed below:
- JJ path coverage:
It determines whether all the jump to jump kind of paths has been executed.
- Linear code sequence and jump coverage: It determines whether or not all LCSAJs have been executed.
- Entry coverage:
It determines whether or not every possible call has been executed.
- Exit coverage:
It determines whether or not every possible return of the functions has been executed.
- Path coverage:
It determines whether or not every possible path through a given unit has been executed.
- Loop coverage:
It determines whether or not all the loops have been executed.

Testing often achieves 100 percent of code coverage. This is verified through safety critical applications. Attaining full one kind of coverage is practically impossible. Most parts in a program are such that they cannot be accessed easily and therefore remain void of code coverage.There are certain things that are needed to be considered while implementing a code coverage type:

- Requirements of the code coverage for the certification of the finished product.
- Level of code coverage required.
- Testing of code coverage against the tests to verify the requirements.
- Direct generation of object code.

Some best practices that contribute to improved software testing Part III

There is always a search for best practices going on. Some are well known and some hidden. Testing does not stand alone. It is intimately dependant on the development practices. These practices have come from many sources. These practices can be divided in three parts:
- Basic Practices
- Foundation Practices
- Incremental Practices

The incremental practices include:
- Teaming Testers with Developers
This practice should understand the kinds of teaming that are beneficial and the environments in which they are employed. This practice should be more than just a concept.
- Code coverage
Code coverage is the numerical metric that measures the elements of code. this practice should include the information about the tools and methods of how to employ code coverage and track results.
- Automated Environment Generator
Setting up test environments to execute test cases is the most difficult task. This practice should capture the issues, tools and techniques that are associated with setting up the environment, the break down and automatic running of test cases.
- Testing to help ship on demand
Testing process should be viewed as one that enables changes that occur late and handles market pressures and still do not break the product or ship schedule. This practice should identify how to work this concept in organizations.
- State task diagram
State transition diagrams are used to capture functional operations of an application. It allows you to create test cases automatically. This practice has more than one application and one need to capture the tools, methods and uses.
- Memory Resource Failure Simulation
This practice addresses loss of memory because of poor management or lack of garbage collection. It should develop methods and tools for use on different platforms and language environments.
- Statistical Testing
the concept of statistical testing is to use software testing as a means to assess the reliability of software as opposed to a debugging process. It needs to work on the software along an operational profile and measuring interfailure times used to estimate reliability.
- Semiformal Methods
A semi formal method is one where specifications that are captured may be in state transition diagrams or tables that can be used for even test generation.
- Check-in tests for code
Check-in tests couple an automatic test program with the change control system. the chances of the code breaking the build are minimized.
- Minimizing regression test cases
To minimize regression tests, several methods are there out of which one method looks at code coverage and distill test cases to a minimal set. Sometimes, it does confuse a structural metric with a functional test.
- Instrumented versions for MTTF
Mean time between failures(MTTF) can be measured if the failures are recorded and returned to vendor. It enhances the quality of the product that is meaningful to user. It also captures first failure data that benefits the diagnosis and problem determination.
- Benchmark Trends
This practice could be initiated by benchmarking and then advance the practice to include a large pool with customers and competitors.
- Bug Bounties
These are the initiatives that charge the organization with a focus on detecting software bugs.

What is white Box Testing and why we do it ?

White box testing involves looking at the structure of the code. When you know the internal structure of a product, tests can be conducted to ensure that the internal operations are performed according to the specifications and all the internal components have been adequately exercised. In other words, white box testing tends to involve the coverage of the specification in the code.

The control structure of the procedural design to derive test cases is used during white box testing. Using the methods of WBT, a tester can derive the test cases that guarantee that all independent paths within a module have been exercised at least once, exercise all logical decisions on their true and false values, execute all loops at their boundaries and within their operational bounds and exercise internal data structures to ensure their validity.

White box testing is done because black box testing uncover sorts defects in the program. These defects are:
- Logic errors and incorrect assumptions are inversely proportional to the probability that a program path will be executed. Errors tend to creep into our work when we design and implement functions, conditions or controls that are out of the program.
- The logical flow of the program is sometimes counter intuitive, meaning that our unconscious assumptions about flow of control and data may lead to design errors that are uncovered only when path testing starts.
- Typographical errors are random, some of which will be uncovered by syntax checking mechanisms but others will go undetected until test begins.

All we need to do in white box testing is to define all logical paths, develop test cases to exercise them and evaluate results i.e. generate test cases to exercise the program logic exhaustively. We need to know the program well, the specifications and the code to be tested, related documents should be available to us.

Branch Coverage or Decision Coverage in White Box Testing

Code coverage analysis is the process of finding areas of a program not exercised by a set of test cases, creating additional test cases to increase coverage, and determining a quantitative measure of code coverage, which is an indirect measure of quality.
A Decision is a program point at which the control flow has two or more alternative routes. A decision coverage is the percentage if the decision outcomes that have been exercised by a test suite.
Branch coverage testing helps in validating of all the branches in the code and making sure that no branching leads to abnormal behavior of the application. It is a better practice as compared to statement coverage. It goes deeper into the code as compared to statement coverage.
It states whether the the boolean expressions are tested in control structures. It ensures for having adequate number of test cases for every program to ensure execution of every decision or branch at least once.

What are the advantages of branch coverage?
- To validate that all the branches in the code are reached.
- To ensure that no branches lead to any abnormality of the program’s operation.
- It eliminate problems that occur with statement coverage testing.

What are disadvantages of branch coverage?
- There may be other condition that can be used for decision making.
- This metric ignores branches within boolean expressions which occur due to short-circuit operators.

Testing Approaches: Functional Testing versus Structural Testing and Mutation testing

In functional testing, functionality of the module is tested and structure is not considered. Test cases based on specifications and internals of modules are not considered. This type of testing is also known as black box testing. The objective here is to ensure that `correct results'' are obtained when `good inputs'' are applied to the various parts of the design, and when `bad inputs'' are applied the design operates in a predictable manner. Functional testing can therefore be considered as concentrating on checking that the data paths operate correctly.

Structural testing is used to test the implementation of the program. Structural testing, also known as white box or open box testing, is normally applied to sequential HDL code and concentrates on checking that all executable statements within each module. Structural testing involves :
- Statement coverage : This ensures that each and every statement is tested. Software tools called profilers are used to carry this statement coverage.
- Branch coverage : Each and every condition is taken, and inputs are given in such a way that each branch is executed at least once.
- Path Coverage : To test loops, this is required. For example the statement
for(i=o;i<=100;i++) is executed 101 times.

Mutation testing is required to ensure that the software does not fail. It is a good debugging mechanism. The program is modified slightly to obtain mutants of the program. Different mutants are tested with the same test cases. If the mutant fails, and the actual program works correctly, confidence is gained in the program and test cases are considered good. To produce mutants, mutation operators are defined.
- Constant replacement
- Variable replacement
- Arithmetic operator replacement
- Relational operator replacement
- Goto label replacement

Code coverage tools

So you have decided that you need to ensure that your software development cycle has code coverage, and want to move ahead. In addition to making sure that team members have awareness about the principles and about the methods used, another important decision is about the tools to use. There are a wide variety of tools that are available, and here is a listing of the tools. It is hard to recommend a tool, since the actual tool depends on the targets, depends on the developed language (C++, Python, Java, Web language, etc). Here is a sampling of some of the popular tools.

- LDRA Testbed: Automatic instrumentation techniques
- Cobertura: Cobertura is a free Java tool that calculates the percentage of code accessed by tests. It can be used to identify which parts of your Java program are lacking test coverage. It is based on jcoverage.
- Quilt: Quilt is a Java software development tool that measures coverage , the extent to which unit testing exercises the software under test. It is optimized for use with the JUnit unit test package, the Ant Java build facility, and the Maven project management toolkit.
- Clover: Commercial tool. Clover is designed to measure code coverage in a way that fits seamlessly with your current development environment and practices, whatever they may be. Clover's IDE Plugins provide developers with a way to quickly measure code coverage without having to leave the IDE. Clover's Ant and Maven integrations allow coverage measurement to be performed in Automated Build and Continuous Integration systems and reports generated to be shared by the team.
- NUnit: Meant for unit testing. NUnit is a unit-testing framework for all .Net languages. It is written entirely in C# and has been completely redesigned to take advantage of many .NET language features, for example custom attributes and other reflection related capabilities. NUnit brings xUnit to all .NET languages.
- NCover: Commercial tool. NCover 3 includes Symbol and Branch Point Coverage, but it also has two new metrics: Cyclomatic Complexity and Method Visit Coverage. Cyclomatic complexity describes the number of independent paths through your methods. It's a great indicator of when you should re-factor. Code with a low cyclomatic complexity will help you guarantee that new developers on your teams can get up to speed fairly quickly. While method visit coverage previously existed in NCover Explorer, it only worked when you had symbol points for a method.
- Jester: Jester finds code that is not covered by tests. Jester makes some change to your code, runs your tests, and if the tests pass Jester displays a message saying what it changed. Jester includes a script for generating web pages that show the changes made that did not cause the tests to fail.
- Emma: EMMA is an open-source toolkit for measuring and reporting Java code coverage. EMMA distinguishes itself from other tools by going after a unique feature combination: support for large-scale enterprise software development while keeping individual developer's work fast and iterative.
- Koalog Code Coverage: Commercial license. Koalog Code Coverage is a code coverage computation application written in the Java programming language. Koalog Code Coverage allows you to measure the efficiency of your tests suite, but also to discover dead code in your project.
- EclEmma: EclEmma is a free Java code coverage tool for Eclipse, available under the Eclipse Public License. Internally it is based on the great EMMA Java code coverage tool, trying to adopt EMMA's philosophy for the Eclipse workbench.
- GroboCodeCoverage: There are several commercially available code coverage tools for Java, but they all require a large fee to use. This is a 100% Pure Java implementation of a Code Coverage tool. It uses Jakarta's BCEL platform to post-compile class files to add logging statements for tracking coverage. An old tool.
- Hansel: Hansel is an extension to JUnit that adds code coverage testing to the testing framework. What makes Hansel different from other code coverage tools? Most tools try to generate code coverage reports from a test-run of all available tests. But a much more useful information is how much of the code which a test is supposed to test is covered. Hansel gives you this information.
- CodeCover: CodeCover is a free glass-box testing tool developed in 2007 at the University of Stuttgart. CodeCover measures statement, branch, loop, and MC/DC coverage. CodeCover uses the template engine Velocity.
- rcov: code coverage for Ruby: 20-300 times faster than previous tools. Multiple analysis modes: standard, bogo-profile, "intentional testing", dependency analysis. Detection of uncovered code introduced since the last run ("differential code coverage")
- Insure++: a coverage of source code of application tested with functional tests.
- iSYSTEM winIDEA: measures coverage on a wide variety of embedded processors. It works by recoding execution directly on hardware, without instrumenting code or modifying the program and in real-time.
- LDRA Testbed: Measures statement coverage, branch/decision coverage, LCSAJ Coverage, procedure/function call coverage, branch condition coverage, branch condition combination coverage and modified condition decision coverage (MC/DC) for DO-178B Level A.
- VB Watch: Visual Basic code coverage and performance analysis tool
- BullseyeCoverage: C and C++ code coverage tool
- XDebug: PHP debugging tool, including code coverage

If you know of more tools, please add in the comments.

How important is 100% code coverage ?

It can be hard to achieve 100% code coverage, and there are a number of proponents who do not believe in going the extra bit it takes to achieve that. There are people who do not feel comfortable unless they reach a 100% code coverage, and are passionate about it. What are some of the benefits of trying to attain a 100% code coverage ? What is actually 100% code coverage ?
Let's start with the meaning of what code coverage is (if you missed my previous post where I explained code coverage in more detail). Code coverage is a measurement that answers the question about how much of the source code for an application is run when the unit tests for the application are run, and more important, how much code is missed by the unit cases. If 100% of the lines of source code are run when the unit tests are run, then you have 100% code coverage. Seems fairly simple, right. Have you ever wondered why Service Level Agreements don't seem to promise 100%, but do 99%, or even 99.9% ? This example was just a statement thrown in between to demonstrate how getting to 100% is not easy in any field. Let's take this further.
When you write test cases, the principle behind writing the unit test cases and then executing them is to ensure that an application works as expected. So, if your test cases cover more of the code, then it means that the quality of a higher percentage of the code has been verified through testing. In practice, not everything goes by theory, so while 100% code coverage does not guarantee that an application has no bugs, your confidence in the quality of your code increases significantly the higher your code coverage is.
There are many people who feel that trying to target 95% code coverage is a futile target; at the same time, I believe that targeting 95% code coverage is not the way to go. 95% is a great interim target if you have not done code coverage before, but anything less than 100% is a path that you do not want to take. If you testing of the code is only 95%, you are still risking that 5% of the code is untested - this section of the code may be harmless, but may also have some big bugs that could screw up your happiness later. You allow exceptions to creep in, and then pretty soon, pressure causes you to make more exceptions, and an example has already been set. There will always be a niggling feeling that the untested portions of the code can come back to haunt you later.
Reaching there is not easy. It takes a lot of hard work to get everybody to target the 100% code coverage, and needs determination when the temptation is strong to leave things be in the 90% (even more so when sections of the code are older code that may not have seen many changes).

What is code coverage ?

People know what software testing is, and most people in the profession can differentiate between white box and black box testing. However, when you get into more details, and look to identify how testing can provide even greater value, the benefit of measures such as code coverage become apparent. At the same time, there would be a large number of software professionals who are not even aware of what code coverage is, and what are its key benefits. So the idea of this article is to try and articulate some of the benefits.
So what is Code Coverage ?
Well, Code coverage is a type of measure used in software testing which tries to answer the questions about the degree to which the entire source code of an application has been tested. Traditional black box testing, with its focus on functional testing cannot even come close to trying to answer this question, although White Box testing does come closer to trying to answer this question. In fact, if you examine code coverage practices in detail, it is possible to say that code coverage is a form of testing that inspects the code directly and is therefore a form of white box testing.
Code coverage techniques were amongst the first techniques invented for answering the question of systematic software testing. For those proponents of the text plan / case based method of testing, code coverage testing works on the principle that it is entirely possible that sections of a software application remain untouched by test data, and as a result, then it is not possible to say, with any degree of certainty, that these sections do not contain residual errors.
How do you go ahead with actually trying to do code coverage ?
Code Coverage requires support from engineering to proceed. Why is this so ? Getting code coverage in practise requires a different procedure from the normal software build - The target application / software is configured to be built with special options / libraries and/or run under a special environment such that every function that is exercised (executed) in the program(s) is mapped back to the function points in the source code. Doing this systematic process (although requires effort and the value does not seem immediately clear to individual developed and QE) allows developers and quality assurance personnel to look for parts of a system that are rarely or never accessed under normal conditions (error handling and the like) and helps reassure test engineers that the most important conditions (function points) have been tested. Once this exercise has been done, the output is further analysed to see what areas of code have not been exercised and the tests are updated to include these areas as necessary. Once this exercise has been completed (and it may need to be done on a regular basis as the code is in the process of being developed), it gives a much higher level of confidence about the overall quality of the code.