What is the entry and exit criterion for user acceptance testing?

The acceptance is quite an important choice for the clients or the customers. It plays a very important role when it comes to the addressal of the issues related to the acceptance of the software system or application by the client or the customer.

Like any other testing, the acceptance testing also has some of its pre defined entry and exit criteria that a software system or application needs to satisfy before it can undergo the acceptance testing process.

This article is focussed up on those entry and exit criteria only but first let us take a glimpse of what acceptance testing really is so that it becomes easy for us to understand the entry and exit criterion for the acceptance testing.

About User acceptance Testing

- For the field of software engineering, this kind of testing has been termed as the user acceptance testing since it is carried out in order to obtain confirmation from the user or the client that the developed software system or application meets its specified and agreed up on requirements and specifications.

- This confirmation is provided by the SME or the subject matter expert who is the owner of the software system or application under testing after carrying out several trials and reviews.

- The user acceptance is therefore one of the final software testing methodologies that is carried out before handing over the software system or application to its owner.

- The user acceptance testing is preferably carried out via the users of that software system or application which are in the contact of the client or mentioned in the users requirements specification document.

- As many as the formal tests required are created by the test developer or designer based on the different levels of the severity of the errors and flaws.

- Typically, for an ideal acceptance testing the test designer should handle the creation of the formal system and integration test cases for the same software system or application.

- The user acceptance testing serves as a means of final verification of the well functioning of the software system or application by creating the real world conditions for the its usage as it will used by the customer and required business function under process.

- The system needs to perform as intended because then only it can be subjected to its reasonable extrapolation in the process of product at the same level of the stability and reliability.

- Unlike other software testing methodologies, the test cases of the user acceptance testing do not serve to identify the simple problems, errors and show stopper defects (system crashes, failures and hangs etc).

- It is so because all such defects are fixed by the testers and developers in the earlier stages of the software testing life cycle.

- There is another reason for this testing to be performed which is to give confidence and assurance to the client or the customer that the system will perform well in the production phase.

- Some contractual or legal requirements are also signed at the end of the acceptance testing.

Entry Criterion for User Acceptance Testing

1. The transition meeting of the integration testing must be signed off.
2. The functional requirements and the business requirements have been met and verified in the integration testing.
3. The test cases for the user acceptance testing are ready to be executed.
4. The test environment for the UAT should be ready.
5. Required access of the resources for testing should be granted.
6. All the critical bugs have been previously addressed.
7. The reports of the previous testing should be handed over to the client.

Exit Criterion for User Acceptance Testing
1. No defects are found.
2. Defects with the medium priority are found.
3. There is no hindrance in the business process.
4. The UAT meeting is signed off.

What are different characteristics of beta testing?

Beta testing is carried out after the successful completion of the alpha testing. As alpha testing can be regarded as the internal form of user acceptance testing, similarly beta testing can be regarded as the external form of the user acceptance testing.

- A few versions of the finished software application are released. These testing versions are called the beta versions of the software system.
- These beta versions of the software system are released to a certain limited audience.
- The audience consists of the people who are not a part of the programming team or the software development team.

- This is done so because the more number of people, there will be more exploitation of the software. This is done to ensure that the number of bugs is minimum.
- Sometimes, the software manufacturer company may decide to release the beta versions of the software to a huge open public so as to get more feedback on the quality of the software application.
- Alpha testing is the internal pilot test and beta testing is called the external pilot test.

- Beta testing is carried out on only on the software that has successfully passed the first level unit testing, integration testing, system testing, internal pilot test and removal of the faults or the bugs.
- Beta testing is carried out because the finished software product may still have some minor errors and bugs. In order to find them, user participation is very much needed.
- The beta versions are released to some selected customers only to simulate a normal execution environment for the software application to make it run normally and to spot the problems in such an environment.

- Beta testing phase starts when the software development is complete.
- Beta testing can be thought of as a way of incorporating usability testing.
- The process of releasing the beta versions to the selected customers and other suitable audience is known as beta release.

- Usually after the development of the software system, this is the first time for which the software becomes available to the public.
- The audience selected for carrying out beta testing on the beta version of the software system is called beta testers.
- Beta testers are generally the prospective customers of that particular software manufacturing company who are willing to test the software system free of charge. As a reward for this, they are given the completed and finished software free of cost or sometimes at a reduced price.

- The beta testing phase is known by many names such as prototype, preview, early access or technical preview (TP).
- One is normal beta testing, there’s one more kind of called perpetual beta testing.
- In perpetual beta testing, new features and functionalities are continually added to the software system and therefore, not declaring the beta version as the final release of the company.

- In the context of software development, beta testing is considered as the second phase of the software testing stage.
- Beta testing is nothing but the pre release testing. Beta versions provide a preview of the final release of the firm.
- Alpha testing and beta testing collectively form the acceptance testing. So we can say that the beta testing is the second phase of the acceptance testing.

- Beta testing is conducted in the customer environment by multiple customers.
- Beta testing also ensures that the behavior of the software system is same in the development environment and in the customer environment.
- Beta testing is time consuming and may take weeks and months. The whole application is installed in the customer environment and the development of the software system has been 100 percent completed.

What are different characteristics of acceptance testing?

In software engineering and development, acceptance testing is defined as the testing which is carried out to check if all the kinds of requirements specified by the customer in the contract have been fulfilled or not.

Acceptance is a composition of 3 kinds of tests namely:
- Physical tests
- Chemical tests
- Performance tests

These are the 3 mostly used tests. Other than these, acceptance testing some times may include black box testing to be performed on the software system before the delivery of the software product.

Such kind of black box testing carried out is known by many names such as functional testing, QA testing, final testing, confidence testing, validation testing, factory acceptance testing etc.

It is important to distinguish between the acceptance testing done by the client, user or the customer and acceptance testing done by the system provider prior the delivery of the software artifact i.e., it should be done before the ownership is transferred from the software developer company to the client.

Here the acceptance testing carried out by the customer is called user acceptance testing or UAT in short form. There are other names also for this kind of testing like end user testing, field acceptance testing or site acceptance testing.

The smoke test is the first step of the acceptance testing. It is carried out just before the main and most important step of the acceptance testing.

Acceptance testing generally involves execution of combination of test cases on the completed and finished software system or the application where each and every individual test case implements a particular condition for the operation of the program based on the environment of the user and on the features of the program.

Each test case results in either a pass condition or a fail condition. This output is of the Boolean type. Its very important that the testing environment of the software system should resemble as much closely as possible the environment as anticipated by the user or the client.

It should include the extreme features of user’s anticipated environment without fail. It’s not possible for any testing environment to exactly resemble the user’s anticipated environment. Therefore there is no degree of success or failure defined here.

The test cases are accompanied by the data values which have to be entered as input and a brief description of the functional activity which has to be performed. It also contains a brief description of the expected outcomes and results.

- Acceptance test cases and conditions are created by the business clients or the customers.
- The agreement or the contract is written in some business domain language.
- The acceptance test is carried out against the given input data with the test case and sometimes an acceptance test script is used to give directions for the testing.
- The results thus obtained are then compared with the expected results.
- If both the results show up a proper match, then the test case is said to be successful. If the two results don’t match, then the software system is either accepted on the former conditions or it is rejected.
- Acceptance testing is aimed at providing the confidence to the client that the produced software artifact meets all the standards specified in the contract.
- The acceptance testing is also known as the final gateway.
- Acceptance testing works on the principal that acceptance testing once completed successfully i.e., all the conditions have been met successfully, the contractors and the software developers will declare the software system as satisfying, and the client pays off the manufacturer.

Therefore UAT can be defined as a process to confirm that a software system satisfies all the requirements of the contract.

What comprises Test Ware Development : Test Plan - Acceptance Test Plan (ATP)

The client at their place performs the acceptance testing. It will be very similar to the system test performed by the software development unit. Since the client is the one who decides the format and testing methods as part of acceptance testing, there is no specific clue on the way they will carry out the testing. But it will not differ much from the system testing. Assume that all the rules, which are applicable to the system test, can be implemented to acceptance testing also.

Since this is just one level of testing done by the client for the overall product, it may include test cases including the unit and integration test level details.

Test Plan Outline

- BACKGROUND: This item summarizes the functions of the application system and the tests to be performed.
- INTRODUCTION
- ASSUMPTIONS: Indicates any anticipated assumptions which will be made while testing the application.
- TEST ITEMS: List each of the items(programs) to be tested.
- FEATURES TO BE TESTED: List each of the features(functions or requirements) which will be tested or demonstrated by the test.
- FEATURES NOT TO BE TESTED: explicitly lists each feature, function, or requirement which would not be tested and why not.
- APPROACH: Describe the data flows and test philosophy. This section also mentions all the approaches which will be followed at the various stages of the test execution.
- ITEM PASS/FAIL CRITERIA: Itemized list of expected output and tolerances.
- SUSPENSION/RESUMPTION CRITERIA: Must the test run from start to finish? Under what circumstances it may be resumed in the middle? Establish check-points in long tests.
- TEST DELIVERABLES: What, besides software, will be delivered? It includes test report and test software.
- TESTING TASKS: It includes functional and administrative tasks.
- ENVIRONMENTAL NEEDS: It includes security clearance, office space and equipment and hardware/software requirements.
- RESPONSIBILITIES: It includes what are the tasks in section 10? What does the user do?
- STAFFING AND TRAINING
- SCHEDULE
- RESOURCES
- RISKS AND CONTINGENCIES
- APPROVALS

The schedule details of the various test pass such as unit tests, integration tests, system tests should be clearly mentioned along with estimated efforts.

Validation phase - User Acceptance Testing and Installation Testing

User Acceptance Testing occurs just before the software is released to the customer. The end-users along with the developers perform the User Acceptance Testing with a certain set of test cases and typical scenarios.

Installation testing is often the most under tested area in testing. this type of testing is performed to ensure that all installed features and options function properly. It is also performed to verify that all necessary components of the application are, indeed, installed. Installation testing should take care of the following points:
- To check if while installing product checks for the dependent software/patches.
- The product should check for the version of the same product on the target machine, say the previous should not be over installed on the newer version.
- Installer should give a default installation path.
- Installation should allow user to install at location other than the default installation path.
- Check if the product can be installed "Over the Network".
- Installation should start automatically when the CD is inserted.
- Installer should give the Remove/Repair options.
- When uninstalling, check that all the registry keys, files, DLL, shortcuts, activeX components are removed from the system.
- Try to install the software without administrative privileges.
- Try installing on different operating system.
- Try installing on system having non-compliant configuration such as less memory/RAM/HDD.

Advantages and Disadvantages of Black Box Testing and what are its types.

In black box testing, the internals of the system are not taken into consideration. The testers do not have access to the source code. A tester who is doing black box testing generally interacts through a user interface with the system by giving the inputs and examining the outputs.
The advantages of black box testing includes that it is very efficient for large segments of code. It clearly separates user's perspective from developer's perspective. The code access not required. It is very easy to execute.
The disadvantages of black box testing includes limited code path coverage as limited number of inputs can be checked. It cannot control targeting code segments or paths which may be more error prone than others.

There are different kinds of testing that are associated with black box testing :
1. Smoke Testing
2. User Input Testing
3. User Acceptance testing
It includes :
- Alpha testing
- Beta testing
4. System Testing
- Functional testing
- User interface testing
- Usability testing
- Compatibility testing
- Model based testing
- Error exit testing
- User help testing
- Security testing
- Capacity testing
- Performance testing
- Sanity testing
- Regression testing
- Reliability testing
- Recovery testing
- Installation testing
- Maintenance testing
- Accessibility testing

Overview of User Acceptance testing - what does it mean and what are its forms.

User Acceptance Testing is often the final step before rolling out the application. Usually the end users who will be using the applications test the application before ‘accepting’ the application.
A formal product evaluation performed by a customer as a condition of purchase. The testing can be based upon the User Requirements Specification to which the system should conform. Use Acceptance testing is black box testing.

USER : System developers cannot do it, as although they are expert in writing software, they are unlikely to know anything about the realities of running the organisation, other then what they have acquired from requirements specifications, and similar documents.

Acceptance : The acceptance of a system means you are confident it will give benefit to the organisation. It does not mean that it only meets the original specification as requested.

Testing : Whenever people are asked what testing is, many of them say it is to prove the system works.
The point of UAT is for business users to try and make a system fail, taking into account the real organisation it will be working in.

User Acceptance testing can be in the form of :
Alpha Testing - Tests are conducted at the development site by the end users. Environment can be controlled a little bit in this case.
Beta Testing - Tests are conducted at customer site and development team do not have any control on the test environment.