What is application programming interface testing?

An application programming interface or API as it is known by its abbreviated form can be defined as a specification based on source code. Typically application programming interface is intended to be used as an interface in the software components in order to communicate with each other. An application programming interface includes specifications and requirements basically for required data structures, routines, variables, and object classes.

The requirements and specifications of application programming interface can take many forms such as POSIX which is an international standard or Microsoft windows API which is vendor documentation or standard template library or STL (in java AP or C++) which is the library of the programming languages.

Application programming interface and application binary interface are in great contrast to each other. The basic difference is that the application programming interface is based on source whereas the application binary interface is a binary interface. The best example is given by POSIX and Linux standard base. The POSIX is an application programming interface, whereas the Linux standard base in an application binary interface.

An application programming interface has got any features.
- It is language dependent. It means that the application programming interfaces is only available by using the elements and syntax of a particular programming language. This feature makes the application programming interface more easy and convenient to use.

- An application programming interface can also be independent of language. This feature helps in calling the application programming interface from several programming languages. This is the most desirable feature for an application programming interface that is service oriented. Such an API is not bound to specific system or process. It may be provided as a web service or remote procedure calls.

- Sometimes the term “application programming interface” is used to refer to a complete programming interface, a set of APIs that have been provided by an organization or a single function. Therefore, the scope of the meaning of an API is usually determined according to its usage.

- In some cases an application programming testing may describe the way in which a task is performed. An action is usually meditated by a function call in procedural programming languages like C. hence; here API usually describes all the functions and routines provided.

- Other times the application programming interface can be interpreted as a collection of the files included by the C language and its readable descriptions.

- There are various programs development environments that provide the documentations associated with an application programming interface in digital format. Example of such language is Perl which comes with a tool perldoc.

When it comes to object oriented languages, application programming interface provides a description of definitions and behaviors associated with a set of class.

- An application programming interface generally prescribes the methods using which one can interact or handle the class derived objects.
- An application programming interface is closely related to a software library.
- Library is the actual implementation of the rules that are set by the application programming interface.
- Like other this API factor also needs to be tested. For this, API testing is employed.
- API testing is somewhat different from other types testing since GUI is rarely involved in it. for API testing, one needs to set up the testing environment and invoke the API with its respective parameters and then analyze the result.
- The problems come while setting up the environment as the GUI is not involved. For cases that don’t return anything one needs to have some mechanism to check the API’s behavior.

There are 3 main challenges faced by API testing:
- Parameter combination
- Parameter selection
- Call sequence

What are different kind of application programming interface testing tools?

There are many testing tools available. Depending on the level of testing required, different tools could be used. Some of the API testing tools available are:

- JVerify: This is from Man Machine Systems. JVerify is a java class/API testing tool that supports a unique invasive testing model. The invasive model allows access to the internals of any Java object from within a test script. The ability to invade class internals facilitates more effective testing at class level, since controllability and observability are enhanced. This can be very valuable when a class has not been designed for testability.

- JavaSpec: JavaSpec is a SunTest's API testing tool. It can be used to test java applications and libraries through their API. JavaSpec guides the users through the entire test creation process and lets them focus on the most critical aspects of testing. Once the user has entered the test data and assertions, JavaSpec automatically generates self-checking tests, HTML, test documentation, and detailed test reports.

To automate API testing, the assumptions are as follows:
- Test engineer is supposed to test some API.
- The APIs are available in form of library(.lib).
- Test engineer has the API document.

There are mainly two things to test in API testing:
Black box testing of the APIs :
In this, we have to test the API for outputs. When we give a known input then we also know the ideal output. So, we have to check for the actual output against the idle output.
A simple program in C can be written as:
- Take the parameters from text file.
- Call the API with these parameters.
- Match the actual and idle output and also check the parameters for good values that are passed with reference.
- Log the result.

Interaction/ integration testing of the APIs.
Suppose there are teo APIs say
Handle h = handle createcontext(void);
When the handle to the device is to be closed then the corresponding function
Bool bishandledeleted = bool deletecontext(handle &h);

Here, we have to call the two APIs and check if they are handled by the created createcontext() and are deleted by the deletecontext().
This will ensure that these two APIs are working fine.For this we can write a simple C program that will do the following:
- Call the two APIs in the same order.
- Pass the output parameter of the first as the input of the second.
- Check for the output parameter of the second API.
- Log the result.

Step 4 To test API : Call Sequencing, Step 5 To Test API : Observe the output

Step 4: Call Sequencing
When combinations of possible arguments to each individual call are unmanageable, the number of possible call sequences is infinite. Parameter selection and combination issues further complicate the problem call-sequencing problem. Faults caused by improper call sequences tend to give rise to some of the most dangerous problems in software. Most security vulnerabilities are caused by the execution of some such seemingly improbable sequences.

Step 5: Observe the output
The outcome of an execution of an API depends upon the behavior of that API, the test condition and the environment. The outcome of an API can be at different ways i.e. some could generally return certain data or status but for some of the APIs. It might not return or shall be just waiting for a period of time, triggering another event, modifying certain resource and so on.

The tester should be aware of the output that needs to be expected for the API under test. The outputs returned for various input values like valid/invalid, boundary values etc needs to be observed and analyzed to validate if they are as per the functionality. All the error codes returned and exceptions returned for all the input combinations should be evaluated.

Step 3 To test API : Identify the combination of parameters

Identify the combination of parameters
Parameter combinations are extremely important for exercising stored data and computation. In API calls, two independently valid values might cause a fault when used together which might not have occurred with the other combinational values. Therefore, a routine called with two parameters requires selection of values for one based on the value chosen for the other. Often the response of a routine to certain data combinations is incorrectly programmed due to the underlying complex logic.

The API needs to be tested taking into consideration the combination of different parameter. The number of possible combinations of parameters for each call is typically large. For a given set of parameters, if only the boundary values have been selected, the number of combinations, while relatively diminished, may still be prohibitively large. For example, consider an API which takes three parameters as input. The various combinations of different values for the input values and their combinations needs to be identified.

Parameter combination is further complicated by the function overloading capabilities of many modern programming languages. It is important to isolate the differences between such functions and take into account that their use is context driven. The APIs can also be tested to check that there are no memory leaks after they are called. this can be verified by continuously calling the API and observing the memory utilization.

Step 1 and Step 2 To test API : Identify the initial condition and Input/Parameter Selection

STEP 1: Identify the initial condition
The testing of an application programming interface (API) would depend largely on the environment in which it is to be tested. Hence, initial condition plays a very vital role in understanding and verifying the behavior of the API under test. the initial conditions for testing APIs can be classified as:

- Mandatory Pre-setters
The execution of an API would require some minimal state, environment. These type of initial conditions are classified under the mandatory initialization for the API. For example, a non-static member function API requires an object to be created before it could be called. This is an essential activity required for invoking the API.

- Behavioral Pre-setters
To test the specific behavior of the API, some additional environmental state is required. These types of initial conditions are called the behavioral pre-setters category of initial condition. These are optional conditions required by the API and need to be set before invoking the API under test thus influencing its behavior. Since these influence the behavior of the API under test, they are considered as additional inputs other than the parameters.

Thus, to test any application programming interface, the environment required should also be clearly understood and set up. Without these criteria, API under test might not function as required and leave the tester's job undone.

STEP 2: Input/Parameter Selection
the list of valid input parameters need to be identified to verify that the interface actually performs the tasks that it was designed for. While there is no method that ensures this behavior will be tested completely, using inputs that return quantifiable and verifiable results is the next big thing. The techniques like boundary value analysis and equivalence partitioning need to be used while trying to consider the input parameter values. The boundary values or limits that would lead to errors or exceptions needs to be identified.

It would also be helpful if the data structures and other components that use these data structures apart from the API are analyzed. The data structure can be loaded by using the other components and the API can be tested while the other component is accessing these data structures.

The availability of the source code to the testers would help in analyzing the various input values that could be possible for testing the API. It would also help in understanding the various paths which could be tested. Therefore, not only are testers required to understand the calls, but also all the constants and data types used by the interface.

What is the strategy that is needed to test an API?

By analyzing the problems faced by the testers, a strategy needs to be formulated for testing the application programming interface.
- The API to be tested would require some environment for it to work. Hence, it is required that all the conditions and prerequisites understood by the tester.
- The next step would be to identify and study its points of entry. The graphical user interfaces would have items like menus, buttons, check boxes, and combo lists that would trigger the event or action to be taken.
- Similarly, for APIs, the input parameters, the events that trigger the API would act as the point of entry. Subsequently, a chief task is to analyse the points of entry as well as significant output items. The input parameters should be tested with the valid and invalid values using strategies like the boundary value analysis and equivalence partitioning.
- The fourth step is to understand the purpose of the routines, the contexts in which they are to be used. Once all this parameter selections and combinations are designed, different call sequences need to be explored.

The steps can be summarized as follows:
- Identify the initial conditions required for testing.
- Identify the parameters i.e. choosing the values of individual parameters.
- Identify the combination of parameters i.e. pick out the possible and applicable parameter combination with multiple parameters.
- Identify the order to make the calls i.e. deciding the order in which to make the calls to force the API to exhibit its functionality.
- Observe the output.

How can the testing of API calls be done?

Testing of API calls can be done in isolation or in sequence to vary the order in which the functionality is exercised and to make the API produce some useful results from these tests. Designing tests is essentially designing sequences of API calls that have a potential of satisfying the test objectives.This in turn boils down to designing each call with specific parameters and to building a mechanism for handling and evaluating return values.
Designing of test cases depends on the following criteria:
- What value should a parameter take?
- What values together make sense?
- What combination of parameters will make APIs work in a desired manner?
- What combination will cause a failure, a bad return value, or an anomaly in the operating environment?
- Which sequences are the best candidates for selection?

Some of the interesting problems for testers are:
- Ensuring that the test harness varies parameters of the API calls in ways that verify functionality and expose failures. This includes assigning common parameter values as well as exploring boundary conditions.
- Generating interesting parameter value combinations for calls with two or more parameters.
- Determining the content under which an API call is made. This might include setting external environment conditions like files, peripheral devices and also the internal stored data that affect the API.
- Sequencing API calls to vary the order in which the functionality is exercised and to make the API produce useful results from successive calls.

Understanding Application Programming Interfaces(APIs)

Application Programmable Interfaces (APIs) are collections of software functions or procedures that can be used by other applications to fulfill their functionality. APIs provide an interface to the software component.These form the critical elements for the developing the applications and are used in varied applications from graph drawing packages, to speech engines, to web-based airline reservations systems, to computer security components.

Each API is supposed to behave the way it is coded i.e. it is functionality specific. These APIs may offer different results for different type of the input provided. The errors or the exceptions returned may also vary. However, once integrated within a product, the common functionality testing/integration testing may cover only those paths. By considering each API as a black box, a generalized approach of testing can be applied. But, there may be some paths which are not tested and lead to bugs in the application. Applications can be viewed and treated as APIs from a testing perspective.

The distinctive attributes that make testing of APIs slightly different from testing other common software interfaces like GUI testing.
- Testing APIs requires a thorough knowledge of its inner workings: Some APIs may interact with the operating system kernel, other APIs, with other software to offer their functionality. Thus, an understanding of the inner workings of the interface would help in analyzing the call sequences and detecting the failures caused.
- Adequate programming skills: API tests are in form of sequences of calls, namely, programs.Each tester must possess expertise in the programming language that are targeted by the API.
- Lack of domain knowledge: Involve the testers from the initial stage of development. This would help the testers to have some understanding on the interface and avoid exploring while testing.
- No documentation: Without the documentation, it is difficult for the test designer to understand the purpose of calls, the parameter types and possible valid/invalid values, their return values, the calls it makes to other functions and usage scenarios. Hence, proper documentation would help test designer design the tests faster.
- Access to source code: The availability of the source code would help tester to understand and analyze the implementation mechanism used; and can identify the loops or vulnerabilities that may cause errors.
- Time constraints: Thorough testing of APIs is time consuming, requires a learning overhead and resources to develop tools and design tests.