Ensuring that there are clear agreements over deliveries from third parties

In today's world of software development, most large products use input components from a variety of sources. So, if you are building a new version of a video editing application, it tends to incorporate a large number of external components - some of these dealing with the ability of read views of different formats and codecs, then there will be the components dealing with burning what you created to a disc for which there are additional components, and so on. In all such cases, it always makes sense to use components that have a specialized function rather than try to build it yourself. In the current case, see how it difficult it would have been if you team was trying to write code to read the different video formats in existence today. You would have mostly given up trying to do such a thing. Even otherwise, there are other components that are used - if you are working for a company that has multiple applications, the company would ensure that common components are built at one place and included in the various application (this is the most efficient way of building such components); for example, you would be using a Help system, your application would be using Installers for installing the application onto the machine, you would have having a licensing system, you might be having a common system for generating the UI of the application, and so on.
Now, this sounds like  the most efficient way of building such an application. However, for somebody who has been there and done that, there are many slips between getting a robust system in place that integrates all these components in your schedule. For example, one of the biggest problems that we used to run into was the quality level of these components. We had a contract with one of our component makers who would supply us a component (and they used to supply the same component to our rivals and other applications in the video space). From time to time, we would run into problems where these components would not be delivered in time, or where we would have to reject the quality level of the component, because we found a critical bug.
Why were all these problems happening ? I am talking about a situation which used to happen a few years back, and it was extremely frustrating. However, it turned out that we had been working with them for many years, and we had never really tried to set some kind of process for these deliveries of the component and hence all the problems. So, we started out by working the timelines for when we need these components, and then did more analysis while working with the development and testing teams about the level of testing that we expected from the vendor end. Now, all this was being done from our end, and we had only aired some level of frustration with the vendor, but never really got in a problem solving discussion.
We set up a series of meetings with the vendor, talking them through what the current system was, pointed out the problems it was causing for us (and for them as well, since when we rejected a version of the component, it would have meant more effort from their end as well). We talked them through some of the solutions that we were looking at, we got our development and testing teams to have discussions with their respective teams on a regular basis. Now, this was not a magic bullet; they did not do everything that we had asked for, and in some cases, it meant that we had to pay around 10% more because of the extra effort on their side. However, in the end, we had a more strict agreement for the delivery of the component along with a quality level, and this allowed up stability in a part of our schedule, which was well worth it. 

What are the challenges of testing Web Applications? How to use Silk Test with Web Applications?

Web applications are a rage these days and thus a fast delivery of web applications is in urgent need. But at the same time it is important to keep the security of these web applications in check. 

However, web testers continue to face some challenges while testing the web applications and this is what that we are going to discuss in this article. 

What are Web Applications?

- As we are familiar with the growing popularity of the web applications, with the time, new and advance features are being introduced in the web applications.

- It makes them one of the most popular means for the development of the commercial systems. 

- Nowadays, almost all the companies opt for developing web based software wherever and for whatever possible. 

- This is so because with the help of web applications the company can easily cater to the needs of a large number of people. 

- Also, once the infrastructure of the application is put in place the deployment of the apps becomes quite easy. 

- The web applications are powerful assets and using them, a feature rich content can be provided to a large audience across the globe at quite an economical cost.

- Hence, it becomes more and more daunting to test these complex applications. 

- For the web applications the two factors continue to be the most important:

1. Accessibility and
2. Security

Challenges in testing Web Application

While on one side, the company wants to cater to the users all over the world, on the other side their web application can fall prey to some grave security issues. Thus, only web application testing stands a chance to secure the data as well the organization. 

- The design and nature of every web application is somewhat unique to it and so it is possible for the users to follow different paths of usage.

- In such cases, it is necessary to ensure that all paths are working well by testing all the permutations and combinations. 

- Also, all the software applications are not self explanatory. 

- Users of a web application are from various back grounds and those who are from a non technical background may find it hard to use the application. 

- This does affects the design of the application and therefore should be tested during the usability testing of the application. 

- For intranet based web applications, it is possible for the developers to make assumptions regarding the audience but this is impossible for the internet based web applications. 

- Today, there are many web browsers available and so different browsers are used by different users. 

- Thus, there is a need to test this aspect since it can be ensured that the web application is compatible with all the platforms.

How is web testing done in SilkTest?

- Web testing is possible using the Borland’s silk test. 

- Till now, it has been proven to be the leading functional testing tool in the software industry when it comes to testing e- business applications either based up on windows, web, java or client or server based and so on. 

- Silk test is used for the following purposes:

1. Test planning
2. Test management
3. Direct data base access and validation

- It also comes with a built – in recovery system for the purpose of unattended testing of the web applications across multiple browsers, platforms and technologies.

What are different kind of browser bugs?

It is a universal fact that every thing in this world is infected with some discrepancies. So does it hold well for the web browsers! This article is here to discuss about the bugs associated with the web browsers.

Till date so many bugs of the web browsers like Mozilla, internet explorer, Netscape, opera and so on have been discovered. Let us check out some of the prominent errors:

Multiple browsers bugs

Bugs discovered under this category are:

(a) Font variant ligatures are not implemented by most of the bugs. The font module level 3 of the CSS defines the property of the font variant ligatures to specify the use of ligatures. Till now no such browser has been developed that supports this property fully.

(b) Font weight is not implemented consistently by the browsers. This font weight property of CSS lays down the specifications regarding the numeric values and key words.

(c) Data tables are not managed properly by the browsers. Most of the table properties are not supported by the browsers.

(d) Layout affected by outlining property.

(e) Styling legend tags

(f) Bugs in Mozilla Firefox
Styling is not applied and problems have been experienced with the horizontal scroll bar.

(g) Bugs in Internet Explorer
When the cursor is hovered over some elements, they did not work properly with the forms; the left origin of the positioning coordinates is incorrectly set.

(h) Bugs in Opera
&rsquo and &isquo entities were not recognizable by the opera, but this was later fixed in the 8th version.

(i) Bugs in Safari
These crashes were reported when : hover :: after was used.

(j) The borders separating the head and the body sections are often placed incorrectly by the opera.

(k) Multi-column ordered list remembering.

(l) Backgrounds show through invisible tables.

(m) Buttons with images cannot be aligned with those having only text.

(n) In some browsers even the fixed elements align along with the adjacent elements.

WHAT CHALLENGES ARE FACED WHILE DEVELOPING A WEBSITE

- When you start developing or designing your web sites choose carefully between the CSS and HTML.

- Don’t go for such an advance version of the languages because if you did then you may run in to the problems with the incompatibility of the browser.

- There is so much of competition going around the whole software market.

- The browser developers are releasing new browsers at very fast pace without even testing them properly which then becomes a headache for the web site designers.

- The new languages being used today like HTML 5 and CSS 3 are now gaining very much popularity though they are pretty much complex then their preceding versions.

- Number of features to be implemented is quite huge which is the major cause of the bugs.

- Whenever you come across a bug, do not forget to report it since it may be so very rare that nobody else would have stumbled up on it.

- One thing you should always remember is to keep the problem in the reduced form as much as possible.

- For doing this you an simply make up a copy of the code and from that remove the java script or CSS files one by one and ultimately you’ll come at a point when the problem will go away.

- Now you add that file and remove the others from the ones that you did previously.

- If you find that the bug is now going away even after removing all the files then it is likely that the bug lies in HTML coding.

- Now after you know that which aspect houses the problem, you need to locate that file or code.

What are different manual testing challenges?

Everything in this world has got some positive sides and some negative sides, some advantages and some disadvantages, and of course challenges! This holds utterly true with the technology too. Manual testing being so unsophisticated faces challenges too.

WHAT IS MEANT BY MANUAL TESTING?

- Manual testing involves a tester who is supposed to carry out the testing processes manually for finding out the errors and bugs.
- The tester here tests a software system or application with a view of an end user.
- All the features and functionalities of the software system or application are exploited to the most possible extent.
- As the tester cannot mentally track the whole testing process, he follows a written test plan.
- This also ensures that no important test case is missed.
- Even today after the invention of many modern testing technologies, most of the software engineering projects rely on manual testing since it involves a rigorous testing procedure.
- Manual testing works a long way in digging out more defects.
- In a typical manual testing a high level testing plan is followed.
- All the resources like software licenses and skilled people and computers are identified.
- The test cases are written in detail along with the procedure to be followed and the expected outcome.
- Different test cases are assigned to different testers who carry out the testing manually.
- A detailed test report is prepared.
- Manual testing demands skills because without skill the tester might falter.

CHALLENGES FACED IN MANUAL TESTING

1. Manual testing cannot be used to test out the whole application. It can only be used for some parts. The test cases are so large in numbers that it becomes impossible to execute all of them manually. If you were to execute all of those test cases, testing will take too much of time. You won’t be able to complete the testing within the stipulated period of time.

2. Always pay attention to the company defined processes. You should be well informed with the purpose these processes serve. Often following the company defined processes leas to incomplete software testing. The company processes often don’t keep up with the tester’s methodologies or test plan.

3. Manual testing requires good skills. The main skills required are of trouble shooting, analyzing and communication.

4. As the tester gets more hold of the software system or application testing, more and more test cases and errors and bugs come in to the scene and it become pretty much difficult to keep on testing the software system or application further. This is where the regression testing comes in to the play.

5. You should be careful while choosing the team members. They all should be skilled. Unskilled testers can further aggravate the problem rather than simplifying it. This also leads to inappropriate testing.

6. Manual testing should be governed by the time constraint. There is no time for executing each and every test case. The tester usually focuses on the completion of the task rather than focusing on the quality of the testing. There are a whole lot of the tasks to be performed like executing, documentation, automation and reviewing the test scenario.

7. The problem of sorting the test cases according to the priority often comes in way while following the manual testing. Defining a criterion for the sorting of the test cases well in accordance to the priority.

8. In manual testing the requirements of the software system or application are often misunderstood.

9. The reuse of test scripts is a difficult task in manual testing.

These challenges require analyzing skills rather than any other kind of skills.

What are different automated testing challenges?

Automated testing though being a modern testing methodology, faces challenges. Test automation is used whenever a large number of test cases are to be executed. A huge number of test cases cannot be executed manually.

AUTOMATED TESTING
- In automated testing, an automation tool or software is employed to handle the execution of the test cases.
- Such software even compares the actual outcome to the expected outcome.
- It efficiently generates the test data and also the preconditions.

MANUAL AND AUTOMATED TESTING
- Test automation process is mainly used to automate a manual testing process which is typically a much formalized one.
- Although manual testing is way more effective than the automated testing, it consumes too much of time and requires a larger effort.
- Though, manual testing is able to discover more errors and bugs, there are certain kinds of bugs that cannot be discovered up by manual testing.
- For automated testing a computer program is written that will take up the whole testing process which otherwise would have required manual work.
- Manual testing sounds like quite a drudgery! Isn’t it?
- Once the automated software is ready, the test cases can be executed quickly and can also be repeated any number of times without much fuss.
- Automated testing proves to be time, efforts and cost effective method for the software projects that are way too lengthy and need more maintenance.

CHALLENGES FACED IN AUTOMATED TESTING
Test automation tools are expensive and therefore they are employed in certain definite combinations with the manual testing. There are few challenges faced in automated testing:

1. COST EFFECTIVE
One of the challenges that automated testing faces is that it becomes cost effective only when used in the long term i.e., for regression testing of a software system or application.

2. Requirements
- The requirements for a software system or application are identified from the user stories.
- User stories are basically a description like of the software product.
- These requirements are sorted according to the priority as follows:

(a) High priority Requirements
These are the most critical requirements of a software system or application and have to be included in to the software before it is released to the public.
(b) Medium priority Requirements
These requirements are not so critical to the software system or application and hence can be worked around later. But, it should be done before the implementation.
(c) Low priority Requirements
These requirements are added just to enhance the overall quality of the software system or application. But, the software can do without these requirements.

After sorting up on the requirements in the order of their priority, the iterations for the release are planned and each iteration release takes about 1-2 months. Meanwhile too much of change occurs in the requirements of the software. Many a times such changes cause the iterations to bump. This is indeed a great challenge that hinders the implementation of the automating processes.

3. Tools
- The traditional tools don’t seem to work for all kinds of automated testing processes.
- For example, the traditional tools cannot be used for the agile automation process since they are capable of solving only the traditional problems.
- This makes it difficult to implement the automation in the early stages of the testing cycle.
- The deployment of the automated testing processes become simpler only after the whole software comes to the verge of completion since by then many issues had been settled.
- So, making a wise selection of tools is crucial for the automation and also poses a big challenge.

4. Effective resource management as the program grows poses another problem.
5. Lack of effective communication among the team members.

Some of the challenges involved in the process of using test automation tools

Description of automated decision making problems

In the past decade or so, test automation has jumped up to being a solution touted by a number of experts for improving quality. Even when you consider the case of institutions that provide training on testing concepts, their courses that cost the most are those related to automated testing tools and how to become an expert on using them. However, it should be considered that taking a decision to go for automated testing needs to be a thoroughly considered decision, with the cost and benefits needing to be calculated (in one small company, I realized that the decision to go in for automated testing was made based on the fact that the company had hired 2 people who had done some amount of automated testing earlier). After all, it is very easy to mockup a Excel file and Powerpoint with the benefits of introducing automated testing, but as I said earlier, this should not be a decision taken in haste.

Here are a few points that you should be aware of before you go in for automated testing tools

- Resources: If you are going in for automated testing, then consider the fact that you will need to have more people for your testing team (this also means that some of the people you need will have to have experience with using automated testing). The typical work for a testing team is normally challenging enough, without having to spend the additional time required to automates the test cases.
- Time required: Automation of test cases can take some time to happen. It takes time to learn how to use a testing tool, it takes time to create a proper automation framework, and then starting to write the proper automation scripts and cover all your test cases (or atleast the desired level of test cases that are considered proper for automation).
- Skill set differences. For a team comprising of black box testers, as well as a manager of such people, having a team of people expert in automation testing means a different skill sets. This can cause various morale issues, as well as a lack of understanding of the various specialized needs of the testing team.
- When the application needs to change frequently. During the course of the product development process, the product can change, including the UI. I have seen cases in the past where the UI changes, and the product automation no longer works. In such cases, the script needs to be re-worked, which needs more effort.
- Maintainability of such scripts. Typically, over a period of time, automation scripts need to be maintained, and as people move on in the company, maintaining the same scripts (and ensuring that new people are able to understand the entire structure of the test automation framework) can take increasing periods of effort and time.
- Interlap between test automation areas and manual testing areas. This can be hard to figure for many products where the product functionality cannot be broken down into clean separable areas. In such cases, duplication of testing efforts can happen.
- Expensive. From my experience, the cost needed for test automation is not a one-time effort. We had bought a fairly expensive tool, but then realized that we still had to pay around 40% of the cost annually for the AMC and for the ability to get regular updates.
- Platform compatibility. This was something new that we learnt. We had a product that was only available on Windows, and after a couple of cycles, we also moved onto the Mac and onto Linux because of the potential of getting more sales. And then we realized that our testing tool did not support Linux, and we either needed to get a new tool for Linux (and which meant that we needed to port all our test scripts), or we needed to go in for manual testing (in which case also we did not get the benefits of automated testing on Linux).

Product Development - planning a minor / dot release - Challenges

In the last post, I had talked about why a minor (dot) release is needed, as well as some of the reasons as to why doing a dot release is an inconvenience. However, if the decision has been made to do a dot release, then it is necessary to understand the process of planning and executing a dot release, including some of the difficulties (challenges) that emerge in such a minor release.
What are some of the activities that need to be done ?
1. You need to finalize the changes that need to be a dot release. If this release is because of some known changes, then the changes need to be analysed, and the engineering response (and design) needs to be worked out.
2. If the changes are still unknown, for example, if your release is failing some security tests or some certification, then you need to figure out what can be done. If you take an example where the earlier release is failing some new certification norms (and for those who know how certification works, it can be a lot of effort to prepare the infrastructure and execute all the cases. As an example, you may need to take the help of tools for certification, and those tools may need an upgrade of memory. In other cases, the amount of testing required may be huge, and the actual calendar time needed may stretch the schedule of the dot release.
3. For companies where a lot of work is handled by support teams (configuration, build, release, internationalization, etc), the required overhead of handling all those teams and getting their support takes both budget allocation and time.
4. Too many minor releases causes cynicism in the market about the initial stability of the software release. For example, Microsoft releases many service packs for its software, and there are many people who do not migrate onto a newer version until they see 1 or 2 service packs because they would rather wait for some of the important bug fixes.
5. You run into issues where there are multiple releases for around the same version (with say versions 8.0, 8.1, 8.2, etc for a product). Once you have multiple versions, you get into support issues whereby issues are different for these versions, and support may be a nightmare.
6. Newer dot versions, in many cases need to be deployed to the retail channel (replacing the software already in the retail channel), and to the web store (including to many online software sellers who resell the software). All of these steps involve huge logistical nightmares and / or costs.
7. Branching strategies. Getting a dot (minor release) in process needs major configuration issues (getting branches in place, especially when work is also going for the next release), and takes a fair amount of explanation to both the Development and Quality teams.

Of course, if you folks can suggest more issues that make minor releases a challenge, please add comments.