Subscribe by Email


Showing posts with label Testing. Show all posts
Showing posts with label Testing. Show all posts

Tuesday, August 6, 2019

Allocating Adequate Time for Software Testing: Ensuring Quality in Development

Introduction

In the realm of software development, testing is not merely a phase but a pivotal process that ensures the delivery of a high-quality product. Despite its significance, testing often faces challenges related to time allocation and resource management. This article delves into the necessity of allocating adequate time for testing, the common hurdles encountered, and strategies to enhance the testing process.


The Important Role of Testing in Software Development

Every software application, regardless of its complexity, is susceptible to defects. Even seasoned developers acknowledge the inevitability of bugs during the coding process. Recognizing this, effective testing becomes indispensable to identify and rectify these issues before the software reaches the end-user.

High-quality software is characterized by its reliability, efficiency, and user-friendliness. Achieving this standard necessitates a thorough testing process that uncovers and addresses defects, ensuring the software performs as intended under various conditions.


Challenges in Allocating Time for Testing

One of the predominant challenges in software projects is ensuring sufficient time is allocated for testing. Project managers and stakeholders often face pressure to expedite development cycles, leading to compressed testing schedules. This haste can compromise the thoroughness of testing, potentially allowing defects to persist into the final product.

Moreover, testing teams frequently encounter skepticism regarding their time estimates. This skepticism often stems from a lack of understanding of the testing process, especially among decision-makers without a background in quality assurance. Consequently, testing teams may feel compelled to reduce their estimated timelines, potentially undermining the effectiveness of the testing process.


Strategies for Effective Testing Time Estimation

Accurate estimation of testing time is crucial for project planning and execution. Here are some strategies to enhance the estimation process:

1. Leverage Experienced Testers

Seasoned testers bring invaluable insights into the estimation process. By reviewing project requirements, even at a high level, experienced testers can provide informed estimates based on their familiarity with similar projects and potential complexities.

2. Analyze Historical Data

Organizations can benefit from analyzing data from previous projects. By examining the testing timelines and challenges of similar past projects, teams can make more informed estimates for current endeavors.

3. Develop Detailed Test Plans

Creating comprehensive test plans based on available requirements allows for a more accurate estimation of the testing effort. These plans serve as a foundation for identifying necessary resources and timelines.

4. Conduct Rigorous Reviews

Subjecting test plans to thorough reviews helps identify potential gaps and areas that may require additional attention. This process ensures that estimates are grounded in a realistic understanding of the testing scope.


Importance of Management Support

For testing efforts to be effective, they must be supported by management. This support includes acknowledging the importance of thorough testing, allocating appropriate resources, and respecting the time estimates provided by testing teams. When management understands and values the testing process, it fosters an environment where quality is prioritized.


Conclusion

Allocating adequate time for software testing is not a luxury but a necessity. It ensures that the final product meets quality standards and performs reliably for users. By understanding the challenges and implementing strategic estimation practices, organizations can enhance their testing processes, leading to the successful delivery of high-quality software.


Recommended Reading

For those interested in delving deeper into software testing and estimation techniques, consider the following books:


Informative Videos

To further understand the nuances of software testing and time estimation, the following videos provide valuable insights:

Software Testing Tutorial Introduction and Course Topics - Software Testing Bootcamp




5 Types of Testing Software Every Developer Needs to Know!





By embracing the importance of allocating sufficient time for testing and implementing effective estimation strategies, software development teams can significantly enhance the quality and reliability of their products.


Thursday, April 11, 2019

Prioritizing Early Delivery of Major Features for Effective Testing in Software Projects

Delivering high-impact features early in the software development lifecycle is a practice that often sounds too obvious to be worth repeating. Yet, in project after project, this simple principle is either overlooked or under-executed, resulting in repeated delays, strained relationships between development and quality engineering (QE) teams, and ultimately, avoidable chaos as deadlines loom.

This post highlights why ensuring early delivery of major features is critical, what challenges typically block this goal, and how teams can address those challenges with practical, sustainable planning.

The Reality of Feature Delivery Timing

When working on a software project, not all features are created equal. Some carry more weight than others. They may be background engines like a tax calculation module in an accounting application or the image-rendering engine in a graphic design suite. Or they could be marquee features—a brand-new capability intended to be a headline feature in a new release.

These major components often form the backbone of the user experience or system functionality. That means they deserve extra scrutiny, deeper testing, and more time to stabilize. Yet, paradoxically, they are often the last to be handed off to the QE team.

Why? Because they are usually the most complex, and complexity tends to lead to delays.

Why Early Delivery Matters

1. More Time to Discover and Fix Defects: New features, especially ones being built from scratch, are naturally prone to bugs. Delivering them earlier in the cycle allows time for:

  • Comprehensive test coverage

  • Identification of corner-case issues

  • Regression testing against legacy code

2. Resolving Workflow Disagreements: Even with the most thorough requirement documents, ambiguity or misinterpretation is inevitable. A minor issue like the wording of an error message can cause days of delay if found late. Early QE feedback enables Product Managers to step in and mediate before these discrepancies become blockers.

3. Reducing Localization and Documentation Bottlenecks: Localization and documentation are downstream activities. Their accuracy and completeness depend heavily on the stability of the feature. Early delivery means that:

  • Strings and flows can be finalized for translation

  • User manuals and help guides can be drafted and reviewed on time

4. Mitigating Risk in Final Stages: When critical features are delivered late, they often carry unresolved defects that threaten release readiness. This increases the risk of either:

  • A compromised user experience

  • Delayed product release

Common Barriers to Early Delivery

Even when the intention to deliver early exists, reality can get in the way. Common challenges include:

  • Scheduling Conflicts: The team may be handling multiple dependencies that push key feature work toward the end of the cycle.

  • Inter-Team Dependencies: Some features rely on libraries or APIs being ready, which introduces delay.

  • Changing Priorities: Business or product direction may shift, affecting delivery timelines.

  • Scope Creep: Continuous additions or revisions to the feature can extend development timelines.

Strategic Solutions

To ensure critical features are delivered early enough for meaningful testing, teams can adopt several practices:

1. Planning from the Start: Feature prioritization must be embedded in the planning phase. Identify the top 3-5 components that are high-risk or high-visibility and schedule them for early design, implementation, and review.

2. Feature Slicing: Break large features into testable parts. Even if the full engine or module can't be delivered early, its building blocks often can.

3. Parallelization: Where possible, split the work so some teams handle foundational components while others focus on complementary tasks.

4. Early Involvement of QE: Involve the QA and testing team during design and coding to:

  • Define test scenarios early

  • Identify potential risks or ambiguities

5. Build Internal Checkpoints: Set up mid-cycle reviews or dry runs focused only on the major features. Use these to assess readiness, gather feedback, and refine schedules.

Practical Tips for Teams

  • Document Readiness Gates: Define what it means for a feature to be “ready for test.” This can include UI stability, API responses being mock-tested, or error handling being implemented.

  • Maintain a Defect Buffer: Allocate buffer time for defect triage and fix cycles for high-impact features.

  • Promote Transparency: Keep stakeholders updated. Use dashboards to show which features are blocking QE or causing downstream delays.

  • Encourage Debriefs: Post-release, conduct retrospectives focused on the timing and testing of major features. Capture learnings for future sprints or releases.

Final Thoughts

It's tempting to treat each feature as just another ticket in the backlog. But some features are simply more important. They define the release. They define the user experience. They carry more risk.

By identifying these early and giving them the schedule, priority, and attention they deserve, teams can drastically improve testing effectiveness, product stability, and overall project success.

Early feature delivery isn’t just about code readiness. It’s about de-risking your release cycle and giving your team the runway to succeed.

Suggested Amazon Books on Software Testing and Agile Planning


Sunday, April 7, 2019

The Late Defect Dilemma: Fostering Collaboration Over Blame in High-Pressure Software Releases

The Late Defect Dilemma: Fostering Collaboration Over Blame in High-Pressure Software Releases (And Why a Post-Mortem Review is Crucial)


As a software development team navigates the final, often frenetic, stages of a project, a palpable shift in atmosphere is common. The tension levels in the team can suddenly change drastically, and more often than not, they increase. There's a collective holding of breath, an anticipation that, despite meticulous planning and execution, something unexpected might still go wrong, something that could derail carefully laid milestones and unyielding deadlines. When the team reaches those critical days just before the scheduled completion of development and testing, every new testing cycle brings forth a mixture of hope and trepidation. Leads and managers fervently hope that the testing is thorough, yet simultaneously pray that no major, showstopper defect emerges that could catastrophically impact the impending release.

The discovery of any major or high-severity defect near the end deadline carries the potential for severe impact. The dilemma is stark: the risk of not making a fix is releasing a buggy, potentially unstable product that could damage user trust and the company's reputation. However, rushing a fix under immense pressure carries its own significant risks. Any last-minute code change, no matter how seemingly small, has the potential to cause an undesired change in existing functionality or, worse, introduce a new, even more insidious defect – something that may not be easily captured by hurried, targeted testing. With the relentless pressure of deadlines looming, unless more time is miraculously granted, even rigorous code reviews and focused impact testing can only provide a certain level of confidence that there are no adverse effects from the fix. A lingering risk always remains.

The Peril of Pressure: When Tension Leads to Blame

What I have consistently observed in these high-pressure, end-of-cycle situations is that this inherent tension can unfortunately cause people to start "flipping out" when things inevitably start going wrong. It's a human reaction to stress, but one that can be incredibly damaging to team morale and counterproductive to resolving the actual issue.

I recall a specific instance that perfectly illustrates this. A young, diligent tester on the Quality Engineering (QE) team unearthed a severe defect almost at the eleventh hour, just days before the scheduled release. There was no sugarcoating it; the defect was critical, and its impact was undeniable. The team was immediately thrown into crisis mode. There was an urgent need to make a fix, meticulously evaluate the impact of that fix across the system, conduct multiple, thorough code review cycles, and deploy multiple testers to rigorously check all potentially impacted areas. And, as was almost inevitable in such a scenario, there was a pushing out of the release deadlines by a couple of crucial days.

The reaction from one of the senior managers was, to put it mildly, one of extreme irritation. He publicly and pointedly dressed down the QE lead, questioning why this severe defect was not caught much earlier in the testing cycle. The implication, verging on an outright accusation, was that the QE team had somehow failed to do their job thoroughly. The atmosphere became charged, and the focus shifted, albeit temporarily, from collaborative problem-solving to defensive posturing and, for some, a feeling of being unfairly targeted.

The Power of Retrospection: Uncovering Root Causes, Not Scapegoats

Once the release was successfully, albeit slightly delayed, completed, a crucial step was taken: a post-mortem review. A dedicated review team was assembled to go through the various development and testing documents, trace the defect's origin, and understand the process breakdowns. This objective examination revealed a far more complex picture than initial, heat-of-the-moment reactions suggested. It turned out there was a subtle but significant mix-up right from the start, originating in the developer's design documents. These flawed design documents were then, in good faith, used by the QE team as a basis for creating their test cases. The test cases, therefore, were validating against an incorrect design. Ironically, it was a lucky, ad-hoc exploratory test conducted by that young tester – going beyond the scripted test cases – that finally uncovered the critical defect.

As a valuable byproduct of this comprehensive review, the senior manager who had earlier ascribed blame was also advised – gently but firmly – that such public blaming does not help the situation. In fact, it can have the opposite effect, potentially discouraging team members (like the tester who found the critical bug) who were, in reality, only doing their jobs, and in this case, doing them in a particularly diligent and ultimately beneficial manner. It was a learning moment not just for the technical processes, but for managerial approach as well.

The Importance of Withholding Judgment: Why Blame Culture is Destructive

The instinct to find someone or some group to blame when a high-stakes deadline is threatened by a last-minute defect is understandable, but it's a path fraught with negative consequences:

  1. Demoralizes the Team: When individuals or teams feel unfairly blamed, morale plummets. It creates an environment of fear rather than one of open collaboration.

  2. Discourages Transparency: If finding a bug leads to a dressing down, team members might become hesitant to report issues in the future, especially if they perceive it might reflect negatively on them or their colleagues. This can lead to defects being hidden or downplayed, which is far more dangerous.

  3. Shifts Focus from Solution to Defense: Energy that should be spent on analyzing the defect, understanding its impact, and implementing a robust fix is instead diverted to defending actions or deflecting blame.

  4. Erodes Trust: A blame culture erodes trust between team members, between teams (e.g., Development vs. QE), and between management and their teams.

  5. Masks Root Causes: Blaming an individual or a single team often prevents a deeper investigation into systemic issues or process flaws that might have contributed to the defect escaping detection earlier. The gas station sign analogy from a previous discussion applies here – if the sign is unreadable, is it the driver's fault for not seeing it, or the designer's for making it unreadable? Often, the issue lies in the system or process.

  6. Hinders Learning and Improvement: True improvement comes from understanding root causes and implementing corrective actions in processes, tools, or training. A blame culture stifles this learning process.

A Constructive Approach: Responding to Last-Minute Defects

When a critical defect surfaces late in the cycle, a more constructive and ultimately more effective approach involves:

  1. Stay Calm and Assess: The initial reaction should be to calmly assess the severity and impact of the defect. Panic rarely leads to good decisions.

  2. Focus on the Problem, Not the Person/Team: The immediate priority is to understand the defect, reproduce it, and determine the best way to fix it safely.

  3. Collaborative Triage: Involve key stakeholders (developers, testers, product managers, relevant leads) in a quick triage meeting to discuss the defect, its impact, and potential fix strategies.

  4. Thorough Impact Analysis: Before any fix is implemented, a careful analysis of its potential impact on other parts of the system is crucial. What are the regression risks?

  5. Rigorous Code Review and Testing (Even Under Pressure): While time is short, skimping on code reviews for the fix and thorough testing of the fix and surrounding areas is a recipe for introducing new problems. This is where experience and focused effort are key. Sometimes, this means making the hard decision to push the deadline, as in the example.

  6. Clear Communication: Keep all relevant stakeholders informed about the defect, the plan to address it, and any potential impact on the release schedule. Transparency is vital.

  7. Post-Release Retrospective (The "No-Blame" Review):

    • Once the immediate crisis is over and the product is released, conduct a thorough, no-blame retrospective or post-mortem.

    • The goal of this review is not to assign blame but to understand:

      • What was the root cause of the defect?

      • Why was it not caught earlier in the development or testing process?

      • Were there gaps in the requirements, design, development practices, or testing strategies?

      • What process improvements can be implemented to prevent similar defects from occurring or from reaching such a late stage in future releases?

    • This review should involve representatives from all involved teams and focus on learning and continuous improvement.

The Nuance of Managerial Involvement and Team Dynamics

As highlighted in the initial reflection, the dynamics of when and how managers or leads get involved in defect resolution can vary. This isn't necessarily a reflection of a team's "maturity or values," but rather "how the dynamics of the group have become established."

  • Some teams might empower developers and testers to manage and resolve many defects independently, only escalating to managers for critical issues or those requiring broader decisions (like shifting deadlines).

  • Other teams or projects might have a more hands-on managerial approach, with leads or managers involved in the triage and decision-making for most significant defects.

Neither approach is inherently superior; effectiveness depends on the team's experience, the complexity of the product, and the established working culture. However, what remains constant is the need for clear roles, responsibilities, and open lines of communication, especially when critical issues arise. The manager's role in such situations is crucial: to facilitate problem-solving, provide support, make tough decisions when necessary (like delaying a release), shield the team from undue external pressure, and, importantly, to foster a culture where finding and fixing problems is seen as a collective responsibility, not an opportunity for blame.

Conclusion: Building Resilience Through Process and Culture

The appearance of last-minute, high-severity defects is an almost inevitable reality in the complex world of software development. While a desirable goal is to catch all critical issues much earlier, the final stages of integration and system testing can sometimes unearth problems that previously lay dormant. The true test of a team and its leadership is not whether such defects occur, but how they respond when they do.

Rushing to ascribe blame in these high-tension moments is a natural human tendency, but it is a counterproductive one. It stifles transparency, erodes morale, and distracts from the crucial tasks of fixing the immediate problem and, equally importantly, understanding the systemic reasons for its late discovery. A culture that prioritizes objective root cause analysis through blameless post-mortem reviews, that encourages diligent testing and reporting (even if the news is unwelcome), and that sees defects as opportunities for process improvement is far more likely to build resilient, high-performing teams and consistently deliver quality software.

The young tester who found that critical bug, despite the initial uncomfortable reaction from management, was ultimately a hero for that release. Her ad-hoc, curious testing prevented a faulty product from reaching customers. Supporting and encouraging such diligence, rather than reacting with frustration, is the hallmark of a mature and effective development organization. It’s about focusing on the "what" and "why" of the problem, not the "who."

Further References & Learning:


Books on Software Quality, Testing, Team Dynamics, and Blameless Culture (Available on Amazon and other booksellers):

"Lessons Learned in Software Testing: A Context-Driven Approach" by Cem Kaner, James Bach, and Bret Pettichord (Buy book - Affiliate link): A classic that discusses the realities of software testing and finding bugs.

"Agile Retrospectives: Making Good Teams Great" by Esther Derby and Diana Larsen (Buy book - Affiliate link): Provides frameworks for conducting effective, blameless retrospectives.

"Debugging Teams: Better Productivity through Collaboration" by Brian W. Fitzpatrick and Ben Collins-Sussman (Buy book - Affiliate link): Focuses on the human and social aspects of software development and dealing with problems.

"Peopleware: Productive Projects and Teams" by Tom DeMarco and Timothy Lister (Buy book - Affiliate link): Emphasizes the importance of the social environment for productive software development.

"Software Engineering at Google: Lessons Learned from Programming Over Time" by Titus Winters, Tom Manshreck, Hyrum Wright (Buy book - Affiliate link): Contains insights into Google's culture of blameless postmortems and continuous improvement.


Tuesday, June 21, 2016

Supporting previous versions of the software (Part 6)

The previous post (Supporting previous versions of the software (Part 5)) talked about a team trying to make a decision about whether to drop support for a previous version of the software or not. If you are a user who is impacted by dropping support for the version of the software you are using, you would think that the team has taken such a decision without thinking about users. However, such decisions are not taken without a detailed discussion, and maybe after thinking about this decision at multiple levels before the final decision on dropping support is taken. However, it can be the exact reverse decision that could happen. With some serious defects having risen, the team has proposed that these cannot be fixed easily and also because these problems are for an older version; however, when a decision is finally taken, the decision is the exact opposite. The support for the previous version is not removed, and infact, the team is told that with these severe defects, there needs to be a fix provided to users.
With this decision, it can provide quite a challenge for the team to provide the dot release / patch for the release, but not impossible and hence they have to plan for it. There are a number of parameters that need to be considered, evaluated and a solution found for all of these. Some of these are (this is not an exhaustive list, if you see more parameters, please let me know through comments):
- Whether to release a dot or patch: What is the difference ? The terminology could vary, but typically consider 2 different kinds of releases ? One is where you have a simple small executable that just updates the few files required for the fix, while the other is a complete release that essentially replaces the application that they have on their machine. The reasons for deciding which method to use is also complex; it depends on the number of files to be updated, as well as the actual mechanics of issues such as upgrades, dependencies, etc. This decision is typically an engineering decision.
- Number of resources to be used for the release: In most cases, teams do not get additional resources for handling such updates unless they are doing this on a regular basis. So, some amount of work estimation needs to be done to determine the number of resources needed, as well as the schedule on which they will be needed.
- Support from external teams: For any product release, there is support needed from additional teams such as testing, installer and release, documentation, etc. Even if there is a small update to be provided, there is a need to ensure that these external teams are in place and committed to the effort, This can be a challenge sometimes.
- The timeframe for this effort needs to be decided: This can be a challenge sometimes. Typically a dot or a patch needs to be released pretty quickly, and there is a need to optimize to ensure that the existing engineering work for the current version is not impacted. This schedule can be a challenge, since the priority will remain the work for the existing release, and yet there needs to be an update.
- Update problems / engineering issues: Whenever an update / patch / dot release for a previous release needs to be released, it needs to be checked with the application installed on the user's machine. This includes ensuring that the application continues to work fine with the update, including the standard checks. This means that the team needs to install previous versions of the application and even replicate the operating system version that some of the customers might be having.
- Informing the users through support teams, forums and so on. There needs to be a campaign to inform users about the impending update release and ensure that feedback needs to be taken quickly when the update is released.


Monday, October 13, 2014

Bug fixing: Ensuring that there is regular contact with outside users

Bug fixing is an integral part of the product development process, and unless the bug finding and bug fixing process starts tapering down near the end of the product development cycle, you are in deep trouble. Near the end of the cycle, is one of the most stressful time for the product development team. After all, when you are nearing the end of the development cycle, any new defect that comes through can cause huge problems for the team and for the schedule. If the defect was serious, then there are many issues that come to light. One needs to figure out whether the area was tested well enough before during the earlier parts of the schedule, as well as figure out what the risk was in terms of making the fix of the defect. If the fix can impact a large enough area, then the team might well want to not make the fix and take the defect. If a serious defect fix has been made near the end of the schedule, then there is a compromise that needs to be made in terms of the amount of time available for testing of the fix.
But this post was not about this problem. The problem is more of a derived problem, In my experience with multiple development cycles, I have seen defects come near the end, or become more serious near the end, where the defect was seen earlier with external users (these are users who are not part of the core team, but could be users who are part of a restricted pre-release or a public release). The challenge comes in terms of recognizing these defects as valid defects.
The advantage of outside users is the amount of diversity that they have. People outside the core team who are testing the product do a lot more adhoc testing, trying out different combinations that the core team would not be testing. In addition, the diversity of equipment that outside users have far strips the diversity of equipment available with the core testing team - they would have more types of machines, different sets of equipment, and so on. This provides an advantage that the core team would be well equipped to use.
At the same time, there is a cost associated with the pre-release users. Some of the defects that they find are already known to the core team, other defects may not be able to be replicated by the core testing team, and yet others are significant defects that the core team takes up for fixing. For a number of defects, it may have been critical for the outside user, but the core team would make a choice that either the defect was not been able to be replicated, or the area was not of a high priority to be taken up for fixing.
However, this is where we had problems and had to make changes. Near the end of the cycle, we would find some defects during the final stage of adhoc testing that had been found by outside users but which the team had dismissed. The amount of impact to the schedule and increasing the stress level of the team was one of the byproducts. To control this problem, we decided to make a lot of effort to evaluate the defects raised by the outside users, including remote control of their machines to try and find the defect cause, spending much more time to analyze the defect, and we had some success in this kind of effort that we were putting. We were able to figure out some of these more serious defects and this in turn reduced the chance of getting some of these defects near the end of the cycle, and also had the byproduct where the outside users felt that their defects were getting more serious attention and produced some great defects.


Friday, August 22, 2014

What are some of the problems of Test Driven development?

The Test driven development (TDD) does have a lot of benefits, but you would expect some issues, and there are some problems too. Regression testing, though sufficient and good enough for most cases of software development, there are sometimes situations where testing requires full functional testing. In these situations, partial functional testing cannot be used for determining success or failure since a large number of unit tests are used. The types of software facing this problem includes user interfaces, programs using databases and certain network configurations. This is so because of the feature that is typically an advantage of TDD, is a disadvantage too in this situation. This feature is that TDD encourages programmers to write modules with minimum amount of code with maximum logic that can be tested using mock – ups and fakes. The outside world of the module is also represented using these mock – ups. Another shortcoming of TDD is that it cannot do without management support. It is essential for its efficient existence. If the entire organization does not believes that TDD will improve their product, it may seem to management that it is wasting time in writing the tests. Also in TDD, the programmer who is writing the code is responsible for writing the units tests which means – a number of blind spots in the code will go untested. It is always a good practice to have unit tests written by someone else. Certain things to be checked might not come to the notice of the developer leaving the code partially tested.
Take another example where the requirements of a module are misinterpreted by the developer. In this case both the code as well as the unit tests that he will write will be wrong. So the even though incorrect, the tests will pass and the developer will assume that his code is correct. Also if these unit tests are many in number, they will pass giving a wrong sense of security which might result in less testing of the software i.e., compliance testing, integration testing etc. writing and executing tests adds to the maintenance overhead of the software. Tests with bad code such as error strings or code which is prone to failure are quite expensive when it comes to maintenance.
The same is the case of fragile tests. They involve risk that tests likely to generate failures regularly will be neglected. Therefore, when a failure occurs in actual, it won’t be detected. Though a bit difficult, it is possible to write code that requires low maintenance. This can be done by reusing the error strings wherever possible. Code refactoring should focus on code reusability. Writing and executing a lot of tests takes a lot of time too. However, some flexible modules having few tests might incorporate new requirements without having to make any changes in the tests. It is because of this reason that it is easy to adjust with few number of tests and little data than with big and complex tests. With advanced factor analysis and sampling skills, developers can be warned of excessive – testing.
The repeated TDD cycles achieve a level of testing and coverage that cannot be recreated later. Therefore, this original coverage is very much essential and has to be fixed early. On the other hand if the testing strategy is poor, changes must be fixed individually. And this should be done carefully as just deleting or disabling them can cause holes in the coverage. TDD insists on writing tests for every part of your code which instead of making it better, might make it contorted and damage it. 


Tuesday, August 19, 2014

How does Test Driven development benefit developers?

Test driven development (TDD) as a process has proven to be a boon for developers time and again. Businesses tend to change rapidly with the time and so does their requirements from the software that they are using. If we develop these software products using the traditional development methodologies, it is obvious that later it will be more difficult for us to maintain them as requirement changes. If you suggest making changes to the existing model developed using traditional method, it might create havoc in unpredictable ways. As a result of these consequences, organizations often don’t go for modification of the existing software as it might hamper their productivity and effectiveness. But this is no problem for companies who have developed their products using TDD. This is so because it is like a continuous integration model in which testing modules are added. Then it becomes easy for the organizations to make modifications to their products, without fearing any breakdown.
First benefit of TDD to developers is that it is easy for them to maintain the software by virtue of its extensibility and flexibility. Since both testing and development go hand in hand in TDD at the lowest level, it guarantees testing of every single logical piece and can even be changed. Once the development is done, the application is tested once again with thousands of tests. After making a change to the application, associated tests are run to see if there is any impact on the other parts of the application. With this approach there occurs no problem in modifying the existing legacy applications. Apart from benefiting developers, this has benefits for organizations seeking growth by making it easier for them to update their systems.
The codebase is streamlined along with test coverage which is unparallel. In TDD, writing a test first before writing code is mandatory. That is how it provides unparalleled coverage. Further, regression testing and refactoring make the code as minimal and economical as possible. This plays a big role in streamlining the codebase. If for a functionality there is no use case, the test is not written and no code and thus there is no growth in the codebase. This is also another reason behind easy maintenance.
TDD provides a clean interface throughout the development process. Since the tests are written first, the APIs thus produced are from the perspective of an API – user. That’s why it is very easy to use these APIs when compared to the APIs written by programmer’s perspective.
The code refactoring process is central to the success of TDD and makes sure that the codebase is strengthened preventing the software from getting outdated and monolithic.

TDD aims at improving the code and is particularly useful in the following:
> Addition of new feature or functionality: With TDD the programmer feels confident in changing part of a large application (even otherwise the programmer might feel confident but this feeling may not be shared by other stakeholders). If it wasn’t flexible, then we would be adding functionality to the application but without any proper integration. This would have definitely caused many problems.
> Changes in the technical infrastructure: Developers are always thinking of making technical changes to the code for increasing its elegance and make it more extensible.
The use cases used in TDD are actually tests which can be used by the other developers as examples of how the code works and how it can be used. Thus TDD provides executable documentation. Updating the software without any painstaking efforts has been possible because of TDD.
An organization can be successful only if it embraces the changes and makes improvements. Test driven development makes all this possible with its extensability, maintainability and flexibility. 


Friday, December 13, 2013

Some white box testing techniques – part 2

This post is a continuation of the discussion on white box testing techniques from part 1 of this article (link).

- Path testing: This is a typical application program that consists of 100s of paths between two points of the program i.e., the entry and the exit. The number of paths goes on multiplying by the number of decisions that are encountered during the execution. This number is again increased by the execution of the loops. Each of these paths has to be tested to ensure there are no bug. This seems quite easy for a straight line having 50 – 100 lines of code but what about a large program? Counting gets difficult. With a limited number of resources (and most software projects have a limited number of resources), all the possible paths have to be covered. Path testing can be made a little easy by not consider the redundant code in the program as far as possible. Path testing is responsible for checking whether or not each and every path is getting executed at least once. Today we also have a new type of path testing, called the basis path testing which is a mixture of branch testing and path testing. Even the path testing makes use of the control flow graph. It involves calculating the cyclomatic complexity. For each & every path, a test case has to be designed.

- Statement coverage: We also know this white box testing technique by other names such as segment coverage or line coverage. This coverage methodology provides coverage only to the conditions that are true. The false conditions are neglected. This helps in the identification of the statements that are often executed in the program. It also helps in finding out the areas of the program where there is no data flow because of no execution. But all the lines of code are examined and executed. Statement coverage is important for verifying whether the code is serving its desired purpose or not. In other words it gives us a measure of the code quality. We can consider it as a partial form of path testing. Disadvantages include: false conditions are left untested; it gives no report about the loop termination and does not consider the logical operators. One line of code might be a full statement, a partial one or blank or may contain many statements. There are nested statements too. The statement coverage has to be measured first and then the test cases should be designed.

- Decision coverage: Another name for decision coverage is “all – edges coverage”. Here all the conditions that result because of a logical decision are considered. It does not matter whether the conditions are true or false. This is just the opposite of statement coverage. The outcome of a decision is nothing but a branch. So the branch coverage gives a measure of extent up to which the decisions in a program have been tested. This is why most of us often get confused with branch coverage and decision coverage and use them interchangeably. But it is more than statement coverage. The decision statements include loop control statements, if statements, if – else statements, switch – case statements i.e., the statements that result in either a true or a false value based up on the logical condition in that statement. This white box testing technique is helpful in validation of branch coverage that is whether or not all branches have been covered. It helps in ensuring that no branch gives an abnormal result that may disturb the operation of the whole program. There are certain drawbacks of the statement coverage as we saw above. These drawbacks are eliminated by decision testing. 


Thursday, December 12, 2013

Some white box testing techniques – part 1

In this article we shall give a brief description of the various techniques that are used in white box testing.

- Control flow testing: This is more of being similar to structural testing and is based on the control flow model of the application that is being testing. In control flow testing certain paths among all are selected judiciously so as to provide maximum coverage. The paths are chosen so that a decided thoroughness in the testing process is maintained. This means that paths that have been selected should be enough so as to cover all the statements at least once. It is usually applied to the new software systems that have to be put under unit testing (unit level white box testing). This technique assumes that there are correct specifications, correctly assessed and defined data, no bugs in the program except those in the control flow. The number of control flow bugs is fewer in the programs that are written using object – oriented or structured programming languages. This technique makes use of a flow graph usually called the control flow graph that represents the control structure of the program.

- Data flow testing: Flowing data in a program is what keeps a program active. If there is no data flow, the program won’t be able to perform any operation. The program’s data flow needs to be tested to ensure that it is constant and efficient. Data flow testing also requires a data flow graph. This graph is required to know where and why the data is going or if it is going to correct destination or not. This testing helps in uncovering any anomalies that might restrict the flow of data in a program. Knowing these anomalies also helps in branch and path testing. A number of methods and techniques go in to data flow testing for exploring various events that are taking place in the program, whether right or wrong. It is used for checking whether all the data objects have been initialized or not and makes sure that they are used at least once in the whole program. Arrays and pointers are considered to be the two major elements that play a critical role in data flow testing. These two elements cannot be neglected. Data flow testing might include static anomaly detection, dynamic anomaly detection and anomaly detection through compliers.

- Branch testing: As the name suggests, this testing technique is used for testing all the branches of all the loops within a program. Here branch coverage plays a great role. It has to be made sure that each and every branch in the program is executed once. Thus the test cases are designed in such a way that all the branches are covered. This technique is just for complementing the white box testing at unit testing level. The programmers aim that their test cases must provide 100 percent branch coverage. Also the coverage has to be proper otherwise it can lead to other potential problems such as removal of the code that was actually correct and insertion of faulty code. But providing 100 percent coverage is not possible and we are always left with some bugs and errors that never come to the scene. Branch testing lets you uncover errors which lie in those parts of the program that are least executed or never executed. But there is a potential drawback too; which is; that it is very ineffective in uncovering errors in interactions of structures and decisions. Because of this drawback, the testers usually prefer to go for path testing.

Read more about this in Part 2 - White Box testing techniques.


Tuesday, December 3, 2013

What is Orthogonal Array testing? - An explanation

There are a number of black box testing techniques; the one that is being discussed in this post is the orthogonal array testing. This technique provides a systematic as well as statistical strategy for testing software. The number of inputs to the system in this technique is small but large enough for testing each and every possible input as in exhaustive testing. This technique has proved to be quite helpful in discovering errors that show a faulty logic in the software systems. Orthogonal arrays can be applied in various testing types such as following:
- User interface or UI testing
- System testing
- Regression testing
- Configuration testing
- Performance testing and so on.

The permutations for the factor levels that consist of only one treatment have to be chosen in an uncorrelated way, so that every single treatment gives you a piece of information that is different from the others. The advantage of organizing testing in such a way is that a minimum number of experiments are required for gathering same information. Orthogonality is a property exhibited by the orthogonal vectors. Properties exhibited by the orthogonal vectors are mentioned below:
- Information conveyed by each vector is different from the information by other vectors in the sequence. That is, as we mentioned information conveyed by each treatment is unique to it. This is important otherwise there will be redundancy.
- It is easy to separate these signals on a linear addition.
- All the vectors are statistically independent of each other which means that there is no correlation between them.
- When these individual components are added linearly, the result is an arithmetic sum.

Suppose a system is having 3 parameters each of which has 3 values. We require total 27 test cases for testing all of the parameter combinations which is quite time consuming. So we use an orthogonal array for selecting a combination subset from these combinations. As a result of using the orthogonal array testing, maximization of the test coverage area is possible. At the same time this minimizes the number of test cases that have to be considered for testing. The pair that is selected is assumed to have the maximum number of defects. The technique works based up on this assumption. These many combinations are sufficient for catching the fault. The interaction of the input parameters between themselves has also to be considered. The array is said to be orthogonal because the occurrence of all pair wise combinations is once. The results of the test cases are assessed as follows:
- Single mode faults
- Double mode faults
- Multimode faults

Below mentioned are the major benefits of using this technique:
- The testing cycle time is reduced.
- The analysis process gets simpler
- Test cases are balanced which means that the defect isolation and performance assessments are straightforward.
- Saves up on costs when compared to the pair-wise testing. The coverage to all the defects can only be provided by testing all the combinations that are possible. But our schedule and budget often do not permit this. Therefore we are forced to select only a sample of the combinations from the test domain. Orthogonal array testing is a means for generating samples that provide high coverage for the validation of test domain effectively. This has made the technique particularly useful in the integration testing and testing of the configurable options. Software testers often face a dilemma during selection of the test cases. The quality of the software cannot be tested but only the defects can be detected. And the exhaustive testing is difficult even in the small systems. 


Monday, December 2, 2013

Some advantages and disadvantages of white box testing

White box testing can be applied at the following levels in software testing process:
- Unit testing
- Integration testing
- System testing

However, commonly it is carried out at the unit level. White box testing at this level is used for exploiting the paths through the program units, and for testing paths between different units, and on the system level for testing paths between the various sub-systems. In this article we discuss about the advantages and disadvantages of using this testing methodology.

Advantages: This testing methodology ranks among the widely used testing methodologies, with an increasing number of people using it, although it does require technical knowledge and capabilities. We have three major advantages from it.
First, in this technique it is beneficial to have knowledge about the source code unlike the other methodologies where the tester should not know much about the source code. This helps in thoroughly testing the application.
Second, this methodology has made it possible to optimize the code by helping in uncovering the hidden errors and removing them.
Third, it provides an opportunity for introspection.
Another plus point is that the testing can be started with just one developed unit at hand. We don’t have to wait for the whole program to be ready with GUI. Majority of the paths are covered in white box testing.

Disadvantages: Just as white box testing has advantages; it has its own minus points too.
The first major disadvantage is that it gets very complex once you start with it. This happens so because every path in the program has to be tested and so all the paths have to be identified which can become very time consuming and difficult indeed. For this the programmer as well as the tester must have a great deal of knowledge in this level of detail.
Second major disadvantage is that white box testing consumes too much time. In every project it is not possible to testing each and every single path. It is certain that some of the paths will go unnoticed. This is why the white box test cases are very complex and can only be implemented with a thorough knowledge of the application and code. Maintenance of the test scripts proves to be a burden as the complexity increases and changes have to be made in the implementation.

This testing methodology requires many testing tools which might not be available at hand instantly. The work of a mechanic is analogous to white box testing i.e., the programmer examines the source code as to see why it is not working.
First the tester or the programmer must analyze the source code with the help of comprehensive software documentation, code and the samples.
Second, the tester needs to think for the ways or methods by which he can disrupt the normal functioning of the application or what are the input factors that can cause the program to go awry? Now based up on these assessments, the white box testing techniques can be implemented. The assessments have to be made carefully for white box testing to be successful. In simple words white box testing is just a means for the verification of the source code.
The logic and the structure of the code must be known to the person who is testing it. Logical decisions are implemented. Each logical decision tests a different path. In white box testing programmers can only substitute for testers i.e., they only can test the application. If other testers are hired, they will take some time in understanding the source code of the program or need a high degree of technical knowledge.


Sunday, December 1, 2013

Testing: A brief summary of white box testing

We all know white box testing by many other names as well such as glass box testing, clear box testing, structural testing, transparent box testing etc. This testing methodology is used for testing the internal structure of the software or to be precise to test how the application actually works and whether it works as desired or not. This is just the opposite of black box testing that is responsible for testing the system’s functionality only, not getting into the internal structure. The test cases for the white box testing are designed based up on the programming skills and internal perspective from which the system is viewed. From the various inputs, some are selected for exercising all paths through the program and determining whether they are producing the desired outputs or not.
A number of errors can be discovered using this testing methodology. But it is not capable of detecting the parts of the specification that have not been implemented or are missing. The following testing techniques are included under the white box testing:

- Data flow testing
- Path testing
- Decision coverage
- Control flow testing
- Branch testing
- Statement coverage

This testing methodology is used for testing the source code of the application using the test cases that are derived using the above mentioned techniques. All these techniques are used as guidelines for the creation of an error free environment. Any fragile piece of code in the whole program can be examined using white box testing. These techniques act as the building blocks for white box testing and its essence lies in carefully testing the source code so that no errors occur later on. Using these different testing techniques all the paths through the source code that are visible can be exercised and an error free environment can be created. But the tester should have the ability to know which path of the code is being tested and what the output should be. Now we shall discuss about the various levels in white box testing:

- Unit testing: White box testing done during this phase ensures that code is working as desired before it is integrated with the units that have already been tested. This helps in catching the errors at an early stage so that later it does not becomes difficult to find them when the units are integrated and the system complexity increases.
- Integration testing: Test cases at this level are meant to test how the interface of various units interact with each other i.e., the system’s behavior is tested in an open environment. During such testing any interaction that is not familiar to the programmer comes to light.
- Regression testing: This phase uses the recycled test cases from white box testing.

For the white box testing to be effective the tester should understand the purpose of the source code so that they are tested well. The programmer must understand the application so that he/ she can create effective and appropriate test cases.
The programmer working on preparing the test cases needs to know the application very well in order to ensure that he / she can prepare cases to handle every visible path for testing purposes. With the source code having been understood, this understanding of the course code lets the team use this analysis of the source code for preparing test cases. And for making these test cases, these are the steps that need to happen in white-box testing:
- Input, that may include functional requirements, specifications, source code etc.
- Processing unit, that carries out the risk analysis, prepares a test plan, executes tests and gives results.
- Output, that provides the final result.


Facebook activity