Now that previous posts have discussed to some details about the need for the test plan, let's talk a bit about what the test plan covers:
Test plans, also called test protocol, are formal documents that typically outline requirements, activities, resources, documentation and schedules to be completed.
A sample of what a test plan (a template) is available at the following location: sqatester.com. The actual word document template is available at this link.
A lot of details about what a test plan covers, including the contents of the test plan, references (documents that support the test plan including project plan, SRS, design documents (HLD, LLD)), issues and risks, features included for testing in the test plan and to be excluded from the test plan, and overall approach are all included in the Wikipedia page. This is a good reference page to read if you want to get a lot of detail about what a test plan should cover.
For a shorter reading of what a test plan should cover, refer to this paragraph below where the test plan is broken down into 3 main sections:
1. Beginning of the plan: This contains background information such as header information, title, author, date, project number/code, project description, references to related documents, an explanation of the test objectives, definition, key words.
2. Middle section of the plan: This section starts to contain some meat about the plan, including a data collection / sampling plan, test environment including setup, resources to be used, assumptions that need to be made, customer specific needs, and how to report test-related problems such as failures.
3. Ending section of the plan: This section typically contains analysis information, statistical techniques, what was the original testing need, a definition for test success or failure, how to conduct data analysis, what to do in case testing results in insufficient information, how to present the final information, references.
Sunday, July 29, 2007
Software test plan and templates
Posted by
Ashish Agarwal
at
7/29/2007 01:59:00 PM
0
comments
Labels: Development, Plan, Processes, SDLC, Software, Strategy, Template, Testing
|
| Subscribe by Email |
|
Software test plan - process
What are the various testing types that a software test plan can be used for ?
The Test Plan is used to perform system testing, subsystem testing, assembly testing, subassembly testing, module testing, user acceptance testing. The Test Plan includes step by step instructions for the various types of testing that will occur during the project lifecycle; it also defines the required test equipment, calibration requirements, test facility requirements, and other key factors.
A formal test plan is a document that provides and records important information about a test project, for example:
* project and quality assumptions
* project background information
* resources
* schedule & timeline
* entry and exit criteria
* test milestones
* tests to be performed
* use cases and/or test cases
Posted by
Ashish Agarwal
at
7/29/2007 12:55:00 PM
0
comments
Labels: Development, Processes, SDLC, Software, Testing
|
| Subscribe by Email |
|
Creating a software test plan - Need
Anybody who has worked in the field of software development would have heard of a software test plan. How many of you have actually understood the need for a software test plan?
Some of the reasons for making a comprehensive test plan are:
* Preparation: To assure that all reasonable aspects of running a test have been considered including getting test data in place, getting the required hardware and software.
* Communication and Training: To train those who need to assist with the test and to enable them to understand the overall perspective.
* Effectiveness: To provide a mechanism for outlining test needs, limitations (listing assumptions), and justification for purposes of setting expectations, acquiring resources, investigating unexpected results, assuring normalcy and effectiveness.
* Legal and Regulatory Prudence: To enable replication and protection of discoveries made, to mitigate potential litigation costs from use of those discoveries, and to help provide evidence to show regulatory bodies of efficacy. A test plan is an important of various software quality process such as ISO and CMM, and helps provide internal or external customers with the confidence that the testing activity is being well planned
* Improve testing coverage and efficiency. Without some form of test plans and scripts available, the testing effort typically degenerates into an ad-hoc effort where no one knows what their colleagues are testing and management can never say for sure that the application has been tested sufficiently.
* Test plans are useful tools to organize, manage and schedule the testing effort. Since the complexity of software and Web applications continues to grow, a well thought-out test plan can greatly facilitate the testing work.
So what is a test plan?
The Test Plan describes the overall approach to development, integration, qualification, and acceptance testing. It describes plans for testing software systems; test environment to be used for the testing; identifies tests to be
performed, and provides schedules for test activities. The process of preparing a test plan is a useful way to think through the efforts needed to validate the acceptability of a software product. The completed document will help people outside the test group understand the 'why' and 'how' of product validation. It should be thorough enough to be useful but not so thorough that no one outside the test group will read it.
Details of a test plan:
In software testing, a test plan gives detailed testing information regarding an upcoming testing effort, including
* Scope of testing
* Schedule
* Test Deliverables
* Release Criteria
* Risks and Contingencies
Posted by
Ashish Agarwal
at
7/29/2007 12:47:00 PM
0
comments
Labels: Development, Processes, SDLC, Software, Testing
|
| Subscribe by Email |
|
Thursday, July 26, 2007
Validation and verification explained
There is typically a lot of confusion about what the terms of validation and verification mean with respect to software testing. Both of these are valid terms, used in the context of software testing, but mean different things. If you are looking for a one-line definition of both:
Verification is about building the product right.
Validation is about building the right product.
Together, these 2 activities ensure that the testing process is complete and the software product that is built is correct and meets all the user requirements. Some more details of these terms:
Verification:
- Ensure that the process being used is correct and is being followed. The review of interim work steps and interim deliverables during a project to ensure they are acceptable. To determine if the system is consistent, adheres to standards, uses reliable techniques and prudent practices, and performs the selected functions in the correct manner.
- Low level activity.
- Performed during development on key artifacts, like walkthroughs, reviews and inspections, mentor feedback, training, checklists and standards.
Validation:
- To ensure that the product meets all the specifications and requirements. Determining if the system complies with the requirements and performs functions for which it is intended and meets the organization’s goals and user needs. It is traditional and is performed at the end of the project.
- High level activity
- Determination of correctness of the final software product by a development project with respect to the user needs and requirements
Posted by
Ashish Agarwal
at
7/26/2007 05:14:00 PM
0
comments
Labels: Development, Processes, Software, Strategy, Terms, Testing
|
| Subscribe by Email |
|
Sunday, July 22, 2007
Need for software testing
Let us first talk about the process of software development. Software development is the process of developing a software application, built to order on the basis of a set of requirements. These requirements could be from a client (where the software is built on the requirements particular to the client), or could be based on a mix of market needs and competitor analysis when a shrink-wrapped software product is being built.
An integral part of the software development process is the software testing process. The scope and details of the software development practice is huge by itself, covering by itself a number of different processes. And what does this software testing process do ? There is the need for a process to ensure that the requirements against which the software is being built is adhering to the requirements as much as possible. In more technical words, the process of testing ensures that a process of validation and verification is being carried out during the entire development process. In less technical words, software testing ensures that somebody is checking that the development process is happening in the right way, that the application is matching the needs of the client.
Testing follows a process to ensure that the process is effective, with the preparation for testing entailing preparation of a test plan and detailed test cases. And there are many additional processes that form a part of software testing, such as test automation, white box testing, black box testing, etc. The usage of specific testing processes during a software development project depends on the kind of project it is. For example, in a project where there is no exposure of details of the code to the testing team, there is no question of white box testing, and so on.
Posted by
Ashish Agarwal
at
7/22/2007 04:33:00 PM
0
comments
Labels: Development, Problems, Processes, Software, Strategy, Testing
|
| Subscribe by Email |
|
Saturday, July 14, 2007
Prototyping process
How to go about prototyping:
1. Identify the base set of requirements
2. Develop the initial prototype
3. Get the prototype reviewed by end-users and customers
4. Based on this review and other work, revise the prototype until it has broad agreement with the customers and end-users
2 main types of prototypes:
1. Evolutionary prototypes: The objective of evolutionary prototyping is to deliver a working system to end-users. Used for systems where the specification cannot be developed in advance such as user interface systems and where verification is impossible because there is no specification.
2. Throw-away prototypes: The objective of throw-away prototyping is to validate or derive the system requirements. Throwaway or Rapid Prototyping refers to the creation of a model that will eventually be discarded rather than becoming part of the finally delivered software. After preliminary requirements gathering is accomplished, a simple working model of the system is constructed to visually show the users what their requirements may look like when they are implemented into a finished system.
Posted by
Ashish Agarwal
at
7/14/2007 07:56:00 PM
0
comments
Labels: Development, Iterative, Processes, Prototyping, SDLC, Software
|
| Subscribe by Email |
|
Saturday, June 23, 2007
Iterative Software Development: The Process of Continuous Feedback, Build, Improve
Iterative Software Development: A Practical Guide to Agile, Iterations, and Incremental Delivery
Introduction: Why Iteration Took Over from Waterfall
Iterative software development became more prominent to overcome the weaknesses of the waterfall development process, namely the issues related to inability to incorporate changes at a later date, as well as the fact that in most client-driven projects, having all the requirements complete in the beginning is not realistic in a number of cases. Waterfall assumes we know everything up front and that change is expensive. In reality, we rarely know enough at the beginning, and change is inevitable and valuable.
So what is iterative software development? Agile software development is a conceptual framework for undertaking software engineering projects that embraces and promotes evolutionary change throughout the entire life cycle of the project. In the iterative cycle, the developer has the chance to develop a software system incrementally, allowing the people involved to learn from previous iterations and make those changes in the next iterations. Key steps in the process are to start with a simple implementation of a subset of the software requirements and iteratively enhance the evolving sequence of versions until the full system is implemented.
This post unpacks those ideas, shows what really happens inside a 2–5 week iteration, and shares practical examples, pitfalls, and tips you can apply in your team.
What Is Iterative Software Development?
At its core, iterative development delivers software in small, time-boxed cycles. Each cycle (iteration) produces a working, testable slice of the product. You don’t wait months to see value; you see it every few weeks. You don’t guess what users want; you learn from each iteration and adjust the plan.
Each iteration is a self-contained mini-project composed of activities such as requirements analysis, design, programming, and test. The goal for each iteration is to develop a system that is stable and testable, but does not necessarily have all the functionality incorporated. Most iteration releases are internal, a baseline primarily for the benefit of the development team—they are not released externally. Hence, each succeeding iteration will typically have more functionality included. Typically, an iteration will be between 2 and 5 weeks.
This approach works well with methods like Scrum (which calls iterations “sprints”) and Kanban (which emphasizes continuous flow but can still use time-boxed evaluation points). The key is short, repeatable cycles with feedback loops.
Why Waterfall Struggles—and How Iteration Fixes It
- Changing requirements: Waterfall assumes the plan is stable. Iterative development assumes change and welcomes it with minimal disruption.
- Late feedback: In waterfall, feedback often arrives at the end when it’s expensive to fix. Iterations pull feedback forward every few weeks.
- Risk exposure: Big-bang integration hides risks until late. Iteration reduces risk by building and testing vertical slices early.
- Value delay: Waterfall defers value delivery. Iteration delivers the most valuable slices first, so stakeholders see tangible progress.
Core Principles of Iterative Development
- Incremental value: Deliver the highest-value, smallest slice that can stand on its own.
- Frequent feedback: Review working software with stakeholders every iteration.
- Time-boxing: Constrain work to a short, predictable window (2–5 weeks).
- Test early and often: Bake in testing and test automation from day one.
- Evolving design: Start with a simple design and improve it as needs emerge.
- Transparency and adaptation: Use retrospectives, metrics, and reviews to adapt process and scope.
The Anatomy of an Iteration (2–5 Weeks)
Think of an iteration as a loop with clear entry and exit points. A typical iteration includes:
- Iteration planning (2–4 hours for a two-week cycle)
- Select a realistic set of user stories from the backlog.
- Clarify acceptance criteria; split large stories into smaller ones.
- Agree on a sprint goal: a short statement of value to achieve.
- Daily sync (10–15 minutes)
- Surface blockers, align on priorities, adapt the plan.
- Build, test, integrate (the bulk of the time)
- Design and code in small increments; keep branches short-lived.
- Write tests alongside code (unit, integration, and acceptance).
- Integrate often with a CI server; keep the main branch always releasable.
- Use feature flags to keep incomplete work out of production.
- Review and demo (1–2 hours)
- Demonstrate working software to stakeholders.
- Confirm acceptance criteria and record feedback.
- Retrospective (1 hour)
- Inspect what helped or hindered the team.
- Choose 1–2 concrete improvements to try next iteration.
- Backlog refinement (ongoing)
- Clarify upcoming work, refine estimates, and ensure the next iteration is ready.
A Practical Example: Iteratively Building a Simple Feature
Scenario: You’re building a task manager app. The product goal is “help users capture and complete tasks quickly.”
Iteration 1 (2 weeks): Create and list tasks
- Stories:
- As a user, I can create a task with a title so I can remember what to do.
- As a user, I can see a list of my tasks sorted by creation time.
- Acceptance criteria:
- Title is required; tasks persist after app restart.
- Empty state shows “No tasks yet.”
- Implementation outline:
- API endpoint: POST /tasks, GET /tasks
- Data model: Task { id, title, createdAt, completed=false }
- UI: Simple form and list view
- Tests:
- Unit tests for model validation
- Integration test for POST/GET flow
- Demo outcome:
- Stakeholders can add and view tasks. Feedback suggests adding completion status next.
Iteration 2 (2 weeks): Complete and filter tasks
- Stories:
- As a user, I can mark tasks complete so I can focus on what remains.
- As a user, I can filter tasks by status.
- Acceptance criteria:
- Completed tasks visually distinct; filter persists across sessions.
- Implementation outline:
- API: PATCH /tasks/:id { completed: true/false }
- UI: Checkbox per item; filters: All, Active, Completed
- Tests:
- Unit tests for status toggling
- UI test for filter behavior
- Demo outcome:
- Stakeholders suggest due dates and reminders later, but prioritize search first.
Iteration 3 (2 weeks): Search
- Stories:
- As a user, I can search tasks by title keywords.
- Acceptance criteria:
- Search is case-insensitive and fast on lists up to 1,000 items.
- Implementation outline:
- Client-side search with debouncing; consider server-side later.
- Tests:
- Performance test: search completes under 100ms on typical devices.
- Demo outcome:
- Users satisfied with core experience; next, explore due dates as a separate iteration.
Notice how each iteration produces something stable and testable, even if the full vision isn’t done. Stakeholder feedback directly shapes what comes next.
Roles and Collaboration in Iterative Teams
- Product Owner or Business Analyst: Owns the backlog, prioritizes by value, clarifies acceptance criteria.
- Developers/Engineers: Design, code, test, and integrate features; pair or mob program where helpful.
- QA/Test Engineers: Collaborate on test strategy, build automation, and explore the product for edge cases.
- UX/UI Designers: Prototype flows, validate with users, and iterate on interaction details.
- DevOps/Platform Engineers: Keep CI/CD healthy, manage environments, enable safe releases.
- Stakeholders/Customers: Provide timely feedback and help define “value.”
Key Artifacts and Deliverables
- Product backlog: Ordered list of user-centric work (stories, defects, spikes).
- User stories and acceptance criteria: Clear, testable definitions of value.
- Definition of Done (DoD): Shared checklist that ensures quality, e.g., “code reviewed, tests passing, integrated, documented, and deployable.”
- Iteration plan/board: Visible workflow with WIP limits to prevent multitasking.
- Test suite and coverage: Unit, API, UI tests run automatically in CI.
- Demo notes and retrospective actions: Captured decisions and improvements.
Metrics That Matter (Without Gaming)
- Velocity: Completed story points per iteration. Use for capacity planning, not for comparing teams.
- Cycle time: Time from “started” to “done.” Lower is generally better.
- Lead time: Time from request to delivery. Reflects true responsiveness.
- Burn-down or burn-up: Visualize progress and scope changes.
- Defect escape rate: Bugs found in production versus pre-release.
- WIP (Work In Progress): Lower WIP reduces context switching and increases flow.
Be careful not to weaponize metrics. Their purpose is to guide improvement, not to police people.
Reducing Risk with Iteration
- Risk-first scheduling: Do the riskiest or most uncertain work early (e.g., scalability, integrations).
- Spikes: Time-boxed research to answer unknowns and reduce uncertainty before committing to a solution.
- Architectural runway: Build just enough architecture to support the next few iterations, evolving as needs become clear.
- Tracer bullets: Implement a thin end-to-end slice to validate the path from UI to database early.
Tooling and Automation Essentials
- Version control with frequent integration: Short-lived branches; prefer trunk-based development when possible.
- CI/CD pipeline: Automatic build, test, and deploy to test environments on every change.
- Feature flags: Deploy incomplete features safely; toggle on when ready.
- Code review: Lightweight, fast reviews to improve quality and share context.
- Test automation pyramid: Emphasize many fast unit tests, fewer integration tests, and targeted end-to-end tests.
Common Pitfalls and How to Avoid Them
- Mini-waterfall inside the iteration: If you batch “all design, then all coding, then all testing,” you lose fast feedback. Instead, slice vertical features and finish them end to end.
- Carryover every sprint: Chronic spillover signals overcommitment or unclear acceptance criteria. Reduce scope and clarify done.
- Oversized iterations: Longer than 5 weeks reduces feedback frequency. Keep cycles short and predictable.
- Scope creep mid-iteration: Protect the iteration goal; put new requests into the backlog unless they are critical.
- Ignoring design: Iteration isn’t an excuse for chaos. Use refactoring and design reviews to keep quality high.
- Skipping retrospectives: Without reflection, problems calcify. Make small, continuous improvements.
When Iterative Development Might Not Be Ideal
- Strictly regulated, contractually fixed-scope projects with heavy upfront sign-off may constrain iteration. You can still prototype and iterate internally before baselining.
- Hardware-first or embedded systems with long lead times can limit release frequency. Combine iterative software development with simulation and staged integration.
- Safety-critical domains demand rigorous validation. Iterations can still help, but incorporate formal verification and traceability from day one.
Scaling Iterative Practices Across Teams
- Align on a shared cadence for integration and demos to reduce integration pain.
- Establish shared standards (coding, testing, observability) to keep quality consistent.
- Use a common backlog with clear ownership when multiple teams contribute to a product.
- Invest in enabling teams (platform, tooling, test automation) to reduce friction.
FAQs: Quick Answers
- Iterative vs. incremental: Iterative means you revisit and refine the same areas over multiple cycles; incremental means you add new slices of functionality. Most Agile teams do both: add increments iteratively.
- Iteration vs. sprint: In Scrum, an iteration is called a sprint. The concepts are equivalent.
- How long should an iteration be? Typically 2–5 weeks. Choose a length that balances feedback frequency with overhead and stick to it.
- Can requirements change mid-iteration? Urgencies happen, but minimize churn. Add changes to the backlog unless critical.
- What about documentation? Document just enough to support maintenance and compliance. Lightweight architecture decisions, API docs, and user-facing guides evolve with the product.
A Simple Iteration Checklist
- We have a clear sprint goal.
- Each story has acceptance criteria and a test approach.
- Work is sliced into small, independent tasks.
- CI is green; main branch is releasable.
- We demo working software, not slideware.
- We run a retrospective and commit to one improvement.
Bringing It All Together
- Iterative development replaces guesswork with learning.
- By delivering working, testable increments every 2–5 weeks, teams embrace change, reduce risk, and focus on value.
- Start with a simple slice, test it, learn from it, and let each iteration inform the next.
- With clear goals, strong collaboration, solid automation, and honest retrospectives, iteration becomes your engine for continuous improvement.
Amazon Book Recommendations on Waterfall, Agile, and Iterative Development
- Agile Estimating and Planning by Mike Cohn: Practical techniques for planning in evolving environments, with story points, velocity, and release forecasting. (Buy from Amazon link; I get a small commission for every purchase made through this link)
- User Stories Applied by Mike Cohn: How to craft user stories and acceptance criteria that capture intent and invite collaboration. (Buy from Amazon link; I get a small commission for every purchase made through this link)
- Scrum: The Art of Doing Twice the Work in Half the Time by Jeff Sutherland: The story and mechanics of Scrum, focusing on speed, feedback, and continuous improvement.(Buy from Amazon link; I get a small commission for every purchase made through this link)
- Continuous Delivery by Jez Humble and David Farley(Buy from Amazon link; I get a small commission for every purchase made through this link)
- Lean Software Development: An Agile Toolkit — Mary and Tom Poppendieck. Lean principles that complement iterative delivery. (Buy from Amazon link; I get a small commission for every purchase made through this link)
Helpful YouTube Videos and Channels Explaining the Concept
- What is Scrum? | Agile
.
- Mastering Agile Estimation: How to Perfect Story Points Estimation
Posted by
Ashish Agarwal
at
6/23/2007 06:37:00 PM
0
comments
Labels: Agile, Agile development, Development, Iterative, Processes, Software
|
| Subscribe by Email |
|