Test Activities and Tasks

Test planning

Test planning is a crucial aspect of the software testing process, as it helps ensure that testing is carried out effectively and efficiently. The main purpose of a test plan is to provide a roadmap for the testing effort, outlining the objectives, scope, approach, resources, and schedule for testing.

The contents of a test plan can vary depending on the project and its requirements, but some typical sections include:

1. Introduction: A brief overview of the purpose and scope of the test plan.
2. Test objectives: A description of the goals and objectives of the testing effort.
3. Scope: A definition of what is included and excluded from the testing effort.
4. Test approach: A description of the overall approach to testing, including the types of testing to be performed and the techniques to be used.
5. Test environment: A description of the hardware, software, and other resources needed for testing.
6. Test schedule: A detailed timeline for the testing effort, including milestones and deadlines.
7. Test deliverables: A list of the documents and artifacts that will be produced during the testing effort.
8. Test risks and contingencies: A description of the potential risks to the testing effort and the contingency plans in place to mitigate them.
9. Test team roles and responsibilities: A description of the roles and responsibilities of the members of the testing team.
10. Approval and sign-off: A section for obtaining approval and sign-off from stakeholders.

Here's an example test plan for a credit card application project:

1. Introduction
   • Purpose and scope of the test plan
   • Overview of the credit card application project
2. Test objectives
   • Ensure that the credit card application meets functional requirements
   • Validate that the application performs accurately and efficiently
   • Confirm that the application is user-friendly and intuitive
   • Ensure that the application is secure and reliable
3. Scope
   • In-scope: Credit card application modules, user interface, and functionality
   • Out-of-scope: Credit card processing and transactions
4. Test approach
   • Test types: Functional, regression, usability, security
   • Test techniques: Black-box, boundary value analysis, equivalence partitioning
   • Test automation: Automated functional and regression tests
   • Test data management: Use of real and synthetic data for testing
5. Test environment
   • Hardware: Standard office workstations and servers
   • Software: Operating systems, browsers, databases, test automation tools
   • Other resources: Test data, user credentials
6. Test schedule
   • Milestones: Test plan approval, test case design, test execution, defect management
   • Deadlines: Completion of testing before application launch date
7. Test deliverables
   • Test plan, test cases, test reports, defect reports, test automation scripts
8. Test risks and contingencies
   • Risks: Security vulnerabilities, testing delays, resource constraints
   • Contingencies: Use of third-party testing services, additional resources allocated for testing, defect triage and prioritization
9. Test team roles and responsibilities
   • Test manager: Overall responsibility for testing effort
   • Test analysts: Test case design and execution
   • Test automation engineers: Development and maintenance of automated test scripts
   • Business analysts: Requirements gathering and validation
   • Developers: Defect resolution and retesting
10. Approval and sign-off
    • Sign-off from project sponsor and stakeholders on the test plan and associated deliverables.

Test planning is further explained in Test Plan Design

Test monitoring

Test monitoring is an ongoing process of comparing the actual progress of testing against the planned progress as defined in the test plan. It involves the use of test monitoring metrics to track the progress of testing and identify any deviations from the planned schedule, test coverage, or quality criteria.

Test control is the process of taking actions necessary to ensure that testing is meeting its objectives as defined in the test plan. This may involve adjusting the testing approach or test plan based on feedback from the monitoring process, or taking corrective actions to address any issues or defects that are discovered during testing.

The evaluation of exit criteria is an important aspect of both test monitoring and test control. Exit criteria are the defined set of conditions that must be met before testing can be considered complete, such as achieving a certain level of test coverage or meeting a specific level of product quality.

Test progress is communicated to stakeholders through test progress reports, which provide information on the progress of testing, any deviations from the test plan, and any decisions made regarding the continuation or completion of testing. Effective test monitoring and control are essential to ensuring the success of the testing process and the delivery of a high-quality product.

Test monitoring and control are further explained in Test Monitoring

Test analysis

Test analysis is a crucial activity in the software testing process that helps in identifying the testable features and defining associated test conditions. The primary objective of test analysis is to determine "what to test" in terms of measurable coverage criteria. It involves analyzing the test basis that includes requirement specifications, design and implementation information, the implementation of the component or system itself, and risk analysis reports.

During test analysis, the test basis is evaluated to identify defects of various types such as ambiguities, omissions, inconsistencies, inaccuracies, contradictions, and superfluous statements. The identified defects are documented, tracked, and reported to ensure that they are fixed in the subsequent phases of the testing process.

Test analysis also involves identifying features and sets of features to be tested and defining and prioritizing test conditions for each feature based on analysis of the test basis. The test conditions consider functional, non-functional, and structural characteristics, other business and technical factors, and levels of risks. Bi-directional traceability between each element of the test basis and the associated test conditions is captured to ensure that all requirements are adequately tested.

The application of black-box, white-box, and experience-based test techniques can be useful in the process of test analysis to reduce the likelihood of omitting important test conditions and to define more precise and accurate test conditions. In some cases, test analysis produces test conditions that are used as test objectives in test charters, which are typical work products in some types of experience-based testing.

The identification of defects during test analysis is an essential potential benefit, especially where no other review process is being used and/or the test process is closely connected with the review process. Test analysis activities verify whether the requirements are consistent, properly expressed, and complete, and also validate whether the requirements properly capture customer, user, and other stakeholder needs.

For example, techniques such as behavior-driven development (BDD) and acceptance test-driven development (ATDD) involve generating test conditions and test cases from user stories and acceptance criteria prior to coding. These techniques also verify, validate, and detect defects in the user stories and acceptance criteria.

Test implementation

Test implementation is the phase in software testing where the testware necessary for test execution is created or completed. This includes developing and prioritizing test procedures, creating test suites from test procedures and automated test scripts, arranging the test suites within a test execution schedule, building the test environment, preparing test data, and verifying bi-directional traceability. Test design and implementation tasks can be combined, especially in exploratory testing and other experience-based testing, where test design and implementation occur during test execution. The goal of test implementation is to ensure that everything is in place to run the tests efficiently and effectively. Test design answers the question "how to test?" while test implementation answers the question "do we now have everything in place to run the tests?"

Test Execution

Test execution, it may be necessary to:

• Manage test environments, including setting up and tearing down test environments between test executions, or restoring a test environment to a specific baseline state

• Manage test data, including creating, modifying, or deleting test data as needed during test execution

• Monitor and manage test progress, including tracking progress against test completion criteria, logging results, and reporting status to stakeholders 

• Manage test incidents, including triaging, investigating, and resolving incidents that arise during test execution

• Manage test documentation, including updating test cases, procedures, and other testware based on feedback from test execution

• Manage test resources, including scheduling and coordinating resources required for test execution, such as human resources, equipment, and facilities.

it may be necessary to adapt the tests or test procedures as issues arise or new information is discovered. This could include modifying test data, adjusting test steps, or adding new tests.

Test execution is typically performed multiple times during the testing process, including during functional testing, integration testing, system testing, acceptance testing, and regression testing.

It is important during test execution to have clear and concise documentation of the test results and any defects found, as well as any modifications made to the tests or test procedures. This information can be used to inform future testing efforts and to help identify trends and patterns in the defects found.

Test completion

Test completion activities are important as they provide a chance to evaluate the testing effort and improve future testing processes. The activities help to ensure that all defects are properly documented and that the test environment and other testware are properly archived for future use.

Creating a test summary report is an important activity as it helps to communicate the testing results to stakeholders. The report should include information such as the testing objectives, scope, testing techniques used, and the results of the testing effort. It should also highlight any issues or risks that were identified during testing.

Analyzing lessons learned is another important activity. This involves reviewing the testing effort to identify areas for improvement. This can include identifying areas where testing techniques can be improved, or where more effective test tools could be used. The goal is to identify ways to improve the overall testing process so that future testing efforts are more effective and efficient.

Overall, test completion activities are important for ensuring that the testing process is properly concluded and that the results are properly documented and communicated. By analyzing the results of the testing effort, it is possible to identify areas for improvement and to continuously improve the testing process.