Test reports are distributed and generated during the test performance cycles.
After finishing all the test cycles the final summary report is made. The final report has the next structure:
1. The test project’s introduction. The purpose of the test report is summarized according to the test plan in the test project introduction section. The following information is included:
- Project’s name
- Software image’s name
- Document’s revision history of this
- Definitions & Terminology
- Testing staff
- Document’s scope
2. Test results summary. The test results for each test category, which are identified in the test plan, are summarized in the test results section. The test results are summarized in the form of the numbers of test cases failed, passed, invalid, untested and blocked in each test cycle. And the reasons for not carrying out some test cases in software testing process. Several reasons can cause the unchecked test such as difficulty in creating the test scenario using the available test bed setup and unavailability of equipment. The last case may be possible only in at a real beta-testing site. The following data are given in this section for each test cycle:
- Completion and Start dates
- Number of defects filed
- Defects number in different states, such as FAD, irreproducible, shelved, closed, postponed, duplicate, open and assigned.
The problems are still present in the system, including any likelihood of failure or shortcoming. They are gathered in this section in terms of their potential impact on the customers and their levels of severity.
3. Characteristics of performance. The system response time, resource utilization, delay measurement results and throughput are described in the performance characteristics section of the document along with the test tools and environments used in the measurement process.
4. Limitations of scaling. The system’s limits are formulated in the section of scaling limitation of the document based on the scalability testing findings. The facilities and tools that are used in scaling testing are described here.
5. Observations of stability. The stability observation section includes the following information:
- Hours number spent in the system
- Crashes number observed by different groups, such as SIT, software development team and ST, in the test cycles weekly.
- Descriptions of defects symptoms, which causing system crash that are in the state, that cannot be reproduced.
6. System’s interoperability. We point in the section of interoperability, those third-party hardware and software systems, which were subjected to the system interoperability tests.
7. Software/hardware reconcilable matrix. A table demonstrates what software versions are reconcilable with what hardware revisions in the software/hardware reconcilable matrix section.
8. Status of compliance requirement. At last, in the section of compliance requirement status, there is a compliance statistics summarized by generating them from the requirements database.
Making corresponding queries on the database produces the next reports.
- Requirements that are in noncompliance, compliance or partial compliance with the software image.
- Requirements that are checked by testing, inspection, analysis and demonstration.
- Requirements covered by each test case.