What is Software Testing Auditing Process

Introduction: To ensure transparency and reliability of the IT systems it may be necessary to audit the Software Development Processes including the most important aspect – Software Testing Process. it ensure that the Testing processes are followed as defined.

Types of Testing Process Audits
The Audits are performed to achieve certain following goals :

Audit to verify compliance: To judge if the process complies with a standards. In this, the actual testing conducted is compared with the documented process.

Audit for process improvement/problem solving: To audit and trace the various steps in the process and try to weed out process problems. For instance it is observed that too many software defects escaped detection even though the testing process was apparently followed. So the audit is done as a preliminary step to collect facts and analyze them.

Audit for Root Cause Analysis : To audit the testing process is to find a Root Cause of a specific problem. For example the customers discovered a high severity defect in software. So we retrace our testing steps to find out what went wrong in this specific case.

Internal Audits – The internal audits are initiated from within the organizations

External Audits – External Audits are done by and initiated by external agencies

Need of Software Testing Process Audit –
It is a management activity & helps the management understand if the process is being followed as specified.

Testing audit may be peforemd for one or more of the following factors:
• To ensure continued reliability and integrity of the process
• To verify compliance of standards (ISO, CMM, etc)
• To solve process related problems
• To find the root cause of a specific problem
• To detect or prevent Fraud
• To improve the Testing process

Auditing of the Testing process may also be done if the Software Product is a mission critical one such as used for Medical Life Support Systems. This is done to prevent any loop holes or bugs in the system.

How to Audit
Typically the Audit of the Testing Process will include the following steps:

• reviewing the Testing process as documented in the Quality Manual. This helps the auditor
understand the process as defined.
• Reviewing the deliverable documents at each step
• Document reviewed include
– Test Strategy
– Test Plans
– Test Cases
– Test Logs
– Defects Tracked
– Test Coverage Matrix
– any other relevant records

Each of the above document provides a certain level of traceability that the process was followed and the necessary steps were taken

• Interviewing the Project Team at various levels – PM, Coordinator, Tester.

Interviewing the Project Team members gives an understanding of the thought process prevalent in those conducting the Testing Process. This can provide valuable insights over an above what was actually documented

Whether test cases created covered all requirements/use cases
This analysis reveals if the test coverage was sufficient. It indicates that whether the testing team did the best to provide adequate amount of testing

Whether all Defects were fixed – The Status of all the Defects logged is checked to verify if all were fixed and verified

Whether there are any known bugs in the software released – Sometimes all the defects may not be fixed, the software may be released with known problems. Test Logs would indicate the actual results and evidence of any bugs being present.

Whether the levels of testing was effective enough – If Defects pass thru the various levels of testing undetected, it may reflect poorly on the effectiveness of the testing process
• What were the number of defects (Defect Leaks) that went by undetected in each phase
• Number of iterations of testing in each level
• Time taken to test each module/component
• This data may be used for process improvement
• Versions of source code actually tested

The Test Logs and Defect Logs indicates (if the information was captured) the actual versions of code/components tested. This information is valuable in root cause analysis.