In Software projects ,it is most important to measure the quality ,cost and effectiveness of the project and the processes. Without measuring these ,project can't be completed successfully.
What are Software Testing Metrics?
A Metric is a quantitative measure of the degree to which a system, system component, or process possesses a given attribute.
What are Software Testing Metrics?
A Metric is a quantitative measure of the degree to which a system, system component, or process possesses a given attribute.
Suppose, in general, “Kilogram” is a
metric for measuring the attribute “Weight”. Similarly, in software, “How many
issues are found in thousand lines of code?”, here No. of issues is one
measurement & No. of lines of code is another measurement. Metric is
defined from these two measurements.
Test
metrics Could be:
- How many defects are existed within the module?
- How many test cases are executed per person?
- What is the Test coverage %?
Who is accountable for For Metrics: In general , Test Lead /Manager are responsible for generating the test metrics.
Advantages:
- Take the decision for next phase of activities such as, estimate the cost & schedule of future projects.
- Understand the kind of improvement required to success the project
- Take decision on process or technology to be modified etc.
- Test Metrics are the most important to measure the quality of the software.
If we don't have any software metrics in our project ,then how can we measure the quality of work done by Software Tester/QA.
For
Example: A Test Analyst has to,
- Design the test cases for 5
requirements
- Execute the designed test cases
- Log the defects & need to
fail the related test cases
- After the defect is resolved,
need to re-test the defect & re-execute the corresponding failed test
case.
In above scenario, if metrics are
not followed, then the work completed by the test analyst will be subjective
i.e. the test report will not have the proper
information to know the status of his work/project.
If Metrics are involved in the
project, then the exact status of his/her work with proper numbers/data can be
published.
I.e. in
the Test report, we can publish:
1. How many test cases have been designed per requirement?
2. How many test cases are yet to design?
3. How many test cases are executed?
4. How many test cases are passed/failed/blocked?
5. How many test cases are not yet executed?
6. How many defects are identified & what is the severity of those defects?
7. How many test cases are failed due to one particular defect? etc.
1. How many test cases have been designed per requirement?
2. How many test cases are yet to design?
3. How many test cases are executed?
4. How many test cases are passed/failed/blocked?
5. How many test cases are not yet executed?
6. How many defects are identified & what is the severity of those defects?
7. How many test cases are failed due to one particular defect? etc.
Based on the project needs we can
have more metrics than above mentioned list, to know the status of the project
in detail.
Based on
the above metrics, test lead/manager will get the understanding of the below
mentioned key points.
a) %ge of work completed
b) %ge of work yet to be completed
c) Time to complete the remaining work
d) Whether the project is going as per the schedule or lagging? etc.
a) %ge of work completed
b) %ge of work yet to be completed
c) Time to complete the remaining work
d) Whether the project is going as per the schedule or lagging? etc.
Based on the metrics, if the project is not
going to complete as per the schedule, then the manager will raise the alarm to
the client and other stake holders by providing the reasons for lagging to
avoid the last minute surprises.
Examples of Software Testing Metrics:
Let’s take an example to calculate
various test metrics used in software test reports:
Below is
the table format for the data retrieved from the test analyst who is actually
involved in testing:
Definitions
and Formulas for Calculating Metrics:
#1) %ge
Test cases Executed:
This metric is used to obtain the execution status of the test cases in terms
of %ge.
%ge Test cases Executed = (No.
of Test cases executed / Total no. of Test cases written) * 100.
So, from the above data,
%ge Test cases Executed = (65 / 100) * 100 = 65%
%ge Test cases Executed = (65 / 100) * 100 = 65%
#2) %ge
Test cases not executed:
This metric is used to obtain the pending execution status of the test cases in
terms of %ge.
%ge Test cases not executed = (No.
of Test cases not executed / Total no. of Test cases written) * 100.
So, from the above data,
%ge Test cases Blocked = (35 / 100) * 100 = 35%
%ge Test cases Blocked = (35 / 100) * 100 = 35%
#3) %ge
Test cases Passed:
This metric is used to obtain the Pass %ge of the executed test cases.
%ge Test cases Passed = (No.
of Test cases Passed / Total no. of Test cases Executed) * 100.
So, from the above data,
%ge Test cases Passed = (30 / 65) * 100 = 46%
%ge Test cases Passed = (30 / 65) * 100 = 46%
#4) %ge
Test cases Failed:
This metric is used to obtain the Fail %ge of the executed test cases.
%ge Test cases Failed = (No.
of Test cases Failed / Total no. of Test cases Executed) * 100.
So, from the above data,
%ge Test cases Passed = (26 / 65) * 100 = 40%
%ge Test cases Passed = (26 / 65) * 100 = 40%
#5) %ge
Test cases Blocked:
This metric is used to obtain the blocked %ge of the executed test cases. A
detailed report can be submitted by specifying the actual reason of blocking
the test cases.
%ge Test cases Blocked = (No.
of Test cases Blocked / Total no. of Test cases Executed) * 100.
So, from the above data,
%ge Test cases Blocked = (9 / 65) * 100 = 14%
%ge Test cases Blocked = (9 / 65) * 100 = 14%
#6) Defect
Density = No. of Defects identified / size
(Here “Size” is considered as
requirement. Hence here the Defect Density is calculated as number of defects
identified per requirement. Similarly, Defect Density can be calculated as
number of Defects identified per 100 lines of code [OR] No. of defects
identified per module etc.)
So, from the above data,
Defect Density = (30 / 5) = 6
Defect Density = (30 / 5) = 6
#7) Defect
Removal Efficiency (DRE) = (No.
of Defects found during QA testing / (No. of Defects found during QA testing
+No. of Defects found by End user)) * 100
DRE is used to identify the test
effectiveness of the system.
Suppose, During Development & QA testing, we have identified 100 defects.
After the QA testing, during Alpha & Beta testing, end user / client identified 40 defects, which could have been identified during QA testing phase.
Suppose, During Development & QA testing, we have identified 100 defects.
After the QA testing, during Alpha & Beta testing, end user / client identified 40 defects, which could have been identified during QA testing phase.
Now, The DRE will be calculated as,
DRE = [100 / (100 + 40)] * 100 = [100 /140] * 100 = 71%
DRE = [100 / (100 + 40)] * 100 = [100 /140] * 100 = 71%
$8) Defect
Leakage: Defect Leakage is the Metric which
is used to identify the efficiency of the QA testing i.e., how many defects are
missed / slipped during the QA testing.
Defect Leakage = (No. of
Defects found in UAT / No. of Defects found in QA testing.) * 100
Suppose, During Development & QA
testing, we have identified 100 defects.
After the QA testing, during Alpha & Beta testing, end user / client identified 40 defects, which could have been identified during QA testing phase.
After the QA testing, during Alpha & Beta testing, end user / client identified 40 defects, which could have been identified during QA testing phase.
Defect Leakage = (40 /100) * 100 =
40%
#9)
Defects by Priority:
This metric is used to identify the no. of defects identified based on the
Severity / Priority of the defect which is used to decide the quality of the
software.
%ge Critical Defects = No. of
Critical Defects identified / Total no. of Defects identified * 100
From the data available in the above table,
%ge Critical Defects = 6/ 30 * 100 = 20%
From the data available in the above table,
%ge Critical Defects = 6/ 30 * 100 = 20%
%ge High Defects = No. of High
Defects identified / Total no. of Defects identified * 100
From the data available in the above table,
%ge High Defects = 10/ 30 * 100 = 33.33%
From the data available in the above table,
%ge High Defects = 10/ 30 * 100 = 33.33%
%ge Medium Defects = No. of Medium
Defects identified / Total no. of Defects identified * 100
From the data available in the above table,
%ge Medium Defects = 6/ 30 * 100 = 20%
From the data available in the above table,
%ge Medium Defects = 6/ 30 * 100 = 20%
%ge Low Defects = No. of Low Defects
identified / Total no. of Defects identified * 100
From the data available in the above table,
%ge Low Defects = 8/ 30 * 100 = 27%
From the data available in the above table,
%ge Low Defects = 8/ 30 * 100 = 27%
Conclusion:
The metrics provided in this article
are majorly used for generating the daily/weekly status
report with
accurate data during test case development/execution phase & this is also
useful for tracking the project status & Quality of the software.


