Test Estimation Technique: Hybrid of Story Points and Test Points Analysis

Test Estimation is one field of Software Engineering where little work has been done. Much work is still to be done to mature the field.
The most scientific approach known to date for Test Estimation is Test Point Analysis. However in its current form, it is mostly based upon the Project Size (arrived at by Functional Point Analysis or Use Case Point Analysis).
The accuracy of Test Estimates worked upon by Test Point Analysis depends upon the accuracy (and relevance) of the Project Size. If the Project Size is not known (not calculated or not available) it becomes very hard to estimate the Test Effort. Also, for Scrum based projects where estimates of Project Size is usually calculated using Planning Poker (Story Points); test estimation is very hard.

Recently, for a client we had to come up with some numbers about Test Effort required for the UAT with no readymade information about the project size available. The actual Project Size was something the vendor was supposed to come up with. From our end there was no time and capacity to start with the Project Size Estimation and then extend it to Test Estimation due to lack of resources, SMEs and lower level project details available.

The idea of breeding Story Points with Test Point Analysis struck my mind and I started working on this hybrid model.

So here's the final model (I am yet to validate the accuracy levels):

1. List all the high level test aspects (test stories).

2. Assign Story Points (relative) to all test stories. (Follow all the conventions attached to Story Points/ Planning Poker et al).

3. Identify the Dynamic Test Factors relevant to your project situation. (Limit this to 5 only). This is a limitation since we are going to modify the Test Point Analysis methodology a bit to suite our needs (deriving the estimate from Story Point instead of Use Case based Project Size called Use Case Points).

4. For the Dynamic Test Factors assign weights like:
Dynamic Test Factor #1: Low=3, Medium=6, High=12
Dynamic Test Factor #2: Low=2, Medium=4, High=12
Dynamic Test Factor #3: Low=2, Medium=4, High=8
Dynamic Test Factor #4: Low=3, Medium=6, High=12
Dynamic Test Factor #5: Low=3, Medium=6, High=12

5. You can reverse the weight assignment if the Dynamic Test Factor has a reverse impact on the test effort. For example, if Dynamic Test Factor # 3's High intensity decreases the test effort whereas, for other test factors the High intensity increases the test effort, then you can have Low= 8, Medium = 4, High = 2 for Dynamic Test Factor #3.

6. Now identify the 8 Environmental Test Factors impacting your project, and rate them on a scale of 0 to 5.

7. Calculate the Environmental Test Factor for the project by the formula

ETF= 1.4 + (-0.03*Sum of all Environmental Test Factors)

9. To each Test Story identified in step 1, assign Dynamic Test Factor weights (out of the Low, Medium High).

10. For each Test Story, You will get 5 distinct values -- one each for each Dynamic Test Factor. Now you need to consolidate these five values into a single value. For this apply the following formula (Sum all 5 values of Dynamic Test Factors, and divide it by 26).

Weighted Dynamic Test Factor= Sum(Dynamic Test Factor Values)/26

The value 26 is the nominal value of the Dynamic Test Factor weights. By doing so, you will get a single weighted Dynamic Test Factor value for each Test Story.

11. Now calculate Adjusted Dynamic Test Factor by multiplying the Weighted Dnamic Test Factor by its Story Points.

Adjusted Dynamic Test Factor = Weighted Dynamic Test Factor * Story Points

12. Now calculate Test Points for each Test Story by the following formula:

Test Points= Adjusted Dynamic Test Factor * ETF

13. The final Test Points for the entire project would be the sum of all the Test Points.


Now in order to do the Effort Estimation you need to factor the staff's Productivity.
The Productivity has the industry standard range of 0.7 to 2.0. (0.7 is for the highly productive team, whereas 2.0 is for a poorly productive team).

Effort = Test Points * Productivity

Let us say that the total Test Points you arrived at were 400, and your team's productivity is 1.0 then
Person Hours= 400 *1.0

Hence you need 400 person hours of test effort for this project with your current team.