Test Estimation Technique: Hybrid of Story Points and Test Points Analysis

Test Estimation is one field of Software Engineering where little work has been done. Much work is still to be done to mature the field.
The most scientific approach known to date for Test Estimation is Test Point Analysis. However in its current form, it is mostly based upon the Project Size (arrived at by Functional Point Analysis or Use Case Point Analysis).
The accuracy of Test Estimates worked upon by Test Point Analysis depends upon the accuracy (and relevance) of the Project Size. If the Project Size is not known (not calculated or not available) it becomes very hard to estimate the Test Effort. Also, for Scrum based projects where estimates of Project Size is usually calculated using Planning Poker (Story Points); test estimation is very hard.

Recently, for a client we had to come up with some numbers about Test Effort required for the UAT with no readymade information about the project size available. The actual Project Size was something the vendor was supposed to come up with. From our end there was no time and capacity to start with the Project Size Estimation and then extend it to Test Estimation due to lack of resources, SMEs and lower level project details available.

The idea of breeding Story Points with Test Point Analysis struck my mind and I started working on this hybrid model.

So here's the final model (I am yet to validate the accuracy levels):

1. List all the high level test aspects (test stories).

2. Assign Story Points (relative) to all test stories. (Follow all the conventions attached to Story Points/ Planning Poker et al).

3. Identify the Dynamic Test Factors relevant to your project situation. (Limit this to 5 only). This is a limitation since we are going to modify the Test Point Analysis methodology a bit to suite our needs (deriving the estimate from Story Point instead of Use Case based Project Size called Use Case Points).

4. For the Dynamic Test Factors assign weights like:
Dynamic Test Factor #1: Low=3, Medium=6, High=12
Dynamic Test Factor #2: Low=2, Medium=4, High=12
Dynamic Test Factor #3: Low=2, Medium=4, High=8
Dynamic Test Factor #4: Low=3, Medium=6, High=12
Dynamic Test Factor #5: Low=3, Medium=6, High=12

5. You can reverse the weight assignment if the Dynamic Test Factor has a reverse impact on the test effort. For example, if Dynamic Test Factor # 3's High intensity decreases the test effort whereas, for other test factors the High intensity increases the test effort, then you can have Low= 8, Medium = 4, High = 2 for Dynamic Test Factor #3.

6. Now identify the 8 Environmental Test Factors impacting your project, and rate them on a scale of 0 to 5.

7. Calculate the Environmental Test Factor for the project by the formula

ETF= 1.4 + (-0.03*Sum of all Environmental Test Factors)

9. To each Test Story identified in step 1, assign Dynamic Test Factor weights (out of the Low, Medium High).

10. For each Test Story, You will get 5 distinct values -- one each for each Dynamic Test Factor. Now you need to consolidate these five values into a single value. For this apply the following formula (Sum all 5 values of Dynamic Test Factors, and divide it by 26).

Weighted Dynamic Test Factor= Sum(Dynamic Test Factor Values)/26

The value 26 is the nominal value of the Dynamic Test Factor weights. By doing so, you will get a single weighted Dynamic Test Factor value for each Test Story.

11. Now calculate Adjusted Dynamic Test Factor by multiplying the Weighted Dnamic Test Factor by its Story Points.

Adjusted Dynamic Test Factor = Weighted Dynamic Test Factor * Story Points

12. Now calculate Test Points for each Test Story by the following formula:

Test Points= Adjusted Dynamic Test Factor * ETF

13. The final Test Points for the entire project would be the sum of all the Test Points.


Now in order to do the Effort Estimation you need to factor the staff's Productivity.
The Productivity has the industry standard range of 0.7 to 2.0. (0.7 is for the highly productive team, whereas 2.0 is for a poorly productive team).

Effort = Test Points * Productivity

Let us say that the total Test Points you arrived at were 400, and your team's productivity is 1.0 then
Person Hours= 400 *1.0

Hence you need 400 person hours of test effort for this project with your current team.

Traveling Onsite: Reasons (contd.): # 5

Reason 5: Strategic Staff Exchange

Long term relationships between a client and a vendor; joint ventures and partnerships; merger / acquisitions trigger the need for a strategic staff exchange program. The motive of such staff exchange program can vary from basic training knowledge sharing to a more intangible benefits like cultural sensitization, international exposure to staff (in order to scale up their abilities) or job enrichment (to keep the morale high).

Post-merger Consolidation measures often trigger large scale staff exchange programs wherein the teams of the two business entities work together in order to gel the systems, processes and work approaches together.
I once happened to witness some client side engineers to travel to India to work with their vendor's team just because they wanted to add 'international exposure' in their CV. Their complaint was that often Indians travel to US which inflates their profile whereas the professionals in US miss on this international exposure tag because company does not send them to India. So it was more of an HR issue which the client had to deal with. Not so often seen but yes, it has started to happen in US :-).

Traveling Onsite: Reasons (contd.): # 4

Reason 4: Project Knowledge Gathering

As resources are inducted to an existing project or as a new project is initiated, the resources are asked to visit onsite for knowledge gathering purpose. The reason is induction in simplified terms. Resources are exposed to the new project environment (and organization culture, processes, standards etc) so that they can become productive in the project from offshore.
Usually this kind of travel is organized in groups involving formal or informal means of training or induction. Usually it is a low key affair and the skill set required is very much specific to the project requirements. This kind of travel is often done for a group of team members or batches (in case of operations outsourcing). As evident, this kind of onsite travel is a very low key affair and does not impact much on the individual's profile since not much depends upon an individual's capabilities. However, the resources gain substantial knowledge in terms of exposure, experience and cultural diversity which can be very useful in future.