Test Estimation is one field of Software Engineering where little work has been done. Much work is still to be done to mature the field.
The most scientific approach known to date for Test Estimation is Test Point Analysis. However in its current form, it is mostly based upon the Project Size (arrived at by Functional Point Analysis or Use Case Point Analysis).
The accuracy of Test Estimates worked upon by Test Point Analysis depends upon the accuracy (and relevance) of the Project Size. If the Project Size is not known (not calculated or not available) it becomes very hard to estimate the Test Effort. Also, for Scrum based projects where estimates of Project Size is usually calculated using Planning Poker (Story Points); test estimation is very hard.
Recently, for a client we had to come up with some numbers about Test Effort required for the UAT with no readymade information about the project size available. The actual Project Size was something the vendor was supposed to come up with. From our end there was no time and capacity to start with the Project Size Estimation and then extend it to Test Estimation due to lack of resources, SMEs and lower level project details available.
The idea of breeding Story Points with Test Point Analysis struck my mind and I started working on this hybrid model.
So here's the final model (I am yet to validate the accuracy levels):
1. List all the high level test aspects (test stories).
2. Assign Story Points (relative) to all test stories. (Follow all the conventions attached to Story Points/ Planning Poker et al).
3. Identify the Dynamic Test Factors relevant to your project situation. (Limit this to 5 only). This is a limitation since we are going to modify the Test Point Analysis methodology a bit to suite our needs (deriving the estimate from Story Point instead of Use Case based Project Size called Use Case Points).
4. For the Dynamic Test Factors assign weights like:
Dynamic Test Factor #1: Low=3, Medium=6, High=12
Dynamic Test Factor #2: Low=2, Medium=4, High=12
Dynamic Test Factor #3: Low=2, Medium=4, High=8
Dynamic Test Factor #4: Low=3, Medium=6, High=12
Dynamic Test Factor #5: Low=3, Medium=6, High=12
5. You can reverse the weight assignment if the Dynamic Test Factor has a reverse impact on the test effort. For example, if Dynamic Test Factor # 3's High intensity decreases the test effort whereas, for other test factors the High intensity increases the test effort, then you can have Low= 8, Medium = 4, High = 2 for Dynamic Test Factor #3.
6. Now identify the 8 Environmental Test Factors impacting your project, and rate them on a scale of 0 to 5.
7. Calculate the Environmental Test Factor for the project by the formula
ETF= 1.4 + (-0.03*Sum of all Environmental Test Factors)
9. To each Test Story identified in step 1, assign Dynamic Test Factor weights (out of the Low, Medium High).
10. For each Test Story, You will get 5 distinct values -- one each for each Dynamic Test Factor. Now you need to consolidate these five values into a single value. For this apply the following formula (Sum all 5 values of Dynamic Test Factors, and divide it by 26).
Weighted Dynamic Test Factor= Sum(Dynamic Test Factor Values)/26
The value 26 is the nominal value of the Dynamic Test Factor weights. By doing so, you will get a single weighted Dynamic Test Factor value for each Test Story.
11. Now calculate Adjusted Dynamic Test Factor by multiplying the Weighted Dnamic Test Factor by its Story Points.
Adjusted Dynamic Test Factor = Weighted Dynamic Test Factor * Story Points
12. Now calculate Test Points for each Test Story by the following formula:
Test Points= Adjusted Dynamic Test Factor * ETF
13. The final Test Points for the entire project would be the sum of all the Test Points.
Now in order to do the Effort Estimation you need to factor the staff's Productivity.
The Productivity has the industry standard range of 0.7 to 2.0. (0.7 is for the highly productive team, whereas 2.0 is for a poorly productive team).
Effort = Test Points * Productivity
Let us say that the total Test Points you arrived at were 400, and your team's productivity is 1.0 then
Person Hours= 400 *1.0
Hence you need 400 person hours of test effort for this project with your current team.
Friday, July 31, 2009
Friday, July 24, 2009
Traveling Onsite: Reasons (contd.): # 5
Reason 5: Strategic Staff Exchange
Long term relationships between a client and a vendor; joint ventures and partnerships; merger / acquisitions trigger the need for a strategic staff exchange program. The motive of such staff exchange program can vary from basic training knowledge sharing to a more intangible benefits like cultural sensitization, international exposure to staff (in order to scale up their abilities) or job enrichment (to keep the morale high).
Post-merger Consolidation measures often trigger large scale staff exchange programs wherein the teams of the two business entities work together in order to gel the systems, processes and work approaches together.
I once happened to witness some client side engineers to travel to India to work with their vendor's team just because they wanted to add 'international exposure' in their CV. Their complaint was that often Indians travel to US which inflates their profile whereas the professionals in US miss on this international exposure tag because company does not send them to India. So it was more of an HR issue which the client had to deal with. Not so often seen but yes, it has started to happen in US :-).
Long term relationships between a client and a vendor; joint ventures and partnerships; merger / acquisitions trigger the need for a strategic staff exchange program. The motive of such staff exchange program can vary from basic training knowledge sharing to a more intangible benefits like cultural sensitization, international exposure to staff (in order to scale up their abilities) or job enrichment (to keep the morale high).
Post-merger Consolidation measures often trigger large scale staff exchange programs wherein the teams of the two business entities work together in order to gel the systems, processes and work approaches together.
I once happened to witness some client side engineers to travel to India to work with their vendor's team just because they wanted to add 'international exposure' in their CV. Their complaint was that often Indians travel to US which inflates their profile whereas the professionals in US miss on this international exposure tag because company does not send them to India. So it was more of an HR issue which the client had to deal with. Not so often seen but yes, it has started to happen in US :-).
Wednesday, July 22, 2009
Traveling Onsite: Reasons (contd.): # 4
Reason 4: Project Knowledge Gathering
As resources are inducted to an existing project or as a new project is initiated, the resources are asked to visit onsite for knowledge gathering purpose. The reason is induction in simplified terms. Resources are exposed to the new project environment (and organization culture, processes, standards etc) so that they can become productive in the project from offshore.
Usually this kind of travel is organized in groups involving formal or informal means of training or induction. Usually it is a low key affair and the skill set required is very much specific to the project requirements. This kind of travel is often done for a group of team members or batches (in case of operations outsourcing). As evident, this kind of onsite travel is a very low key affair and does not impact much on the individual's profile since not much depends upon an individual's capabilities. However, the resources gain substantial knowledge in terms of exposure, experience and cultural diversity which can be very useful in future.
As resources are inducted to an existing project or as a new project is initiated, the resources are asked to visit onsite for knowledge gathering purpose. The reason is induction in simplified terms. Resources are exposed to the new project environment (and organization culture, processes, standards etc) so that they can become productive in the project from offshore.
Usually this kind of travel is organized in groups involving formal or informal means of training or induction. Usually it is a low key affair and the skill set required is very much specific to the project requirements. This kind of travel is often done for a group of team members or batches (in case of operations outsourcing). As evident, this kind of onsite travel is a very low key affair and does not impact much on the individual's profile since not much depends upon an individual's capabilities. However, the resources gain substantial knowledge in terms of exposure, experience and cultural diversity which can be very useful in future.
Traveling Onsite: Reasons (contd.): # 3
Reason 3: Providing Consultancy
A reason of onsite travel which stands much higher in the value chain is providing consultancy. The seasoned Subject Matter Experts travel onsite for providing consultancy. The nature of consultancy and the impact on client's business are key drivers towards the kind of resource required and the duration of onsite stay. It may range from a specific project's topical area or may be a detailed recommendation fuelling a strategic decision.
Most of us feel onsite travellers as consultants (and most of those traveling onsite are indeed consultants in some way) however there are some big differences which need to be noticed.
Pure-play Consultants vs Service Delivery Consultants
If someone tells you 'how should you do something' or 'what will be impact of X on Y' from a strategic or long term perspective then it can be considerred as Pure Play consultant. However, if someone tells you 'how this thing can be done in this project' or 'what will be impact of this feature X on feature Y' then it is a consultancy provided during service delivery.
Pure-play consultants solve problems with global or larger impact; Service Delivery executives may provide topical consultancy with local impact.
A reason of onsite travel which stands much higher in the value chain is providing consultancy. The seasoned Subject Matter Experts travel onsite for providing consultancy. The nature of consultancy and the impact on client's business are key drivers towards the kind of resource required and the duration of onsite stay. It may range from a specific project's topical area or may be a detailed recommendation fuelling a strategic decision.
Most of us feel onsite travellers as consultants (and most of those traveling onsite are indeed consultants in some way) however there are some big differences which need to be noticed.
Pure-play Consultants vs Service Delivery Consultants
If someone tells you 'how should you do something' or 'what will be impact of X on Y' from a strategic or long term perspective then it can be considerred as Pure Play consultant. However, if someone tells you 'how this thing can be done in this project' or 'what will be impact of this feature X on feature Y' then it is a consultancy provided during service delivery.
Pure-play consultants solve problems with global or larger impact; Service Delivery executives may provide topical consultancy with local impact.
Traveling Onsite: Reasons (contd.): # 2
Reason 2: Establishing Long-term Relationships
Long term contracts once signed trigger the immediate travel of the middle level executives to the onsite location for establishing the relationship. The nature of such onsite visit is very different from winning new accounts (described earlier). Here the contract has already been signed, bringing in some certainty and confidence in vendor. However, the mid-level executive team is sent to setup the core team for a long term . Some of the key expectations from this core team's travel are:
1. Sufficient absorption of the client's technical environment, processes, and infrastructure.
2. Understanding the client organization's culture, environment and way or working.
3. Establishing communication channels within the client organization's segments.
4. Establish a positive sentiment among the (a bit nervous) staff of client organization due to recent offshore decision.
5. Chalk out the long term communication, collaborative work plans between the two business entities for a healthy business relationship.
Another key point of this kind of onsite travel is that it is non-project specific even though it may appear to be for a particular project. Such contracts are often done under a framework agreement between two companies, and under this umbrella framework agreement multiple project specific agreements are signed. For the first project the core team who travels has few members specific for the project in hand and some key resources for the higher level long term relationship establishment (as mentioned above).
Long term contracts once signed trigger the immediate travel of the middle level executives to the onsite location for establishing the relationship. The nature of such onsite visit is very different from winning new accounts (described earlier). Here the contract has already been signed, bringing in some certainty and confidence in vendor. However, the mid-level executive team is sent to setup the core team for a long term . Some of the key expectations from this core team's travel are:
1. Sufficient absorption of the client's technical environment, processes, and infrastructure.
2. Understanding the client organization's culture, environment and way or working.
3. Establishing communication channels within the client organization's segments.
4. Establish a positive sentiment among the (a bit nervous) staff of client organization due to recent offshore decision.
5. Chalk out the long term communication, collaborative work plans between the two business entities for a healthy business relationship.
Another key point of this kind of onsite travel is that it is non-project specific even though it may appear to be for a particular project. Such contracts are often done under a framework agreement between two companies, and under this umbrella framework agreement multiple project specific agreements are signed. For the first project the core team who travels has few members specific for the project in hand and some key resources for the higher level long term relationship establishment (as mentioned above).
Friday, June 05, 2009
Traveling Onsite: Reasons (contd.): # 1
Reason 1: Winning New Accounts
Onsite travel for serving a prospect adds highest value in a person's professional profile.
A typical large outsourcing deal takes 3-9 months before it finalizes. Vendor selection is the most exhaustive task during this phase. Most organizations typically follow multi-level shortlisting process, starting from the vendor selection due-diligence, understanding vendor's capabilities, service offerings, past experience, costing, productivity, utilization, and the overall total cost of ownership. Once the probable vendors are shortlisted, the organization assign them a small project to evaluate their delivery. Such projects are called 'pilot projects'.
The vendor who shocases best performance in the pilot project wins the account. It is during these pilot projects, the vendor sends the skilled resources to the onsite location for showcasing the skills, understand requirements and more importantly build cotarie with the client organization.
Resources traveling onsite for such assignments are usually smart enough to understand the client requirements, power centers of the organization, assess the unquoted factors of the evaluation. These resources are often expected to take the spot decisions after aptly judging the situation. Vendor's chances of winning or losing the account directly depend upon these resources.
Selecting the resources for pilot project onsite visit:
Since the chances of winning the account depend upon the technical competency, delivery excellence and smartness of the onsite resource, due care must be taken in selecting the right resource for such purposes.
1. Resource's technical capabilities must exceed the project demands.
2. Besides technical skills, the soft skills play a larger role in winning the deals. Hence, the person must have great interpersonal skills.
3. Capability to judge the organization's way of working, its needs and actual requirements must exist in the person.
4. The person should be articulative enough to hide the vendor's weaknesses/ negative aspects. However, the capability to understand where to draw the line between ethical non-disclosure of vendor's weaknesses or honest confession must be made by this person. So high level of maturity is involved. (It is always prudent to take time and include opinion from vendor's senior management before giving important commitments).
5. Outsourcing decision is usually painful for the client organization's employees (due to fear of losing job). Hence, the person should be smart to obtain best utilization of client's current staff and gather all valuable details.
6. Person should be strong to handle extreme pressure, especially when the stakes are high; multiple vendors are involved and the evaluation is taking place centrally.
More to come on this...
Onsite travel for serving a prospect adds highest value in a person's professional profile.
A typical large outsourcing deal takes 3-9 months before it finalizes. Vendor selection is the most exhaustive task during this phase. Most organizations typically follow multi-level shortlisting process, starting from the vendor selection due-diligence, understanding vendor's capabilities, service offerings, past experience, costing, productivity, utilization, and the overall total cost of ownership. Once the probable vendors are shortlisted, the organization assign them a small project to evaluate their delivery. Such projects are called 'pilot projects'.
The vendor who shocases best performance in the pilot project wins the account. It is during these pilot projects, the vendor sends the skilled resources to the onsite location for showcasing the skills, understand requirements and more importantly build cotarie with the client organization.
Resources traveling onsite for such assignments are usually smart enough to understand the client requirements, power centers of the organization, assess the unquoted factors of the evaluation. These resources are often expected to take the spot decisions after aptly judging the situation. Vendor's chances of winning or losing the account directly depend upon these resources.
Selecting the resources for pilot project onsite visit:
Since the chances of winning the account depend upon the technical competency, delivery excellence and smartness of the onsite resource, due care must be taken in selecting the right resource for such purposes.
1. Resource's technical capabilities must exceed the project demands.
2. Besides technical skills, the soft skills play a larger role in winning the deals. Hence, the person must have great interpersonal skills.
3. Capability to judge the organization's way of working, its needs and actual requirements must exist in the person.
4. The person should be articulative enough to hide the vendor's weaknesses/ negative aspects. However, the capability to understand where to draw the line between ethical non-disclosure of vendor's weaknesses or honest confession must be made by this person. So high level of maturity is involved. (It is always prudent to take time and include opinion from vendor's senior management before giving important commitments).
5. Outsourcing decision is usually painful for the client organization's employees (due to fear of losing job). Hence, the person should be smart to obtain best utilization of client's current staff and gather all valuable details.
6. Person should be strong to handle extreme pressure, especially when the stakes are high; multiple vendors are involved and the evaluation is taking place centrally.
More to come on this...
Wednesday, June 03, 2009
Traveling Onsite: Reasons
Being in outsourcing industry you get to travel a lot. People find it valuable to quote such travel experiences in their professional resume. There is a great 'demand' for skilled resources having international exposure- at least in India. Prospective employees with 'onsite experience' are valued more as compared to equally qualified people with no onsite exposure. This trend is more prominent in India.
I got several chances to travel various locations of US and Europe for serving client organizations. The most common reason for such travel is to gather knowledge, get training from onsite resources and then come back to your offshore location and carry out the work (and extend the training to other offshore resources). When I visited abroad for the first time, this was the very reason for my travel. However, over a period of time, I have seen more pervasive reasons for such onsite travels. These reasons are rarely known to most of the people and lesser understood by the recruiting agencies.
The reasons are (in their order of value addition):
1. Winning new accounts by showcasing skills to (still a prospective) client.
2. Establishing the building blocks of a long term outsourcing relationship.
3. Providing consultancy to clients
4. Knowledge gathering for new project (to be executed at offshore)
5. Visits under staff exchange program as a strategic step for long term relationships (between vendor and client).
Watch this space for interesting findings about the nature of onsite visit, and the recruitment perspective attached with them...
I got several chances to travel various locations of US and Europe for serving client organizations. The most common reason for such travel is to gather knowledge, get training from onsite resources and then come back to your offshore location and carry out the work (and extend the training to other offshore resources). When I visited abroad for the first time, this was the very reason for my travel. However, over a period of time, I have seen more pervasive reasons for such onsite travels. These reasons are rarely known to most of the people and lesser understood by the recruiting agencies.
The reasons are (in their order of value addition):
1. Winning new accounts by showcasing skills to (still a prospective) client.
2. Establishing the building blocks of a long term outsourcing relationship.
3. Providing consultancy to clients
4. Knowledge gathering for new project (to be executed at offshore)
5. Visits under staff exchange program as a strategic step for long term relationships (between vendor and client).
Watch this space for interesting findings about the nature of onsite visit, and the recruitment perspective attached with them...
Tuesday, June 02, 2009
First Disconnenct-27 March 2007 to 2 June 2009
On 28 March 2007 it was not thought of. Till 2 June 2009 it was seemingly unintentional disconnect. Today on 2 June 2009 it seems that the enlightenment has come. The Disconnect must fade away.
I am talking about the disconnect I imposed upon myself from the digital world by not blogging. Forgot password? Nope, I didn't. Shifted priorities? Somewhat, not exactly. Then what?
'The First Attempt' is the place where I tell you about whatever I attempt for the first time. However, during this period of disconnect, I slid myself into the cocoon just to explore my inner self. Exploring inner self by becoming a monk is common- several saints have done it, many more are doing it. I explored myself but did not disconnect from the outside world including friends & family.
Did you see the disconnect in my disconnect from the digital world and the outside world? Does that mean that the digital world is not the outside world for me? I don't know exactly, but digital world is the one which helps me to stay with myself more freely. Outside world imposes restrictions.
Enough about disconnect, lets us see what all I explored about myself and how I looked at the various facets of life, world, nature, business, people and economy et al.
I am talking about the disconnect I imposed upon myself from the digital world by not blogging. Forgot password? Nope, I didn't. Shifted priorities? Somewhat, not exactly. Then what?
'The First Attempt' is the place where I tell you about whatever I attempt for the first time. However, during this period of disconnect, I slid myself into the cocoon just to explore my inner self. Exploring inner self by becoming a monk is common- several saints have done it, many more are doing it. I explored myself but did not disconnect from the outside world including friends & family.
Did you see the disconnect in my disconnect from the digital world and the outside world? Does that mean that the digital world is not the outside world for me? I don't know exactly, but digital world is the one which helps me to stay with myself more freely. Outside world imposes restrictions.
Enough about disconnect, lets us see what all I explored about myself and how I looked at the various facets of life, world, nature, business, people and economy et al.
Subscribe to:
Posts (Atom)
