Using Hypothesis Testing in IT

How the use of hypothesis testing shall be a great asset in my company?

We do decision making everyday in our personal and professional life but most of the times it’s not backed by sufficient data to arrive at a confident decision. For example, we do choose different routes to reach a particular destination, but this decision most of the time is based on our likes of the route or experience with that route. This doesn’t provide us the sufficient confidence with objective evidence that with certain confidence level the route will result in saving travel time. Let me take another example from professional life, every time a software development project is won in an IT industry, we deploy various resources for the execution of project based on our estimate of size and effort required to complete the project. Here again our decision is based on the overall productivity and we get the number of resources to be deployed. We donot take into account what experience level of resources is going to have what impact on the execution effort and time for the project.

Hypothesis testing is the testing of any assumptions regarding a population with the sample data. There are broadly two types of hypothesis testing viz. Null & Alternate hypothesis.

The null hypothesis testing tests the claim of equality or of no difference, while alternate hypothesis tests inequality or of difference between two or more populations.

The assumptions made in the null hypothesis can be accepted / rejected on the basis of p-value comparison with significance level or alpha.

There can be two types of errors which can be committed while doing hypothesis evaluation primarily called Type I or alpha error and Type II or beta error.

Alpha error is committed when we fail to accept null hypothesis, while actually it is true.

Beta error is committed when we fail to accept alternate hypothesis, while actually it is true.

Hypothesis is a technique which can help us in arriving at a suitable mix of different experience level of resources to be deployed for the project with certain level of confidence of execution.

This technique offers us the opportunity to compare the productivity level of resources with different experience level and arrive at a conclusion if there is any significant difference in the productivity at different experience level. Knowing the productivity level of resources at various experience level, we can suitably deploy resources of different experience level in the project and thereby can reduce the cost in completing the project and improve project profitability.

There are other scenarios in IT industry where hypothesis test can be effectively and easily deployed to garner the success through this technique e.g. comparing the quality level of different modules, evaluating the quality of deliverable w.r.t. target etc.

One of the other opportunity I see in deploying hypothesis technique in IT industry is in the evaluation of various review techniques which organizations employs in review the work products e.g. Offline review, walkthrough review and inspections. Many a times these reviews are planned in the projects without considering their review yields and comparing the effectiveness data from various review techniques which unnecessarily results in spending more effort for certain types of work products where less stringent review could be have been deployed to capture the similar number of defects.

Thus hypothesis testing provides a very good technique in our quality arsenal which can be effectively and efficiently used in various decisions at different levels. Comments invited!

Sanjay Shah