Case for Statistical Significance – an example
Let’s consider the following data that is the age of 10 employees.
42, 35, 24, 31, 33, 41, 33, 31, 33, 32.
Assume these 10 data points are a sample that represents a large population of say, over 5000 employees. Now, using this available information we are asked a question “whether the average age of the employees in this population can be considered equal to 30 years?”
The quickest thing that anyone would do is to compute the average of the samples, which comes to 33.5. Since this is 3.5 more than 30, can we say that the population average age will be more than 30? These are situations where there is bound to be judgmental subjectivity and likelihood of reaching incorrect conclusions.
This is a simple example of a situation where a test of hypothesis may be done and the concept of statistical significance helps to reach an objective conclusion.
Statistical Significance – what does it imply?
Statistical significance implies that the difference that is under evaluation, (whether it is a population average being compared to a specified value, or the averages of two populations are being compared, or the variances of two populations are being compared, etc.) can be considered as a difference that is significantly larger than what a chance cause variation would have caused.
Since what we have is a sample data, it is to be noted that for different set of samples, the sample average is expected to vary with in certain limits for the same population (and same population average). The limits are governed by the variance of the population.
The test of significance will evaluate, with the given set of data, whether the sample average is falling within the confidence limits or not. So long as the sample mean falls within the confidence limits, the conclusions will be that there not sufficient reason to believe that the population average represented by this sample is different from the specified value.
Usage of Statistical Significance
In today’s world the application of tests of significance has been simplified using statistical software such as Minitab. Once we give the inputs depending upon the case being studied, the application comes out with a P value, which is used to determine the significance of the results. Smaller the p-value, the evidence against the null hypothesis becomes stronger. Usually a p-value < 0.05 is used as the criteria for rejecting the null hypothesis; i.e. the difference is considered significant.
As part to problem solving, tests of significance are integral part of Hypothesis testing, Analysis of Variance, Design of Experiments and other tools. It helps to take objective decisions with small samples. These methods are particularly useful during the Analyze phase where it helps to narrow down on short listed causes; and improve phase where the effectiveness of identified solutions could be validated.