Benchmark Six Sigma Expert View by Venugopal R
When we have to compare the averages for two samples, it could be for different reasons:
1. To estimate whether two existing populations are different with respect to their average values of the characteristics of interest.
Examples:
To compare the average life span of bulbs produced by two different companies
Average marks scored by male students vs that of female students
2. To estimate whether the effect of some change on a given population is significant or not.
Examples:
Performance of a group before training and after training
Average mileage of cars for one type of fuel vs another
From the above, we can see that for point-1, the two samples being compared can never be the same, since the reason for comparison is a difference based on the very nature of the sample itself. In such situations, we have to use 2-sample 't' test, and no ‘pairing' is possible.
For the point-2, we have a possibility of subjecting the same set of samples to the first treatment and then to the second treatment and compare the difference in performance for each same sample. In such situations, Paired ‘t’ test is the ideal comparative statistical tool to be used.
We may also come across some situations, where the paired sampling would not be practically possible. For example, let’s take the case of evaluating the average life of bulbs from the same company before and after doing a process improvement. Since the life testing of bulbs is a destructive test, the same samples will not be available for doing a paired 't' test and hence we have to use a different set of samples, and hence, only 2-sample 't' test.
Another example would be to compare the effect of two vaccines on a set of people. Once they are subject to vaccine-1, they would have developed immunity and we cannot subject the same set of people to vaccine-2, ruling out the possibility of a paired 't' test.
A paired ‘t’ test is recommended over 2-sample ‘t’ test whenever the situation permits, considering the advantages. Let me statistically illustrate certain advantages of paired test using the below example.
As part of a medical research study, the heart rates of 20 athletes were studied before and after subjecting them to a running program. Since heart rates of the same athletes were studied before and after the treatment, a paired test is possible. We will however, carryout the paired test and the unpaired 2-sample 't' test for the same sets of data and compare the results. The mean heart rate before the treatment was 74.5 and after treatment was 72.3.
The Minitab outputs for both the tests are given below:
From the above results, it can be seen from the p values that for the same set of data, the paired t test has shown significance, where as the 2-sample t test has not shown significance. Thus, the 2-sample t test for the same data exhibits higher ‘Type 2’ error.
Now, let us fix the required power of the test as 0.8 and determine the sample size requirements for both these tests, all data remaining same:
The above information are the outputs based on ‘Power & Sample size’. For both the type of tests, the sample size was determined based on a difference of 2, target power of 0.8 and standard deviation of 4.29. The paired test requires a sample of 39 whereas the 2-sample test requires a sample of 74.
Hence, the paired t test is preferable, whenever practically possible, from the sampling size requirement as well.