September 13, 20232 yr Recently came across a case when testing the effectiveness of an improvement. Each transaction in the "before" Data Set was matched with a transaction in the "After" data set, and a Paired t Test showed that the null hypothesis could be rejected. Out of curiosity, when the two sample t test was run on the same data, it surprisingly showed that the null hypothesis could not be rejected. The differences in the statistical concepts behind the two types of t tests apart, what would be the decision on the floor?
October 14, 20232 yr The paired t-test uses the statistical concept of "blocking random contributions". Its aim is to decrease the variability of the dataset, but this decrease comes with the cost of loosing degrees of freedoms. Here the example which I commonly use: Suppose we measure the height of people a) without their shoes, and b) with their shoes. Our hypothesis would surely be that the height of people is larger if they wear their shoes, but because the variation between the peoples height is much larger than the effect size we try to measure, we will have difficulties to obtain a clear result. (I tried to include a graph, but I'm not sure if it worked) The solution is to block the variation between the subjects, and only to consider the variation within the subjects. This is what the paired t-test does. However, the blocking costs half the degrees of freedoms. Thus, the blocking is a trad-off and only works in our favour, if we the blocked variation is large compared to the decrease of resolution. There are formulas describing this ration.
October 18, 20232 yr Solution Consider the practical significance of rejecting the Null Hypothesis. If rejecting Ho results in practical gains in the process, then by all means reject it. Paired T test is statistically more powerful as compared to 2 sample T test, provided the external conditions are same. If the processing was done by same agent and on the same transaction (in context of service industry), then we can do the Paired T test. If the agents or transactions were different then 2 sample T test is a better option.
Create an account or sign in to comment