Leaderboard

Effect size indicates the practical significance of a research outcome. it tells you how meaningful the relationship between two variables or the difference between groups is. A large effect size means that a research finding has practical significance. While statistical significance shows that an effect exists in a study, practical significance shows the effect is large enough to be meaningful in the real world. Statistical significance is denoted by p-value, whereas the practical significance is represented by effect size. Statistical significance alone can be misleading as it is influenced by sample size i.e. increasing the sample size will always make it more likely to find a statistical significant effect, no matter how small the effect truly is in the real world. In contrast to this, effect size is independent of the sample size which makes it relevant to showcase in order to represent the practical significance of a finding. Let us understand the difference in statistical & practical significance through an example:- In a study, we are comparing two weight loss methods with 13000 subjects each in two groups. One group let’s say uses method I of weight loss & the other group uses method II of weight loss. Now basis the results, the mean weight loss in Kg for one group is 10.6 kg with standard deviation of 6.7 kg, which is marginally higher compared to the mean weight loss in Kg for the other group which is 10.5 kg with a standard deviation of 6.8 kg. Statistically these results are significant at p=0.01, however a difference of only 0.1 kg between the groups is negligible & doesn’t really tell you that which of the weight loss method is more effective. Here adding a measure of practical significance can showcase the differences in the two methods. There are various measures of effect size. Let us see some of the common ones:- Cohen’s d : Cohen’s d is designed for comparing two groups, it basically takes the difference between two means & expresses them in standard deviation units. It shows how many standard deviation lie between two means. Cohen’s d is calculated with the below formula:- d = (x̅1 - x̅2) / s where x1-bar is the mean of one group, x2-bar is the mean of the other group & s is the standard deviation. In general, greater the value of cohen’s d, the larger the effect size. Considering the above weight loss example, let us calculate cohen’s d for both the groups:- d = (10.6 - 10.5) / 6.8 = 0.015, now with this value of cohen’s d, there’s limited to no practical significance that one group findings are more effective than the other group’s findings. Pearson’s r : It is also known as the correlation coefficient & it measures the extent of a linear relationship between two variables. The main premise is to compute how much of the variability of one variable is determined by the variability of the other variable. A value of pearson’s r closer to -1 or +1 indicates a larger effect size. Below is the representation of the magnitude of the effect size in terms of both Cohen’d d as well as Pearson’s r methods:- Effect Size : Small, Cohen’s d : 0.2, Pearson’s r : +/- 0.1 to 0.3 Effect Size : Medium, Cohen’s d : 0.5, Pearson’s r : +/- 0.3 to 0.5 Effect Size : Large, Cohen’s d : >=0.8, Pearson’s r : >= 0.5 or <= -0.5 It is always helpful to calculate effect size before commencing any study & post data collection completion. The reason behind this statement is that within an expected effect size, one can figure out the minimum sample size required in order to have enough statistical power to detect an effect of that magnitude. If we don’t ensure enough power in a study, we may not be able to detect a statistically significant result even though it has practical significance, thus it is helpful to perform a power analysis, so that one can use a set effect size & significance level to determine the required sample size.Once data is collected, one can calculate & report the actual effect size.