Q 434. There are 2 different ways of doing a 2 proportion test - one based on normal approximation and the other based on Fisher's exact test. Why do we need these 2 different tests? Explain the difference between them using a sample data set.

 

Note for website visitors - Two questions are asked every week on this platform. One on Tuesday and the other on Friday.

• All questions so far can be seen here - https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/
• Please visit the forum home page at https://www.benchmarksixsigma.com/forum/ to respond to the latest question open till the next Tuesday/ Friday evening 5 PM as per Indian Standard Time. Questions launched on Tuesdays are open till Friday and questions launched on Friday are open till Tuesday. 
• When you respond to this question, your answer will not be visible till it is reviewed. Only non-plagiarised (plagiarism below 5-10%) responses will be approved. If you have doubts about plagiarism, please check your answer with a plagiarism checker tool like https://smallseotools.com/plagiarism-checker/ before submitting. 
• The best answer is always shown at the top among responses and the author finds honorable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term

5 hours ago, Vishwadeep Khatri said:

Q 434. There are 2 different ways of doing a 2 proportion test - one based on normal approximation and the other based on Fisher's exact test. Why do we need these 2 different tests? Explain the difference between them using a sample data set.

 

Note for website visitors - Two questions are asked every week on this platform. One on Tuesday and the other on Friday.

• All questions so far can be seen here - https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/
• Please visit the forum home page at https://www.benchmarksixsigma.com/forum/ to respond to the latest question open till the next Tuesday/ Friday evening 5 PM as per Indian Standard Time. Questions launched on Tuesdays are open till Friday and questions launched on Friday are open till Tuesday. 
• When you respond to this question, your answer will not be visible till it is reviewed. Only non-plagiarised (plagiarism below 5-10%) responses will be approved. If you have doubts about plagiarism, please check your answer with a plagiarism checker tool like https://smallseotools.com/plagiarism-checker/ before submitting. 
• The best answer is always shown at the top among responses and the author finds honorable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term

The 2 proportions test is a self explanatory and best practices to decide on the best of two samples , products etc. These can be used to determine the best of two which comes in very close to most of its results and details .  These can be actively seen in lots of cricket matches when the compare the players last match performance with the current and rate it accordingly. And we use it every day on the products we buy to get the best of two for a low price which provides all the benefits of the one with higher price.

A two-proportion test is a hypothesis test to check if the differences between two population proportions are statistically significant. For example, are the proportion of girls to boys in a school significantly different. The null hypothesis is that there is no difference between the population proportions. It can be tested against an alternate hypothesis that can be two-tailed or left/right-tailed.

The output of a 2 proportions test has two p-values, one is the normal approximation based on the Z statistic and the other is the Fisher’s Exact Test. As their names indicate, the normal approximation test is an approximation and has greater error for smaller sample sizes and becomes more accurate as the sample size increases whereas the Fisher’s Exact test is always exact irrespective of the sample size however is more difficult to calculate as the sample size increases.

The Fisher’s Exact Test is calculated using the hypergeometric distribution. The factorials in the formula make it more and more difficult to calculate the p-value as the sample size increases since it runs every possible combination from the sample, and calculates the total number of successes and failures at that given sample size. It then calculates the p-value from the total successes and failures. 

Thus, for larger samples, it is not only easier to calculate the p-value using the Normal Approximation Test, but the results are closer to Fisher’s Exact test results.

Since we no longer do manual calculations and statistical software have the ability to quickly calculate the p-value from the Fisher’s Exact Test, it makes more sense to use the Fisher’s Exact Test irrespective of the sample size.

For a small number of expected values, when compared to the Chi-Square or G-Test of independence, the Fisher’s exact test is more accurate.

The Normal Approximation Test (Z-test), is not accurate when the number of events/non-events are < 5. This is based on the rule that N*P or N(1-P) should be >5 (where N is the No. of trials and P is the proportion of successes. In other words, the normal distribution can be used in place of the binomial distribution when the sample size is large. If N is small and P is small, the binomial distribution will be skewed and the normal distribution cannot be taken to represent it. This is evident from Table 1 where N is increased and Table 2 where P is increased. It can be seen from these 2 tables, that as N and P are increased the Normal Approximation approaches the Fisher’s Exact Test.

It is also evident from Table 2 that for small samples/P, the Normal Approximation test may indicate that a difference between the population exist when no difference exists.

References

https://stats.stackexchange.com/questions/234010/2-sample-proportions-z-test-vs-fishers-exact-test

https://blog.minitab.com/en/quality-data-analysis-and-statistics/two-p-values-for-a-2-proportions-test-am-i-seeing-double

http://www.biostathandbook.com/fishers.html

Excellent explanation by Johanan and hence his answer is selected as the best answer.