Q. 165  In statistics, causal relationships need to be examined carefully before making any inferences. Simpson's paradox is one of the phenomenon that is observed if such causal relationships are ignored. What is Simpson's Paradox? Explain with the help of examples.

 

Note for website visitors - Two questions are asked every week on this platform. One on Tuesday and the other on Friday.

• All questions so far can be seen here - https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/
• Please visit the forum home page at https://www.benchmarksixsigma.com/forum/ to respond to the latest question open till the next Tuesday/ Friday evening as per Indian Standard Time.  
• The best answer is always shown at the top among responses and the author finds honorable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term.

 

 

A wiki defition of Simpson's paradox, also known as yule-effect states that, it is a phenomenon in statistics, in which a trend appears in several different groups of data but disappears or reverses when these groups are combined.

 

How this happens?
We always try to see relationship between the dependent (Y) and independent (X) variables. But there could be also a factor- called the confounding variable which can influence both your X and Y which we might not have considered while taking decisions. This plays a hand on the reversal of the trend!!  

 

Let us see some examples

 

Example:1 Imagine a hypothetical case. 

Elections	Ruling Party % won	Opposition Party % won
Candidate for Local Councillor  Election	50%	45%
Candidate for City Mayor Election	50%	40%
Candidate for State Assembly Elections	40%	60%

In a state(or province), two elections happens in some of the major cities - first councillor elections happen for the place/area, you live within a city, followed by mayor election for your city (due to unforeseen circumstances). Now sometime later, state assembly election comes. Even as in the first two elections, the members of the ruling party wins, the crucial assembly elections the ruling party loses!!  

 

While there is a correlation between a party and the candidate who represents the party , for each of the election, there  could be other confounding factors such as people's perception of the party, in matters close to their heart or liking or any other lurking factors whose effects would have been failed to be studied or noticed.

 

I also noticed some sites like https://www.britannica.com/topic/Simpsons-paradox, and also wikipedia for this topic , for further understanding and good examples.

 

Conclusion:
It is very important to identify confounding factors which can impact the causal relationships. Failure to do so, can create the Simpson's paradox. This could even confuse the decision makers to take a firm decision. The positive side of this is that it will highlight that there is something more to be explored to!! 

Simpson paradox is a situation in the world of statistics in which a particular pattern or trend is witnessed from the sets & subsets of data from two or more groups. But as soon as we combine all the sub groups or sub groups , the trend reverses dramatically. This is essential in order to have a logical and rational causal relationship the cause and effect. 

Simpson paradox can be understood well with few of following few examples:

a. In Pharmaceutical company the turnaround time of product A,B,C......n will give a particular type of trend once these are combined in a common group the trend gives a different picture altogether. This might happen because of various intrinsic features or parameters which are generic to the product range and type.

b. Another  example can be understood from the example of data to deliver the product from the warehouse of an e commerce company to the house of customers. 

 

Simpson paradox should actually be considered as a check point to validate whether our approach for drawing the causal relationship is rationale and logical or not. This will help to make sure we follow structured and analytical methodology towards identifying the root cause and make develop the pattern or trend more meaningful.

Many Thanks

The chosen best answer is that of R Rajesh for clear definition and complete example.  Read through Sreyash's answer to understand the paradox further.