Q 355. Problem solving methods focus more on statistical significance, whereas a good solution should also be practically significant. What is practical significance? Support your answer with suitable examples.

 

Note for website visitors - Two questions are asked every week on this platform. One on Tuesday and the other on Friday.

• All questions so far can be seen here - https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/
• Please visit the forum home page at https://www.benchmarksixsigma.com/forum/ to respond to the latest question open till the next Tuesday/ Friday evening 5 PM as per Indian Standard Time. Questions launched on Tuesdays are open till Friday and questions launched on Friday are open till Tuesday. 
• The best answer is always shown at the top among responses and the author finds honorable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term

In the world of statistics, we use hypothesis testing (based on sample) to check 'Statistical Significance' and draw conclusion about the population or universe. As we know that these hypothesis testing is subjected to some sort of inevitable errors (type-1 error & type-2 error), hence influence our decision making process. If our decision is solely based on the result of hypothesis testing, it might lead us to wrong path/decision when applied to real world situations due to numerous factors which might not been considered while forming hypothesis. On the contrary, Real world situation/scenarios are different from those conducted on pilot/trial basis. Therefore, we need to check the 'Practical Significance' of the solution drawn from statistical test by leveraging SMEs and other expertise. In practical significance, we have to take in to account both type-1 and type-2 error to check whether percieved effect (derived from hypothesis testing) is really significant or not. Therefore, we need to use SMEs' knowledge in-conjuction with statistical result, derived from hypothesis testing, for faster and accurate decision making while deploying solution to real world scenario. 

 

Example:

For a typical steel plant, the main raw materials are iron ore, coal and fluxes. During procurement of iron ores from mines, the R&D team of steel plant collects samples from various mines and conduct trials. If trials are successful, they approve the procurement.

 

In india, there are three major belts of iron ore: orissa Jharkhand belt, Durga Bastar Chandrapur Belt and Bellary Chitradurga Belt. Let us say, a steel plant is located in the Karnataka region of the country and researcher wants to conduct the study with respect to the Quality of the iron ore. The Quality of the iron ore is determined by the %Fe present in the iron ore. The %Fe varies from 48 to 63% depending upon the geography of the region. The researcher collects samples from these three belts and formulate following hypothesis testing.

 

Ho: The mean value of %Fe is equal.

Ha: At least one value is different.

After trials, the reasercher find p-value>0.05, thus accepts null hypothesis and conclude that the mean value of %Fe is same in all three belts. And he suggests to purchase iron ore from Karnataka Belt. This decision is purely based on statistical significance.

 

Before making procurement decision, the company consults with geologist and finds that Bellary-Chitradurag Belt has low grade iron ore in comparison to Durga Bastar Chandrapur and Jharkhand Belt. Therefore, the company assimilates both statistical significance and SMEs to arrive at better decision. The company decide to purchase the raw materials from all regions based on the result of SMEs and Statistical Significance. This is known as practical significance which take in to account both type-1 and type-2 error for selecting best decision.

 

Finally, we can conlude that statistical significance should not be treated as one factor of a decision. There could be multiple factors related practical significance as we have seen in the above example.

Practical significance:

 

Practical significance relates to whether the result from a statistical hypothesis test is useful or not.

Hypothesis testing tests for statistical significance. It means the effect observed in the sample was unlikely to occurred due to chance alone. In other words, it would be very unlikely to see what was observed in the sample if the null hypothesis is true.

 

Example:

If a Call Centre claims their average wait time is 30 seconds. We decide to test:

                Ho: mean=30

                Ha: mean>30

We find X (sample mean) = 30.6, and a p-value of 0.002

 

We decided to test that the population mean is 30 and the alternate hypothesis is greater than 30. We find a sample mean of 30.6 sec and get the resulting p-value is 0.002. p-value size shows very strong evidence against the null hypothesis, in this case population mean is greater than 30sec, but in the sample we observed in the sample mean is 30.6, a difference of only .6 sec from the null hypothesis. In this situation most people would feel the difference of 0.6 sec doesn’t really have any practical meaning. But perhaps the company feel differently, the company want to make sure the wait time is no more than 30sec on average and so they feel 0.6 difference is important, we simply don’t know. We can’t say in statistics that this p-value is very small, giving strong evidence against the null hypothesis. We leave up to others to decide that is practical importance. How could be such a strong evidence against the null hypothesis if the difference from the hypothesized is so small.

 

Statistical significance is strongly related to sample size. If the sample size is large enough, even tiny differences from the hypothesized value will be found statistically significant. If the sample size is small, then it can be difficult to find the statistically significant difference.

 

In statistics we determine if there is Statistical significance and let experts in the field of interest determine whether the results have any practical importance. In addition to the results of hypothesis test, it is best to also report an appropriate confidence interval.

 

The interval illustrates the size of the effect and can help to determine if the effect has any practical significance.

Both the published answers have given good examples to highlight the difference between statistical and practical significance. Hence, both have been selected as the winners.