In the world of statistics, we use hypothesis testing (based on sample) to check 'Statistical Significance' and draw conclusion about the population or universe. As we know that these hypothesis testing is subjected to some sort of inevitable errors (type-1 error & type-2 error), hence influence our decision making process. If our decision is solely based on the result of hypothesis testing, it might lead us to wrong path/decision when applied to real world situations due to numerous factors which might not been considered while forming hypothesis. On the contrary, Real world situation/scenarios are different from those conducted on pilot/trial basis. Therefore, we need to check the 'Practical Significance' of the solution drawn from statistical test by leveraging SMEs and other expertise. In practical significance, we have to take in to account both type-1 and type-2 error to check whether percieved effect (derived from hypothesis testing) is really significant or not. Therefore, we need to use SMEs' knowledge in-conjuction with statistical result, derived from hypothesis testing, for faster and accurate decision making while deploying solution to real world scenario.
Example:
For a typical steel plant, the main raw materials are iron ore, coal and fluxes. During procurement of iron ores from mines, the R&D team of steel plant collects samples from various mines and conduct trials. If trials are successful, they approve the procurement.
In india, there are three major belts of iron ore: orissa Jharkhand belt, Durga Bastar Chandrapur Belt and Bellary Chitradurga Belt. Let us say, a steel plant is located in the Karnataka region of the country and researcher wants to conduct the study with respect to the Quality of the iron ore. The Quality of the iron ore is determined by the %Fe present in the iron ore. The %Fe varies from 48 to 63% depending upon the geography of the region. The researcher collects samples from these three belts and formulate following hypothesis testing.
Ho: The mean value of %Fe is equal.
Ha: At least one value is different.
After trials, the reasercher find p-value>0.05, thus accepts null hypothesis and conclude that the mean value of %Fe is same in all three belts. And he suggests to purchase iron ore from Karnataka Belt. This decision is purely based on statistical significance.
Before making procurement decision, the company consults with geologist and finds that Bellary-Chitradurag Belt has low grade iron ore in comparison to Durga Bastar Chandrapur and Jharkhand Belt. Therefore, the company assimilates both statistical significance and SMEs to arrive at better decision. The company decide to purchase the raw materials from all regions based on the result of SMEs and Statistical Significance. This is known as practical significance which take in to account both type-1 and type-2 error for selecting best decision.
Finally, we can conlude that statistical significance should not be treated as one factor of a decision. There could be multiple factors related practical significance as we have seen in the above example.