Q. 95  What is “P-value” in hypothesis testing? Mention the most common misunderstandings with respect to P values?

 

 

Note for website visitors - Two questions are asked every week on this platform. One on Tuesday and the other on Friday.

• All questions so far can be seen here - https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/
• Please visit the forum home page at https://www.benchmarksixsigma.com/forum/ to respond to the latest question open till the next Tuesday/ Friday evening as per Indian Standard Time.  
• The best answer is always shown at the top among responses and the author finds honorable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term.

P-value is the probability of obtaining an effect at least as extreme as the one in  sample data, assuming the truth of the null hypothesis.

When a P value is less than or equal to the significance level, you reject the null hypothesis. 

 

If p > .10 → “not significant”

If p ≤ .10 → “marginally significant”

If p ≤ .05 → “significant”

If p ≤ .01 → “highly significant.”

 

On basis of this we can conclude that The P value, or calculated probability, is the probability of finding the observed, or more extreme, results when the null hypothesis (H₀) of a study  is true – the definition of ‘extreme’ depends on how the hypothesis is being tested. P is also described in terms of rejecting H0 when it is actually true, however, it is not a direct probability of this state.

Definition of P-value

It is the probability of getting the observed values if the null hypothesis is actually true. It can also be written in this way.  Let us see an example .

Suppose there is a magician who has a coin in her hand.Then let us say that

 

Null hypothesis (Ho) : a proper coin (which can show 'Heads' and 'Tails' regularly, when flipped)

Alternate Hypothesis (Ha): a magical coin which can show only 'Heads'.  

 

Now let us see the observed values each time when the coin is flipped. Let us denote Observed Value as OV , Probability as P. 

When the coin is tossed for multiple times, assume that below are the cumulative probability results :

a).OV1 = H; P1=1/2=0.50

b).OV2=H;P2=0.25

c).OV3=H;P3 = 0.12.

d).OV4=H;P4=0.06

e).OV5=H;P5=0.03 .....

 

As we traverse through this further, we find that this coin is not normal!!. We come up with a conclusion (with the above information as strong evidence) that this is a magical coin and therefore reject the null hypothesis.

 

Common Misunderstandings with respect to 'P' values.

1. 'P' value compared with 0.05.

Often P value is compared with a value of 0.05, which is a pre-defined value given for alpha- which determines the statistical significant value for the given

hypothesis.Always we put the equation if P<= 0.05, then we say the alternate hypothesis (Ha) is true. or P > 0.05 indicates that Null hypothesis is true(fail to reject null hypothesis). The point is 0.05 is not a pre-defined value for all cases. It can vary and it depends on how much significance level you want for that study

(hypothesis case) to be. Therefore to say, P > 0.05 will ensure the null hypothesis to be true is incorrect. It should be ideally said as P > the chosen 'alpha' value should result in the 'null hypothesis' being deemed true.

 

Eg: Let us see with an example  as how this alpha value can vary .

Null hypothesis(Ho): The person is innocent

Alternate Hypothesis(Ha): The person is guilt.

Imagine which would be the biggest crime ? Freeing a guilty person or jailing an innocent. You want to ensure to the maximum hilt, that an innocent person should not be jailed and that  percentage of making that mistake should be extremely low(which is what type 1 error is all about- rejecting null hypothesis when it is actually true). Therefore, in this study, we want to lower the alpha value so that there is a negligible chance of an innocent person getting jailed.  So in this case, ideally you want your alpha value to be something like 0.01 or less than that .

 

This is the reason why the jury (in a civil court) asks for strong evidences and the fact that it should convince the judges(because they got strong evidences(or circumstantial evidences) to provide the right verdict. 

 

Note: In Indian law, it is slightly different. A person can be charged on suspicion which is altogether a different issue and the person or his/her has to defend his/her innocence. But most of the countries follow a rule where your are innocent until proven guilty. The takeaway point here is that the p-value should not be statically compared with 0.05 (which is a pre-defined level of significance value used in general) . The significance value (alpha) can vary.  It depends on the consequences that type1 error and type2 error can produce. In general , if there is a greater impact/consequence on getting type1 error, then alpha value should be minimal (lower than may be 0.05 , in many such cases).

 

2.Showing implications of the 'P' value

It can indicate that there is a difference (when P is low and is < alpha) but it does not portray the practical implications or impact that it has.

 

3. Low P-value and rejecting of Null hypothesis.
When P-value is low and is less than the alpha value, say P < 0.05(assuming this alpha value), then it means a statistical evidence to reject the null hypothesis is shown. It however does not mean that this will prove the alternate hypothesis .

 

Conclusion:

It is easy to get confused or misunderstood that p-value is all about stating either null hypothesis is right or wrong.  It helps us in understanding the statistical inference that we get from a study. Also the alpha value and also sample size needs to be decided before the data collection is done. Alpha value is to be decided on the nature of the study and it depends on the consequences that the type 1 and type 2 errors can make.

R Rajesh has provided complete and correct answer for the question. Other answers were disqualified due to plagiarism. Congratulations Rajesh!