Q. 206 Why is a test of equivalence (also called noninferiority testing) considered the opposite of Hypothesis testing? Give examples where noninferiority testing is a better approach as compared to usual hypothesis testing.

 

 

Note for website visitors - Two questions are asked every week on this platform. One on Tuesday and the other on Friday.

• All questions so far can be seen here - https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/
• Please visit the forum home page at https://www.benchmarksixsigma.com/forum/ to respond to the latest question open till the next Tuesday/ Friday evening 5 PM as per Indian Standard Time.  
• The best answer is always shown at the top among responses and the author finds honorable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term.

Null hypothesis assumes that the population mean is the same as a target value or another population mean. In equivalence testing, the null hypothesis assumes the population mean differs from a target value or other population mean.

 

For example, difference between a 2-sample t-test (Hypothesis) and a 2-sample equivalence test can be best illustrated as,

 2-sample t-test to test whether the means of two populations are different. The hypotheses for the test are as follows:

• Null hypothesis (H₀): The means of the two given populations are the same.
• Alternative hypothesis (H₁): The means of the two given populations are different.

If the p-value for the test is less than alpha (α), then the null hypothesis is rejected and concluded as the means are different.

In contrast, 2-sample equivalence test is used to test whether the means of two populations are equivalent. Equivalence for the test is defined by a range of values that you specify (also called the equivalence interval). The hypotheses for the test are as follows:

·         Null hypothesis (H₀): The difference between the means is outside equivalence interval. The means are not equivalent.

·         Alternative hypothesis (H₁): The difference between the means is inside the equivalence interval. The means are equivalent.

If the p-value for the test is less than α, then you reject the null hypothesis and conclude that the means are equivalent.

 

Small differences between products are not always functionally or practically important. For example, a difference of 1 mg in a 200 mg dose of a drug is unlikely to have any practical effect. When an equivalence test is done we must enter equivalence limits that indicate how large the difference must be to be considered important. Smaller differences, which are within the equivalence limits, are considered unimportant. In this way, an equivalence test evaluates both the practical significance and statistical significance of a difference from the population mean.

 

To choose between an equivalence test and a standard t-test, consider what needs to be proven or demonstrated. 

 

The objective of hypothesis test is to conclude the samples are different but when we want to prove that the samples are equivalent we use equivalence test.

 

Equivalence testing is a better approach as compared to usual hypothesis testing when

 

New food item meant to be a substitute 

New generic drug compared to old standard (bioequivalence)  

 

This process makes more sense logically because more samples gives us more power for detecting ‘equivalence’.

An alternative to the two-sample t-test is TOST, designed specifically for bioequivalence testing of pharmaceutical products. It has recently been expanded into broader applications in pharmaceutical science, process engineering, psychology , medicine , chemistry and environmental science.

 

An equivalence test forces us to identify from a practical perspective how big of a difference is important and puts the burden on the data to reach a conclusion of equivalence.

Benchmark Six Sigma Expert View by Venugopal R

 

One of the important prerequisites to answer this question is to have a good grasp about the concepts, methods and interpretation of tests of hypotheses. My discussion below is based on the assumption that the readers are reasonably conversant with TOH.

 

Hypothesis testing is a popular and well known method among Lean Six Sigma practitioners. The fundamental rule that is applied for hypothesis testing is that a null hypothesis, or a 'hypothesis of no difference’ is compared against an alternate hypothesis to examine whether sufficient evidence exists to reject the null hypothesis. For instance, if we want to compare whether the mean value between two sets of data, for example, the weights of sachets of medicinal solution packed by two different methods. In this case:

H_O: Mean weights of sachets by method 1 = Mean weights of sachets by method 2

H_A: Mean weights of sachets by method 1 ≠ Mean weights of sachets by method 2

 

If we run a two sample test, depending on the p value, we would either reject H_O or ‘fail’ to reject H_O.

While inferring the TOH results, the important point to note is that even when we fail to reject H_O, it does not necessarily mean that H_O is true. It is just that we do not have sufficient evidence to prove that H_O can be rejected and does not conclude equivalence.

 

Equivalence tests allow us to conclude equivalence within a confidence interval. For an example like the one above, for equivalence tests, we need to specify the extent of difference between the group averages that is considered important. Ideal difference is zero; so we have to specify the interval that spans on either side of zero. Then, the differences that fall within the range are considered insignificant and equivalence may be concluded. The largest difference that is permissible by the specified interval is known as ‘Equivalence Interval’. The Hypothesis statements for the equivalence test will be:

H_O: The difference of mean between groups is outside the equivalence interval

H_A: The difference of mean between groups is inside the equivalence interval

 

Please see typical graphical illustrations that are obtained while performing equivalence test using Minitab – for situations where equivalence can be claimed and cannot be claimed 

 

 

Equivalence tests are known as “Opposite of hypothesis testing” because of the fact that conventional hypothesis tests look for evidence to prove differences or ‘non-equivalence’, where as Equivalence tests look for evidence to prove ‘Equivalence’.

 

When our aim is to look for equivalence between groups or with respect to a standard, the ‘Equivalence tests are more advantageous.

 

 

 

 

 

 

 

 

 

 

 

Looking at the above differences, it becomes clear as to why Test of Equivalence is considered as opposite of Hypothesis testing.

 

Having laid down the differences, there are some similarities as well

 

1. Both work with samples and apply the concepts of Inferential Statistics (Significance Level, Confidence Intervals etc.)

2. Researcher is interested in Alternate Hypothesis in both (even though the alternate hypothesis are opposite in the two)

 

Choice between hypothesis testing and equivalence will depend on the purpose of the study. Equivalence tests are most commonly used in pharma industry to check if a generic drug (lower cost option) has the same efficacy as the patented drug. 

 

To summarize, equivalence tests could be used wherever we want to use substitutes to an original item without significantly impacting the final outcome. Some e.g. that I could think off

1. Construction - Substituting building materials without impacting the compressive strength

2. Chemical / oil / pharma - Substituting chemicals without impacting the reaction time

3. Medical devices - substituting the type of laser without impacting the burning efficiency and precision

4. Tyre industry - substituting the rubber components without affecting the grip or the life of the tyre

Both Natwar Lal and T Jayaram are winners for responses to this question. Natwar Lal has done a good comparison and has provided examples where equivalence testing is preferred. Jayarm has highlighted Two One Sided T tests (TOST) as another term used for equivalence testing. 

 

Do read the response from Benchmark Expert Venugopal for additional learning.