Q 502. How does different Normality Tests - Anderson Darling, Shapiro-Wilk, Kolmogorov-Smirnov, Chi Square - compare with each other. Provide examples to illustrate the usage of all four. 

 

Note for website visitors - Two questions are asked every week on this platform. One on Tuesday and the other on Friday.

• All questions so far can be seen here - https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/
• Please visit the forum home page at https://www.benchmarksixsigma.com/forum/ to respond to the latest question open till the next Tuesday/ Friday evening 5 PM as per Indian Standard Time. Questions launched on Tuesdays are open till Friday and questions launched on Friday are open till Tuesday. 
• When you respond to this question, your answer will not be visible till it is reviewed. Only non-plagiarised (plagiarism below 5-10%) responses will be approved. If you have doubts about plagiarism, please check your answer with a plagiarism checker tool like https://smallseotools.com/plagiarism-checker/ before submitting. 
• The best answer is always shown at the top among responses and the author finds honorable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term

One of the most common requirements for statistical test procedures is that the data used must be normally distributed thus it is very important to understand whether the data belongs to a normal distribution or not. In order to achieve this, there are various normality tests available for testing the normality of the data, the common ones are:-

 

• Anderson Darling Test
• Kolmogorov Smirnov Test
• Chi Square Goodness of Fit Test
• Shapiro Wilk Test

 

However there are graphical methods also available such as normal probability plot or Q-Q plot for assessing normality of a data set, for this discussion we will be focussing our attention on the tests mentioned above. Let us understand each of these along with examples:-

 

Anderson Darling Test :

 

The Anderson-Darling Goodness of Fit test basically compares the empirical cumulative distribution function or ECDF (an estimator of the cumulative distribution function which allows to plot a variable in order from least to greatest & see how the variable is distributed across the data set) of the sample data & compares it with the expected distribution if the data was normal. If the observed difference is sufficiently large, the assumption of normality is rejected. Below is the null & alternative hypothesis :

 

Ho : The data follows a normal distribution

Ha : The data does not follow a normal distribution

 

The Anderson-darling Test Statistic is defined as :

 

A^2 = -N-S

 

where S = Σ[i = 1 to N] (2i-1)/N (ln F(Yi) + ln(1-F(Y(N+1-i)) & F is the CDF of the specified distribution & Yi is the orderd data.

 

Also ECDF is given by : E(N) = n(i)/N where we have N order data points Y1, Y2, …, YN.

 

 

This test is a one-sided test with the hypothesis that the distribution is normal is rejected if the test statistic A-Sq is greater than the critical value.

 

Amongst all the above test, the Anderson Darling Test is the most effective when non normality is due to variation in the tails of the distribution & also the tails are the most critical part of the distribution when it comes to checking its normality.

 

Let us say we generate 1000 random numbers each for four different distributions Y1, Y2, Y3, Y4 & apply the test to ascertain whether the data follows a normal distribution. Below are the results:-

 

      H0:  the data are normally distributed
      Ha:  the data are not normally distributed

      Y1 adjusted test statistic:  A2 =   0.2576
      Y2 adjusted test statistic:  A2 =   5.8492
      Y3 adjusted test statistic:  A2 = 288.7863
      Y4 adjusted test statistic:  A2 =  83.3935

      Significance level:  α = 0.05
      Critical value:  0.752   
      Critical region:  Reject H0 if A2 > 0.752

 

Looking at the above results the data sets Y1, Y2 & Y4 follows normal distribution as we fail to reject the null hypothesis in these cases.

 

Kolmogorov Smirnov Test :

 

Similar to Anderson Darling Test, the Kolmogorov Smirnov Goodness of Fit Test is also used to decide whether a sample comes from a normal distribution. Here also we compare the ECDF of the sample data with the expected normal distribution if the data were normal & we reject the assumption of normality if there is a significant difference observed. This test has the same set of null & alternate hypothesis as mentioned above for the Anderson Darling Test.

 

However there is a marked difference in terms of its sensitivity as it tends to be more sensitive near the center of the distribution compared to Anderson Darling Test which tends to be more sensitive at the tails of a distribution. Also this test does not depend on the sample size in order for the assumptions to be valid thus making it an exact test. Also the critical value of the test statistic does not depends on the underlying distribution being tested as opposed to the Anderson Darling Test.

 

The Kolmogorov Smirnov Test Statistic is defined as :

 

D = max[1 <= i <= n](F(Yi) - ((i-1)/N), i/N - F(Yi)) here F is the theoretical CDF of the distribution which is being tested. 

 

Also the parameters for distribution i.e. shape, location & scale must be fully specified & cannot be estimated from the data.

 

Let us again generate 1000 random numbers each for four different distributions Y1, Y2, Y3, Y4 & apply the test to ascertain whether the data follows a normal distribution. Below are the results:-

 

      H0:  the data are normally distributed
      Ha:  the data are not normally distributed

      Y1 test statistic:  D = 0.0241492  
      Y2 test statistic:  D = 0.0514086 
      Y3 test statistic:  D = 0.0611935
      Y4 test statistic:  D = 0.5354889

      Significance level:  α = 0.05
      Critical value:  0.04301    

             Critical region:  Reject H0 if D > 0.04301 

 

So the null hypothesis got is not rejected for the first data set Y1 & got rejected for the remaining three which is Y2, Y3 & Y4. Thus the data set Y1 follows a normal distribution.

 

Chi-Square Test :

 

The Chi-Square Goodness of Fit test is used to test if a sample of data come from a population which follows a specific distribution , in our case it is the normal distribution. It is applied to binned data where the data is put into classes & we calculate the Chi-Square Test Statistic & compare that to the critical value in order to reject or accept our hypothesis which is whether the data follows the normal distribution or not.

 

The Chi-Square Test Statistic is defined as :

 

Chi-Sq = Σ[i= 1 to k] (Oi - Ei)^2 / Ei where data is divided into k bins & Oi is the observed frequency of the bin & Ei is the expected frequency of the bin.

 

Here Ei = N(F(Yu) - F(Yl)) where F is the CDF of the distribution being tested, Yu is the upper class limit & Yl is the lower class limit & N is the sample size.

 

The test statistic follows approximately a chi-square distribution with (k-c) degrees of freedom.

 

Let us again generate 1000 random numbers each for four different distributions Y1, Y2, Y3, Y4 & apply the test to ascertain whether the data follows a normal distribution, here we applied the chi-square test with 32 bins, also c = 2+1 = 3, as we have two parameters in a normal distribution i.e. mean & standard deviation.

 

H0:  the data are normally distributed
Ha:  the data are not normally distributed  

Y1 Test statistic:  Χ 2 =   32.256
Y2 Test statistic:  Χ 2 =   91.776
Y3 Test statistic:  Χ 2 =  101.488
Y4 Test statistic:  Χ 2 = 1085.104

Significance level:  α = 0.05
Degrees of freedom:  k - c = 32 - 3 = 29
Critical value:  Χ 21-α, k-c = 42.557
Critical region: Reject H0 if Χ 2 > 42.557

From the above results, we can infer that for data set Y1, we fail to reject the null hypothesis, thus we can conclude that the data set is normally distributed.

 

Shapiro Wilk Test :

 

The Shapiro Wilk Normality Test basically quantifies the similarity b/w the observed & normal distributions by superimposing a normal curve over the observed distribution. It then computes which percentage of our sample overlap with the normal curve. Thus this test computes the probability of finding this similarity percentage. It calculates the W statistic that tests whether a random sample comes from a normal population. Small values of W showcases the evidence of deviation from normality. This test statistic is calculated as shown below :

 

W=(∑ni=1aix(i))2∑ni=1(xi−x¯)2

 

where xi are the ordered sample values & ai are constants generated from the means, variances & covariances from these ordered sample of size n from a normal distribution.

 

 

Let us take an example where we collected the reaction times of a sample of people who have appeared for a typing test. Let us analyze the descriptive statistics computed for these samples:-

 

Reaction Time : N = 233, Mean = 969.97, Median = 932.00, Std Dev = 275.32, Skewness = 0.341, Kurtosis = -0.394

 

Here we can observe from the results that the skewness & kurtosis are closer to zero, thus resembling data closer to a normal distribution, next the actual test is performed & below are the test results:-

 

Reaction Time : Statistic = 0.984, df = 233, Sig(p) = 0.075

 

Since p > 0.05, we can conclude that the data follows normal distribution.

Normality test is a statistical hypothesis test used to determine if the sample has been drawn from a normally distributed population. Graphically, normal distribution resembles a bell with a symmetric distribution around the mean value. It is unimodal with mean, median & mode being equal. Commonly used tests for normality are  Anderson Darling, Shapiro-Wilk, Kolmogorov-Smirnov & Chi Square. 

 

Shapiro-Wilk test is sensitive to sample size. It is generally used for sample sizes less than 50. For larger sample sizes, the result is always statistically significant. 

 

Kolmogorov-Smirnov is generally used for sample sizes greater than 50. It doesn't requires to know the underlying distribution of the population before running the test. It requires us to enter location, scale and shape parameters for running the test. It can't be used for discrete distributions.

 

Anderson Darling is a modification to Kolmogorov-Smirnov test with more weight to the tails than Kolmogorov-Smirnov test. It doesn't has a good power as that of Shapiro-Wilk test but offers better power than other tests.

 

Chi Square normality used when the sample has discrete set of data points. It is used when the expected value of number of sample of observations is greater than 5. The data must be randomly extracted and the variable of study is categorical.

 

 

What is Normality Test -

Normality tests determine whether or not a collection of data is distributed in a manner that is consistent with a normal distribution. They are often tests of a null hypothesis, that the data are chosen from a normal population. As a result, while it is possible to conclude definitively that a collection of data is not normally distributed (by rejecting the null hypothesis), the most that can be said if the null hypothesis is not rejected is that the data may conceivably originate from a normally distributed population.

 

The primary tests for determining normality are Anderson Darling, Shapiro-Wilk, Kolmogorov-Smirnov and Chi Square. 

 

Here let's look into these tests in detail.

 

Anderson Darling Test -

 

The Anderson-Darling normality test is one of the universal normality tests that are meant to identify all deviations from normality. While it is commonly referred to be the most powerful test, no one test is superior to all others, and the other tests also have equivalent potency. 

 

When the p-value is less than or equal to 0.05, the test rejects the normality hypothesis. If the normality test fails, you may say with 95% certainty that the data does not fit the normal distribution. Passing the normality test merely permits you to say that there was no substantial deviation from normality.

 

The AD test statistic is as follows:

While the Anderson-Darling test has good theoretical features, it has a significant weakness when applied to real-world data. Ties in the data caused by low accuracy have a significant impact on the Anderson-Darling test. When there are a large number of ties, the Anderson-Darling will commonly reject the data as non-normal, regardless of how closely it fits the normal distribution. The data below was created using the normal distribution but was rounded to the closest 0.5 to create ties. A tie arises when the same value appears more than once in the data set:

 

 

 

Shapiro-Wilks Normality Test -

 

The Shapiro-Wilks normality test is one of the generic normality tests that are meant to identify all deviations from normality. Its power is equivalent to the other tests.

When the p-value is less than or equal to 0.05, the test rejects the normality hypothesis. If the normality test fails, you may say with 95% certainty that the data does not fit the normal distribution. Passing the normality test merely permits you to say that there was no substantial deviation from normality.

The Shapiro-Wilks test is not as affected by ties as the Anderson-Darling test, but is still affected.

 

Kolmogorov-Smirnov Test -

The Kolmogorov-Smirnov Test (K-S test) compares your data to a known distribution and indicates whether or not they have the same distribution. Although the test is nonparametric (it makes no assumptions about the underlying distribution), it is widely used as a normality test to determine whether your data is normally distributed. It is also used in Analysis of Variance to test the assumption of normality.

More specifically, the test compares a known hypothetical probability distribution (e.g., the normal distribution) against the empirical distribution function created by your data.

 

 

Chi Square Test -

The Chi-Square Test for Normality determines if a model or hypothesis has an approximately normal distribution. The Chi-Square Test for Normality is not as effective as other more precise tests. Still, it's a quick and easy technique to verify for normalcy, especially when you have a distinct set of data points.

To apply the Chi-Square Test for Normality to any data set, assume that your data is sampled from a normal distribution and use the Chi-Square Test. You must compute the anticipated values under the normal distribution for each data point given your mean and standard deviation.

 

 

The formula is then used to calculate the chi-square statistic. When you have your degrees of freedom and desired alpha level, compare this to the crucial chi-square value from a chi-square table. If your chi-square statistic exceeds the table value, you may assume that your data is abnormal.

All the published answers are brilliant. While Kiran's answer is short and to the point and clearly tells us where to use which test, Soji's answer highlights a weakness in the most popular Normality Test. Both answers have been selected as the winners. Congratulations to both!