Hi Friends,

I would appreciate some inputs on : Example : I am trying to test if two variants of fertilizers A & B have similar impact on the length of Sunflower stems applied in 9 plots(5 plots applied with Fertilizer A, 4 plots applied with Fertilizer . The individual stems lengths and averages are calculated.

The commonly accepted Significance Level 0.05 is selected.

The p value obtained is 0.03. If P value is low. The null must go. Hence, we say there is significant variance between the mean length of sunflower stems.

However, what if i want to run the test at 0.01 Significance level. At 0.01 level, we have to accept Null Hypothesis. p value is greater than 0.01. To justify the Significance level of 0.01, should i not increase my sample size?

From past understanding, i remember that for lower levels of Significance like 0.01, the sample size needs to be increased i.e. being very very critical. Eg : In Space Shuttle, the cost of failure is very very high.

This is part 1. The next doubt i have is, for a Space Shuttle whose number of trips itself are very very few from statistical point of view, do they fulfill the 0.01 or lesser(0.001 too) significance level requirements through simulations?, as they can't test it in real manner. This is part 2.

I hope i was clear with my caselet. I appreciate your inputs

 

Hi Kiran,

1. Would like to understand what is the objective of doing above hypothesis? why the length of stem is critical? is it the sales/ reveneue that would be generated or which plot is more fertile or which fertilizer has a better yield?

2. well, if your hypothesis is to check the impact of fertilizer type on stem length then the conclusion (seeing p value =0.03 with significance level .05) should be " Fertilizer type has an impact on stem length" and not "the mean length of stems has a variation".

3. Yes, in order to have a narrower Confidence Interval, the sample size needs to be increased but then we really need to know the criticality of judging a process capability based on sampling.

a. if decisioninig needs to be precise and can't afford error then we definitely need statistically calculated sample. Moreover in space shuttle or nuclear reactor kind of cases the overall poulation itself is finite and there sample would be very close to population (ofcourse in simulation world)

b. But if risk of error can be taken then a more practical way of sampling can be taken which makes a business sense

views are welcome .....

Have a great day!

My views are as follows.

• I presume the discussion is about 2 sample t test where we are trying to test the significance of difference in mean length of stem due to change in fertilizer.
  
• Alpha value (significance level) is something we fix before testing a hypothesis. So, I do not think we should be interested in checking the analysis results with different alpha values after the test. However, the question is valid for academic interest.
  
• Continuing with the point 2 above, we would have fixed an alpha value of 5% if we are willing to accept a 5% probability of rejecting a true null hypothesis (this means maximum 5% chances of concluding that the height of stems are different when they actually are same). If we wish to accept a lower risk in such a conclusion, we may have chosen a smaller alpha of 1%. How much risk we are willing to accept is based on our own assessment of seriousness of impact of error.
  
• Sample Size calculation should be done with predetermined alpha value.
  

If I use Minitab for sample size calculation for the same example, (by clicking on options I can change the alpha value. Alpha 0.05 gives me a sample size of 23. Alpha value of 0.01 gives me a sample size of 32.

I have assumed a standard deviation of 2 and a power value of 0.9.

• I agree with Kiran - We like to run sufficient simulations to cover sample size in situations where real data cannot be generated. As we gather more knowledge (with Chandrayaan - I for example), our simulations become more realistic with better assumptions and facts.
  

Regards,

VK