Statist ical Aids to Hypothes is Testing

Hypothesis testing is the basis for many decisions made in science and engineering. To explain an observation, a hypothetical model is advanced and tested experimentally to determine its validity. The hypothesis tests that we describe are used to determine if the results from these experiments support the model. If they do not support our model, we reject the hypothesis and seek a new one. If agreement is found, the hypothetical model serves as the basis for further experiments. When the hypothesis is supported by sufficient experimental data, it becomes recognized as a useful theory until such time as data are obtained that refute it.
Experimental results seldom agree exactly with those predicted from a theoretical model. As a result, scientists and engineers frequently must judge whether a numerical difference is a result of a real difference (a systematic error) or a consequence of the random errors inevitable in all measurements. Statistical tests are useful in sharpening these judgments. Tests of this kind use a null hypothesis, which assumes that the numerical quantities being compared are, in fact, the same. We then use a probability distribution to calculate the probability that the observed differences are a result of random error. Usually, if the observed difference is greater than or equal to the difference that would occur 5 times in 100 by random chance (a significance level of 0.05), the null hypothesis is considered questionable, and the difference is judged to be significant. Other significance levels, such as 0.01 (1%) or 0.001 (0.1%), may also be adopted, depending on the certainty desired in the judgment. When expressed as a fraction, the significance level is often given the symbol a. The confidence level, CL, as a percentage is related to α by CL= (1 - α) × 100%. Specific examples of hypothesis tests that scientists often use include the comparison of (1) the mean of an experimental data set with what is believed to be the true value, (2) the mean to a predicted or cutoff (threshold) value, and (3) the means or the standard deviations from two or more sets of data. The sections that follow consider some of the methods for making these comparisons. Section 7C treats comparisons among more than two means (analysis of variance).