Setting Meaningful Limits

At the recent Bioelectromagnetics Society Annual Meeting an observation was made that great emphasis is placed on statistical testing of research results to determine if a measure in an exposed group of subjects is statistically different from the results in a similar control population. The test is usually made to determine whether there are significant differences in the results in the exposed population compared to the results found in the control population, that is, the null hypothesis.

Under the null hypothesis, there is no difference in results between an exposed population compared to a control population. This is considered true until evidence indicates that the null hypothesis is unlikely. The objective then is to establish the possibility that a Type I error occurs, that is, a false positive finding. This is equivalent to saying the false positive rate is equal to the significance level. In simple comparisons, significance levels between 0.01 and 0.05 are established demarcations for statistical significance, that is, the rejection of the null hypothesis when it is true.

There is another aspect of the comparison between exposed and control samples that is often overlooked. This second, complementary aspect of statistical testing is the concern about accepting the null hypothesis when in fact there is a difference between the two groups. This error is called a Type II error. The probability of a Type II error is called beta. The power (the probability of rejecting the false hypothesis) is 1-beta. If beta is large, then we are not confident that the null hypothesis is true even though we were unable to reject the null hypothesis.

Questions were raised at the meeting regarding the power of tests when the null hypothesis was not rejected. The expectation that the power of tests needs to be presented along with null results is growing, and it appears to be a reasonable expectation. This emerging issue is based on a concern that some results may delay interventions, both for application to hazard risk assessment and to therapeutic efficacy, that would otherwise lead more rapidly to beneficial health outcomes.