When Words Matter

by Cindy Sage, Sage Associates

A Comparison of Recent EMF Reviews and How Expert Groups Can Come to Opposite Conclusions Looking at the Same Evidence

In judging the sufficiency of scientific evidence to determine the likelihood that an environmental exposure is related to a health outcome, the answer one will arrive at very often depends on the professional background and training of the person interpreting it.

Approaches and terminology that are used in assessing, communicating and managing risks commonly differ in the view and practices of the various professional groups that can be involved in judging science. Evaluating the strengths of scientific evidence on public health and environmental hazards requires that these differences are recognized and transparent in expert reviews. Decision-makers and the public will be best served by a clear recognition that differing standards of evidence and levels of proof are expected and justified, and that making them explicit in review processes will go far to defusing misunderstandings and unnecessary conflicts.

Real life decisions are made every day about how to take EMF into account, with the evidence in hand. Societal judgments about where and whether to commit new resources for development needs to incorporate wise planning for EMF exposures given what we know (and do not know) today. However, waiting until all is known about EMF before we build new homes, schools, day-care and pre-school facilities and the like just will not happen. Its been the same question for nearly 20 years now.

Different approaches to evaluating scientific evidence have resulted in diametrically opposed conclusions about what the evidence means on EMF and health impacts. How can we look at the same evidence and come to such different conclusions about what it means? And, perhaps more importantly,what to do about it?

I recently looked at five reviews, each by recognized experts in the field, to determining what the current science tells us. These included the BioInitiative Report, the Scientific Committee on Emerging and Newly Identified Health Risks (SCENIHR), the WHO ELF Monograph, the UK SAGE Report and the UK Royal Commission on Environmental Pollution (RCEP) documents. All were released in 2007.

Comparing the conclusions of the BioInitative Report and the SCENIHR review, the BioInitiative Working Group concluded that the existing public safety limits (ICNIRP and FCC/IEEE) are insufficiently protective of public health. Further, the Bio-Initiative Report recommends development of new, biologically-based public exposure standards because the substantial evidence we already have suggests that chronic, long-term exposures to ELF and to new wireless technologies are not satisfactorily handled under the old thermally-based safety limits. The SCENIHR review concludes that no changes in the ICNIRP limits are warranted. The WHO ELF Monograph reviews essentially the same science as the BioInitiative Report and the SCENIHR (limited to ELF, of course) and also finds no reason to suggest changes in the ICNIRP limits (even confirming that the IARC 2B designation for ELF is valid). The WHO ELF Monograph and SCENIHR conclusions can only be explained if they adhere to a standard of evidence requiring virtual certainty* (scientific certainty, proof, adverse health risks are established, a causal link is demonstrated). How could so many experts come to so many different conclusions? We examine that question below.

In each case, the outcome was determined in large part by the approach taken. A careful reading of each document, together with the original “instructions” to each working group or committee can illustrate why results differ so widely. There are four important questions to ask about each one.

How is the central question framed?
What standard of evidence (level of proof) is employed?
What terminology guides the assessment?
What level of evidence is used to recommended action?

Framing the Question – What are we asking?
How the question is originally framed to the working group is the most central issue in defining the outcome. Whether the group actually sticks with this definition, or shifts it during the evaluation, is also of paramount importance.

For example, if the question is “is there a health risk demonstrated?” this virtually guarantees that causal scientific evidence (the highest burden of proof) will be required before the resulting analysis will say “EMF causes health risks”.

But, if the question is “is there a possible effect on health?” or some similar phrasing, then evaluation of the same evidence will result in a conclusion that has a lower burden of proof, and is far more likely to result in a conclusion that “EMF is likely to present health risks”.

How the question is structured, and whether this question is answered largely determines whether a report will say “no, EMF is not proven to be a health risk” or “yes, it appears possible or probable”.

* Conformance with EU policies would rule out using a strictly scientific standard of evidence. To be consistent with the EU Constitutional Principle on Health (Section 3.1) and the European Union Treaties Article 174, the WHO ELF Monograph process needed to be consistent with a public health/precautionary principle-based approach to judging the evidence. These two policies require that the precautionary principle be the basis for environmental protection for the public, and that protecting public health and taking preventive action before certainty of harm is proven is the foundation of the Precautionary Principle.

What Standard of Evidence (Level of Proof) is Used?
What is the implicit assumption about which levels of proof will be used going into an assessment of the EMF data? It needs to be made transparent at the outset, because if it is not, there will likely be significantly different views about “when we have enough information” to make policy changes or to require new regulatory action.

There are four basic professional approaches to judging the sufficiency of evidence in order to “take action” appropriate to their professional training and experience.

  • Scientific standard (95% - 99% certainty/causal)
  • Legal standard (51%+ - possible/probable range)
  • Environmental standard (10% - 30% potential for impact)
  • Public Health standard (variable, depends on both how much evidence there is, and severity of impact, if true).

The most rigorous is a scientific standard, where virtual proof of causation is typically required by scientists to arrive at consensus about an effect. This approach works best in mathematics, physics and chemistry. In biological systems this is rarely possible.

The second level of proof is the standard applied in civil legal proceedings, which is “more likely than not”. This is to say if there is a 50%+ likelihood of harm, this is taken as evidence for a relationship. It is not necessary that there be conclusive evidence of harm, nor is some uncertainty of causation a reason to conclude that no relationship exists between exposure and harm. In fact, some uncertainty is allowable even under the more stringent (criminal) standard of evidence, which is “beyond a reasonable doubt”. No legal standard requires complete certainty of effect in order to make a defensible judgment on the evidence at hand.

Environmental decision-making requires only the potential for a significant impact. National and state environmental quality acts (The National Environmental Policy Act) and various state environmental quality acts (SEQAs) require that assessments use a standard which is a relatively low level of certainty (10% to 30%). The potential for a significant impact requires that mitigation strategies be developed, i.e, require precautionary or preventative actions when only the potential for risk is present. We plan for environmental risks all the time without certainty that an adverse risk will occur (seismic events, landslides, flooding, etc).

The standard of evidence in Public Health Policy decision-making should reasonably be based on many factors, including how widespread the risk, how dread the disease, the cost of inaction (doing nothing until there is proof, but many may be harmed and so on). A slim showing of evidence coupled with a highly adverse public health impact (large numbers of individuals harmed, a very large public health impact if the early warnings are ignored) may warrant early precautionary actions. When the public health consequences are not so severe by ignoring early warnings, waiting for more evidence may be warranted since the consequences of doing nothing immediately may not be so great. For potential risks of small overall magnitude, waiting for substantially more evidence before taking actions (particularly costly actions) is reasonable.

A key to understanding how the same evidence can be so differently weighed and judged, in order to set a course for action, is to understand the way in which different professional groups approach this task. As noted above, this has not always been made explicit because different disciplines inherently use different standards.

What Terminology Guides the Assessment?
Selecting and defining key words to be used in an assessment is vital before the review starts. These typically include:

• evidence (as opposed to proof)
• effect
• adverse effect or risk
• proof
• consistent (or inconsistent)
• certainty (or uncertainty)
• plausible (as in mechanism)
• demonstrated or established (as in proven)
• acute versus chronic

What level of evidence is used to recommend action?
This could also be phrased as “what is the conclusion or outcome of the review?” and is it consistent with the original question asked, the standard of evidence which is appropriate to answer that question, and whether the terminology used in the assessment and conclusions are both transparent and consistent with answering the question.

Risk assessments are not only done by scientific expert review panels, but depend on many other stakeholders’ viewpoints. Thus, it is wrong to let the highest burden of proof (the scientific standard for judging the sufficiency of evidence) be a pre-condition to taking action, precisely because valid clinical and public health approaches for assessing the evidence set the bar far lower in terms of certainty, and in judging when it is sufficient to take action.

Whether an environmental or public health policy standard, as opposed to a strict scientific certainty standard is appropriate is a matter of debate and great consequence. The costs of inaction in terms of public health and resources will be enormous on a global scale if we ignore, or worse, refuse to use the proper screening assessments to determine these outcomes. At a minimum we should expect transparency in reviews – what is being asked and answered, whether the right levels of proof are being used, and whether the answers given follow from the evidence in light of these explicit approaches can be validated.