Applying the ‘Bradford Hill’ criteria to assess GMO safety

& | January 23, 2014 |
Image via Innovative Science Solutions. Image via Innovative Science Solutions.

It’s easy to refute critics of genetically modified organisms (GMOs) when they rely on anecdotal case reports, isolated mechanistic studies or unsubstantiated expert opinions. After all, to effectively and systematically evaluate a safety concern from a scientifically sound public health standpoint, it is essential to follow an established set of principals, such as the Bradford Hill (“Hill”) criteria.

Two anti-GMO advocacy groups attempted to use the Hill criteria to demonstrate a link between GMOs and adverse health effects; however, the report released by the Institute of Science in Society (ISiS) and the report released by the American Academy of Environmental Medicine (AAEM) both reflect a limited understanding of the Bradford Hill criteria and prove nothing to advance the claim that GMOs cause any health problem whatsoever

The Bradford Hill criteria are widely used to establish causation between an environmental factor and a disease. A number of review articles discuss the historical impact of the Hill criteria in the field of public health (see here, here, here and this recent post by us). Briefly, Hill proposes that a body of evidence should be evaluated using the following nine criteria:

1. Strength

2. Consistency

3. Specificity

4. Temporality

5. Biological Gradient

6. Biological Plausibility

7. Coherence

8. Experiment

9. Analogy

As more criteria are met, the likelihood of a causal relationship increases. The evidence does not have to meet all nine criteria to establish causation, but it is not possible to infer a causal relationship exists if none or few of the criteria is met. Although ISiS and AAEM acknowledged the importance of satisfying the Hill criteria in their reports, they misapplied the Hill principles. In fact, the Hill criteria were established to avoid the very types of errors that these organizations made in their reports.

Let’s take a look at exactly how these organizations missed the mark when addressing each of the Hill criteria.

1. Strength 

Strength refers to the statistical association between an exposure and an adverse health effect. For example, numerous studies demonstrate a strong association between thalidomide and birth defects. 

ISiS states that it is not possible to determine the strength of the association between GMOs and adverse health events because proper studies cannot be performed. Rather than providing evidence of a strong association, ISiS essentially dismisses the first – and perhaps most significant – principal. AAEM states that a strong association between GM foods and disease “is confirmed in several animal studies.” However, associations must be reported in human populations to satisfy the first principal. Both ISiS and AAEM failed to find a strong association between GMOs and adverse health events because none have been reported.

2. Consistency

To meet this criterion, different studies – conducted by different investigators under different conditions – must identify associations consistent with one another. This principal is sometimes referred to as “replication” or “reproducibility.” For example, lung cancer has been associated with smoking in numerous studies conducted in countless countries, different age groups, and under different conditions of exposure.

ISiS and AAEM inappropriately attempt to satisfy this criterion by citing uncontrolled, anecdotal reports. For example, ISiS contends that people and livestock have been harmed on farms, but cites uncontrolled news reports. Both organizations cite animal laboratory studies. But findings from animal studies belong in the “biological plausibility” criterion, discussed below. Neither organization identified a single study demonstrating an association between GMOs and adverse health effects, let alone several different studies with consistent findings.

3. Specificity

To satisfy this criterion, the evidence must show that a specific population at a specific site developed a disease and no other likely explanation exists. A clear disease definition is required to meet this criterion. For example, numerous controlled studies showed that individuals exposed to asbestos in specific locations had an increased risk of mesothelioma, a clearly defined lung disease. (However, many of the early asbestos studies were confounded by smoking.)

ISiS and AAEM attempt to satisfy the specificity criterion with nebulous, anecdotal examples of poorly defined adverse health events occurring under different circumstances. For example, ISiS states, “The effects reported on cattle and on farm workers are not observed with other new crops.” Yet the study provides no evidence to support this statement. What were the specific effects on cattle? Did GMO-fed cattle experience more instances of these supposed effects compared to non-GMO-fed cattle? What were the conditions under which GMO-fed cattle experienced these effects? ISiS and AAEM fail to provide specific evidence of adverse health effects in specific populations in specific conditions.

4. Temporality

To satisfy temporality, the evidence must establish that the exposure occurred before the adverse health event.

ISiS addresses temporarily as follows: “This criterion is satisfied because there is no doubt that the genetic modification occurred before the harm.” But temporality is not a given. It is entirely possible that the cattle in the previous example would have gotten sick even if they hadn’t eaten GMO crops. We can’t know for sure because a proper study was never conducted. Neither ISiS nor AAEM provide evidence to meet the temporality criterion.

5. Biological Gradient

To satisfy biological gradient, sometimes referred to as dose response, the evidence must show that larger exposures produce a more pronounced, statistical association compared to smaller exposures. For example, dose response relationships have been well documented in lead exposures. Larger lead doses have been shown to cause higher levels of neurocognitive deficits in populations of children exposed to lead paint.

ISiS explicitly concedes that no dose response relationships have been documented with respect to GMOs and any adverse health effect. The AAEM cites a single animal study that purports to show a dose response relationship. Thus neither assessment presents compelling evidence demonstrating that higher levels of GMO exposure are associated with more serious health effects.

6. Biological Plausibility

To meet this criterion, the research must establish a biological mechanism through which the exposure leads to the adverse health event. For example, there is a biologically plausible context in which crystalline silica particles cause silicosis in individuals with sufficient silica exposure.

Neither organization cites evidence of biological plausibility in its report.

7.  Coherence

To satisfy coherence, the information we have on the biology of the disease must not conflict with the interpretation of the data on the exposure-disease relationship. For example, there is a coherent pathway through which diethylstilbestrol (DES) may cause clear cell adenocarcinoma of the vagina in female offspring exposed to the drug in utero.

Neither ISiS nor AAEM provide evidence to satisfy coherence for GMOs and any adverse health effect.

8. Analogy

This principal uses information about similar exposure-disease relationships to inform a causal inference.

As Hill states: “With the effects of thalidomide and rubella before us we would surely be ready to accept slighter but similar evidence with another drug or another viral disease in pregnancy.” Analogy is arguably the least rigorous of the Hill criteria. In fact, a scientist’s ability to find a suitable analogy is limited only by their imagination. Neither ISiS nor AAEM offers a suitable analogy in its report related to GMOs and any adverse health effect.

9. Experiment

To meet this criterion, experimental intervention (e.g., reducing exposure) will result in diminished disease rates.

Neither organization provides evidence to demonstrate that the removal of GMO exposure led to reduced symptoms in human populations.

Conclusion

To properly conduct a Bradford Hill analysis, all of the statistical associations that have been reported must be collected and evaluated based on the nine established criteria. It is not possible to conduct a Bradford Hill analysis of GMOs at this time because no statistical associations have been reported between GMOs and adverse health events in human populations and no reliable animal studies exist. In fact, ISiS and AAEM neglected to mention the plethora of research that fails to demonstrate adverse health effects in animal models (see here, here, and here). The bottom line is, the analyses performed by ISiS and AAEM fail to satisfy the Hill criteria.

David H. Schwartz, PhD is founding partner of Innovative Science Solutions, LLC, a scientific consulting firm. Judith L. Steinman, PhD is a consultant with Innovative Science Solutions, LLC and teaches epidemiologic principles at the University of Hawaii Hilo.