How the public misunderstands vaccine side effect statistics

Rare side effects are undermining vaccine confidence. The problem lies in how we (mis)interprets the data

Published May 1, 2021 2:00PM (EDT)

Doctor giving first aid bandage after vaccination to senior woman (Jasmin Merdan/Getty Images)
Doctor giving first aid bandage after vaccination to senior woman (Jasmin Merdan/Getty Images)

It is a towering scientific achievement that there were multiple effective and safe vaccines against the novel coronavirus within fifteen months of its discovery. These vaccines have the potential to reduce the misery of the pandemic to a distant memory.

Yet two of the major vaccines are marred by rare yet troubling adverse side effects — something that scientists in the field call "safety signals." Cerebral venous sinus thrombosis (CVST), a type of blood clot, was observed in some patients after they were given either the AstraZeneca COVID-19 vaccine or Johnson & Johnson's. Despite the rarity of these side effects, and a lack of clarity over whether there was a causal link to the vaccine, such news threatens to undermine vaccine confidence worldwide.

Faced with uncertain data, a schism has emerged among regulators and scientists, who have two distinct schools of thought on the matter. The first, subscribed to by regulatory bodies like the Food and Drug Administration (FDA), quite reasonably insists that investigatory pauses born of an "abundance of caution" are crucial to maintaining public confidence. The other equally reasonable position, embodied by figures like American statistician Nate Silver, argue the opposite: that restrictions such as the recent "pause" of the Johnson & Johnson vaccine are damaging over-reactions to events that occur at most a handful of times per million.

The furor is a microcosm of a long-standing problem of how we convey risk, and the lurking disconnect between how risks are understood by scientists and the public. As these fears cannot be divorced from the context of a global pandemic that has killed millions and frozen the world in a terrible inertia, quantifying the extent of the issue is critical.

For  AstraZeneca's vaccine, we know that of the 54 million doses administered in the EU and UK by early April, 223 cases of CVST blood clots were reported — an incidence of roughly 4 events per million doses. With background incidence of CVST ranging from 5-15 cases per million people per year, it is tempting to infer there is no significant elevation due to the vaccine. But emergent cases of post-vaccination CVST seem to coincide with low platelet count, an unusual combination potentially hinting at a deeper association.

Yet making a causal connection is a fraught affair. Both typical CVST and the vaccine-associated CVST are so vanishingly rare that even a handful of recorded events can skew interpretations, rendering estimates of their true incidence intrinsically uncertain. Incidence itself varies with age, sex, and other risk factors – the conceptive pill, for instance, is associated with a 7-fold increase in CVST risk for women aged 15-50. Available data is transient and subject to change: originally it was thought this condition might only affect females, a position which has evolved with growing evidence. Complicating things further, COVID-19 itself is associated with both increased risk of CVST and reduced platelet count. This in effect blurs the picture, making it less clear whether associations might be due to the vaccine or the pandemic itself.

Want more health and science stories in your inbox? Subscribe to Salon's weekly newsletter The Vulgar Scientist.

In this sound and fury, regulators have the unenviable task of trying to strike a pragmatic balance between hypothetical risk and the very real harms COVID-19 has already wracked on the world. The European Medicine Agency (EMA) has repeatedly insisted that the benefits of the AstraZeneca vaccine outweigh the risks. Yet even estimating potential harm is not straightforward, as it pivots on other rapidly changing factors, from age-profile to levels of COVID in the community. It is also entirely possible to fixate on the wrong aspects; current data suggests the youngest cohort (20-29) have the highest apparent incidence of post-vaccination CVST at roughly 11 in a million cases. Understandably concerning as this is, the fact remains that COVID kills an estimated 1/2500 people aged 25-44, or 400 in every million infected.

These comparisons, however, are not always obvious, and there is an emotive aspect underlying numbers that tends to confound us. Risk is rarely intuitive, and our collective innumeracy often tempts us siren-like towards spectacularly wrong inferences. The spectacle of risk intuition utterly failing us is something demonstrated repeatedly during the pandemic. A recurrent argument from anti-mask protesters and lockdown skeptics worldwide downplays the virus as relatively harmless, "only" killing 1% or so of those infected. Yet few would willingly go to sold-out events at Michigan stadium (capacity: 107,000) if 1,000 people randomly died each game. Even though this is roughly the same level of risk, the latter feels far more visceral to us.

Communicating risk too is supremely difficult, and scientists have a chequered history in their attempts to do so. Perhaps most infamous is the WHO's International Agency for Research on Cancer (IARC), whose cancer risk classification system has been so misinterpreted by the public as to result in the spread of misinformation over things like red meat consumption and cell phone radiation.

The IARC has a laudable mission: to gauge the strength of evidence for cancer hazard from different substances. But IARC's classification system is based not on degree of risk, but instead gauges strength of evidence for a risk. In other words, classifications do not convey how dangerous something might be; only our certainty that it might be dangerous.  A substance that yielded a hundred-fold increase in cancer risk could be classified Group 1 (strong evidence of risk), the same as a substance that only increased risk a negligible amount — if there were strong evidence that both were dangerous carcinogens.

This has caused wide-scale confusion. To wit: a 2015 IARC communique classifying processed red meat as a group 1 carcinogen was widely misunderstood, with headlines the world over mistakenly proclaiming it to be as much of a cancer risk as smoking and asbestos. In reality, the absolute risk increase in for bowel cancer in heaviest consumers of processed meat was only 1% relative to those who ate none at all (6.5% versus 5.5% lifetime incidence), not remotely comparable to highly carcinogenic agents like cigarette smoke. "Group 1" only implied that there was strong statistical evidence to quantify that the risk was real.

Things become even more nebulous with Group 2A and 2B agents, which are respectively "probably" and "possibly" cancer-causing. In practice, this translates to non-existent or ambiguous evidence, rendering this classification an inadvertent epidemiological dumping ground. This can be readily weaponized by bad-faith actors; classification of radiofrequency radiation as class 2B, for example, has been misrepresented by anti-5G campaigners to falsely imply cell communications cause cancer, despite this being completely at odds with scientific consensus.    

Such disingenuous tactics are grimly successful; public perception is highly malleable, and those hawking them have zero compunction over harnessing a faux-veneer of scientific legitimacy to push profoundly pseudoscientific narratives. 

Anti-vaccine activists have long stood at the apex of this mendacious pantheon; since the advent of the internet, they have proven themselves cynically adept at weaponizing social media. Exposure to anti-vaccine propaganda has a deeply negative impact on parental intentions to vaccinate, with the induced fear of phantom dangers often enough to blind people to the very real harms that vaccines prevent. Anti-vaccine propaganda has been so damaging to public health, in fact,  that in 2019, the dark renaissance of once nigh-on conquered diseases worldwide forced the WHO to declare vaccine hesitancy a top 10 threat to public health

This context is critical to understand Nate Silver's lamentation that the FDA decision to suspend the J&J vaccine was "going to get people killed..and going to create more vaccine hesitancy." Predictably, anti-vaccine activists are already twisting the transparency of regulators into their propaganda, presenting this as a proof-positive that all vaccines are dangerous.  In Europe, even after suspensions have been largely lifted, a miasma of doubt still lingers; perceptions that the AstraZeneca vaccine is subpar or dangerous has resulted in the vaccine being rejected in some instances by vulnerable cohorts.  Wrong-headed conceptions of danger threaten to nudge uptake dangerously in the wrong direction.

Yet to some extent, regulators are in an impossible position. Even if risk is virtually negligible and vastly exceeded by benefits, a failure to investigate a potential side-effect would be negligent and unethical. Several national regulators have invoked the "precautionary principle" as a rationale in temporarily suspending vaccine recommendations. But this is perhaps a misunderstanding of the concept, as such a course of action is not risk free; a misguided suspension of the cancer-preventing human papilloma virus (HPV) vaccine in Japan in 2013 led to a confidence crisis, which drove uptake down from 70% to less than 1% within a year —  a situation which will cost an estimated 11,000 lives.   

Suspending administration of COVID vaccines is thus not a zero-risk option. Quite aside from the potential for long-standing damage to public perception, suspensions leave vulnerable people unprotected from the virus, prolonging the pandemic. They also do nothing to mitigate the spread of COVID, nor stem the tide of hospital admissions and needless deaths the vaccine could prevent. Well-intentioned as it is, the precautionary principle is no substitute for evidence-based decision making.   Perhaps a more pragmatic approach is than outright pauses might be a fusion of both schools, a cautious vigilance where vaccinated individuals are monitored for warning signs of these rare CVSTs so that ill-effects can be circumvented without impeding vaccine drives.

The reality is that nothing is risk-free. Measures to mitigate one risk must be balanced with competing hazards; your car seatbelt may decapitate you in an accident, but it is far more likely to save your life. Risk can seem an abstract, nebulous concept, but it is vital perspective is maintained - and that our positions shift with emerging evidence. Risk can be reduced, but never eliminated - any attempt to do is a fool's errand. But it would benefit all of us to better understand it, especially when a modicum of reflection might save a lot more lives than a knee-jerk reaction.

By David Robert Grimes

David Robert Grimes (Ph.D) is a cancer researcher, physicist and author. His latest book, "Good Thinking: Why Flawed Logic Puts Us All at Risk and How Critical Thinking Can Save the World," is available from The Experiment. Find him on Twitter: @drg1985

MORE FROM David Robert Grimes

Related Topics ------------------------------------------

Astrazeneca Blood Clots Covid-19 Explainer Johnson And Johnson Side Effects Vaccine Confidence