AMA Journal of Ethics^®

Abstract

A growing chorus of academicians, public health officials, and other science communicators have warned of what they see as an ill-informed public making poor personal or electoral decisions. Misinformation is often seen as an urgent new problem, so some members of these communities have pushed for quick but untested solutions without carefully diagnosing ethical pitfalls of rushed interventions. This article argues that attempts to “cure” public opinion that are inconsistent with best available social science evidence not only leave the scientific community vulnerable to long-term reputational damage but also raise significant ethical questions. It also suggests strategies for communicating science and health information equitably, effectively, and ethically to audiences affected by it without undermining affected audiences’ agency over what to do with it.

Current Myopia About Misinformation

“I believe that misinformation is now our leading cause of death,” US Food and Drug Administration Commissioner Robert Califf tweeted in April 2022, “and we must do something about it.”¹ Diagnoses like this one are understandable. Normative democratic ideals assume that the best available scientific evidence informs both societal and individual decisions and crowds out misperceptions, disinformation, or conspiratorial thinking. Because of what it sees as a recent deviation from these norms in our current information environment, the World Health Organization has warned about an “infodemic,”² or a deluge of information—some of it inaccurate—that makes it difficult for citizens to separate signal from noise.

The assumption that misinformation is a new problem and Califf’s diagnosis of it raise several questions. Are there, in fact, reliable bodies of evidence that demonstrate that misinformation among public audiences is (a) more widespread now than it has been in the past and (b) causally rather than correlationally linked to behavioral choices or attitudes that might be harmful to societal or individual well-being? As we have showed elsewhere, the answer to both questions, at the moment, is “no.”³ Given the problem’s uncertain diagnosis and prognosis, the answers to various questions regarding solutions quickly become complicated. What, if anything, constitutes an appropriate way to intervene on misinformation and its behavioral correlates, such as vaccine hesitancy? Does this answer change for different audiences (eg, medical professionals vs nonexperts)? When, if at all, should informational correctives or interventions targeting behavioral correlates of misinformation occur? To whom should they be delivered, in what form, and with what degree of certainty? And who should deliver them—scientists, the government, or other actors?

The complexity of the problem of misinformation and the weak evidence base for both its diagnosis and prognosis have not stopped many in the academic community from urging large-scale interventions to stop the spread and uptake of misinformation about science and health.^4,5 These interventions raise a number of ethical concerns, all of which we explain in more detail below. First, some of these interventions are driven by a (sub)conscious desire among scientists to shape rather than inform public policy and to change how the public consumes public health information. This is not to say that scientists should not engage in continuous dialogue with policy makers⁶ about how science can inform policy options or with other “publics”⁷ about individual citizen choices. Some scientists’ strategy of urging policy makers or citizens to “just follow the science,” however, is not only naïve with respect to its likely success but also normatively at odds with how democratic societies make policy choices. As Stevens succinctly put it: “[T]he process of organising knowledge for policy through advisory committee is political, as well as scientific.… So when a government claims to be ‘following the science’ in response to a global pandemic, we need to treat this claim with caution.”⁸ We will return to this concern below.

The intention to influence policy choices or change citizens’ behavior by solely following the science, unmediated by politics—concerning in itself—is even more troubling in light of the second ethical issue: corrective interventions for misinformation can have unintended effects that undermine scientists’ credibility, raise ethical dilemmas, or create additional vulnerabilities for some populations. Finally, many of these interventions—ironically—are at odds with the best available scientific evidence about how to effectively achieve different science communication goals across diverse populations and in new media environments driven by algorithms.

Informing Policy vs Social Engineering

Scientists’ instinctual desire to intervene in what they see as a worsening misinformation problem comes with a temptation to claim authority over policy questions that do not solely have scientific answers. For example, should we mandate COVID-19 vaccines in elementary schools if those vaccines have been proven to be safe for children between 5 and 12 years of age? The second part of this question—ie, whether vaccines are “safe” according to a given technical standard—is one that science has the competence and authority to answer. However, the first part of this question—the vaccine mandate—is a question of public policy that should be determined by democratic processes involving diverse groups of stakeholders. Although science can function in policy-related deliberations as a factual evidentiary authority—for example, by providing insights as to the comparative health risks and benefits of vaccines—the weight of scientists’ authority relative to that of others in policy making processes is determined by democratic institutions.

This distinction between empirical and policy questions is captured in the term policy itself. In the polis, or the Greek ideal city-state, communities are run by citizens. Scientists and scientific institutions, as one component of our larger polis, are uniquely positioned to inform policy decisions with reliable evidence, but scientific evidence is just one of many input streams to policy choices. As such, scientific evidence competes in democratic decision-making processes with other kinds of considerations, including societal values, strategic goals, and social norms.

As we argued early in the pandemic,⁹ science will risk losing public support—especially among some partisan groups—if it blurs the boundaries between empirical questions that it is qualified to answer and policy questions that can only be addressed as part of broader public deliberations about facts, values, and societal priorities. Anthony Fauci, director of the National Institute of Allergy and Infectious Diseases and chief medical advisor to the US president, demonstrates how to successfully walk this very fine line. When asked whether he would recommend a vaccination requirement for domestic travel in December of 2021, for example, he responded: “The President takes all recommendations, all discussions, and, as a group, we make a decision about what’s best to do.”¹⁰

Recognizing the need for science to be seen as a good-faith arbiter of reliable evidence to inform policy, some in the academic community have shifted their attention to maximizing public uptake of reliable information and preventing the spread of misleading claims. The COVID-19 pandemic, not surprisingly, put significant strain on this approach, given that scientists were faced with the temptation to counter misinformation that they knew to be wrong with rapidly changing scientific evidence, which often turned out to be wrong or incomplete. For example, former President Trump’s misleading claims about the benefits of hydroxychloroquine were quickly debunked by fact checkers based on an article that had been published in the Lancet, which was later retracted.^3,11 The retraction led some of Trump’s defenders to label fact-checks and articles published in medical journals contradicting Trump’s claims as essentially biased political attacks on Trump.¹² The damage was done, even though a recent study published in the Lancet Regional Health Americas found that hydroxychloroquine has no benefit in reducing risk of hospitalization for COVID-19.¹³ As this example shows, the possible benefits we reap from fact-checking scientific claims in contexts of high scientific uncertainty may simply fail to outweigh the risks and unintended consequences of undermining scientific authority.

Attempts to manage public opinion by socially engineering the ways that citizens use and interpret information are riddled with ethical pitfalls. It might be tempting to believe that, by “sticking to the facts,” scientific communicators remain neutral brokers of information whose purview encompasses interventions to combat misinformation. However, it is possible for scientific actors to use the “facts” to communicate in ethically questionable ways.¹⁴ For example, there has been increasing attention to communication approaches designed to “inoculate” members of the public against developing misperceptions about science as a sort of preventive intervention strategy.¹⁵  Psychological inoculation involves warning audiences about the existence of certain misinformation claims before they are exposed to them (eg, “over 31,000 scientists have signed a petition that there is no scientific evidence for human-caused global warming”), and then following that warning with a factual refutation (eg, “97% of climate scientists have concluded that human-caused global warming is happening”).¹⁶ Rooted in propaganda research conducted during World War II,¹⁷ inoculation research is designed to influence a priori how people will process false claims when they encounter them in order to prevent the uptake of certain beliefs (typically beliefs that are considered by scientists to be false or misleading).¹⁶ Yet attempts to limit public discourse, no matter how misguided that discourse may seem, are antithetical to the very idea of science, which, over time, builds an increasingly sound epistemological account of the world through the open contestation of competing truth claims. Attempts by scientific institutions to steer or moderate public debate through inoculation therefore risk doing irreparable reputational damage to science’s claim to be a neutral arbiter of truth.

Some might argue that scientists are uniquely positioned to steer public debate, given their reputation as neutral observers whose conclusions—in an ideal world—are only bound by facts. This reputation does not always align with scientific data, however, as scientists exhibit motivated reasoning and identity-driven conclusions similar to that of non-expert citizens. For instance, male scientists rate research as lower in quality if it challenges gender bias within the academy.¹⁸ Similarly, research conducted by the third author (D.A.S.) and colleagues shows that scientists’ policy judgments about regulation of new technologies are shaped by their personal ideology and other belief systems, even after controlling for their scientific judgments about potential risks and benefits.¹⁹

When, if at all, should informational correctives or interventions targeting behavioral correlates of misinformation occur? 

Perhaps more importantly, however, inoculation and related interventions are techniques that by design rely on audiences not consenting to the “treatment.” If scientists told audiences that they selectively exposed them to small dosages of counter-messages in news or social media in order to change the way they interpret information down the road, the corrective effects might evaporate. More perniciously, scientists would rightfully come under immediate attack from political actors for not staying in their “lane.” The undermining of scientific authority would be exacerbated by a constantly changing knowledge base that might create ethical dilemmas for scientists if—in rare cases—they were aware they might inoculate against messages that turned out not to be completely false as more science emerged, as occurred early in the COVID-19 pandemic.¹³ In other words, informational inoculation without consent—which would be unthinkable for vaccines or other medical treatments—is both normatively troubling and potentially disastrous as a public relations problem for science.

Our concern about social engineering approaches, such as inoculation, however, should not be interpreted as advocacy for a naïve understanding of a marketplace of ideas in which “false” claims will eventually give way to “true” claims in public discourse. They will not. Even if true claims were easily identifiable (which they often are not),³ evidence-backed claims that fail to meaningfully connect to societal values and preferences are unlikely to win hearts and minds when they compete in modern media ecosystems against well-packaged falsehoods.^14,20 It therefore makes sense to use research insights from the behavioral and social sciences, for example, to frame science-related messaging in ways that will resonate with specific audiences who might otherwise be unmotivated to recognize certain issues as relevant, important, or valuable to them.²¹ However, rather than packaging information in ways that increase target audiences’ likelihood of considering a given argument or of seeing certain facts in new ways, inoculation essentially enables the institutions who use it to paternalistically decide that certain claims are blights on public discourse and then to stealthily diminish the power of those claims by encouraging people to discount them and thereby dramatically reduce their spread.

Similar charges of paternalism can be leveled against “nudging” initiatives. Nudging, which is sometimes euphemistically described as “enhanced active choice,”²² is a social engineering strategy that shapes information delivery and decision-making processes in ways that push people’s behavior in a desired direction. To be clear, nudging is not designed to educate, as inoculation sometimes is. Instead, nudging is intended primarily to orchestrate outcomes of interest, typically outside conscious awareness.²³ An example of nudging is when transportation authorities set organ donation as the default when people obtain their driver’s license—ie, rather than asking people to opt-in to organ donation, people must instead opt out. When former President Barack Obama’s Social and Behavioral Sciences Team implemented nudging strategies—eg, by redesigning communications to encourage military service members to contribute to their retirement plans²⁴—the policies were, unsurprisingly, criticized as affronts to individual liberty on the part of condescending government institutions. “To be clear,” noted Richard Williams in Politico, “Congress did not pass legislation authorizing such activity; this is something dreamt up by bureaucracies to force their own preferences on citizens.”²⁵

The concerns we have expressed about nudging are not meant to detract from the importance and power of social science research. Nudges can serve as valuable tools to achieve certain goals, such as mask-wearing during a public health crisis like the COVID-19 pandemic, but the question as to whether orchestrating any given outcome qualifies as using “nudging for good” is often difficult to answer and will be dependent on cost-benefit analyses that incorporate competing value systems and priorities. Is nudging people to wear masks for the sake of public health a justifiable use of nudging, if evidence about possible detrimental effects of mask-wearing on the psychological development of young children²⁶—or on racial profiling of Black people—remains unclear, or at least difficult to quantify?²⁷ The answer is likely still “yes, nudging to encourage mask-wearing is important,” but, as this example suggests, we need to consider the perspectives of diverse stakeholders (including scientists in specialties other than public health) in our decision-making process, as well as in the careful design and dissemination of nudging communications.

While reasonable arguments can be made both in favor of and against policy makers or government institutions using nudging strategies, the idea of scientists themselves trying to nudge publics either in their information processing or behaviors is riddled with ethical landmines. As we discussed earlier, the reputational risks that scientists and scientific institutions face when they engage in such social engineering are serious, and they will only intensify in contexts in which the science at hand is controversial or the scientific evidence underlying the social engineering strategy is rapidly shifting. Indeed, scientists who engage in inoculation or nudging will likely be perceived by some as condescending or paternalistic, as participating in an unethical overreach of their institutional authority, or even as hypocritically undermining the open contestation of knowledge that is core to scientific philosophy and epistemology.⁹

Communicating Science

When Congress established the US land grant system with the passage of the Morrill Act over 160 years ago, it did so with the intent to support not only the growth of scientific knowledge but also the communication of scientific information.²⁸ As Congressman Morrill put it: to “give intelligence to those who will esteem it.… Let us have such colleges … to announce facts and fixed laws … and broadcast that knowledge.”²⁹ As we have argued in this essay, the communication of reliable scientific information in our current highly politicized and competitive information ecologies has many ethical pitfalls. Relying on the best available social science evidence to help guide these efforts is therefore foundational to science’s ability to fulfill what some have called its “social contract.”³⁰ It follows that it would be unethical for scientists not to do everything they can in order to ensure that the benefits of their work reach all cross-sections of society.

This ethical mandate—and the utility of social science research in fulfilling it—is illustrated powerfully by concerns within the scientific community about a lack of trust among African American communities and other populations that historically have been at the receiving end of unethical treatment (or the lack of treatment) by parts of the medical community. Calls to rebuild trust are often well-intentioned but focus on symptoms rather than the underlying causes. For example, expectant African American mothers continue to face up to 3 times higher mortality rates than White mothers.³¹ A lack of trust in the medical community, in other words, might be much less a function of historical mistreatment than of current inequities in health outcomes. Any attempt to rebuild trust through outreach and communication without first addressing these kinds of inequities is disingenuous at best and unethical at worst. But we know from decades of social science research that citizens with higher income and education levels will benefit much more from health information campaigns than people with lower levels of income or education.³² These “knowledge gaps,” as sociologist Phil Tichenor and colleagues called them in the 1970s,³³ will widen as more information becomes available, favoring the already information rich and leaving already vulnerable populations less (accurately) informed.

Given some communities’ lack of trust in science and the existence of knowledge gaps, new information needs to be framed in ways that align with how different publics make sense of information. Decades of research in communication science, sociology, political science, and psychology have shown that the same information is interpreted very differently by audiences when presented in ways that either resonate or do not resonate with their respective interpretive schemas and worldviews.²¹ When scientists communicate without providing audience-relevant context and framing that resonate with citizens’ (rather than their own) value and belief systems, their messaging is likely to favor groups who are already most interested in science and aligned with the scientific community, leaving behind groups that are often most vulnerable and underserved by paywalled science journalism in elite media outlets. Data collected during the pandemic about a lack of public buy-in for “vaccine passports” provide powerful proof of how effective alternative framings can be. While conservative audiences were concerned about the term passport, which resonated with their concerns about government overreach and federal oversight, they were much more likely to support vaccine “verification,” which frames the issue of showing vaccination cards as one of individual choice and responsibility.³⁴

Of course, not every frame is meaningful to all publics, especially in an era when hyperpartisanship is the new normal. We all engage in motivated reasoning, especially when processing information that contradicts our values.³⁵ Similarly, many of us navigate online environments at least partly defined by filter bubbles that echo voices and sources consistent with our prior views and preferences.^36,37 We need to start taking these realities into account rather than seeing those whom we are trying to persuade as the only ones using motivated reasoning and blaming them and the filter bubbles they are in for adverse outcomes or even for exacerbating the problem. For instance, the scientific community has a poor track record of meaningfully communicating the value of science to some sectors of society, such as religious people or conservative audiences.³⁸ And some of these wounds might be self-inflicted: we should not be surprised when science is perceived as partisan when prominent scientist communicators regularly mock Republicans and communities of faith on social media.^38,39

COVID-19 has demonstrated powerfully how unprepared science was to give answers to rapidly emerging and urgent policy problems. Not only were there high-profile retractions of published research¹¹ and preprint-based overclaims in popular media, but science during the pandemic was also conducted much faster than normal and under immense public scrutiny.⁹ Even with these challenges, however, science continues to be the best way that societies have for producing and curating reliable information. Scientists seeing members of the public as partners in solving large societal challenges, such as COVID-19, rather than as patients with attitudinal or behavioral pathologies that need to be fixed will be a prerequisite for scientists’ continued ability to inform the urgent policy choices that are coming our way.