(This post is written in collaboration with another author who does not have a forum account and prefers to be anonymous)
Science progresses through knowledge revolutions, and not incremental discoveries, argues the philosopher of science Thomas Kuhn1. So too does society, argue numerous thinkers, including anthropologist David Graeber2 and philosopher Amia Srinivasan3. The quality of life we can expect to enjoy today has improved vastly over the course of recorded history. If this has been primarily driven by paradigm shifts, large scale reimaginings of how the world works and society should function, we have a duty to support and foster those.
Paradigm shifting does not fall neatly within the framework of impact measurement imposed by Open Philanthropy. It is difficult, perhaps impossible, to attach a dollar value to social movements, to quantify intangible yet groundbreaking emancipation in quality-adjusted life years. How many lives were saved through reimagining gender equality and the subsequent introduction of laws for domestic violence, which largely came to be around the 1990s? How many person-hours of productivity were gained and how much wellbeing was increased after the reimagining of labour and subsequently through the successful lobbying of labour unions to have a 5 day working week?
Here, we first focus on a series of arguments detailing why it might be particularly difficult (and not always useful) to assess cost-effectiveness in conventional ways for paradigm shifting. We do this by highlighting economics research on randomised controlled trials and policy, as well research on nudging. Further, we review examples of successful paradigm shifts (smallpox and social movements) and contextualise their impact. We summarise the outlined challenges and benefits, and, finally, we suggest a possible set of funding recommendations.
Against pure ‘cost-effectiveness’ calculations for systemic change
Experiments, Randomised Controlled Trials (RCTs), and Policy
In taking a narrow, quantitative approach to the operationalization of effectiveness, we inadvertently restrict the gamut of cause prioritisation. Similar arguments have recently been wedged in developmental economics, including by economics Nobel Prize winner, Angus Deaton4,5, that methodological preferences, e.g. for randomised controlled trials, implicitly shape research and policy priorities6.
Randomised Controlled Trials (RCTs) are a popular form of experiment, common across medicine, economics, and social sciences. In many “evidence pyramids” they are the highest form of individual experiment level evidence (that is, excluding “meta” evidence like systematic reviews, meta-analyses or reviews of reviews). RCT data is also commonly used for cost-effectiveness calculations.
Deaton & Cartwright, among others, argue that there are essential caveats to RCTs that should make us cautious about treating them as the single most preferred level of evidence (what they term “magical thinking” about RCTs), such as for instance that:
a) randomisation does not balance confounders,
b) RCTs may improve the unbiasedness of estimates, but this may be of limited practical value (e.g. compared to improving precision or generalizability),
c) improvements in unbiasedness require orthogonality which is not guaranteed by randomization in and of itself,
c) treatment effects likely have asymmetric distributions that are a challenge for significance testing
Overall, this research ultimately argues that RCTs are an important part of a research agenda, that on their own they are a weak ground for inferring ‘what works’, and so that the overall agenda needed to move forward should employ a triangulation of methods that allow us to ascertain more robustly “what works” but also address “why things work”
A related line of research, again by Deaton, strongly argues that RCTs have “no unique advantages or disadvantages over other empirical methods in economics… [t]hey do not simplify inference, nor can an RCT establish causality.” More importantly, this work highlights important ethical considerations. We will focus on this next.
Ethics of experimentation
As RCTs are privileged in the evidence pyramid and often the preferred method of economists to infer cost-effectiveness, it is crucial to consider ethical implications. Ruling out other options for evidence that may pose fewer risks to participants or that perhaps might be more informative is a likely downside, particularly for those who believe in the “magical RCT thinking”. Ignoring other important information is a disservice to science and ethics.
More broadly, most RCTs interventions in the field of global health are delivered by better-educated, better-off people who have more protection and resources and live different lives than the people they target through interventions (who tend to be lower income, less-educated, less protected). Deaton, like other scholars, has questioned how many current experiments, particularly within the USA, are carried out with the interests of rich taxpayers who want to prevent the worst of poverty and want to minimise the cost of doing so.
Further, there has been discussion that institutional review boards are not a sufficient ethical clearance level. Some interventions (which had obtained American IRB approval) carried out by western researchers on extremely poor people in India likely could not have been carried out in the States. Other work provides no clear benefit to the poor, and may even harm or expose them to further risk immediately or post-intervention. Estimates of cost-effectiveness may be further ‘polluted’ by lack of true informed consent, lack of choice, and lack of acceptability/desirability of intervention.
Oftentimes, RCTs and interventions are not sensitive nor appreciative of the overall context in which they are delivered. Many times an intervention will benefit not only the citizens of a given country but the overall regime in that area. This may be particularly potent and visible in areas of armed conflict and war, where it is not possible to deliver aid without cooperation or payment to the warmongers, which may in turn be prolonging or further worsening the suffering of those meant to receive protection. Though this is likely true for many states not at war, particularly authoritarian regimes where an international aid organisation’s work will at minimum serve as political cover for political despots. It is extremely infrequent that such contextual factors are ever discussed in cost-effectiveness considerations, yet directly factored in.
“Aid agencies are turning a blind eye to political repression so long as the oppressors help check off one the Sustainable Development Goals, preferably as demonstrated by randomized controlled trials. The RCT is in itself a neutral statistical tool but as Dean Spears notes, “RCTs provide a ready and high-status language” that allows “mutual legitimization among funders, researchers, and governments.” When the RCT methodology is used as a tool for “finding out what works,” in a way that does not include freedom in its definition of what works, then it risks supporting oppression.”—Deaton, 2020
Constraints for policy
To condense the above, enforcing hierarchies of evidence and privileging some types of evidence, without carefully accounting for cost, harm, context, and methodological limitations, is ineffective and potentially dangerous, both for research and policy.
Akerlof has further worked on illustrating the limitations for policy. He has argued that economics favours “hard” research—that is, research that is quantitative, causal, involving mathematical models, ideally capturing underlying concepts with precision, rather than “soft research” (qualitative research, or quantitative where models are more simplistic or explained through words, correlational work, work that does not target concepts or is imprecise).
The ‘hardness bias’, according to Akerlof, is linked to (among other pitfalls) a bias against new ideas and overspecialisation.
First, there might be a bias against new ideas because it is easier to do ‘hard’ work, perceived as more valuable, when based on existing topics and old paradigms: it is easier to improve precision in existing work, and rely on established terminologies and conceptual frameworks. Those working on truly new work have a larger load to shoulder, as they will often have to suggest entirely novel frameworks.
Second, a focus on “hard” research diminishes the ability to challenge existing paradigms. Old ideas are denounced when their effects fail to replicate or they are demonstrated to be inferior to other ideas (often time related ones, indicating an improvement in fit). At the same time, it is generally expected that ‘new ideas’ provide testable predictions. This is a widely accepted axiom of research, yet in practice this means we discard ‘softer’ tests of theories (e.g. those that consider models based on their quality, plausibility, efficiency, utility etc. and well as the quality, plausibility, efficiency, utility etc. of their conclusions). As a practical example, this might mean that for an intervention we focus on the overall effect, rather than more carefully considering attrition, mechanisms, context, sub-group effects, participant motivation, constraints, time decay etc.
Third, in terms of overspecialisation, it may be easier to carry out “hard” research as a specialist (meeting the requirements and norms of only one field) than to be a generalist (where there are requirements, norms and a wider body of literature and methods to consider and appreciate). This is not, in and of itself, immediately ‘a bad’ state of affairs. Specialists may have nuanced understanding of field-specific techniques or bodies of research. The caution here, instead, should be that some balance may be needed and if it is not found, certain specialties may be more favoured over time, and others may fall out of favour; that nuance that comes with generalist knowledge may be undervalued. Further, we should more carefully consider the implications of discipline-wide norms that seem to promote or encourage only certain kinds of science.
Overall, such a ‘hardness’ bias may make economists and other researchers more readily willing to accept existing theories, and less willing to propose novel theories. Given existing incentive structures, it is likely harder to produce radically new work that could be a paradigm shift.
As a broader and more tentative note, there might be some evidence that some ideas, considered more “heretical” or “fringe” may be more easily mistrusted both across fields and historically. See this overview from Information is Beautiful on maverick scientists.
Examples of Misused Resources and the Need for Paradigm Shifts
Nudging Research
Another recent analysis7 by prominent academics in psychology and economics, Nick Chater and George Lowenstein, argues that nudging interventions targeting individual behaviour are 1) largely ineffective, and 2) actively harmful in distracting our efforts from the necessary systemic changes. With this lens, we can view the core body of research on individual-level nudging as something that has been inefficient, costly, and that has restricted time, effort, and consideration from researchers and policy-makers that could have otherwise gone to more effective and novel ways to improve society.
Nudging research has focused the attention of researchers, policy-makers and lay-people on individual factors by framing global problems, including climate change, retirement savings, obesity and pollution, in individual rather than systemic terms. Yet, from a rational standpoint, this framing is entirely at odds with the evidence for the root of the problems (e.g. 71% of global CO2 emissions owing to 100 companies) and viable solutions (e.g. compulsory superannuation systems).
Climate Change
Climate change, within the Effective Altruism movement, is generally not considered neglected (e.g. see Summary by 80,000 Hours), however here we would like to suggest that the majority of interventions are largely not tractable nor impactful.
As discussed above, many individual-level nudging interventions are largely inefficient (null or very small effects) that cannot address systemic level issues. Rather than a systematic review of current interventions, here we illustrate general level thinking about these problems through a few concrete examples.
We note as a broader comment that the need for paradigm shifts within climate change may be beneficial for humanity’s longterm future as well: it seems, at face value, that innovation away from fossil fuels and into more sustainable energies can be helpful for equity, global wellbeing, space exploration, etc.
Cows and Seaweed
Recently, the news picked up the finding that feeding seaweed to cattle can reduce their methane emission by about 40%, this was described as a game-changing method to slow methane emissions. It was then proposed that red seaweed (high in bromoform which can reduce belching from cows that lead to added methane) becomes a food additive for cows. At face value, this may seem like a promising and innovative proposal. At the same time, what would it mean for such a proposal to be brought to scale?
Prioritising one culture of seaweed may lead us to create mono-cultures that limit biodiversity and in turn harm other animals and water quality.
Bromoform’s effects on human health are not fully established at higher values.
Beyond considerations of harm, these policies are likely not going to be as effective as more radical policy moves, such as for instance imposing a tax on beef or restrictions to its market availability. Although proposals for food additives appear to target important climate change targets, they are in fact more likely lip service and non-tractable.
Tracking your own consumption
One of the most widely popularised ideas in behavioural economics is that seeing your own consumption levels (e.g. for energy) will help you decrease them, pioneered by Robert Cialdini. This idea has had very wide take-up, such as through a popular Ted Talk given by David Cameron, who described it as “the best way to get someone to cut their electricity bill”.
This has become a common strategy for carbon footprint tracking, with key stakeholders like BP (the world’s second largest non-state-owned oil company) who have aggressively promoted the use and wide adoption of personal carbon footprint trackers and calculators. Through a targeted campaign in the early 2000s, they advocated for such individual calculators. This type of frame-shifting exercise encourages people to think of their own individual responsibility rather than the responsibility of companies, governments, and systems. Framing can become normative and a serious challenge to paradigm shifting and needed transformation.
Interim summary
A significant body of research, funding, and international aid resources are dedicated to proximal issues rather than addressing the core problems (e.g. providing aid to the extremely poor but not addressing the roots of poverty with the aims to eradicate it; climate targeting programs that cannot meaningfully be brought to scale but are used to mark off compliance with targets), many efforts are most likely very ineffective policies (e.g. individual level nudging).
Current incentive structures likely prohibit scholars to carry out work that has paradigm shifting capacity. External funding and incentives could address this.
Cost-effectiveness is difficult to assess for many paradigm-shifting interventions and may not make sense to do so through traditional forms of research such as randomised controlled trials (e.g. not feasible to randomize state-wide/system level policies)
Traditional cost-effectiveness estimates commonly target interventions that are not globally impactful and continue to shift our attention to programs and interventions that will not solve problems.
It may not be clear that all “interventions” (e.g. shifts of social norms, changes in policy) are interventions and what exactly is the critical component (e.g. most direct causal driver of impact, tipping point for change, etc.)
Examples of successful paradigm shifts
Eradication of smallpox
Smallpox existed and plagued humanity for millenia, with some of the earliest records of this disease dating from mummies from the 3rd century BCE. This infectious disease induced fever, vomiting and led to the formation of ulcers and rash covering the entire body. Smallpox was characterised by a high mortality rate, leading to death in about 30% of cases (higher rates in babies and infants). Survivors had scarring of their skin and many were left blind. (CDC)
Smallpox was a large-scale problem, estimated to have killed about 300 million people in the 20th century alone. Donald Henderson estimates that in the last 100 years of its existence, smallpox had killed “at least half a billion people” (that is, 5 million annual deaths on average).
Early smallpox vaccination programs began in the 19th century – such as Bavaria, which was the first country to implement a compulsory vaccination program as of 1807, as well as the States where the U.S. Congress passed the Vaccine Act of 1813 which paved the way for American to receive the vaccine.
In high income countries, vaccination programs continued throughout the 19th century and into the 20th century. By the 1900s many countries had completely eliminated smallpox or otherwise had decreased it to very low prevalence rates. By the half-point of the 20th century, global health and humanitarian programs were rolled out. While these had a positive impact (the 1950s Pan American Health Organisation helped eliminate smallpox in a number of countries in South America), progress was slow.
The start of the paradigm shift came in 1958 with the Ukrainian epidemiologist and virologist Professor Viktor Zhadonov (then Minister of Health for the USSR) who appealed to the World Health Assembly to make a commitment and undertake a global initiative to eradicate smallpox.
Unlike previous programs that largely provided vaccination, aiming to reach low community levels of smallpox, the target of eradication was more ambitious and more challenging. This work required careful case reporting and monitoring, isolation of close contacts and containment (a process known as “ring vaccination”). It required large-scale participation and funding, and remained a significant challenge in the late 50s and early 60s.
Progress was slow until the second wave of this paradigm shift came: Dr. Donald Henderson, an American medical doctor and epidemiologist, formed the Smallpox Eradication Unit in 1966 and championed the cause of smallpox eradication, leading to the World Health Organisation further commiting to this work and increasing their efforts and funding. The global eradication of smallpox was confirmed in December 1979.
Work from the CDC estimates that the cost of eradication work from 1967 to 1979 was approximately $300 million dollars. International donations provided $98 million and countries where smallpox was endemic gave about $200 million. WHO had an annual commitment of about $2.4 million.
More concretely, America was the largest individual contributor, providing approximately $23 million in total. The CDC further estimates that the USA recoups the total of all of its financial support every 26 days because it does not have to vaccinate or treat the diseases. This is evidence that eradicating smallpox not only saved lives but was cost-effective.
Considerations from the case of smallpox eradication
There are a number of important further questions to consider.
Shifting from vaccination to eradication was a radical change and it was not initially successfully picked up solely based on the efforts of Professor Zhadnov. It seems that successful paradigm shifts might require further support, over different individuals, time and space (such as by Dr. Henderson in the case of smallpox). Further, in order for change to actually happen, and paradigm shifts to occur, there are likely a number of logistical, operational, and contextual factors that have to be considered and overcome. Understanding this should be a research priority, as it can allow significant positive changes.
Many programs currently focus on treatment or small-scale prevention (e.g. deworming, anti-malarial net provisions) rather than eradication or, more broady, reimagining the aims of global health and the reach and purposes of humanitarian aid. There might be under-researched near targets for eradication or radical improvements which are not currently well understood or considered.
It may be more difficult to estimate cost-effectiveness on paradigm shifts because costs and implementation may be unclear or difficult to estimate. It may be advisable to consider a variety of range estimates but nevertheless consider making more distributed portfolio investments, where donors (e.g. Open Philanthropy) accept supporting a number of “high risk, high payoff” proposals.
Paradigm shifts in the field of global health may be particularly important not only to reach near targets (such as The Sustainable development Goals) but also with a view to the long termist future and biosecurity hazards.
Social Movements and Their Paradigm Shifts
Feminist activists were first to imagine more concretely what a more equal society could look like, including legal protections, and bring about a paradigm shift for equality. Until the 1800s, most legal systems around the world accepted that a husband had the right to beat his wife and that this was an extension of his overall rights to control the resources available to his wife. In many places this was a de facto assumption, though in others it was explicitly acknowledged, such as the State v. Bradley case which established a positive right to wife-beating (Mississippi Supreme Court, 1824). It was the so called first-wave of feminism in the 19th century that initiated a change, first in the imagination of society, and then legally. In the later half of the 19th century, laws began to emerge that afforded women some protections (such as the UK Matrimonial Causes Act from 1878, allowing British women to seek legal separation from abusive husbands). This later paved the way to changing the legal stance on husband’s previous rights to beat their wives, and, much later, to ensure added protections against domestic violence as well as coercive emotional, financial, and physical control for any partner regardless of sex.
Abolitionism was also a paradigm shift: for the majority of our history, slavery was widely accepted, if not normative. American and English Quackers in the late 18th century, such as James Oglethorpe, were amongst the first to critically oppose the Atlantic slave trade. Although the anti-slavery movement kept growing increasingly popular, the eradication of slavery remained challenging for a prolonged period of time. It was only in 1958 when the Universal Declaration of Human Rights declared slavery to be illegal. Mauritania declared slavery illegal in 1981, becoming the last country to do so.
The civil rights movement (~1954-1968) aimed to abolish institutional racial segregation, racial discrimination and inequalities in the United States. After long periods of slavery, racial discrimination, lynchings, segregational, civil rights activists, through non-violent protests and grassroots campaigns also brought about a paradigm shift. What we consider nowadays to be commonly accepted facts, like the lack of bus segregation or the possibility of interracial marriage, came to be realized and enshrined in law thanks to this movement.
Trade unions have advanced protections for workers, such as two-day weekendays, minimum pay security, benefits, and protections against hazardous working conditions. Many of these changes happened in the latter part of the 19th century and marked a significant shift in employment conditions.
Recap of Challenges & Benefits
Potential challenges for paradigm shifts
It may be more difficult to accept radical, novel, or fringe ideas. See previously Kelsey Piper’s write up on fringe ideas. While Kelsey’s post advocates acceptance of such ideas, this post actively promotes them as necessary. Not all novel ideas will be highly impactful and not all will be translated into policy, but some may have the potential to bring about important paradigm shifts.
Successful implementation of proposed ideas is needed for their transformation into paradigm shifts. This likely requires a number of people agreeing and supporting the ideas in the first instance.
Translation into policy will be challenging, and likely require successful multidimensional cooperation, logistics, organisation, lobbying.
Currently academic and social incentive structures and groupthink may limit our capacity to produce paradigm shifts.
Potential benefits of social paradigms
Paradigm shifts can bring about large scale positive societal and global changes
Paradigm shifts can bring more equal restructuring of society, more rights to humans and animals, safety, healthcare, wellbeing
Paradigm shifts can be important for achieving key near-term (e.g. eradication of poverty) and long-term goals (e.g. space exploration)
Paradigm shifts can speed up progress in high importance areas (e.g. existential and suffering risks)
Conclusion
How does systemic change occur? It happens through paradigm shifts, through radically reimagining how things are and how they could be, through spreading ideas which transform society. Looking back at the last century in the Western world, social movements, like feminism, have not only afforded women suffrage, they have also transformed their lived experience. Domestic violence and marital rape were mostly invisible and tacitly acceptable; feminism exposed them. Dramatic improvements in our collective wellbeing, such as the abolitionist and civil rights movement, the eradication of smallpox happen through the championing of radical ideas and social revolutions. As historian Rebecca Solnit has written8, “[i]magination wields great power” in the social realm. Imagination, however, refuses measurement, and so poses a challenge to estimating the cost-effectiveness of paradigm shifting interventions. In this piece, we have advocated for moving being cost-effective as a sole criterion for intervention assessment. Instead, we have proposed we consider “high risk, high payoff” investments such as promoting paradigm shifts, which can radically alter the course of humanity and the structure of our societies for the better.
What should Open Philanthropy then consider funding?
Paradigm shifting work – or work that has the potential to be such
Research advancing our understanding of society and how it can be transformed, including feminist philosophy, sociology, and anthropology
Advocacy and activism, including groups that advance rights for marginalised communities or promote systemic change, like the Fridays for Future movement
Political movements and lobbying, to enact systemic changes
Further fellowship opportunities, like The Century Fellowship, targeting paradigm shifting work
A research agenda into paradigm shifts, including research
a) summarising successful paradigm shifts,
b) understanding how they came to be (e.g. individual capacities, thinking styles, contextual demands,
c) attempting cost-benefit analysis,
d) evaluating the factors that related to their success (e.g. context, group composition, sociocultural and temporal characteristics).
Broadly, more opportunities (research, prizes, competitions) that encourage “high risk, high payoff” work addressing global challenges in a paradigm shifting perspective
At the lower end, this could include examples like this competition and reviews of effective altruism ideas and positions with a paradigm shifting lens (e.g. looking at the ‘most important century’ with a perspective of how can paradigm shifts ensure a fair and positive societal development for all and where are they most needed)
Key References About Paradigm Shifts
1. Kuhn TS. The structure of scientific revolutions. Vol. 111. Chicago University of Chicago Press; 1970.
2. Graeber D. Revolutions in reverse. Minor Compositions; 2011.
3. Srinivasan A. The right to sex. Bloomsbury Publishing; 2021.
4. Deaton A. Randomization in the Tropics Revisited: a Theme and Eleven Variations [Internet]. Cambridge, MA: National Bureau of Economic Research; 2020 Jul [cited 2022 Jul 30] p. w27600. Report No.: w27600. Available from: http://www.nber.org/papers/w27600.pdf
5. Deaton A, Cartwright N. Reflections on Randomized Control Trials. Soc Sci Med. 2018 Aug;210:86–90.
6. Akerlof GA. Sins of Omission and the Practice of Economics. J Econ Lit. 2020 Jun 1;58(2):405–18.
7. Chater N, Loewenstein GF. The i-Frame and the s-Frame: How Focusing on the Individual-Level Solutions Has Led Behavioral Public Policy Astray. SSRN Electron J [Internet]. 2022 [cited 2022 Jun 3]; Available from: https://www.ssrn.com/abstract=4046264
8. Solnit R. Pandora’s box and the volunteer police force. Men Explain Things Me Chic Haymarket. 2014;109–24.
In Support of Paradigm Shifting
In Support of Paradigm Shifting
Cause Exploration Essay for Open Philanthropy
(This post is written in collaboration with another author who does not have a forum account and prefers to be anonymous)
Science progresses through knowledge revolutions, and not incremental discoveries, argues the philosopher of science Thomas Kuhn1. So too does society, argue numerous thinkers, including anthropologist David Graeber2 and philosopher Amia Srinivasan3. The quality of life we can expect to enjoy today has improved vastly over the course of recorded history. If this has been primarily driven by paradigm shifts, large scale reimaginings of how the world works and society should function, we have a duty to support and foster those.
Paradigm shifting does not fall neatly within the framework of impact measurement imposed by Open Philanthropy. It is difficult, perhaps impossible, to attach a dollar value to social movements, to quantify intangible yet groundbreaking emancipation in quality-adjusted life years. How many lives were saved through reimagining gender equality and the subsequent introduction of laws for domestic violence, which largely came to be around the 1990s? How many person-hours of productivity were gained and how much wellbeing was increased after the reimagining of labour and subsequently through the successful lobbying of labour unions to have a 5 day working week?
Here, we first focus on a series of arguments detailing why it might be particularly difficult (and not always useful) to assess cost-effectiveness in conventional ways for paradigm shifting. We do this by highlighting economics research on randomised controlled trials and policy, as well research on nudging. Further, we review examples of successful paradigm shifts (smallpox and social movements) and contextualise their impact. We summarise the outlined challenges and benefits, and, finally, we suggest a possible set of funding recommendations.
Against pure ‘cost-effectiveness’ calculations for systemic change
Experiments, Randomised Controlled Trials (RCTs), and Policy
In taking a narrow, quantitative approach to the operationalization of effectiveness, we inadvertently restrict the gamut of cause prioritisation. Similar arguments have recently been wedged in developmental economics, including by economics Nobel Prize winner, Angus Deaton4,5, that methodological preferences, e.g. for randomised controlled trials, implicitly shape research and policy priorities6.
Randomised Controlled Trials (RCTs) are a popular form of experiment, common across medicine, economics, and social sciences. In many “evidence pyramids” they are the highest form of individual experiment level evidence (that is, excluding “meta” evidence like systematic reviews, meta-analyses or reviews of reviews). RCT data is also commonly used for cost-effectiveness calculations.
Deaton & Cartwright, among others, argue that there are essential caveats to RCTs that should make us cautious about treating them as the single most preferred level of evidence (what they term “magical thinking” about RCTs), such as for instance that:
a) randomisation does not balance confounders,
b) RCTs may improve the unbiasedness of estimates, but this may be of limited practical value (e.g. compared to improving precision or generalizability),
c) improvements in unbiasedness require orthogonality which is not guaranteed by randomization in and of itself,
c) treatment effects likely have asymmetric distributions that are a challenge for significance testing
Overall, this research ultimately argues that RCTs are an important part of a research agenda, that on their own they are a weak ground for inferring ‘what works’, and so that the overall agenda needed to move forward should employ a triangulation of methods that allow us to ascertain more robustly “what works” but also address “why things work”
A related line of research, again by Deaton, strongly argues that RCTs have “no unique advantages or disadvantages over other empirical methods in economics… [t]hey do not simplify inference, nor can an RCT establish causality.” More importantly, this work highlights important ethical considerations. We will focus on this next.
Ethics of experimentation
As RCTs are privileged in the evidence pyramid and often the preferred method of economists to infer cost-effectiveness, it is crucial to consider ethical implications. Ruling out other options for evidence that may pose fewer risks to participants or that perhaps might be more informative is a likely downside, particularly for those who believe in the “magical RCT thinking”. Ignoring other important information is a disservice to science and ethics.
More broadly, most RCTs interventions in the field of global health are delivered by better-educated, better-off people who have more protection and resources and live different lives than the people they target through interventions (who tend to be lower income, less-educated, less protected). Deaton, like other scholars, has questioned how many current experiments, particularly within the USA, are carried out with the interests of rich taxpayers who want to prevent the worst of poverty and want to minimise the cost of doing so.
Further, there has been discussion that institutional review boards are not a sufficient ethical clearance level. Some interventions (which had obtained American IRB approval) carried out by western researchers on extremely poor people in India likely could not have been carried out in the States. Other work provides no clear benefit to the poor, and may even harm or expose them to further risk immediately or post-intervention. Estimates of cost-effectiveness may be further ‘polluted’ by lack of true informed consent, lack of choice, and lack of acceptability/desirability of intervention.
Oftentimes, RCTs and interventions are not sensitive nor appreciative of the overall context in which they are delivered. Many times an intervention will benefit not only the citizens of a given country but the overall regime in that area. This may be particularly potent and visible in areas of armed conflict and war, where it is not possible to deliver aid without cooperation or payment to the warmongers, which may in turn be prolonging or further worsening the suffering of those meant to receive protection. Though this is likely true for many states not at war, particularly authoritarian regimes where an international aid organisation’s work will at minimum serve as political cover for political despots. It is extremely infrequent that such contextual factors are ever discussed in cost-effectiveness considerations, yet directly factored in.
Constraints for policy
To condense the above, enforcing hierarchies of evidence and privileging some types of evidence, without carefully accounting for cost, harm, context, and methodological limitations, is ineffective and potentially dangerous, both for research and policy.
Akerlof has further worked on illustrating the limitations for policy. He has argued that economics favours “hard” research—that is, research that is quantitative, causal, involving mathematical models, ideally capturing underlying concepts with precision, rather than “soft research” (qualitative research, or quantitative where models are more simplistic or explained through words, correlational work, work that does not target concepts or is imprecise).
The ‘hardness bias’, according to Akerlof, is linked to (among other pitfalls) a bias against new ideas and overspecialisation.
First, there might be a bias against new ideas because it is easier to do ‘hard’ work, perceived as more valuable, when based on existing topics and old paradigms: it is easier to improve precision in existing work, and rely on established terminologies and conceptual frameworks. Those working on truly new work have a larger load to shoulder, as they will often have to suggest entirely novel frameworks.
Second, a focus on “hard” research diminishes the ability to challenge existing paradigms. Old ideas are denounced when their effects fail to replicate or they are demonstrated to be inferior to other ideas (often time related ones, indicating an improvement in fit). At the same time, it is generally expected that ‘new ideas’ provide testable predictions. This is a widely accepted axiom of research, yet in practice this means we discard ‘softer’ tests of theories (e.g. those that consider models based on their quality, plausibility, efficiency, utility etc. and well as the quality, plausibility, efficiency, utility etc. of their conclusions). As a practical example, this might mean that for an intervention we focus on the overall effect, rather than more carefully considering attrition, mechanisms, context, sub-group effects, participant motivation, constraints, time decay etc.
Third, in terms of overspecialisation, it may be easier to carry out “hard” research as a specialist (meeting the requirements and norms of only one field) than to be a generalist (where there are requirements, norms and a wider body of literature and methods to consider and appreciate). This is not, in and of itself, immediately ‘a bad’ state of affairs. Specialists may have nuanced understanding of field-specific techniques or bodies of research. The caution here, instead, should be that some balance may be needed and if it is not found, certain specialties may be more favoured over time, and others may fall out of favour; that nuance that comes with generalist knowledge may be undervalued. Further, we should more carefully consider the implications of discipline-wide norms that seem to promote or encourage only certain kinds of science.
Overall, such a ‘hardness’ bias may make economists and other researchers more readily willing to accept existing theories, and less willing to propose novel theories. Given existing incentive structures, it is likely harder to produce radically new work that could be a paradigm shift.
As a broader and more tentative note, there might be some evidence that some ideas, considered more “heretical” or “fringe” may be more easily mistrusted both across fields and historically. See this overview from Information is Beautiful on maverick scientists.
Examples of Misused Resources and the Need for Paradigm Shifts
Nudging Research
Another recent analysis7 by prominent academics in psychology and economics, Nick Chater and George Lowenstein, argues that nudging interventions targeting individual behaviour are 1) largely ineffective, and 2) actively harmful in distracting our efforts from the necessary systemic changes. With this lens, we can view the core body of research on individual-level nudging as something that has been inefficient, costly, and that has restricted time, effort, and consideration from researchers and policy-makers that could have otherwise gone to more effective and novel ways to improve society.
Nudging research has focused the attention of researchers, policy-makers and lay-people on individual factors by framing global problems, including climate change, retirement savings, obesity and pollution, in individual rather than systemic terms. Yet, from a rational standpoint, this framing is entirely at odds with the evidence for the root of the problems (e.g. 71% of global CO2 emissions owing to 100 companies) and viable solutions (e.g. compulsory superannuation systems).
Climate Change
Climate change, within the Effective Altruism movement, is generally not considered neglected (e.g. see Summary by 80,000 Hours), however here we would like to suggest that the majority of interventions are largely not tractable nor impactful.
As discussed above, many individual-level nudging interventions are largely inefficient (null or very small effects) that cannot address systemic level issues. Rather than a systematic review of current interventions, here we illustrate general level thinking about these problems through a few concrete examples.
We note as a broader comment that the need for paradigm shifts within climate change may be beneficial for humanity’s longterm future as well: it seems, at face value, that innovation away from fossil fuels and into more sustainable energies can be helpful for equity, global wellbeing, space exploration, etc.
Cows and Seaweed
Recently, the news picked up the finding that feeding seaweed to cattle can reduce their methane emission by about 40%, this was described as a game-changing method to slow methane emissions. It was then proposed that red seaweed (high in bromoform which can reduce belching from cows that lead to added methane) becomes a food additive for cows. At face value, this may seem like a promising and innovative proposal. At the same time, what would it mean for such a proposal to be brought to scale?
Prioritising one culture of seaweed may lead us to create mono-cultures that limit biodiversity and in turn harm other animals and water quality.
Bromoform’s effects on human health are not fully established at higher values.
Beyond considerations of harm, these policies are likely not going to be as effective as more radical policy moves, such as for instance imposing a tax on beef or restrictions to its market availability. Although proposals for food additives appear to target important climate change targets, they are in fact more likely lip service and non-tractable.
Tracking your own consumption
One of the most widely popularised ideas in behavioural economics is that seeing your own consumption levels (e.g. for energy) will help you decrease them, pioneered by Robert Cialdini. This idea has had very wide take-up, such as through a popular Ted Talk given by David Cameron, who described it as “the best way to get someone to cut their electricity bill”.
This has become a common strategy for carbon footprint tracking, with key stakeholders like BP (the world’s second largest non-state-owned oil company) who have aggressively promoted the use and wide adoption of personal carbon footprint trackers and calculators. Through a targeted campaign in the early 2000s, they advocated for such individual calculators. This type of frame-shifting exercise encourages people to think of their own individual responsibility rather than the responsibility of companies, governments, and systems. Framing can become normative and a serious challenge to paradigm shifting and needed transformation.
Interim summary
A significant body of research, funding, and international aid resources are dedicated to proximal issues rather than addressing the core problems (e.g. providing aid to the extremely poor but not addressing the roots of poverty with the aims to eradicate it; climate targeting programs that cannot meaningfully be brought to scale but are used to mark off compliance with targets), many efforts are most likely very ineffective policies (e.g. individual level nudging).
Current incentive structures likely prohibit scholars to carry out work that has paradigm shifting capacity. External funding and incentives could address this.
Cost-effectiveness is difficult to assess for many paradigm-shifting interventions and may not make sense to do so through traditional forms of research such as randomised controlled trials (e.g. not feasible to randomize state-wide/system level policies)
Traditional cost-effectiveness estimates commonly target interventions that are not globally impactful and continue to shift our attention to programs and interventions that will not solve problems.
It may not be clear that all “interventions” (e.g. shifts of social norms, changes in policy) are interventions and what exactly is the critical component (e.g. most direct causal driver of impact, tipping point for change, etc.)
Examples of successful paradigm shifts
Eradication of smallpox
Smallpox existed and plagued humanity for millenia, with some of the earliest records of this disease dating from mummies from the 3rd century BCE. This infectious disease induced fever, vomiting and led to the formation of ulcers and rash covering the entire body. Smallpox was characterised by a high mortality rate, leading to death in about 30% of cases (higher rates in babies and infants). Survivors had scarring of their skin and many were left blind. (CDC)
Smallpox was a large-scale problem, estimated to have killed about 300 million people in the 20th century alone. Donald Henderson estimates that in the last 100 years of its existence, smallpox had killed “at least half a billion people” (that is, 5 million annual deaths on average).
Early smallpox vaccination programs began in the 19th century – such as Bavaria, which was the first country to implement a compulsory vaccination program as of 1807, as well as the States where the U.S. Congress passed the Vaccine Act of 1813 which paved the way for American to receive the vaccine.
In high income countries, vaccination programs continued throughout the 19th century and into the 20th century. By the 1900s many countries had completely eliminated smallpox or otherwise had decreased it to very low prevalence rates. By the half-point of the 20th century, global health and humanitarian programs were rolled out. While these had a positive impact (the 1950s Pan American Health Organisation helped eliminate smallpox in a number of countries in South America), progress was slow.
The start of the paradigm shift came in 1958 with the Ukrainian epidemiologist and virologist Professor Viktor Zhadonov (then Minister of Health for the USSR) who appealed to the World Health Assembly to make a commitment and undertake a global initiative to eradicate smallpox.
Unlike previous programs that largely provided vaccination, aiming to reach low community levels of smallpox, the target of eradication was more ambitious and more challenging. This work required careful case reporting and monitoring, isolation of close contacts and containment (a process known as “ring vaccination”). It required large-scale participation and funding, and remained a significant challenge in the late 50s and early 60s.
Progress was slow until the second wave of this paradigm shift came: Dr. Donald Henderson, an American medical doctor and epidemiologist, formed the Smallpox Eradication Unit in 1966 and championed the cause of smallpox eradication, leading to the World Health Organisation further commiting to this work and increasing their efforts and funding. The global eradication of smallpox was confirmed in December 1979.
Work from the CDC estimates that the cost of eradication work from 1967 to 1979 was approximately $300 million dollars. International donations provided $98 million and countries where smallpox was endemic gave about $200 million. WHO had an annual commitment of about $2.4 million.
More concretely, America was the largest individual contributor, providing approximately $23 million in total. The CDC further estimates that the USA recoups the total of all of its financial support every 26 days because it does not have to vaccinate or treat the diseases. This is evidence that eradicating smallpox not only saved lives but was cost-effective.
Considerations from the case of smallpox eradication
There are a number of important further questions to consider.
Shifting from vaccination to eradication was a radical change and it was not initially successfully picked up solely based on the efforts of Professor Zhadnov. It seems that successful paradigm shifts might require further support, over different individuals, time and space (such as by Dr. Henderson in the case of smallpox). Further, in order for change to actually happen, and paradigm shifts to occur, there are likely a number of logistical, operational, and contextual factors that have to be considered and overcome. Understanding this should be a research priority, as it can allow significant positive changes.
Many programs currently focus on treatment or small-scale prevention (e.g. deworming, anti-malarial net provisions) rather than eradication or, more broady, reimagining the aims of global health and the reach and purposes of humanitarian aid. There might be under-researched near targets for eradication or radical improvements which are not currently well understood or considered.
It may be more difficult to estimate cost-effectiveness on paradigm shifts because costs and implementation may be unclear or difficult to estimate. It may be advisable to consider a variety of range estimates but nevertheless consider making more distributed portfolio investments, where donors (e.g. Open Philanthropy) accept supporting a number of “high risk, high payoff” proposals.
Paradigm shifts in the field of global health may be particularly important not only to reach near targets (such as The Sustainable development Goals) but also with a view to the long termist future and biosecurity hazards.
Social Movements and Their Paradigm Shifts
Feminist activists were first to imagine more concretely what a more equal society could look like, including legal protections, and bring about a paradigm shift for equality. Until the 1800s, most legal systems around the world accepted that a husband had the right to beat his wife and that this was an extension of his overall rights to control the resources available to his wife. In many places this was a de facto assumption, though in others it was explicitly acknowledged, such as the State v. Bradley case which established a positive right to wife-beating (Mississippi Supreme Court, 1824). It was the so called first-wave of feminism in the 19th century that initiated a change, first in the imagination of society, and then legally. In the later half of the 19th century, laws began to emerge that afforded women some protections (such as the UK Matrimonial Causes Act from 1878, allowing British women to seek legal separation from abusive husbands). This later paved the way to changing the legal stance on husband’s previous rights to beat their wives, and, much later, to ensure added protections against domestic violence as well as coercive emotional, financial, and physical control for any partner regardless of sex.
Abolitionism was also a paradigm shift: for the majority of our history, slavery was widely accepted, if not normative. American and English Quackers in the late 18th century, such as James Oglethorpe, were amongst the first to critically oppose the Atlantic slave trade. Although the anti-slavery movement kept growing increasingly popular, the eradication of slavery remained challenging for a prolonged period of time. It was only in 1958 when the Universal Declaration of Human Rights declared slavery to be illegal. Mauritania declared slavery illegal in 1981, becoming the last country to do so.
The civil rights movement (~1954-1968) aimed to abolish institutional racial segregation, racial discrimination and inequalities in the United States. After long periods of slavery, racial discrimination, lynchings, segregational, civil rights activists, through non-violent protests and grassroots campaigns also brought about a paradigm shift. What we consider nowadays to be commonly accepted facts, like the lack of bus segregation or the possibility of interracial marriage, came to be realized and enshrined in law thanks to this movement.
Trade unions have advanced protections for workers, such as two-day weekendays, minimum pay security, benefits, and protections against hazardous working conditions. Many of these changes happened in the latter part of the 19th century and marked a significant shift in employment conditions.
Recap of Challenges & Benefits
Potential challenges for paradigm shifts
It may be more difficult to accept radical, novel, or fringe ideas. See previously Kelsey Piper’s write up on fringe ideas. While Kelsey’s post advocates acceptance of such ideas, this post actively promotes them as necessary. Not all novel ideas will be highly impactful and not all will be translated into policy, but some may have the potential to bring about important paradigm shifts.
Successful implementation of proposed ideas is needed for their transformation into paradigm shifts. This likely requires a number of people agreeing and supporting the ideas in the first instance.
Translation into policy will be challenging, and likely require successful multidimensional cooperation, logistics, organisation, lobbying.
Currently academic and social incentive structures and groupthink may limit our capacity to produce paradigm shifts.
Potential benefits of social paradigms
Paradigm shifts can bring about large scale positive societal and global changes
Paradigm shifts can bring more equal restructuring of society, more rights to humans and animals, safety, healthcare, wellbeing
Paradigm shifts can be important for achieving key near-term (e.g. eradication of poverty) and long-term goals (e.g. space exploration)
Paradigm shifts can speed up progress in high importance areas (e.g. existential and suffering risks)
Conclusion
How does systemic change occur? It happens through paradigm shifts, through radically reimagining how things are and how they could be, through spreading ideas which transform society. Looking back at the last century in the Western world, social movements, like feminism, have not only afforded women suffrage, they have also transformed their lived experience. Domestic violence and marital rape were mostly invisible and tacitly acceptable; feminism exposed them. Dramatic improvements in our collective wellbeing, such as the abolitionist and civil rights movement, the eradication of smallpox happen through the championing of radical ideas and social revolutions. As historian Rebecca Solnit has written8, “[i]magination wields great power” in the social realm. Imagination, however, refuses measurement, and so poses a challenge to estimating the cost-effectiveness of paradigm shifting interventions. In this piece, we have advocated for moving being cost-effective as a sole criterion for intervention assessment. Instead, we have proposed we consider “high risk, high payoff” investments such as promoting paradigm shifts, which can radically alter the course of humanity and the structure of our societies for the better.
What should Open Philanthropy then consider funding?
Paradigm shifting work – or work that has the potential to be such
Research advancing our understanding of society and how it can be transformed, including feminist philosophy, sociology, and anthropology
Advocacy and activism, including groups that advance rights for marginalised communities or promote systemic change, like the Fridays for Future movement
Political movements and lobbying, to enact systemic changes
Further fellowship opportunities, like The Century Fellowship, targeting paradigm shifting work
A research agenda into paradigm shifts, including research
a) summarising successful paradigm shifts,
b) understanding how they came to be (e.g. individual capacities, thinking styles, contextual demands,
c) attempting cost-benefit analysis,
d) evaluating the factors that related to their success (e.g. context, group composition, sociocultural and temporal characteristics).
Broadly, more opportunities (research, prizes, competitions) that encourage “high risk, high payoff” work addressing global challenges in a paradigm shifting perspective
At the lower end, this could include examples like this competition and reviews of effective altruism ideas and positions with a paradigm shifting lens (e.g. looking at the ‘most important century’ with a perspective of how can paradigm shifts ensure a fair and positive societal development for all and where are they most needed)
Key References About Paradigm Shifts
1. Kuhn TS. The structure of scientific revolutions. Vol. 111. Chicago University of Chicago Press; 1970.
2. Graeber D. Revolutions in reverse. Minor Compositions; 2011.
3. Srinivasan A. The right to sex. Bloomsbury Publishing; 2021.
4. Deaton A. Randomization in the Tropics Revisited: a Theme and Eleven Variations [Internet]. Cambridge, MA: National Bureau of Economic Research; 2020 Jul [cited 2022 Jul 30] p. w27600. Report No.: w27600. Available from: http://www.nber.org/papers/w27600.pdf
5. Deaton A, Cartwright N. Reflections on Randomized Control Trials. Soc Sci Med. 2018 Aug;210:86–90.
6. Akerlof GA. Sins of Omission and the Practice of Economics. J Econ Lit. 2020 Jun 1;58(2):405–18.
7. Chater N, Loewenstein GF. The i-Frame and the s-Frame: How Focusing on the Individual-Level Solutions Has Led Behavioral Public Policy Astray. SSRN Electron J [Internet]. 2022 [cited 2022 Jun 3]; Available from: https://www.ssrn.com/abstract=4046264
8. Solnit R. Pandora’s box and the volunteer police force. Men Explain Things Me Chic Haymarket. 2014;109–24.