What prevents researchers from prioritising x-risk?
Questions
This proposal aims to answer the following questions:
What are researchers’ existing beliefs about existential risks?
How are their actions concerning existential risk mitigation dependent on their beliefs?
What factors might explain researchers not prioritising existential risk?
Background
Most longtermist EA-inspired organisations advocate for and support research on existential risk (among other topics). They do so, presumably, in the hope that, by providing information and resources to researchers, researchers’ views and efforts will shift in favour of more impactful topics.
Yet, little is known about several factors that appear critical to this theory of change. I am not aware of work concerning:
Researchers’ existing beliefs about the prevalence of and relative concern caused by existential risks.
The extent to which providing information and research support related to existential risk affect researchers’ beliefs and downstream actions.
The barriers to researchers’ prioritising work related to existential risk.
(That said, I am sure that research exists on questions broadly analogous to those above.)
Methods
I first need some context with many researchers. I do not yet have a clear idea of what form this context might take. Academic conferences seem most fitting, but are not practical in the short term. The obvious alternative is to conduct surveys over the internet following personal emails; in this case, one concern is that selection in to and out of the data could be a significant issue.
Regardless of the exact context chosen, I will first design surveys to characterise beliefs about existential risk. In eliciting beliefs, some or all participants could be incentivised to answer with estimates as close to expert opinion as possible. (This is standard procedure in economics, although usually one is trying to elicit a participant’s best guess of something closer to ‘ground truth’.) The survey might also ask participants to estimate small but mundane risks—e.g. the probability that a randomly chosen person might be struck by lightning that day—so that survey responses can be more easily compared, and perhaps even filtered or reweighted.
I will also estimate participant’s preferences over trade-offs related to existential risk. This part of the survey might ask, for example: how many statistical lives saved with certainty is equivalently good to a 1% reduction in existential risk next year? (See slides 20-27 on Alsan et al. (2020)).
Next, I envision a field experiment with researchers. I will test the degree to which researchers (1) value information about existential risk and (2) respond to evidence and support in their beliefs, preferences, and actions relating to existential risk. This “evidence and support” might take the form of risk estimates from Ord (2020), and suggestions from research priorities organisations on how to reorient research towards existential risk reduction.
In the experiment, I will first measure participants’ willingness to pay (WTP) for evidence and support. Second, I will evaluate participants’ response to receiving evidence and support on (a) survey outcomes, immediately after receiving information and again some years in the future, (b) measurable non-survey outcomes. I have yet to settle on which outcomes to include in (b), but suggestions include: quantity of articles produced relating to existential risk, donations to charities relevant to existential risk, and so on.
Finally, upon analysing the data, I will explore mechanisms which might play a role in successfully shifting researchers’ focus towards existential risk. This process may include conducting follow-up experiments.
Uncertainties
I welcome feedback on anything/everything above, but some uncertainties that immediately stand out to me:
Which contexts might be most appropriate and amenable for experiments?
Which mechanisms seem most likely to be important ahead of time? (Such that I could include considerations for them in the main experiment.)
Which other trade-off questions might I include? I want this section to get at: if existential risks are made more salient and/or made to appear more tractable, how do perceptions of trade-offs (to be related to research inputs) change? But maybe there is a clearer question to be asked here.
How might my big-picture questions be misspecified? Relatedly, which big-picture questions might be more interesting and might therefore lead to different experiments/analysis?
Are you just focused on any and all researchers, or researchers in some particular set of fields? Perhaps the fields most relevant to existential risk (e.g., AI, biotech, international relations)?
This “evidence and support” might take the form of risk estimates from Ord (2020)
I think there’s some value in updating one’s beliefs based on experts’ existential risk estimates, such as those from Ord. But I’d also worry about the possibility of anchoring and/or information cascades if other researchers—who might have been able to come up with their own reasonable estimates if they tried—are actively encouraged to adjust their beliefs based on existing estimates. I’d also be wary of heavily relying on any particular estimate or set of estimates, without checking what other experts said about the same topic.
So it might be useful for you to draw on the existential risk estimates in this this database I made, and also to just keep in mind the risks of anchoring and information cascades and try to find ways to mitigate those issues. (I’ll be discussing these topics more in a lightning talk at EAGx, and maybe in an Unconference session too.)
In eliciting beliefs, some or all participants could be incentivised to answer with estimates as close to expert opinion as possible. (This is standard procedure in economics, although usually one is trying to elicit a participant’s best guess of something closer to ‘ground truth’.)
At first, I thought you meant telling the researchers what experts thought, and then incentivising the researchers to say the same. I felt unsure what the point of that would be. But now I’m guessing you mean something like telling them they’ll get some incentive if the estimate they come up with is close to experts’ estimates, to encourage them to think hard? If so, what’s the goal from that? I could imagine this leading to researchers giving relatively high estimates because they expect x-risk experts would do so, rather than it leading to researchers thinking really hard about what they themselves should believe.
Finally, it seems possible that the “crucial questions for longtermists” project I’m working on might be relevant in some way for your idea. For example, perhaps that could provide some inspiration regarding things to ask the researchers about, and regarding what may be underlying the differences in the strategic views and choices of them vs of the existential risk community.
What prevents researchers from prioritising x-risk?
Questions
This proposal aims to answer the following questions:
What are researchers’ existing beliefs about existential risks?
How are their actions concerning existential risk mitigation dependent on their beliefs?
What factors might explain researchers not prioritising existential risk?
Background
Most longtermist EA-inspired organisations advocate for and support research on existential risk (among other topics). They do so, presumably, in the hope that, by providing information and resources to researchers, researchers’ views and efforts will shift in favour of more impactful topics.
Yet, little is known about several factors that appear critical to this theory of change. I am not aware of work concerning:
Researchers’ existing beliefs about the prevalence of and relative concern caused by existential risks.
The extent to which providing information and research support related to existential risk affect researchers’ beliefs and downstream actions.
The barriers to researchers’ prioritising work related to existential risk.
(That said, I am sure that research exists on questions broadly analogous to those above.)
Methods
I first need some context with many researchers. I do not yet have a clear idea of what form this context might take. Academic conferences seem most fitting, but are not practical in the short term. The obvious alternative is to conduct surveys over the internet following personal emails; in this case, one concern is that selection in to and out of the data could be a significant issue.
Regardless of the exact context chosen, I will first design surveys to characterise beliefs about existential risk. In eliciting beliefs, some or all participants could be incentivised to answer with estimates as close to expert opinion as possible. (This is standard procedure in economics, although usually one is trying to elicit a participant’s best guess of something closer to ‘ground truth’.) The survey might also ask participants to estimate small but mundane risks—e.g. the probability that a randomly chosen person might be struck by lightning that day—so that survey responses can be more easily compared, and perhaps even filtered or reweighted.
I will also estimate participant’s preferences over trade-offs related to existential risk. This part of the survey might ask, for example: how many statistical lives saved with certainty is equivalently good to a 1% reduction in existential risk next year? (See slides 20-27 on Alsan et al. (2020)).
Next, I envision a field experiment with researchers. I will test the degree to which researchers (1) value information about existential risk and (2) respond to evidence and support in their beliefs, preferences, and actions relating to existential risk. This “evidence and support” might take the form of risk estimates from Ord (2020), and suggestions from research priorities organisations on how to reorient research towards existential risk reduction.
In the experiment, I will first measure participants’ willingness to pay (WTP) for evidence and support. Second, I will evaluate participants’ response to receiving evidence and support on (a) survey outcomes, immediately after receiving information and again some years in the future, (b) measurable non-survey outcomes. I have yet to settle on which outcomes to include in (b), but suggestions include: quantity of articles produced relating to existential risk, donations to charities relevant to existential risk, and so on.
Finally, upon analysing the data, I will explore mechanisms which might play a role in successfully shifting researchers’ focus towards existential risk. This process may include conducting follow-up experiments.
Uncertainties
I welcome feedback on anything/everything above, but some uncertainties that immediately stand out to me:
Which contexts might be most appropriate and amenable for experiments?
Which mechanisms seem most likely to be important ahead of time? (Such that I could include considerations for them in the main experiment.)
Which other trade-off questions might I include? I want this section to get at: if existential risks are made more salient and/or made to appear more tractable, how do perceptions of trade-offs (to be related to research inputs) change? But maybe there is a clearer question to be asked here.
How might my big-picture questions be misspecified? Relatedly, which big-picture questions might be more interesting and might therefore lead to different experiments/analysis?
Session
EDIT: Happy with either session!
Sounds interesting. A few questions/thoughts:
Are you just focused on any and all researchers, or researchers in some particular set of fields? Perhaps the fields most relevant to existential risk (e.g., AI, biotech, international relations)?
I think there’s some value in updating one’s beliefs based on experts’ existential risk estimates, such as those from Ord. But I’d also worry about the possibility of anchoring and/or information cascades if other researchers—who might have been able to come up with their own reasonable estimates if they tried—are actively encouraged to adjust their beliefs based on existing estimates. I’d also be wary of heavily relying on any particular estimate or set of estimates, without checking what other experts said about the same topic.
So it might be useful for you to draw on the existential risk estimates in this this database I made, and also to just keep in mind the risks of anchoring and information cascades and try to find ways to mitigate those issues. (I’ll be discussing these topics more in a lightning talk at EAGx, and maybe in an Unconference session too.)
At first, I thought you meant telling the researchers what experts thought, and then incentivising the researchers to say the same. I felt unsure what the point of that would be. But now I’m guessing you mean something like telling them they’ll get some incentive if the estimate they come up with is close to experts’ estimates, to encourage them to think hard? If so, what’s the goal from that? I could imagine this leading to researchers giving relatively high estimates because they expect x-risk experts would do so, rather than it leading to researchers thinking really hard about what they themselves should believe.
Finally, it seems possible that the “crucial questions for longtermists” project I’m working on might be relevant in some way for your idea. For example, perhaps that could provide some inspiration regarding things to ask the researchers about, and regarding what may be underlying the differences in the strategic views and choices of them vs of the existential risk community.