I am a researcher at Rethink Priorities’ Worldview Investigations Team. I also do work for Oxford’s Global Priorities Institute. Previously I was a research analyst at the Forethought Foundation for Global Priorities Research. I took the role after completing the MPhil in Economics at Oxford University. Before that, I studied Mathematics and Philosophy at the University of St Andrews.
Find out more about me here.
Hi Oscar, thanks. Yes! Indicator evidence is inserted at the bottom and flows upward. The diagram priors flows down to set expectations, so it’s a common convention to draw it so. Informally you see it as each node splitting different nodes into subnodes, but the arrows don’t mean to imply that information doesn’t travel up.
On to your main point: there’s no question that this is a preparadigmatic field where progress and consensus are difficult to find, and rightly so given the state of the evidence.
A few thoughts on why pursue this research now despite the uncertainty:
First, we want a framework in place that can help transition towards a more paradigmatic science of digital minds as the field progresses. Even if you’re sceptical about current reliability, we think that having a model that can modularly incorporate new evidence and judgements could serve as valuable infrastructure for organising future findings.
Second, whilst which theory is true remains uncertain, we found that operationalising specific stances was often less fuzzy than expected. Many theories make fairly specific predictions about conscious versus non-conscious systems, giving us firmer ground for stance-by-stance analysis. (Though you might still disagree with any given stance, of course!)
Your concern about theory quality might itself be worth modelling as a stance—we thought of something like “Stance X” but it could be “Theoretical Scepticism”—where all features provide very weak support. That would yield small updates from the prior regardless of the system. Already, you can see the uncertainty playing out in the model: notice how wide the bands are for chickens vs humans (Figure 3). Several theories disagree substantially about animal consciousness, which arguably reflects concerns about theory quality.
That said, we’re deliberately cautious about making pronouncements on consciousness probability or evidence strength. We see this as a promising way to start characterising these values rather than offering definitive answers.
(Oh and regarding priors: setting them is hard, and robustness important. it might be helpful to see Appendix E and Figures 9-11. The key finding is that while absolute posteriors are highly prior-dependent (as you’d expect), the comparative results and direction of updating are pretty robust across priors.)
Hi Andrew, I definitely agree. Tracking how these systems evolve over time — both what the models reveal about their changes and how we can formulate projections based on those insights — has been one of our main motivations for this work. We’re in fact hoping to gather data on newer systems and develop some more informed forecasts based on what we find.
Thank you for your comment Kuhanj. I share your belief that the EA movement would benefit from the type of suggestions you outlined on your quick take. I particularly valued seeing more discussions on heuristics, for they are often as limited as they are useful!
Regarding your ‘Being slow to re-orient’ suggestion, an important nuance comes to mind: movements can equally falter by pivoting too rapidly. When a community glimpses promise in a new X direction, there’s a risk of hastily redirecting significant resources, infrastructure, and attention toward it prematurely. The wisdom accumulated through longer reflection and careful evidence collection often contains (at least some) genuine insight, and we should be cautious about abandoning established priorities to chase every emerging “crucial consideration” that surfaces.
As ever, the challenge lies in finding that delicate balance between responsiveness and steadfastness — being neither calcified in thinking nor swept away by every new intellectual current.
Thanks for your comment Richard, I think the discussion is better for it. I agree with your clarification that there are key differences that distinguish EA from more traditional attitudes and that defending cause incommensurability and personal taste are two relevant dimensions.
Like you, it does seem to us that in the early days of EA, many people doing prioritisation of GHD interventions went beyond traditional intervention clusters (e.g. education) and did some cross-cause prioritisation (identifying the best interventions simpliciter).
That said, the times feel different now and we think that, increasingly, people are doing within-cause prioritisation by only trying to identify the best interventions within a given area without it being clearly ‘done in service of the ultimate goal of “cross-cause prioritization”’ (e.g. because they are working for an institution or project with funds dedicated exclusively to be allocated within a certain cause).
Thank you Jack, yes it’s absolutely about acting on it too, research is just step 1.
If you’ve found the ‘snapshot of EA’ section particularly valuable, please flag it under this comment so we can gauge how much we should invest in updating it or expanding it in the future. To clarify:
- Vote agree for “particularly valuable”.
- Vote disagree for “not that valuable”.
Feel free to add any comments.
I agree with you Oscar, and we’ve highlighted this in the summary table where I borrowed your ‘contrasting project preferences’ terminology. Still, I think it could still be worth drawing the conceptual distinctions because it might help identify places where bargains can occur.
I liked your example too! We tried to add a few (GCR-focused agent believes AI advances are imminent, while a GHD agent is skeptical; AI safety view borrows resources from a Global Health to fund urgent AI research; meat-eater; gun rights and another supporting gun control both fund a neutral charity like Oxfam...) but we could have done better in highlighting them. I’ve also added these to the table.
I found your last mathematical note a bit confusing because I originally read A,B,C as projects they might each support. But if it’s outcomes (i.e. pairs of projects they would each support), then I think I’m with you!
Just to flag that Derek posted on this very recently. It’s directly connected to both the present post and Michael’s.
Thanks vasco. And thanks for helping us think through what we can do better. Some thoughts on this:
We considered several framings, scales and options to give experts. Since they were evaluating a lot of stances and we wanted experts to really know what we meant, we prioritised giving them context and then asking them the simplified general question of plausibility, with an intuitive scale. The exact question was: ‘how plausible do you find X stance?’, just after having fully describing X. We also asked them for general notes and comments and they didn’t seem to find that part of the survey particularly confusing (perhaps to your and my surprise). More broadly, I agree with you that sometimes perfectly defining terms and scales can help some people think through it but not everyone, and the science on how much it helps points is mixed.
We didn’t find that people were responding with zero plausibility very much at all. As you can see from the results, almost all respondents found most, if not all, stances at least a little bit plausible. I agree that had we found a lot of concentration around the very high or very low plausibility, having some sort of logarithmic scale could help distinguish results.
I’m not sure what you have in mind in terms of modelling the stances’ weight as distributions instead of point estimates. Perhaps you mean something like leveraging those distributions above via some sort of Monte Carlo where weights are drawn from these distributions and the process is repeated many times, then aggregated. That indeed sounds more sophisticated and could possibly help track uncertainty but I suspect it would very little difference. In particular, I think so because we observed that unweighted pooling of results across all stances is surprisingly similar to the pool when weighted by experts; the same if you squint.