Thanks Gideon and others for creating this, I’m glad it exists. I have only skimmed the other comments, so apologies if the following have already been discussed elsewhere. In my mind it is useful to separate two quite different sentiments, I’d be interested if others agree with/find useful this distinction:
We would prefer a world where demographic markers (perhaps most notably nationality) are not predictive of ability to participate in intellectual communities and knowledge-making.
Given the current very unequal and unjust world, we would prefer to fund/hire/support/etc people who do worse in our evaluation metrics (interviews, work trials, research proposals, etc) but are from more underrepresented backgrounds.
Both are in favour of greater pluralism and diversity. I think 1 is trivially true—of course a more equal and just world would be better (though it would be a further claim that this is something X-risk orgs should focus on). 2 I am more torn about, I think I more disagree than agree though. I think if two people, one from a very intellectually privileged background, and one less so, do equally well on our evaluation metrics, the latter person is likely to be more intelligent/creative/novel as they had a harder road to there and needed to do more for themselves. I also think though that it would be pretty bad (and often clearly illegal) for decision-makers to use demographic markers over and above evaluation metrics.
I think the “In Defense of Merit in Science” paper is relevant here, though I haven’t read more than the abstract and the public discussions of it. I am glad the statement talks about tradeoffs. I think this is another important tradeoff to add: the more dimensions you are optimising over, the less well you can optimise each of them. This seems uncontroversially true. The implication of this is that if we add more demographic dimensions to optimise over this will have a cost on whatever other dimensions we may already be using (novelty of ideas, clarity of writing, mathematical ability, etc). I think for me this is the most important tradeoff.
I’m not quite sure how to address most of your comment, as I think in many ways we are critiquing much of the underlying logic of how we evaluate what we ought to be funded. Its essentially suggesting, not really from a place of justice at all, that the current structures fail to optimise or assess the things that are useful for the community. And I think it may also be implicitly suggesting that centralising this power as is done currently is counter to the aims, and so power in the community ought to be better distributed. One response would be to change these criteria to better optimise for things that we may think are more valuable eg novelty of ideas/approach. Another would be to randomise funding ie every proposal that crosses a certain bar gets entered into a lottery. Ultimately, I’m (not necessarily the other signatories)pretty comfortable making the argument for this on purely utilitarian grounds, although some people may feel a pull towards talking in terms of justice; a community that have metrics and evaluations that better encourage scientific creativity and a pluralism of ideas and approaches will be better off, and evaluation criteria that optimise for these sorts of pluralism are considerable better than the status quo.
I also worry that your focus is essentially on individual epistemics over a small range of positions, rather than the overall questions of funding structures and where it goes. For example, Open Phil could have decided to fund fellowships in developing countries rather than ERA in Cambridge, where I can’t imagine the applicants are any less good but didn’t. I think this expanding of geographic diversity may be a realtively easy win, although definitely isn’t the only criteria we could have followed. (Also, as a side point, I would suggest there is good cost-effectiveness reasons to do this as well; for example, the DEGREES initiative (that OpenPhil funds), has now got the largest geoengineering research team in the world, all based in Developing Countries, with significant impact now on informing policy, for a cost that was less than the budget of ERA I believe. Here, increasing geographic diversity must have played into their decision, again purely from a utilitarian and not justice perspectuve. )
In response to the trade off stuff, to some extent this is true, although I’m unconvinced this is that different to the trade offs we discuss. In response to your examples, I would say that the proposals we make are definitely promoting ‘novelty of ideas’ much heavier than the status quo, and indeed we are making a clear trade off between more ‘novel ideas’ and more depth along the current ‘orthodox’ research paths
I think I am a lot more on board with promoting idea pluralism (I realise I should have said this in my original comment, I was focusing there on what I found more controversial or difficult to think about well). I think science generally would go faster if funders took more risks on heterodox ideas (particularly given most research projects have far larger upside risks than downside risks, so ‘hits-based’ funding could work well). That’s a good point re things being cheaper to run in poorer countries, so more cost-effective all else equal.
I can’t imagine the applicants are any less good
At one level, yes intelligence and creativity are ~evenly distributed worldwide. But I think this gets to my earlier point about educational and other opportunities currently being very unequally distributed, so I think it would be the case (unfortunately) that applicants with access to loads of opportunities to develop their thinking and writing and research skills, disproportionately in the rich world, will be better able to contribute straight away. I think there could also be a strong case to run such fellowships elsewhere with fellows who have had fewer opportunities and are currently less capable, as this is more additional, but this seems like a notably different theory of change.
So I think you are pointing to something real here, although even then I don’t think it actually constitutes a great defence of the status quo of ERS. If we are to get people with preexisting experience in the disciplines we want, as most disciplines are far from equal, and if you are straying into an interdisciplinary soace like this job security and lack of mentorship may mean being more senior is very benefiical, we are likely to not get the sorts of demographic diversity we may want. However, ERS at present also rarely included and reaches out to people with deep expertise in these fields as well, so in some ways it feels we get the worst of both worlds; we bring in relatively young and inexperienced people and yet churn out people who broadly think very similarly and have the ability to influence very similar spaces . Sure, I’m not saying juggling all this is easy, nor that it will be perfect, but this status quo seems really suboptimal
Thanks Gideon and others for creating this, I’m glad it exists. I have only skimmed the other comments, so apologies if the following have already been discussed elsewhere. In my mind it is useful to separate two quite different sentiments, I’d be interested if others agree with/find useful this distinction:
We would prefer a world where demographic markers (perhaps most notably nationality) are not predictive of ability to participate in intellectual communities and knowledge-making.
Given the current very unequal and unjust world, we would prefer to fund/hire/support/etc people who do worse in our evaluation metrics (interviews, work trials, research proposals, etc) but are from more underrepresented backgrounds.
Both are in favour of greater pluralism and diversity. I think 1 is trivially true—of course a more equal and just world would be better (though it would be a further claim that this is something X-risk orgs should focus on). 2 I am more torn about, I think I more disagree than agree though. I think if two people, one from a very intellectually privileged background, and one less so, do equally well on our evaluation metrics, the latter person is likely to be more intelligent/creative/novel as they had a harder road to there and needed to do more for themselves. I also think though that it would be pretty bad (and often clearly illegal) for decision-makers to use demographic markers over and above evaluation metrics.
I think the “In Defense of Merit in Science” paper is relevant here, though I haven’t read more than the abstract and the public discussions of it. I am glad the statement talks about tradeoffs. I think this is another important tradeoff to add: the more dimensions you are optimising over, the less well you can optimise each of them. This seems uncontroversially true. The implication of this is that if we add more demographic dimensions to optimise over this will have a cost on whatever other dimensions we may already be using (novelty of ideas, clarity of writing, mathematical ability, etc). I think for me this is the most important tradeoff.
I’m not quite sure how to address most of your comment, as I think in many ways we are critiquing much of the underlying logic of how we evaluate what we ought to be funded. Its essentially suggesting, not really from a place of justice at all, that the current structures fail to optimise or assess the things that are useful for the community. And I think it may also be implicitly suggesting that centralising this power as is done currently is counter to the aims, and so power in the community ought to be better distributed. One response would be to change these criteria to better optimise for things that we may think are more valuable eg novelty of ideas/approach. Another would be to randomise funding ie every proposal that crosses a certain bar gets entered into a lottery. Ultimately, I’m (not necessarily the other signatories)pretty comfortable making the argument for this on purely utilitarian grounds, although some people may feel a pull towards talking in terms of justice; a community that have metrics and evaluations that better encourage scientific creativity and a pluralism of ideas and approaches will be better off, and evaluation criteria that optimise for these sorts of pluralism are considerable better than the status quo.
I also worry that your focus is essentially on individual epistemics over a small range of positions, rather than the overall questions of funding structures and where it goes. For example, Open Phil could have decided to fund fellowships in developing countries rather than ERA in Cambridge, where I can’t imagine the applicants are any less good but didn’t. I think this expanding of geographic diversity may be a realtively easy win, although definitely isn’t the only criteria we could have followed. (Also, as a side point, I would suggest there is good cost-effectiveness reasons to do this as well; for example, the DEGREES initiative (that OpenPhil funds), has now got the largest geoengineering research team in the world, all based in Developing Countries, with significant impact now on informing policy, for a cost that was less than the budget of ERA I believe. Here, increasing geographic diversity must have played into their decision, again purely from a utilitarian and not justice perspectuve. )
In response to the trade off stuff, to some extent this is true, although I’m unconvinced this is that different to the trade offs we discuss. In response to your examples, I would say that the proposals we make are definitely promoting ‘novelty of ideas’ much heavier than the status quo, and indeed we are making a clear trade off between more ‘novel ideas’ and more depth along the current ‘orthodox’ research paths
I think I am a lot more on board with promoting idea pluralism (I realise I should have said this in my original comment, I was focusing there on what I found more controversial or difficult to think about well). I think science generally would go faster if funders took more risks on heterodox ideas (particularly given most research projects have far larger upside risks than downside risks, so ‘hits-based’ funding could work well). That’s a good point re things being cheaper to run in poorer countries, so more cost-effective all else equal.
At one level, yes intelligence and creativity are ~evenly distributed worldwide. But I think this gets to my earlier point about educational and other opportunities currently being very unequally distributed, so I think it would be the case (unfortunately) that applicants with access to loads of opportunities to develop their thinking and writing and research skills, disproportionately in the rich world, will be better able to contribute straight away. I think there could also be a strong case to run such fellowships elsewhere with fellows who have had fewer opportunities and are currently less capable, as this is more additional, but this seems like a notably different theory of change.
So I think you are pointing to something real here, although even then I don’t think it actually constitutes a great defence of the status quo of ERS. If we are to get people with preexisting experience in the disciplines we want, as most disciplines are far from equal, and if you are straying into an interdisciplinary soace like this job security and lack of mentorship may mean being more senior is very benefiical, we are likely to not get the sorts of demographic diversity we may want. However, ERS at present also rarely included and reaches out to people with deep expertise in these fields as well, so in some ways it feels we get the worst of both worlds; we bring in relatively young and inexperienced people and yet churn out people who broadly think very similarly and have the ability to influence very similar spaces . Sure, I’m not saying juggling all this is easy, nor that it will be perfect, but this status quo seems really suboptimal
Link broken, but I googled and perhaps you meant this https://iopenshell.usc.edu/pubs/pdf/JCI_Merit_final.pdf
whoops, fixed. Thanks.