I don’t want to name individuals on a public forum, but noting that there are at least a couple of individuals at FHI who passed through one of the programmes you mention (I don’t know about counterfactual attribution).
Owen_Cotton-Barratt
Karma: 4,013
I’m actually confused about what you mean by your definition. I have an impression about what you mean from your post, but if I try to just go off the wording in your definition I get thrown by “calibrated”. I naturally want to interpret this as something like “assigns confidence levels to their claims that are calibrated”, but that seems ~orthogonal to having the right answer more often, which means it isn’t that large a share of what I care about in this space (and I suspect is not all of what you’re trying to point to).
Now I’m wondering: does your notion of judgement roughly line up with my notion of meta-level judgement? Or is it broader than that?
For one data point, I filled in the EALF survey and had in mind something pretty close to what I wrote about in the post Ben links to. I don’t remember paying much attention to the parenthetical definition—I expect I read it as a reasonable attempt to gesture towards the thing that we all meant when we said “good judgement” (though on a literal reading it’s something much narrower than I think even Ben is talking about).
I think that good judgement in the broad sense is useful ~everywhere, but that:
It’s still helpful to try to understand it, to know better how to evaluate it or improve at it;
For reasons Ben outlines, it’s more important for domains where feedback loops are poor;
The cluster Ben is talking about gets disproportionately more weight in importance for thinking about strategic directions.
ii) But, actors making up a large proportion of total financial assets may have constraints other than maximising impact, which could lead the community to spend faster than the aggregate of the community thinks is correct:
Large donors usually want to donate before they die (and Open Phil’s donors have pledged to do so). (Of course, it’s arguable whether this should be modeled as such a constraint or as a claim about optimal timing).
Other holders of financial capital may not have enough resources to realistically make up for that.
Thanks for pulling this out, I think this is the heart of the argument. (I think it’s quite valuable to show how the case relies on this, as it helps to cancel a possible reading where everyone should assume that they personally will have better judgement than the aggregate community.)
I think it’s an interesting case, and worth considering carefully. We might want to consider:
Whether this will actually lead to incorrect spending?
My central best guess is that there will be enough flow of other money into longtermist-aligned purposes that this won’t be an issue in coming decades, but I’m quite uncertain about that
What are the best options for mitigating it?
Earning to save is certainly one possibility, but we could also consider e.g. whether there are direct work opportunities which would have a significant effect of passing capital into the hands of future longtermists
Thanks for the thoughtful reply!
On reflection I realise that in some sense the heart of my objection to the post was in vibe, and I think I was subconsciously trying to correct for this by leaning into the vibe (for my response) of “this seems wrongfooted”.
But I do think the post tries to caveat a lot and it overall seems good for there to be a forum where even minor considerations can be considered in a quick post., so I thought it was worth posting.
I quite agree that it’s good if even minor considerations can be considered in a quick post. I think the issue is that the tone of the post is kind of didactic, let-me-explain-all-these-things (and the title is “an argument for X”, and the post begins “I used to think not-X”): combined these are projecting quite a sense of “X is solid”, and while it’s great that it had lots of explicit disclaimers about this just being one consideration etc., I don’t think they really do the work of cancelling the tone for feeding into casual readers’ gut impressions.
For an exaggerated contrast, imagine if the post read like:A quick thought on earning-to-save
I’ve been wondering recently about whether earning-to-save could make sense. I’m still not sure what I think, but I did come across a perspective which could justify it.
[argument goes here]
What do people think? I haven’t worked out how big a deal this seems compared to the considerations against earning to save (and some of them are pretty substantial), so it might still be a pretty bad idea overall.
I think that would have triggered approximately zero of my vibe concerns.
Alternatively I think it could have worked to have a didactic post on “Considerations around earning-to-save” that felt like it was trying to collect the important considerations (which I’m not sure have been well laid out anywhere, so there might not be a canonical sense of which arguments are “new”) rather than particularly emphasise one consideration.
I didn’t downvote, but I also didn’t even understand whether you were agreeing with me or disagreeing with me (and strongly suspected that “would have to” was an error in either case).
I almost feel cheeky responding to this as you’ve essentially been baited into providing a controversial view, which I am now choosing to argue against. Sorry!
That’s fine! :)
In turn, an apology: my controversial view has baited you into response, and I’m now going to take your response as kind-of-volunteering for me to be critical. So I’m going to try and exhibit how it seems mistaken to me, and I’m going (in part) to use mockery as a rhetorical means to achieve this. I think this would usually be a violation of discourse norms, but here: the meta-level point is to try and exhibit more clearly what this controversial view I hold is and why; the thing I object to is a style of argument more than a conclusion; I think it’s helpful for the exhibition to be able to draw attention to features of a specific instance, and you’re providing what-seems-like-implicit-permission for me to do that. Sorry!
I’d say that something doesn’t have to be the most effective thing to do for it to be worth doing, even if you’re an EA.
To be clear: I strongly agree with this, and this was a big part of what I was trying say above.
So donating to a seeing eye dog charity isn’t really a good thing to do.
This is non-central, but FWIW I disagree with this. Donating to the guide dog charity usually is a good thing to do (relative to important social norms where people have property rights over their money), it’s just that it turns out there are fairly accessible actions which are quite a lot better.
Choosing to follow a ve*an diet doesn’t have an opportunity cost (usually). You have to eat, and you’re just choosing to eat something different.
This, I’m afraid, is the type of statement that really bugs me. It’s trying to collapse a complex issue onto simple dimensions, draw a simple conclusion there, and project it back to the original complex world. But in doing so it’s thrown common-sense out of the window!
If I believed that choosing to follow a ve*an diet usually didn’t have an opportunity cost, I would expect to see:
People usually willing to go ve*an for a year for some small material gain
In theory if there was no opportunity cost, even for something trivial like $10, but I think many non ve*ans would be unwilling to do this even for $1000
[As an aside, I think taxes on meat would probably be a good policy that might well be accessible]
Almost everyone who goes ve*an for ethical reasons keeping it up
In fact some significant proportion of people stop
Or perhaps you just think the personal cost to you of being ve*an is substantial enough to offset the harm to the animals.
I certainly don’t claim this in any utilitarian comparison of welfare. But now the argument seems almost precisely analogous to:
“You could help the poorest people in the world a tremendous amount for the cost of a cup of coffee. Since your welfare shouldn’t outweigh theirs, you should forgo that cup of coffee, and every other small luxury in your life, to give more to them.”
I think EA correctly rejects this argument, and that it’s correct to reject its analogue as well. (I think the argument is stronger for ve*anism than giving to the poor instead of buying coffee; but I also think that there are better giving opportunities than giving directly to the poor, and that when you work it through the coffee argument ends up being stronger than the corresponding one for ve*anism.)
---
Again, I’m not claiming that EAs shouldn’t be ve*an. I think it’s a morally virtuous thing to do!
But I don’t think EAs have a monopoly on virtue. I think the EA schtick is more like “we’ll think things through really carefully and tell you what the most efficient ways to do good are”. And so I think that if it’s presented as “you want to be an EA now? great! how about ve*anism?” then the implicature is that this is a bigger deal than, say, moving from giving away 7% of your income to giving away 8%, and that this is badly misleading.
Notes:
There may be some people for whom the opportunity cost is trivial
I think there are probably quite a few people for whom the opportunity cost is actually negative—i.e. it’s overall easier for them to be ve*an than not
I would feel very good about encouragement to check whether people fall into one of these buckets, as in cases where they do then dietary change may be a particularly efficient way to do good
I’d also feel very good about moral exhortment to be ve*an that was explicit that it wasn’t grounded in EA thinking, like:
“Many EAs try to be morally serious in all aspects of their lives, beyond just trying to optimise for the most good achievable. This leads us to ve*anism. You might want to consider it.”
1. Did you make an active decision to shift your priorities somewhat from doing to facilitating research? If so, what factors drove that decision?
There was something of an active decision here. It was partly based on a sense that the returns had been good when I’d previously invested attention in mentoring junior researchers, and partly on a sense that there was a significant bottleneck here for the research community.
2. What do you think makes running RSP your comparative advantage (assuming you think that)?
Overall I’m not sure what my comparative advantage is! (At least in the long term.)
I think:
Some things which makes me good at research mentoring are:
being able to get up to speed on different projects quickly
holding onto a sense of why we’re doing things, and connecting to larger purposes
finding that I’m often effective in ‘reactive’ mode rather than ‘proactive’ mode
(e.g. I suspect this AMA has the highest ratio of public-written-words / time-invested of anything substantive I’ve ever done)
being able to also connect to where the researcher in front of me is, and what their challenges are
There are definitely parts of running RSP which seem not my comparative advantage (and I’m fortunate enough to have excellent support from project managers who have taken ownership of a lot of the programme)
3. Any thoughts on how to test or build one’s skills for that sort of role/pathway?
Read a lot of research. Form views (and maybe talk to others) about which pieces are actually valuable, and how. Try to work out what seems bad even about good pieces, or what seems good even about bad pieces.
Be generous with your time looking to help others with their projects. Check in with them afterwards to see if they found it useful. (Try to ask in a way which makes it safe for them to express that they did not.)
Try your own hand at research. First-hand experience of challenges is helpful for this.
(I’ve focused on the pathway of “research mentorship”; I think there are other parts you were asking about which I’ve ignored.)
Gee, this is really hard to measure.
I’d guess that somewhere between 10% and 30% is done as part of something that we’d naturally call the “standard academic process” ?
I think that there are some good reasons for deviation, and some things that academic norms provide that we may be missing out on.
I think academia is significantly set up as a competitive process, where part of the game is to polish your idea and present it in the best light. This means:
It encourages you to care about getting credit, and people are discouraged from freely-sharing early stage ideas that they might turn into papers, for fear of being scooped
This seems broadly bad
It encourages people to put in the time to properly investigate the ins and outs of an idea, and find the clearest framing of it, making it more efficient for later readers
This seems broadly good
I’d like it if we could work out how to get more of the good here with less of the bad. That could mean doing a larger proportion of things within some version of the academic process, or could mean working out other ways to get the benefits.
There’s also a credentialing benefit to doing things within the academic process. I think this is non-negligible, but also that if you do really high-quality work anywhere, people will observe this and come, so I don’t think it’s necessary to rest on that credentialing.
This is an interesting question, but I don’t think there’s a decent short-answer version; it’s more like investing several hours or not at all.
So I’ll take this as a prompt to consider the several-hour version, but won’t answer for now.
Malevolence seems potentially important to me, although I mostly haven’t been thinking about it (except a bit about psychopathy and its absence). Things more like game-theoretic dynamics are where a good portion of my attention has been … but I don’t want to claim this means they’re more important.
[meta: this is a short answer because while I might have things to say about crisper questions within this space, for saying things-in-general I think it makes more sense to wait until I have coherent enough ideas to publish something.]
Good question.
Of the two options I’d be tempted to say it’s more of a priority to spread the underlying arguments, but actually I think something more nuanced: it’s a priority to keep engaging with people about the underlying arguments, finding where there seems to be the greatest discomfort and turning a critical eye on the arguments there, looking to see if we can develop stronger versions of them.
I think that talking about the tentative conclusions along with this is important both for growing the network of people sympathetic to those, and for providing concrete instantiation of what is meant by the underlying philosophy (too much risk of talking past each other or getting lost in abstraction-land without this)
I guess I think that “decision-making under deep uncertainty” is mostly too broad a category to be able to say useful things about (although maybe we can draw together useful lessons that seem to hold in a variety of more specialised contexts), and we’re better trying to look at more particular setups and reason about those.
I don’t feel like I’m at all an expert in biosecurity careers, but I agree with that directionally they seem more credentialist.
I think this is a consideration against RSP, although it doesn’t feel like an overwhelming one, since:
It could be a reasonable option before a PhD
This is particularly relevant if taking the time to think about what you want to work on allows you to do a PhD in which your work is much closer to things you eventually care about
(similarly it could be a good option for some people after a PhD)
There may well be some roles (now or in the future) which are less credential-locked
Generally Oxford lectures are open to any university members, although:
They wouldn’t generally get “academic credit” for this
They wouldn’t necessarily be able to join accompanying classes (although we might be able to arrange this)
I’ve no idea what the situation is now so many things are remote because of COVID-19
There’s a class of things which feel majorly helpful, but it’s hard to distinguish between whether I was helped by the background in pure mathematics, or whether I have some characteristics which both helped me in mathematics and help me now (I suspect it’s some of both):
Being good at framing things
Turning things over in my head, looking for the angle which makes them most parsimonious, and easiest to comprehend clearly
Relatedly, feeling happy to dive in and try to make up theory, but keep it grounded by “this has to actually explain the things we want to know about”
These are useful skills when faced with domains where we haven’t yet settled on paradigms which we’re satisfied capture the important parts of what we care about
Generally keeping track of precisely what are the epistemic statuses of different claims, and how they interact
This is a useful skill for domains where we’re projecting out beyond things we can easily check empirically
Then there are some cases where I was more directly applying some mathematical thinking, e.g.:
Work on normative uncertainty (chiefly: variance normalization; bargaining)
Estimating the value of research seems really hard to me (and this is significantly true even in retrospect).
That said, some candidates are:
Work making the point that we should give outsized attention to mitigating risks that might manifest unexpectedly soon, since we’re the only ones who can
At the time it didn’t seem unusually valuable, but I think it was relatively soon after (a few months) that I saw some people changing behaviour in light of the point, which increased my sense of its importance
Work on cost-effectiveness of research of unknown difficulty, particularly the principle of using log returns when you don’t know where to start
Felt sort-of important at the time, although I think the kind of value I anticipated hasn’t really manifested
I have felt like it’s been useful for my thinking in a variety of domains, thinking about pragmatic prioritisation (and I’ve seen some others get some value from that); however logarithm is an obvious-enough functional form that maybe it didn’t really add much
Maybe something where it was more about dissemination of ideas than finding deep novel insights (I think it’s very hard to draw a line between what counts as “research” or what doesn’t), such as Prospecting for Gold, or How valuable is movement growth?
Quite a few people have told me that they got something out of one or both of those pieces, although it’s extremely hard to assess the counterfactuals
I felt like I was doing something significant in these cases (particularly when writing the talk Prospecting for Gold)
Overall I’d be hard pressed to decide between choosing one of the above, although I’d tend to guess these are more valuable than most other pieces I’ve done (excepting some recent work that I don’t yet want to judge, and with the caveat that I’m surely forgetting some)
That said, some of the more policy-ish pieces of research might still turn out to be the most valuable, if they got picked up somewhere important, but so far I’ll not count them
Something like: it seems like the people we’re taking on the programme are doing kind of good things, but when we dig into counterfactual analysis it seems like they might on average have done more if they hadn’t joined the programme (perhaps because e.g. normal academic pressures are surprisingly helpful motivationally, or because we’re fostering a community which is too inward-looking).
Something like: it catalysed the creation of a whole stream of major new projects (led by scholars who used the space afforded by the programme to think seriously about possibilities, and who are well-networked with the broader x-risk ecosystem which makes coordination and recruitment easier).
Thanks! This largely seems rather better.
One paragraph where you’ve lost the meaning is:
I’m not claiming that my original was the easiest to follow, but the point that needs justifying is not that the terms cancel (that’s mathematically trivial), but that the decomposition is actually an improvement in terms of ease of understanding or ease of estimation, relative to the term on the left of the equation.