I work on AI Grantmaking at Open Philanthropy. Comments here are posted in a personal capacity.
alex lawsen
Karma: 3,657
[I left 80k ~a month ago, and am writing this in a personal capacity, though I showed a draft of this answer to Michelle (who runs the team) before posting and she agrees it provides an accurate representation. Before I left, I was line-managing the 4 advisors, two of whom I also hired.]
Hey, I wanted to chime in with a couple of thoughts on your followup, and then answer the first question (what mechanisms do we have in place to prevent this). Most of the thoughts on the followup can be summarised by ‘yeah, I think doing advising well is really hard’.
Advisors often only have a few pages of context and a single call (sometimes there are follow-ups) to talk about career options. In my experience, this can be pretty insufficient to understand someone’s needs.
Yep, that’s roughly right. Often it’s less than this! Not everyone takes as much time to fill in the preparation materials as it sounds like you did. One of the things I frequently emphasised when hiring for and training advisors was asking good questions at the start of the call to fill in gaps in their understanding, check it with the advisee, and then quickly arrive at a working model that was good enough to proceed with. Even then, this isn’t always going to be perfect. In my experience, advisors tend to do a pretty good job of linking the takes they give to the reasons they’re giving them (where, roughly speaking, many of those reasons will be aspects of their current understanding of the person they’re advising).
the person may feel more pressure to pursue something that’s not a good fit for them
With obvious caveats about selection effects, many of my advisees expressed that they were positively surprised at me relieving this kind of pressure! In my experience advisors spend a lot more time reassuring people that they can let go of some of the pressure they’re perceiving than the inverse (it was, for example, a recurring theme in the podcast I recently released).
if they disagree with the advice given, the may not raise it. For example, they may not feel comfortable raising the issue because of concerns around anonymity and potential career harm, since your advisors are often making valuable connections and sharing potential candidate names with orgs that are hiring.
This is tricky to respond to. I care a lot that advisees are in fact not at risk of being de-anonymised, slandered, or otherwise harmed in their career ambitions as a result of speaking to us, and I’m happy to say that I believe this is the case. It’s possible, of course, for advisees to believe that they are at risk here, and for that reason or several possible other reasons, to give answers that they think advisors want to hear rather than answers that are an honest reflection of what they think. I think this is usually fairly easy for advisors to pick up on (especially when it’s for reasons of embarrassment/low confidence), at which point the best thing for them to do is provide some reassurance about this.
I do think that, at some point, the burden of responsibility is no longer on the advisor. If someone successfully convinces an advisor they they would really enjoy role A, or really want to work on cause Z, because they think that’s what the advisor wants to hear, or they think that’s what will get them recommended for the best roles, or introduced to the coolest people, or whatever, and the advisor then gives them advice that follows from those things being true, I think that advice is likely to be bad advice for that person, and potentially harmful if they follow it literally. I’m glad that advisors are (as far as I can tell), quite hard to mislead in this way, but I don’t think they should feel guilty if they miss some cases like this.
I know that 80K don’t want people to take their advice so seriously, and numerous posts have been written on this topic. However, I think these efforts won’t necessarily negate 1) and 2) because many 80K advisees may not be as familiar with all of 80K’s content or Forum discourse, and the prospect of valuable connections remains nonetheless.
There might be a slight miscommunication here. Several of the posts (and my recent podcast interview) talking about how people shouldn’t take 80k’s advice so seriously are, I think, not really pointing at a situation where people get on a 1on1 call and then take the advisor’s word as gospel, but more at things like reading a website that’s aimed at a really broad audience, and trying to follow it to the letter despite it very clearly being the case that no single piece of advice applies equally to everyone. The sort of advice people get on calls is much more frequently a suggestion of next steps/tests/hypotheses to investigate/things to read than “ok here is your career path for the next 10 years”, along with the reasoning behind those suggestions. I don’t want to uncritically recommend deferring to anyone on important life decisions, but on the current margin I don’t think I’d advocate for advisees taking that kind of advice, expressed with appropriate nuance, less seriously.
OK, but what specific things are in place to catch potential harm?
There are a few things that I think are protective here, some of which I’ll list below, though this list isn’t exhaustive.
Internal quality assurance of callsThe overwhelming majority of calls we have are recorded (with permission), and many of these are shared for feedback with other staff at the organisation (also with permission). To give some idea of scale, I checked some notes and estimated that (including trials, and sitting in on calls with new advisors or triallists) I gave substantive feedback on over 100 calls, the majority of which were in the last year. I was on the high end for the team, though everyone in 80k is able to give feedback, not only advisors.
I would expect anyone listening to a call in this capacity to flag, as a priority, anything that seemed like and advisor saying something harmful, be that because it was false, displayed an inappropriate level of confidence, or because it was insensitive.
My overall impression is that this happens extremely rarely, and that the bar for giving feedback about this kind of concern was (correctly) extremely low. I’m personally grateful, for example, for some feedback a colleague gave me about how my tone might have been perceived as ‘teacher-y’ on one call I did, and another case where someone flagged that they thought the advisee might have felt intimidated by the start of the conversation. In both cases, as far as I can remember, the colleague in question thought that the advisee probably hadn’t interpreted the situation in the way they were flagging, but that it was worth being careful in future. I mention this not to indicate that I never made mistakes on calls, but instead to illustrate why I think it’s unlikely that feedback would miss significant amounts of potentially harmful advice.
Advisee feedback mechanisms
There are multiple opportunities for people we’ve advised to give feedback about all aspects of the process, including specific prompts about the quality of advice they received on the call, any introductions we made, and any potential harms.
Some of these opportunities include the option for the advisee to remain anonymous, and we’re careful about accidentally collecting de-anonymising information, though no system is foolproof. As one example, we don’t give an option to remain anonymous in the feedback form we send immediately after the call (as depending on how many other calls were happening at the time, someone filling it in straight away might be easy to notice), but we do give this option in later follow-up surveys (where the timing won’t reveal identity).
In user feedback, the most common reason given by people who said 1on1 caused them harm is that they were rejected from advising and felt bad/demotivated about that. The absolute numbers here are very low, but there’s an obvious caveat about non-response bias.
On specific investigations/examples
I worked with community health on some ways of preventing harm being done by people advisers made introductions to (including, in some cases, stopping introductions)
I spent more than 5 but less than 10 hours, on two occasions, investigating concerns that had been raised to me about (current or former) advisors, and feel satisfied in both cases that our response was appropriate i.e. that there was not an ongoing risk of harm following the investigation.
Despite my personal bar for taking concerns of this sort seriously being pretty low compared to my guess at the community average (likely because I developed a lot of my intuitions for how to manage such situations during my previous career as a teacher), there were few enough incidents meriting any kind of investigation that I think giving any more details than the above would not be worth the (small) risk of deanonymising those involved. I take promises of confidentiality really seriously (as I hope would be expected for someone in the position advisors have).
Thanks for asking these! Quick reaction to the first couple of questions, I’ll get to the rest later if I can (personal opinions, I haven’t worked on the web team, no longer at 80k etc. etc.):
I don’t think it’s possible to write a single page that gives the right message to every user—having looked at the pressing problems page—the second paragraph visible on that page is entirely caveat. It also links to an FAQ, where multiple parts of the FAQ directly talk about whether people should just take the rankings as given. When you then click through to the AI problem profile, the part of the summary box that talks about whether people should work on AI reads as follows:As a result, the possibility of AI-related catastrophe may be the world’s most pressing problem — and the best thing to work on for those who are well-placed to contribute.
Frankly, for my taste, several parts of the website already contain more caveats than I would use about ways the advice is uncertain and/or could be wrong, and I think moves in this direction could just as easily be patronising as helpful.
[not on the LTFF and also not speaking for Open Phil, just giving a personal take]
A few reactions:
Which AI systems are conscious seems like a good candidate for an extremely important humanity will need to solve at some point.
studying current systems, especially in terms of seeing what philosophical theories of consciousness have to say about them, seems like a reasonable bet to make right now if you’re excited about this problem being solved at some point.
To the extent you want to bet on a person to help push this field forward, Rob seems like an excellent bet.
On why the question seems important:
Being wrong about which systems are conscious seems dangerous in both directions:
Falsely believing systems are conscious could lead to enormous waste (trying to improve the well-being of such systems).
AI systems advocating for their own rights is a plausible way by which they could gain influence over some humans, given that at least one case of this has already happened.
Not treating genuinely conscious systems as being worthy of moral consideration on the other hand, seems like a good candidate for causing a moral catastrophe of potentially astronomical scale.
Can confirm that:
“sr EAs [not taking someone seriously if they were] sloppy in their justification for agreeing with them”
sounds right based on my experience being on both sides of the “meeting senior EAs” equation at various times.
(I don’t think I’ve met Quinn, so this isn’t a comment on anyone’s impression of them or their reasoning)
So there’s now a bunch of speculation in the comments here about what might have caused me and others to criticise this post.
I think this speculation puts me (and, FWIW, HLI) in a pretty uncomfortable spot for reasons that I don’t think are obvious, so I’ve tried to articulate some of them:
- There are many reasons people might want to discuss others’ claims but not accuse them of motivated reasoning/deliberately being deceptive/other bad faith stuff, including (but importantly not limited to):
a) not thinking that the mistake (or any other behaviour) justifies claims about motivated reasoning/bad faith/whatever
b) not feeling comfortable publicly criticising someone’s honesty or motivations for fear of backlash
c) not feeling comfortable publicly criticising someone’s honesty of motivations because that’s a much more hurtful criticism to hear than ’I think you made this specific mistake’
d) believing it violates forum norms to make this sort of public criticism without lots of evidence
- In situations where people are speculating about what I might believe but not have said, I do not have good options for moving that speculation closer to the truth, once I notice that this might not be the only time I post a comment or correction to something someone says.
Examples:
- If I provide positive reassurance about me not actually implying bad faith with a comment that didn’t mention it, that makes it pretty clear what I think in situations where I’m not ruling it out.
- If I give my honest take on someone’s motivation in any case where I don’t think there’s any backlash risk, but don’t give a take in situations where there is backlash risk, then I’m effectively publicly identifying which places I’d be worried about backlash, which feels like the sort of thing that might cause backlash from them.
If you think for a few minutes about various actions I might take in various situations, either to correct misunderstanding or to confirm correct understanding, I’m sure you’ll get the idea. To start with, you might want to think about why it doesn’t make sense to only correct speculation that seems false.
That’s a very long-winded way of saying “I posted a correction, you can make up your own mind about what that correction is evidence of, but I’d rather you didn’t spend a ton of time publicly discussing what I might think that correction is evidence of, because I won’t want to correct you if you’re wrong or confirm if you’re right”.
My comment wasn’t about whether there are any positives in using WELLBYs (I think there are), it was about whether I thought that sentence and set of links gave an accurate impression. It sounds like you agree that it didn’t, given you’ve changed the wording and removed one of the links. Thanks for updating it.
I think there’s room to include a little more context around the quote from TLYCs.
In short, we do not seek to duplicate the excellent work of other charity evaluators. Our approach is meant to complement that work, in order to expand the list of giving opportunities for donors with strong preferences for particular causes, geographies, or theories of change. Indeed, we will continue to rely heavily on the research done by other terrific organizations in this space, such as GiveWell, Founders Pledge, Giving Green, Happier Lives Institute, Charity Navigator, and others to identify candidates for our recommendations, even as we also assess them using our own evaluation framework.We also fully expect to continue recommending nonprofits that have been held to the highest evidentiary standards, such as GiveWell’s top charities. For our current nonprofit recommendations that have not been evaluated at that level of rigor, we have already begun to conduct in-depth reviews of their impact. Where needed, we will work with candidate nonprofits to identify effective interventions and strengthen their impact evaluation approaches and metrics. We will also review our charity list periodically and make sure our recommendations remain relevant and up to date.
- 's comment on The Happier Lives Institute is funding constrained and needs you! by (Jul 10, 2023, 9:20 PM; 47 points)
- 's comment on The Happier Lives Institute is funding constrained and needs you! by (Jul 10, 2023, 10:22 PM; 45 points)
- 's comment on The Happier Lives Institute is funding constrained and needs you! by (Jul 11, 2023, 8:48 AM; 1 point)
[Speaking for myself here]
I also thought this claim by HLI was misleading. I clicked several of the links and don’t think James is the only person being misrepresented. I also don’t think this is all the “major actors in EA’s GHW space”—TLYCS, for example, meet reasonable definitions of “major” but their methodology makes no mention of wellbys
I find this surprising, given that I’ve heard numbers more like 100-200 $/h claimed by people considerably more senior than top-uni community builders (and who are working in similar fields/with similar goals).
(I’m straight up guessing, and would be keen for an answer from someone familiar with this kind of study)
This also confused me. Skimming the study, I think they’re calculating efficacy from something like how long it takes people to get malaria after the booster, which makes sense because you can get it more than once. Simplifying a lot (and still guessing), I think this means that if e.g. on average people get malaria once a week, and you reduce it to once every 10 weeks, you could say this has a 90% efficacy, even though if you looked at how many people in each group got it across a year, it would just be ‘everyone’ in both groups.
This graph seems to back this up:
https://www.thelancet.com/cms/attachment/2eddef00-409b-4ac2-bfea-21344b564686/gr2.jpg
This is a useful consideration to point out, thanks. I push back a bit below on some specifics, but this effect is definitely one I’d want to include if I do end up carving out time to add a bunch more factors to the model.
I don’t think having skipped the neglectedness considerations you mention is enough to call the specific example you quote misleading though, as it’s very far from the only thing I skipped, and many of the other things point the other way. Some other things that were skipped:
-
Work after AGI likely isn’t worth 0, especially with e.g. Metaculus definitions.
-
While in the community building examples you’re talking about, shifting work later doesn’t change the quality of that work, this is not true wrt PhDs (doing a PhD looks more like truncating the most junior n years of work than shifting all years of work n years later).
-
Work that happens just before AGI can be done with a much better picture of what AGI will look like, which pushes against the neglectedness effect.
-
Work from research leads may actually increase in effectiveness as the field grows, if the growth is mostly coming from junior people who need direction and/or mentorship, as has historically been the case.
And then there’s something about changing your mind, but it’s unclear to me which direction this shifts things:
it’s easier to drop out of a PhD than it is to drop into one, if e.g. your timelines suddenly shorten.
If your timelines shorten because AGI arrives, though, it’s too late to switch, while big updates towards timelines being longer are things you can act on, pushing towards acting as if timelines are short.
-
Most podcast apps let you subscribe to an RSS feed, and an RSS feed of the audio is available on the site
I’m a little confused about what “too little demand” means in the second paragraph. Both of the below seem like they might be the thing you are claiming:
There is not yet enough demand for a business only serving EA orgs to be self sustaining.
EA orgs are making a mistake by not wanting to pay for these things even though they would be worth paying for.
I’d separately be curious to see more detail on why your guess at the optimal structure for the provision of the kind of services you are interested in is “EA-specific provider”. I’m not confident that it’s not, but my low confidence guess would be that “EA orgs” are not similar enough that “context on how to with with EA orgs” becomes a hugely important factor.
I think “different timelines don’t change the EV of different options very much” plus “personal fit considerations can change the EV of a PhD by a ton” does end up resulting in an argument for the PhD decision not depending much on timelines. I think that you’re mostly disagreeing with the first claim, but I’m not entirely sure.
In terms of your point about optimal allocation, my guess is that we disagree to some extent about how much the optimal allocation has changed, but that the much more important disagreement is about whether some kind of centrally planned ‘first decide what fraction of the community should be doing what’ approach is a sensible way of allocating talent, where my take is that it usually isn’t.
I have a vague sense of this talent allocation question having been discussed a bunch, but don’t have write-up that immediately comes to mind that I want to point to. I might write something about this at some point, but I’m afraid it’s unlikely to be soon. I realise that I haven’t argued for my talent allocation claim at all, which might be frustrating, but it seemed better to highlight the disagreement at all than ignore it, given that I didn’t have the time to explain in detail.
Whether you should do a PhD doesn’t depend much on timelines.
I like this
(I’m excited to think more about the rest of the ideas in this post and might have further comments when I do)
Commenting briefly to endorse the description of my course as an MVP. I’d love for someone to make a better produced version, and am happy for people to use any ideas from it that they think would be useful in producing the better version
Now posted as a top-level post here.
[context: I’m one of the advisors, and manage some of the others, but am describing my individual attitude below]
FWIW I don’t think the balance you indicated is that tricky, and think that conceiving of what I’m doing when I speak to people as ‘charismatic persuasion’ would be a big mistake for me to make. I try to:
Say things I think are true, and explain why I think them (both the internal logic and external evidence if it exists) and how confident I am.
Ask people questions in a way which helps them clarify what they think is true, and which things they are more or less sure of.
Make tradeoffs (e.g. between a location preference and a desire for a particular job) explicit to people who I think might be missing that they need to make one, but usually not then suggesting which tradeoff to make, but instead that they go and think about it/talk to other people affected by it.
Encourage people to think through things for themselves, usually suggesting resources which will help them do that/give a useful perspective as well as just saying ‘this seems worth you taking time to think about’.
To the extent that I’m paying attention to how other people perceive me[1], I’m usually trying to work out how to stop people deferring to me when they shouldn’t without running into the “confidence all the way up” issue.
- ^
in a work context, that is. I’m unfortunately usually pretty anxious about, and therefore paying a bunch of attention to, whether people are angry/upset with me, though this is getting better, and easy to mostly ‘switch off’ on calls because the person in front of me takes my full attention.
I don’t think it’s worth me going back and forth on specific details, especially as I’m not on the web team (or even still at 80k), but these proposals are different to the first thing you suggested. Without taking a position on whether this structure would overall be an improvement, it’s obviously not the case that just having different sections for different possible users ensures that everyone gets the advice they need.
For what it’s worth, one of the main motivations for this being an after-hours episode, which was promoted on the EA forum and my twitter, is that I think the mistakes are much more common among people who read a lot of EA content and interact with a lot of EAs (which is a small fraction of the 80k website readership). The hope is that people who’re more likely than a typical reader to need the advice are the people most likely to come across it, so we don’t have to rely purely on self-selection.