I am a big fan of 80k, and have found talking to 80k advisors helpful. But this program feels reminiscent of the excesses of pre-FTX-implosion EA, in that this is a lot of money to be giving people to do something that is not very hard and (in my view) of questionable value, though maybe I’m underestimating the efficacy of 80k’s filtering process, how much these conversations will shift the career paths of the referred parties, how well people will use the career grants, or something else. I’m sure a lot of thought went into doing this, so I’d be curious to see the BOTEC that led to these career grants.
I would guess that thinking more about the value of high impact career plan changes and accelerations would make a $50,000 experiment in scaling them seem not-excessive.
For example, it’s not unusual that hiring manager will say the first-best candidate for a role is 25% better than a second-best candidate. If the role pays $100,000/yr, naïvely, a single plan change coming from these calls pays for half the program. A bit less naïvely, people at top orgs are often paid multiples less than a grantmaker’s willingness to forgo donations, so perhaps one year of work from one change pays back the whole investment.
Now of course advising can’t claim full credit for the change, this is always divided among many sources/investments, but careers are 20+ years long post-change too. So accelerations will count for less and of course we’re hoping to drive multiple changes here.
A summary frame is that getting the right people into the right roles is high stakes and the cost of missing the best people is extremely high.
This all turns on the ability of the program to generate calls and calls to generate changes of course, so for some number of calls, this program will look bad. FWIW, we’re currently near the worst case scenario (10 qualified referrers with 2 calls each), but still within ~7x of average call cost. I’m hoping we can bring that down to 2-3x this month. I think the question then becomes whether you think advising overall is excessive, which is a separate conversation. Though it’s worth noting that everyone who’s looked at the program would ~emphatically disagree.
I take responsibility for the current situation. I substantially overestimated my ability to get this program in as many minds as I’d hoped to. Please help me remedy this!
One last thing I’ll highlight is that I expect people who qualify for and win the grants to be pretty engaged and agentic EAs who will use $5,000 very well to advance their careers and create good down the line. I think this is very substantially different from a cash reward.
Thanks; this is helpful, and I appreciate your candor. I’m not questioning whether 80k’s advising overall is valuable, and am thus willing to grant stuff like “most of the shifts people make as a result of 80k advising are +EV”. My reservations mainly pertain to the following:
does this grant effectively incentivize referrals?
are those referrals of high quality?
contingent on 80k agreeing to meet with a referred party, is that party liable to make career shifts based on the advising they receive?
(to a lesser extent) will the recipients of the career grants use the money well?
I get that it’s easy to be critical of (1) post-hoc, but I think we should subject the general model of “give EAs a lot of money to do things that are easy and that have very uncertain and difficult to quantify value” to a high degree of scrutiny because (as best I can tell based on a small n) this: (a) hasn’t tended to work that well, (b) is self-serving, and (c) often seems to be held to a lower evidentiary standard than other kinds of interventions EAs fund. (A countervailing piece of evidence is that OP does this for hiring referrals, and they presumably do have good evidence re: efficacy, although the benefits there also seem much clearer for the reasons you mention.)
Regarding (2), my worry is that the people who get referred as a result of this program will be importantly different from the general population of people who receive 80k career advising. This is because I suspect highly engaged EAs will have already applied for or received 80k advising. Conversely, people who are not familiar enough with EA to have previously heard of 80k advising—which I think is a low bar, given many people learn about EA via 80k—probably won’t have successful applications. Thus, my model of the median successful referral is “someone who has heard of 80k but not previously opted to pursue 80k advising.” Which brings me to (3): by virtue of these people having not previously opted into a free service, I suspect that they’re less likely to benefit from it. In other words, I suspect that people referred as a result of this program will be less likely (or less able) to make changes as a result of their advising meetings. (Or at least this was the conclusion I came to in deciding who to send my referral links to.)
Regarding (4), I haven’t seen evidence to support the claim that “very engaged and agentic EAs… will use $5,000 very well to advance their careers and create good down the line,” and while this seems prima facie plausible, I don’t think that is the standard of evidence we should apply to this—or any—intervention. (This is a less important point, because if this program generated tons of great referrals, it wouldn’t really matter how the $50k was spent.)
All fair points. My biggest point of disagreement is (2). As a true example, only around a third of EAG(x) attendees have applied for advising. That’s a much more expensive program for both the org and the participant. Tons of relevant people effectively don’t know advising exists (e.g. maybe they’ve heard of it once, but if it didn’t make it to the top of their stack that day, they forget about it). Importantly, our highest counterfactual comes from talking to only modestly engaged people, so maybe people who’ve done some reading but only know 1-2 other EAs – I see this program being aimed at these not-highly-engaged people makes the call more worthwhile, not less. “You were up for a call if someone leaned on you a bit,” seems roughly ideal to me.
And how counterfactual were the (quality) referrals—are the resulting advising sessions unlikely to have ever happened otherwise, or would it be more appropriate to categorize most referrals as causing certain people to receive advising services sooner than they otherwise would have?
Its an interesting question. I agree with the uncomfortability of excess here, but I don’t think the BOTEC would be that hard to justify. $5,000 is basically saving a life—so I would have thought even a tiny redirection of the career of 1 in 10 people they talked with (25,000 dollars worth) would likely reap at least $25,000 of value either in future donations or 5 human lives worth of life-direction-value.
I know that’s a little simplistic and doesn’t account for other costs and work…
Also I don’t necessarily agree with this kind of Botec, but it wouldn’t be too hard to justify
I am a big fan of 80k, and have found talking to 80k advisors helpful. But this program feels reminiscent of the excesses of pre-FTX-implosion EA, in that this is a lot of money to be giving people to do something that is not very hard and (in my view) of questionable value, though maybe I’m underestimating the efficacy of 80k’s filtering process, how much these conversations will shift the career paths of the referred parties, how well people will use the career grants, or something else. I’m sure a lot of thought went into doing this, so I’d be curious to see the BOTEC that led to these career grants.
I would guess that thinking more about the value of high impact career plan changes and accelerations would make a $50,000 experiment in scaling them seem not-excessive.
For example, it’s not unusual that hiring manager will say the first-best candidate for a role is 25% better than a second-best candidate. If the role pays $100,000/yr, naïvely, a single plan change coming from these calls pays for half the program. A bit less naïvely, people at top orgs are often paid multiples less than a grantmaker’s willingness to forgo donations, so perhaps one year of work from one change pays back the whole investment.
Now of course advising can’t claim full credit for the change, this is always divided among many sources/investments, but careers are 20+ years long post-change too. So accelerations will count for less and of course we’re hoping to drive multiple changes here.
A summary frame is that getting the right people into the right roles is high stakes and the cost of missing the best people is extremely high.
This all turns on the ability of the program to generate calls and calls to generate changes of course, so for some number of calls, this program will look bad. FWIW, we’re currently near the worst case scenario (10 qualified referrers with 2 calls each), but still within ~7x of average call cost. I’m hoping we can bring that down to 2-3x this month. I think the question then becomes whether you think advising overall is excessive, which is a separate conversation. Though it’s worth noting that everyone who’s looked at the program would ~emphatically disagree.
I take responsibility for the current situation. I substantially overestimated my ability to get this program in as many minds as I’d hoped to. Please help me remedy this!
One last thing I’ll highlight is that I expect people who qualify for and win the grants to be pretty engaged and agentic EAs who will use $5,000 very well to advance their careers and create good down the line. I think this is very substantially different from a cash reward.
Thanks; this is helpful, and I appreciate your candor. I’m not questioning whether 80k’s advising overall is valuable, and am thus willing to grant stuff like “most of the shifts people make as a result of 80k advising are +EV”. My reservations mainly pertain to the following:
does this grant effectively incentivize referrals?
are those referrals of high quality?
contingent on 80k agreeing to meet with a referred party, is that party liable to make career shifts based on the advising they receive?
(to a lesser extent) will the recipients of the career grants use the money well?
I get that it’s easy to be critical of (1) post-hoc, but I think we should subject the general model of “give EAs a lot of money to do things that are easy and that have very uncertain and difficult to quantify value” to a high degree of scrutiny because (as best I can tell based on a small n) this: (a) hasn’t tended to work that well, (b) is self-serving, and (c) often seems to be held to a lower evidentiary standard than other kinds of interventions EAs fund. (A countervailing piece of evidence is that OP does this for hiring referrals, and they presumably do have good evidence re: efficacy, although the benefits there also seem much clearer for the reasons you mention.)
Regarding (2), my worry is that the people who get referred as a result of this program will be importantly different from the general population of people who receive 80k career advising. This is because I suspect highly engaged EAs will have already applied for or received 80k advising. Conversely, people who are not familiar enough with EA to have previously heard of 80k advising—which I think is a low bar, given many people learn about EA via 80k—probably won’t have successful applications. Thus, my model of the median successful referral is “someone who has heard of 80k but not previously opted to pursue 80k advising.” Which brings me to (3): by virtue of these people having not previously opted into a free service, I suspect that they’re less likely to benefit from it. In other words, I suspect that people referred as a result of this program will be less likely (or less able) to make changes as a result of their advising meetings. (Or at least this was the conclusion I came to in deciding who to send my referral links to.)
Regarding (4), I haven’t seen evidence to support the claim that “very engaged and agentic EAs… will use $5,000 very well to advance their careers and create good down the line,” and while this seems prima facie plausible, I don’t think that is the standard of evidence we should apply to this—or any—intervention. (This is a less important point, because if this program generated tons of great referrals, it wouldn’t really matter how the $50k was spent.)
All fair points. My biggest point of disagreement is (2). As a true example, only around a third of EAG(x) attendees have applied for advising. That’s a much more expensive program for both the org and the participant. Tons of relevant people effectively don’t know advising exists (e.g. maybe they’ve heard of it once, but if it didn’t make it to the top of their stack that day, they forget about it). Importantly, our highest counterfactual comes from talking to only modestly engaged people, so maybe people who’ve done some reading but only know 1-2 other EAs – I see this program being aimed at these not-highly-engaged people makes the call more worthwhile, not less. “You were up for a call if someone leaned on you a bit,” seems roughly ideal to me.
And how counterfactual were the (quality) referrals—are the resulting advising sessions unlikely to have ever happened otherwise, or would it be more appropriate to categorize most referrals as causing certain people to receive advising services sooner than they otherwise would have?
Its an interesting question. I agree with the uncomfortability of excess here, but I don’t think the BOTEC would be that hard to justify. $5,000 is basically saving a life—so I would have thought even a tiny redirection of the career of 1 in 10 people they talked with (25,000 dollars worth) would likely reap at least $25,000 of value either in future donations or 5 human lives worth of life-direction-value.
I know that’s a little simplistic and doesn’t account for other costs and work…
Also I don’t necessarily agree with this kind of Botec, but it wouldn’t be too hard to justify
I expect that they were hoping for this referral program to receive much more referrals than it’s receiving?