Thanks; this is helpful, and I appreciate your candor. I’m not questioning whether 80k’s advising overall is valuable, and am thus willing to grant stuff like “most of the shifts people make as a result of 80k advising are +EV”. My reservations mainly pertain to the following:
does this grant effectively incentivize referrals?
are those referrals of high quality?
contingent on 80k agreeing to meet with a referred party, is that party liable to make career shifts based on the advising they receive?
(to a lesser extent) will the recipients of the career grants use the money well?
I get that it’s easy to be critical of (1) post-hoc, but I think we should subject the general model of “give EAs a lot of money to do things that are easy and that have very uncertain and difficult to quantify value” to a high degree of scrutiny because (as best I can tell based on a small n) this: (a) hasn’t tended to work that well, (b) is self-serving, and (c) often seems to be held to a lower evidentiary standard than other kinds of interventions EAs fund. (A countervailing piece of evidence is that OP does this for hiring referrals, and they presumably do have good evidence re: efficacy, although the benefits there also seem much clearer for the reasons you mention.)
Regarding (2), my worry is that the people who get referred as a result of this program will be importantly different from the general population of people who receive 80k career advising. This is because I suspect highly engaged EAs will have already applied for or received 80k advising. Conversely, people who are not familiar enough with EA to have previously heard of 80k advising—which I think is a low bar, given many people learn about EA via 80k—probably won’t have successful applications. Thus, my model of the median successful referral is “someone who has heard of 80k but not previously opted to pursue 80k advising.” Which brings me to (3): by virtue of these people having not previously opted into a free service, I suspect that they’re less likely to benefit from it. In other words, I suspect that people referred as a result of this program will be less likely (or less able) to make changes as a result of their advising meetings. (Or at least this was the conclusion I came to in deciding who to send my referral links to.)
Regarding (4), I haven’t seen evidence to support the claim that “very engaged and agentic EAs… will use $5,000 very well to advance their careers and create good down the line,” and while this seems prima facie plausible, I don’t think that is the standard of evidence we should apply to this—or any—intervention. (This is a less important point, because if this program generated tons of great referrals, it wouldn’t really matter how the $50k was spent.)
All fair points. My biggest point of disagreement is (2). As a true example, only around a third of EAG(x) attendees have applied for advising. That’s a much more expensive program for both the org and the participant. Tons of relevant people effectively don’t know advising exists (e.g. maybe they’ve heard of it once, but if it didn’t make it to the top of their stack that day, they forget about it). Importantly, our highest counterfactual comes from talking to only modestly engaged people, so maybe people who’ve done some reading but only know 1-2 other EAs – I see this program being aimed at these not-highly-engaged people makes the call more worthwhile, not less. “You were up for a call if someone leaned on you a bit,” seems roughly ideal to me.
And how counterfactual were the (quality) referrals—are the resulting advising sessions unlikely to have ever happened otherwise, or would it be more appropriate to categorize most referrals as causing certain people to receive advising services sooner than they otherwise would have?
Thanks; this is helpful, and I appreciate your candor. I’m not questioning whether 80k’s advising overall is valuable, and am thus willing to grant stuff like “most of the shifts people make as a result of 80k advising are +EV”. My reservations mainly pertain to the following:
does this grant effectively incentivize referrals?
are those referrals of high quality?
contingent on 80k agreeing to meet with a referred party, is that party liable to make career shifts based on the advising they receive?
(to a lesser extent) will the recipients of the career grants use the money well?
I get that it’s easy to be critical of (1) post-hoc, but I think we should subject the general model of “give EAs a lot of money to do things that are easy and that have very uncertain and difficult to quantify value” to a high degree of scrutiny because (as best I can tell based on a small n) this: (a) hasn’t tended to work that well, (b) is self-serving, and (c) often seems to be held to a lower evidentiary standard than other kinds of interventions EAs fund. (A countervailing piece of evidence is that OP does this for hiring referrals, and they presumably do have good evidence re: efficacy, although the benefits there also seem much clearer for the reasons you mention.)
Regarding (2), my worry is that the people who get referred as a result of this program will be importantly different from the general population of people who receive 80k career advising. This is because I suspect highly engaged EAs will have already applied for or received 80k advising. Conversely, people who are not familiar enough with EA to have previously heard of 80k advising—which I think is a low bar, given many people learn about EA via 80k—probably won’t have successful applications. Thus, my model of the median successful referral is “someone who has heard of 80k but not previously opted to pursue 80k advising.” Which brings me to (3): by virtue of these people having not previously opted into a free service, I suspect that they’re less likely to benefit from it. In other words, I suspect that people referred as a result of this program will be less likely (or less able) to make changes as a result of their advising meetings. (Or at least this was the conclusion I came to in deciding who to send my referral links to.)
Regarding (4), I haven’t seen evidence to support the claim that “very engaged and agentic EAs… will use $5,000 very well to advance their careers and create good down the line,” and while this seems prima facie plausible, I don’t think that is the standard of evidence we should apply to this—or any—intervention. (This is a less important point, because if this program generated tons of great referrals, it wouldn’t really matter how the $50k was spent.)
All fair points. My biggest point of disagreement is (2). As a true example, only around a third of EAG(x) attendees have applied for advising. That’s a much more expensive program for both the org and the participant. Tons of relevant people effectively don’t know advising exists (e.g. maybe they’ve heard of it once, but if it didn’t make it to the top of their stack that day, they forget about it). Importantly, our highest counterfactual comes from talking to only modestly engaged people, so maybe people who’ve done some reading but only know 1-2 other EAs – I see this program being aimed at these not-highly-engaged people makes the call more worthwhile, not less. “You were up for a call if someone leaned on you a bit,” seems roughly ideal to me.
And how counterfactual were the (quality) referrals—are the resulting advising sessions unlikely to have ever happened otherwise, or would it be more appropriate to categorize most referrals as causing certain people to receive advising services sooner than they otherwise would have?