It seems quite possible to me have a “parameterized list”. That is, recommendations can take the shape “If X is true of you, Y and Z are good options.” And in fact 80,000 Hours does do this to some degree (via, for example, their career quiz). While this isn’t entirely personalized (it’s based only on certain attributes that 80,000 Hours highlights), it’s also far from a single, definitive list. So it doesn’t seem to be that there’s any insoluble tension between taking account of individual difference and communicating the same message to a broad audience—you just have to rely on the audience to do some interpreting.
So it doesn’t seem to be that there’s any insoluble tension between taking account of individual difference and communicating the same message to a broad audience
I don’t think the tension is between those things. The tension is between saying ‘our research is useful: it tells (group X) of people what it is best for them to do’ and ‘our research does not offering a definite ranking of what it is before for people to do (whether people in group X or otherwise)’. I don’t think you can have this both ways.
While this isn’t entirely personalized (it’s based only on certain attributes that 80,000 Hours highlights), it’s also far from a single, definitive list
Then it seems reasonable to interpret it as (an attempt at) a definitive list if you have those attributes.
I understand why the author is arguing that 80k doesn’t offer a big list but I think that argument is undermines the claim that 80k is useful (“Hey, we’re not telling anyone what to do?” “Really? I thought that was the point”)
The tension is between saying ‘our research is useful: it tells (group X) of people what it is best for them to do’ and ‘our research does not offering a definitive ranking of what it is before for people to do (whether people in group X or otherwise)’.
Though I am saying that 80,000 Hours’ research can’t offer a single, definite ranking of what is best for everyone to do, that doesn’t mean that their research isn’t very useful for people figuring out what it is best for them to do.
The way I might put it: 80,000 Hours research helps people put together their own list of what is best for them to do, by (1) offering lots of information people need to combine with their own knowledge about themselves to build their list—e.g., what certain jobs are like, how people typically get into a particular job, and so on, (2) offering tools for people to use to figure out the information about themselves that they need—like for assessing personal fit, etc., and (3) offering guidance on how to prioritize options according to the impact that people in the roles can have under various different circumstances. 80,000 hours also does things like seek out specific positions and bring them to people’s attention.
All this is really useful, I believe, for helping people do the most good they can with their careers, without any of it amounting to creating a big list of what it’s best for everyone in group x (e.g., the EA community) to do.
Though I am saying that 80,000 Hours’ research can’t offer a single, definite ranking of what is best for everyone to do, that doesn’t mean that their research isn’t very useful for people figuring out what it is best for them to do
Well, they do offer A list of the most urgent global problems. I’ll grant this isn’t a list of what it is best for everyone to do, but it is (plausibly, from their perspective) a list of what it is best for most people to do (or ‘most EAs’ or some nearby specification). Indeed, given 80k has a concept of ‘personal fit’, which is distinct from their rating of the problems, the natural reading of the list is that it provides a general, impersonal ranking of where (average?) individuals can do the most good.
I’m concerned you’re defending a straw man - did anyone ever claim 80k’s list was true for every single possible person? I don’t think so and such a claim would be implausible.
I’m concerned you’re defending a straw man - did anyone ever claim 80k’s list was true for every single possible person? I don’t think so and such a claim would be implausible.
As an anecdote, I’ve always read their list and recommendations as applying to their target audience of talented graduates of elite Western colleges.
To be clear, I don’t know whether they specifically target elite college graduates. I was speaking slightly loosely and don’t have any inside information on 80k. It just seems to me that use elite colleges are a proxy for ambitious graduates.
Last I checked, the career quiz recommends almost everyone (including everyone “early” and “middle” career, no matter their other responses) either “Policy-oriented [UK] government jobs” or “[US] Congressional staffer”, so it hardly seems very reflective of actually believing that the “list” is very different for different people.
Hi lexande, Habryka, Milan — As you note, the quiz is no longer current content. It has been moved way down in the site structure, and carries this disclaimer:
“N.B. The results from this quiz were last reviewed in 2016 and the ranking may no longer reflect our current views. Your top results should be read as suggestions for further research, not recommendations. … The quiz doesn’t tell you what you should do, but can give you ideas to research further.”
Yep, I saw that. I didn’t actually intend to criticize your use of the quiz, sorry if it came across that way. I just gave it a try and figured I would contribute some data.
(This doesn’t mean I agree with how 80k communicates information. I haven’t kept up at all with 80k’s writing, so I don’t have any strong opinions either way here)
Yeah, my response was directed at cole_haus suggesting the quiz as an example of 80k currently providing personalized content, when in fact it’s pretty clearly deprecated, unmaintained, and no longer linked anywhere prominent within the site. (Though I’m not sure what purpose keeping it up at all serves at this point.)
Yeah, I hadn’t realized it was more or less deprecated. (The page itself doesn’t seem to give any indication of that. Edit: Ah, it does. I missed the second paragraph of the sidenote when I quickly scanned for some disclaimer.)
Also, apparently unfortunately, it’s the first sublink under the 80,000 Hours site on Google if you search for 80,000 Hours.
“The page itself doesn’t seem to give any indication of that.”
As I pointed out it says at the top:
“N.B. The results from this quiz were last reviewed in 2016 and the ranking may no longer reflect our current views. Your top results should be read as suggestions for further research, not recommendations. … The quiz doesn’t tell you what you should do, but can give you ideas to research further.”
We’ve been working to get it downgraded from the Google search results, but unfortunately we don’t have full control over that.
I like there being a record of out-of-date recommendations and tools on the 80K site [edit: so I know how they’ve updated, and so I can access the parts of old resources that aren’t out of date].
A curated list of Archive links might work OK as a replacement, I suppose. But in general, given that various pages have accumulated offsite hyperlinks over the years, I think it’s more informative to plaster giant “this content is out-of-of-date because X” disclaimers on the relevant pages, rather than just taking the page down.
That applies to most of the deprecated pages, but doesn’t apply to the quiz, because its results are based on the database of existing career reviews. The fact that it gives the same results for nearly everybody is the result of new reviews being added to that database since it was written/calibrated. It’s not actually possible to get it to show you the results it would have showed you back in 2016 the last time it was at all endorsed.
Yep, the quiz may be an exception! I was commenting on the general thread of discussion on this page “just take everything down that’s out of date,” and the quiz subthread was just the one that caught my eye. My apologies for making it sound like the quiz in particular is the thing I want preserved; I don’t have a strong view on that.
FWIW, I was specifically looking for a disclaimer and it didn’t quickly come to my attention. It looks like a few other people in these subthreads may have also missed the disclaimer.
Congressional staffer and policy oriented jobs are the top two highest weighted profiles of about 20 so everyone will automatically get these (followed by a ranked list of the remaining 15 options, but only the first profile is displayed, possibly a glitch). The biggest filter is quantitative—if you select no it cuts 15 profiles, then a cut of any profiles with weights of 0.
The weights are biased—it’s impossible to get arts or marketing as a final result (because it’s not a recommended job pathway within EA). The basic premise is if you are good at math and science—do these high-impact math and science jobs. If not, do any other non-quantitative high-impact job we recommend.
The results aren’t personal enough to have it be designed like this—a simple list of all 35 profiles with a quantitative skills filter and then ranking profiles by 80k’s weights would be sufficient enough. It’s really not well-suited for a quiz, but a filtered list like the job board might be more effective.
Note that the “policy-oriented government job” article is specific to the UK. Some of the arguments about impact may generalize but the civil service in the UK in general has more influence on policy than in the US or some other countries, and the more specific information about paths in etc doesn’t really generalize at all.
It seems quite possible to me have a “parameterized list”. That is, recommendations can take the shape “If X is true of you, Y and Z are good options.” And in fact 80,000 Hours does do this to some degree (via, for example, their career quiz). While this isn’t entirely personalized (it’s based only on certain attributes that 80,000 Hours highlights), it’s also far from a single, definitive list. So it doesn’t seem to be that there’s any insoluble tension between taking account of individual difference and communicating the same message to a broad audience—you just have to rely on the audience to do some interpreting.
I don’t think the tension is between those things. The tension is between saying ‘our research is useful: it tells (group X) of people what it is best for them to do’ and ‘our research does not offering a definite ranking of what it is before for people to do (whether people in group X or otherwise)’. I don’t think you can have this both ways.
Then it seems reasonable to interpret it as (an attempt at) a definitive list if you have those attributes.
I understand why the author is arguing that 80k doesn’t offer a big list but I think that argument is undermines the claim that 80k is useful (“Hey, we’re not telling anyone what to do?” “Really? I thought that was the point”)
Though I am saying that 80,000 Hours’ research can’t offer a single, definite ranking of what is best for everyone to do, that doesn’t mean that their research isn’t very useful for people figuring out what it is best for them to do.
The way I might put it: 80,000 Hours research helps people put together their own list of what is best for them to do, by (1) offering lots of information people need to combine with their own knowledge about themselves to build their list—e.g., what certain jobs are like, how people typically get into a particular job, and so on, (2) offering tools for people to use to figure out the information about themselves that they need—like for assessing personal fit, etc., and (3) offering guidance on how to prioritize options according to the impact that people in the roles can have under various different circumstances. 80,000 hours also does things like seek out specific positions and bring them to people’s attention.
All this is really useful, I believe, for helping people do the most good they can with their careers, without any of it amounting to creating a big list of what it’s best for everyone in group x (e.g., the EA community) to do.
Well, they do offer A list of the most urgent global problems. I’ll grant this isn’t a list of what it is best for everyone to do, but it is (plausibly, from their perspective) a list of what it is best for most people to do (or ‘most EAs’ or some nearby specification). Indeed, given 80k has a concept of ‘personal fit’, which is distinct from their rating of the problems, the natural reading of the list is that it provides a general, impersonal ranking of where (average?) individuals can do the most good.
I’m concerned you’re defending a straw man - did anyone ever claim 80k’s list was true for every single possible person? I don’t think so and such a claim would be implausible.
As an anecdote, I’ve always read their list and recommendations as applying to their target audience of talented graduates of elite Western colleges.
Have they ever admitted to specifically targeting graduates of elite colleges rather than ambitious graduates generally?
To be clear, I don’t know whether they specifically target elite college graduates. I was speaking slightly loosely and don’t have any inside information on 80k. It just seems to me that use elite colleges are a proxy for ambitious graduates.
Yup. In which case, it is a ‘big list’ for such folks.
Last I checked, the career quiz recommends almost everyone (including everyone “early” and “middle” career, no matter their other responses) either “Policy-oriented [UK] government jobs” or “[US] Congressional staffer”, so it hardly seems very reflective of actually believing that the “list” is very different for different people.
fwiw I’ve never gotten those outcomes when I’ve taken the quiz.
I got them on basically every setting that remotely applied to me.
Hi lexande, Habryka, Milan — As you note, the quiz is no longer current content. It has been moved way down in the site structure, and carries this disclaimer:
Yep, I saw that. I didn’t actually intend to criticize your use of the quiz, sorry if it came across that way. I just gave it a try and figured I would contribute some data.
(This doesn’t mean I agree with how 80k communicates information. I haven’t kept up at all with 80k’s writing, so I don’t have any strong opinions either way here)
Yeah, my response was directed at cole_haus suggesting the quiz as an example of 80k currently providing personalized content, when in fact it’s pretty clearly deprecated, unmaintained, and no longer linked anywhere prominent within the site. (Though I’m not sure what purpose keeping it up at all serves at this point.)
Yeah, I hadn’t realized it was more or less deprecated. (The page itself doesn’t seem to give any indication of that. Edit: Ah, it does. I missed the second paragraph of the sidenote when I quickly scanned for some disclaimer.)
Also, apparently unfortunately, it’s the first sublink under the 80,000 Hours site on Google if you search for 80,000 Hours.
“The page itself doesn’t seem to give any indication of that.”
As I pointed out it says at the top:
We’ve been working to get it downgraded from the Google search results, but unfortunately we don’t have full control over that.
Why don’t you just take it down entirely? It’s already basically non-functional.
I like there being a record of out-of-date recommendations and tools on the 80K site [edit: so I know how they’ve updated, and so I can access the parts of old resources that aren’t out of date].
A curated list of Archive links might work OK as a replacement, I suppose. But in general, given that various pages have accumulated offsite hyperlinks over the years, I think it’s more informative to plaster giant “this content is out-of-of-date because X” disclaimers on the relevant pages, rather than just taking the page down.
That applies to most of the deprecated pages, but doesn’t apply to the quiz, because its results are based on the database of existing career reviews. The fact that it gives the same results for nearly everybody is the result of new reviews being added to that database since it was written/calibrated. It’s not actually possible to get it to show you the results it would have showed you back in 2016 the last time it was at all endorsed.
Yep, the quiz may be an exception! I was commenting on the general thread of discussion on this page “just take everything down that’s out of date,” and the quiz subthread was just the one that caught my eye. My apologies for making it sound like the quiz in particular is the thing I want preserved; I don’t have a strong view on that.
Perhaps the case for keeping the page up has something to do with the page being highly ranked on Google search...
Ah, I see that now. Thanks.
FWIW, I was specifically looking for a disclaimer and it didn’t quickly come to my attention. It looks like a few other people in these subthreads may have also missed the disclaimer.
It’s not as prominent as it should be. We’re going to fix that.
Ah, I hadn’t taken the quiz in a couple years. Looks like they’ve changed it since then.
I just tried 5 different answer-configurations of the quiz: https://80000hours.org/career-quiz/
And got “congressional staffer” or “policy-oriented government job” for all configurations. Guess I should move to DC.
Congressional staffer and policy oriented jobs are the top two highest weighted profiles of about 20 so everyone will automatically get these (followed by a ranked list of the remaining 15 options, but only the first profile is displayed, possibly a glitch). The biggest filter is quantitative—if you select no it cuts 15 profiles, then a cut of any profiles with weights of 0.
The weights are biased—it’s impossible to get arts or marketing as a final result (because it’s not a recommended job pathway within EA). The basic premise is if you are good at math and science—do these high-impact math and science jobs. If not, do any other non-quantitative high-impact job we recommend.
The results aren’t personal enough to have it be designed like this—a simple list of all 35 profiles with a quantitative skills filter and then ranking profiles by 80k’s weights would be sufficient enough. It’s really not well-suited for a quiz, but a filtered list like the job board might be more effective.
Note that the “policy-oriented government job” article is specific to the UK. Some of the arguments about impact may generalize but the civil service in the UK in general has more influence on policy than in the US or some other countries, and the more specific information about paths in etc doesn’t really generalize at all.