Max_Daniel

Karma: 5,813

I’m part of the Global Catastrophic Risks Capacity Building team at Open Philanthropy. Previously I was the Chief of Staff at the Forethought Foundation for Global Priorities Research, participated in the first cohort of FHI’s Research Scholars Programme (RSP), and then helped run it as one of its Project Managers. I also used to be the chair of the EA Infrastructure Fund.

Before that, my first EA-inspired jobs were with the Effective Altruism Foundation, e.g., running what is now the Center on Long-Term Risk. While I don’t endorse their ‘suffering-focused’ stance on ethics, I’m still a board member there.

Unless stated otherwise, I post on the Forum in a personal capacity, and don’t speak for any organization I’m affiliated with.

I like weird music and general abstract nonsense. In a different life I would be a mediocre mathematician or a horrible anthropologist.

Why s-risks are the worst existential risks, and how to prevent them

Max_Daniel2 Jun 2017 8:48 UTC

8 points

1 comment22 min readEA link

(www.youtube.com)

When should EAs allocate funding randomly? An inconclusive literature review.

Max_Daniel17 Nov 2018 14:53 UTC

42 points

10 comments42 min readEA link

Max_Daniel 18 Nov 2018 21:56 UTC
3 points
0 ∶ 0
in reply to: rk’s comment on: When should EAs allocate funding randomly? An inconclusive literature review.
do you have a vague impression of when randomisation might be a big win purely by reducing costs of evaluation?
Not really I’m afraid. I’d expect that due to the risk of inadvertent negative impacts and large improvements from weeding out obviously suboptimal options a pure lottery will rarely be a good idea. How much effort to expend beyond weeding out clearly suboptimal options to me likely seems to depend on contextual information specific to the use case. I’m not sure how much there is to be said in general except for platitudes along the lines of “invest time into explicit evaluation until the marginal value of information has diminished sufficiently”.

Max_Daniel 22 Nov 2018 18:43 UTC
2 points
0 ∶ 0
in reply to: Aaron Gertler’s comment on: When should EAs allocate funding randomly? An inconclusive literature review.
Hi Aaron, thank you for the suggestion. I agree that posting a more extensive summary would help readers decide if they should read the whole thing, and I will strongly consider doing so in case I ever plan to post similar things. For this specific post, I probably won’t add a summary because my guess is that in this specific case the size of the beneficial effect doesn’t justify the cost. (I do think extremely few people would use their time optimally by reading the post, mostly because it has no action-guiding conclusions and a low density of generally applicable insights.) I’m somewhat concerned that more people read this post than would be optimal just because there’s some psychological pull toward reading whatever you clicked on, and that I could reduce the amount of time spent suboptimally by having a shorter summary here, with accessing the full text requiring an additional click. However, my hunch is that this harmful effect is sufficiently small. (Also, the cost to me would be unusually high because I have a large ugh field around this project and would really like to avoid spending any more time on it.) But do let me know if you think replacing this text with a summary is clearly warranted, and thank you again for the suggestion!

Max_Daniel 22 Nov 2018 18:49 UTC
2 points
0 ∶ 0
in reply to: axioman’s comment on: When should EAs allocate funding randomly? An inconclusive literature review.
On your second point: I think you’re right, and that’s a great example. I’ve added a link to your comment to the post.

Max_Daniel 22 Nov 2018 18:57 UTC
2 points
0 ∶ 0
in reply to: axioman’s comment on: When should EAs allocate funding randomly? An inconclusive literature review.
On your first point: I agree that the paper just shows that, as you wrote, “if your decision strategy is to just choose the option you (naively) expect to be best, you will systematically overestimate the value of the selected option”.
I also think that “just choose the option you (naively) expect to be best” is an example of a “nonrandom, non-Bayesian decision strategy”. Now, the first sentence you quoted might reasonably be read to make the stronger claim that all nonrandom, non-Bayesian decision strategies have a certain property. However, the paper actually just shows that one of them does.
Is this what you were pointing to? If so, I’ll edit the quoted sentence accordingly, but I first wanted to check if I understood you correctly.
In any case, thank you for your comment!

Max_Daniel 22 Nov 2018 23:20 UTC
11 points
0 ∶ 0
on: Some cruxes on impactful alternatives to AI policy work
Thank you for posting this, I find it very interesting and useful to have discussions of this kind publicly available!
For now just one point, even though I don’t think it matters much for the high-level disagreement (in particular, I probably still disagree with Ben’s view on the impact of Google, Wikipedia etc.):
I don’t think current IT has had much of an effect by standard metrics of labour productivity, for example.
The context makes me think that maybe by “current IT” you specifically mean things like Facebook or Twitter that became big in the last 10 years. In that case, for all I know the quoted claim may well be correct. I’m not so sure if “current IT” includes e.g. the internet: I believe a prominent view in economics is that IT was a major cause of the US productivity growth resurgence in the 1990s to mid-2000s. For example:
- David Romer’s popular textbook Advanced Macroeconomics (4th ed., p. 32) says:
Until the mid-1990s, the rapid technological progress in computers and their introduction in many sectors of the economy appear to have had little impact on aggregate productivity. In part, this was simply because computers, although spreading rapidly, were still only a small fraction of the overall capital stock. And in part, it was because the adoption of the new technologies involved substantial adjustment costs. The growth-accounting studies find, however, that since the mid-1990s, computers and other forms of information technology have had a large impact on aggregate productivity.
- Gordon (2014, p. 6), who in general argues against techno-optimists and predicts a growth slowdown, describes 1996-2004 as “the productivity revival associated with the invention of e-mail, the internet, the web, and e-commerce”.
More broadly, the sense I got from the literature is that many people would be comfortable endorsing claims like (i) innovation has been and still is a major driver of productivity growth (say responsible for >10% of productivity growth), and (ii) within the last 10 years a significant share (weighted by impact on productivity, say again >10% of the effect) of innovation has happened in IT. (Admittedly, the arguments behind similar claims often seemed a bit handwavy to me and not as data-drived as I’d like.) So even if productivity growth has slowed down considerably and will remain low, IT would be responsible for a significant part of what little growth we have, and the absolute effect wouldn’t be less than one order of magnitude of typical effects of technology on productivity.
I think this all of this is consistent with e.g. the views that IT has increased productivity less than past innovations such as the steam engine, or that most people overestimate the effect of IT. I’d also guess it’s consistent with Cowen’s and Thiel’s views, but I haven’t read the books by them that you mentioned.
(I said “a prominent view” because I don’t have a good sense of whether it’s a majority view. In particular, I wasn’t able to find a relevant IGM Forum survey of economists. My overall impression is based on having engaged on the order of 10 hours with the relevant literature, albeit in an only moderately systematic way, and I don’t have a background in economics. I think there’s a good chance you’re aware of the above points, and I’m partly writing this comments to see if you or someone else can spot a flaw in my current impression.)

Max_Daniel 23 Nov 2018 19:10 UTC
1 point
0 ∶ 0
in reply to: axioman’s comment on: When should EAs allocate funding randomly? An inconclusive literature review.
Ok, thanks, I now say “Prove that a certain nonrandom, non-Bayesian …”.

Max_Daniel 8 Feb 2019 23:32 UTC
5 points
0 ∶ 0
in reply to: Ward’s comment on: Near-term focus, robustness, and flow-through effects
If I recall correctly this paper by Tom Sittler also makes the point you paraphrased as “some reasonable base rate of x-risk means that the expected lifespan of human civilization conditional on solving a particular risk is still hundreds or thousands of years”, among others.

Max_Daniel 26 Feb 2019 21:05 UTC
36 points
0 ∶ 0
on: After one year of applying for EA jobs: It is really, really hard to get hired by an EA organisation
Thank you and respect for having written this. I really appreciate this, particularly you being open about this having been mentally challenging for you and the concrete data on time cost you provide.

Max_Daniel 26 Feb 2019 21:52 UTC
125 points
0 ∶ 0
on: After one year of applying for EA jobs: It is really, really hard to get hired by an EA organisation
Adding some more data from my own experience last year.
Personally, I’m glad about some aspects of it and struggled with others, and there are some things I wish I had done differently, at least in hindsight. But here I just mean to quickly provide data I have collected anyway in a ‘neutral’ way, without implying anything about any particular application.
Total time I spent on ‘career change’ in 2018: at least 220h, of which at least about 101h were for specific applications. (The rest were things like: researching job and PhD opportunities; interviewing people about their jobs and PhD programs; asking people I’ve worked with for input and feedback; reflection before I decided in January to quit my previous job at the EA Foundation by April.) This does neither include 1 week I spent in in San Francisco to attend EAG SF and during which I was able to do little other work nor 250h of self-study that seems robustly useful but which I might not have done otherwise. (Nor 6 full weeks plus about 20h afterwards I spent doing an internship at an EA org, which overall I’m glad I did but might not have done otherwise.)
- Open Phil Research Analyst—rejected after conversation notes test − 16h [edit: worth noting that they offered compensation for the time spent on the trial task]
- OpenAI Fellows program—after more than 6 months got a rejection email encouraging me to apply again within the next 12 months − 5h [plus 175h studying machine learning including 46h on a project I tried to do specifically for that application—I count none of this as application cost because I think it was quite robustly useful]
- BERI project manager application—rejected immediately (the email was ambiguous between a regular desk reject and them actually not hiring at all for that role for now) − 1h
- Travelling to EAG SF from Germany to get advice on my career and find out about jobs - ~1 full week plus something between USD 1,000 and 5,000, which was between 10% and 50% of my liquid runway
- CEA Summer Research Fellowship [NB this was a 6-week internship, not a full-time role] - got an offer and accepted − 4.5h
- 2nd AI safety camp (October) [NB the core of this was a 1-2 week event organized by ‘grassroots’ efforts, and nothing that comes with funding above covering expenses] - got an offer and accepted − 1.2h
- FHI Research Scholars Programme—got an offer and accepted [this is what I’m doing currently] − 30h
- AI Impacts researcher—withdrew my application after the 1st interview because I accepted the FHI RSP offer − 44h [NB this was because I ‘unilaterally’ spent way more time to create a work sample than anyone had asked me to do, and in a quite inefficient way; I think one could have done an application in 1-5h if one had had a shovel-ready work sample. Again I’m excluding an additional 64h of teaching myself basic data processing and visualization skills with R because I think they are robustly useful.]
[I did manual time tracking so there might be some underestimation, with the error varying a lot between applications. A systematic error is that I never logged time spent in job interviews, but this is overall negligible.]
(I feel slightly nervous about sharing this. But I think the chance that it contributes to identifying if there are valuable changes to make in the overall talent/job landscape and messaging is well worth the expected cost; and also that as someone with a fixed-term but full-time job at an EA org I’m well-positioned to take some risks.)
What links here?
- Vaidehi Agarwalla's comment on Vaidehi Agarwalla’s Quick takes by Vaidehi Agarwalla (7 Jan 2021 2:24 UTC; 4 points)

Max_Daniel 26 Feb 2019 22:41 UTC
60 points
0 ∶ 0
on: After one year of applying for EA jobs: It is really, really hard to get hired by an EA organisation
Some related half-baked thoughts:
[Epistemic status: I appreciate that there are people who’ve thought about the EA talent landscape systematically and have access to more comprehensive information, e.g. perhaps some people at 80K or people doing recruiting for major EA orgs. I would therefore place significantly more weight on their impressions. I’m not one of these people. My thoughts are based on (i) having talked 10-100 hours with other EAs about related things over the last year, mostly in a non-focussed way, (ii) having worked full-time for 2 EA organizations (3 if one counts a 6-week internship), (iii) having hired 1-5 people for various projects at the Effective Altruism Foundation, (iv) having spent about 220h on changing my career last year, see another comment. I first heard of EA around October 2015, and have been involved in the community since April 2016. Most of that time I spent in Berlin, then over last summer and since October in Oxford.]
- I echo the impression that several people I’ve talked to—including myself—were or are overly focussed on finding a job at a major EA org. This applies both in terms of time spent and number of applications submitted, and in terms of more fuzzy notions such as how much status or success is associated with roles. I’m less sure if I disagreed with these people about the actual impact of ‘EA jobs’ vs. the next best option, but it’s at least plausible to me that (relative to my own impression) some of them overvalue the relative impact of ‘EA jobs’. E.g. my own guess is that a machine learning graduate course is competitive with most ‘EA jobs’ one could do well without such an education. [I think this last belief of mine is somewhat unusual and at least some very thoughtful people in EA disagree with me about this.]
- I think several people were in fact too optimistic about getting an ‘EA job’. It’s plausible they could have accessed information (e.g. do a Fermi estimate of how many people will apply for a role) that would have made them more pessimistic, but I’m not sure.
- I know at least 2 people who unsuccessfully applied to a large number of ‘EA jobs’. (I’m aware there are many more.) I feel confident that they have several highly impressive relevant skills, e.g. because I’ve seen some of their writing and/or their CVs. I’m aware I don’t know the full distribution of their relevant skills, and that the people who made the hiring decisions are in a much better position to make them than I. I’m still left with a subjective sense of “wow, these people are really impressive, and I find it surprising that they could not find a job”. This contributes to (i) me feeling more pressure to perform well in and more doubtful about the counterfactual impact of my current role because I have a visceral sense that ‘the next best candidate would have been about as good as I or better’ / ‘it would in some sense be tragic or unfair if I don’t perform well’ (these aren’t endorsed beliefs, but still affect me) , (ii) me being more reluctant to introduce new people into the EA community because and I don’t want them to make frustrating experiences, (iii) me being worried that some of my friends and other community members will make frustrating experiences [which costs attention and life satisfaction but also sometimes time e.g. when talking with someone about their frustration—as an aside, I’d guess that the burden of emotional labor of the latter kind is disproportionately shouldered by relatively junior women in the community]. (None of these effects are very large. I don’t want to make this sound more dramatic than it is, but overall I think there are non-negligible costs even for someone like me who got one of the competitive jobs.)
- I agree that identifying and promoting impactful roles outside of EA orgs may be both helpful for the ‘EA job market’ and impactful independently. I really like that the 80K job board sometimes includes such roles. I wonder if there is a diffusion of responsibility problem where identifying such jobs is no-one’s main goal and therefore doesn’t get done even if it would be valuable. [I also appreciate that this is really hard and costs a lot of time, and what I perceive to be 80K’s strategy on this, i.e. focussing on in-depth exploration of particularly valuable paths such as US AI policy, seems on the right track to me.]
- I think communication around this is really hard in general, and something that is particularly tricky for people like me and most EAs that are young and have little experience with similar situations. I also think there are some unavoidable trade-offs between causing frustration and increasing the expected quality of applicants for important roles. I applaud 80K for having listened to concerns around this in the past and having taken steps such as publishing a clarifying article on ‘talent constraints’. I think as a community we can still do better, but I’m optimistic that the relevant actors will be able to do so and certain that they have good intentions. I’ve seen EA leaders have valuable and important conversations around this, but it’s not quite clear to me if anyone in particular ‘owns’ optimizing the EA talent landscape at large, and so again wonder if there is a diffusion of responsibility issue that prevents ‘easy wins’ such as better data/feedback collection from getting done (while also being open to the possibility that ‘optimizing the EA talent landscape’ is too broad or fuzzy for one person to focus on it).
What links here?
- Max_Daniel's comment on My experience on a summer research programme by Jaime Sevilla (9 Oct 2019 12:17 UTC; 38 points)

Max_Daniel 27 Feb 2019 0:01 UTC
75 points
1 ∶ 0
in reply to: Max_Daniel’s comment on: After one year of applying for EA jobs: It is really, really hard to get hired by an EA organisation
One thing that might be worth noting: I was only able to invest that many resources because of things like (i) having had an initial runway of more than $10,000 (a significant fraction of which I basically ‘inherited’ / was given to me for things like academic excellence that weren’t very effortful for me), (ii) having a good relationship to my sufficiently well-off parents that moving back in with them always was a safe backup option, (iii) having access to various other forms of social support (that came with real costs for several underemployed or otherwise struggling people in my network).
I do think current conditions mean that we ‘lose’ more people in less comfortable positions than we otherwise would.
What links here?
- EA Survey 2019 Series: Careers and Skills by Neil_Dullaghan (7 Jan 2020 21:13 UTC; 46 points)
- Jon_Behar's comment on A Framework for Thinking about the EA Labor Market by Jon_Behar (12 May 2019 15:20 UTC; 11 points)

Max_Daniel 27 Feb 2019 11:22 UTC
23 points
1 ∶ 0
in reply to: Kirsten’s comment on: After one year of applying for EA jobs: It is really, really hard to get hired by an EA organisation
I think there are at least two effects where the world loses impact: (i) People in less privileged positions not applying for EA jobs; sometimes one of these would actually have been the best candidate. (ii) More speculatively (in the sense that I can’t point to a specific example, though my prior is this effect is very likely to be non-zero), people in less privileged positions might realize that it’s not possible for them to apply for many of the roles they perceived to be described as highest-impact and this might reduce their EA motivation/dedication in general, and make them feel unwelcome in the community.
I emphatically agree that them taking another potentially impactful job is positive. In fact, as I said in another comment, I wish there was more attention on and support for identifying and promoting such jobs.

Max_Daniel 28 Feb 2019 19:43 UTC
34 points
0 ∶ 0
in reply to: lukeprog’s comment on: After one year of applying for EA jobs: It is really, really hard to get hired by an EA organisation
In a nutshell, I’m worried that the people would not find the options you list exciting from their perspective, and instead would perceive not working in one of the 20 most competitive jobs at explicitly EA-motivated employers as some kind of personal shortcoming, hence the frustration.
I think the OP is evidence that his can happen e.g. because the author reports that
this is the message I felt I was getting from the EA community:
“Hey you! You know, all these ideas that you had about making the world a better place, like working for Doctors without Borders? They probably aren’t that great. The long-term future is what matters. And that is not funding constrained, so earning to give is kind of off the table as well. But the good news is, we really, really need people working on these things. We are so talent constraint… (20 applications later) … Yeah, when we said that we need people, we meant capable people. Not you. You suck.”
Note that I agree with you that in fact “[t]here are lots of exciting things for new EAs” including the options you’ve listed. However, even given this considered belief of mine, I think I was overly focussed on ‘EA jobs’ in a way that negatively affected my well-being.
Even when I consider that my guess is that I’m unusually susceptible to such psychological effects (though not extremely so, my crude guess would be ’80th to 99th percentile’), I’d expect some others to be similarly affected even if they agree—like I—about the impact of less competitive options.
Perhaps with “the kind of thing described in the original post” you meant specifically refer to the issue ‘people spend a lot of time applying for EA jobs’. Certainly a lot of the information in the OP and in one of my comments was about this. In that case I’d like to clarify that it’s not the time cost itself that’s the main cause of effects (i)-(iii) I described in the parent. In fact I somewhat regret to have contributed to the whole discussion perhaps being focused on time costs by providing more data exclusively about this. The core problem as I see it is how the OP, I, and I believe many others, think about and are psychologically affected by the current EA job market and the surrounding messaging. The objective market conditions (e.g. number of applicants for jobs) contribute to this, as do many aspects of messaging by EA orgs and EAs, as do things that have nothing to do with EA at all (e.g. people’s degree of neuroticism and other personality traits). I don’t have a strong view on which of these contributing factors is the best place to intervene.

Max_Daniel 28 Feb 2019 20:01 UTC
34 points
0 ∶ 0
in reply to: lukeprog’s comment on: After one year of applying for EA jobs: It is really, really hard to get hired by an EA organisation
A speculative thought I just had on one possible reason for why some people are overly focussed on EA jobs relative to e.g. the other options you list here:
- Identifying one’s highest-impact career option is quite challenging, and there is no way to easily conclusively verify a candidate answer.
- Therefore (and for other reasons), many people rely a lot on advice provided by 80K and individual EAs they regard as suitable advisors.
- At least within the core of the (longtermist) EA community, almost all sources of advice agree that one of the most competitive jobs at an explicitly EA-motivated org usually is among the top options for people who are a good fit.
- However, for most alternatives there is significant disagreement among the most trusted sources of advice on whether these alternatives are competitive (in terms of expected impact) with an ‘EA job’, or indeed good ideas at all. For example, someone who I believe many people consult for career advice discouraged me from ‘train up as a cybersecurity expert’ - an option I had brought up (and according to my own impression still consider an attractive option) -, at least relative to working at an EA org. Similarly, there are significant disagreements about the value of academic degrees, even in machine learning (and a bunch of hard-to-resolve underlying disagreements e.g. about how much ML experience is essential/useful for AI safety and strategy).
- As a result, people will often be faced with a distribution of views similar to: ‘Everyone agrees working at <EA org> would be great. Many people think a machine learning PhD would be great, one or two even think it’s better for me specifically, but a significant minority thinks it’s useless. One person was excited about cybersecurity, one person was pessimistic, and most said they couldn’t comment on it.’ Perhaps if all of these opinions had been conveyed with maximal reasoning transparency and one was extremely careful about aggregating the opinions this wouldn’t be a problem. But in practice I think this often means that ‘apply to <EA org>’ seems like the top option, at least in terms of psychological pull.
- (Another contributing factor to the large number of applications to EA jobs, perhaps less so for how it affects people, may be that that few EA orgs have a very explicit model of the specific skills they require for their most competitive jobs—at least that’s my impression. As a result, they cannot offer reliable guidance people can use to decide if they’re a good fit apart from applying.)
What links here?

Max_Daniel 28 Feb 2019 21:08 UTC
3 points
0 ∶ 0
in reply to: EA applicant’s comment on: After one year of applying for EA jobs: It is really, really hard to get hired by an EA organisation
I didn’t think people consistently recommended EA orgs over other options
Interesting, thank you for this data point. My speculation was partly based on recently having talked to people who told me something like “you’re the first one [or one of very few among many] who doesn’t clearly recommend me to choose <EA org> over <some other good option>”. It’s good to know that this isn’t what always happens.

Max_Daniel 4 Mar 2019 11:39 UTC
15 points
0 ∶ 0
in reply to: AGB’s comment on: After one year of applying for EA jobs: It is really, really hard to get hired by an EA organisation
FWIW, without having thought systematically about this, my intuition is to agree. I’d be particularly keen to see:
- More explicit models for what trainable skills and experiences are useful for improving the long-term future, or will become so in the future (as new institutions such as CSET are being established).
- More actionable advice on how to train these skills.
My gut feeling is that in many places we could do a better job at utilizing skills and experiences people can get pretty reliably in the for-profit world, academia, or from other established ‘institutions’.
I’m aware this is happening to some extent already, e.g. GPI trying to interface with academia or 80K’s guide on US policy. I think both are great!
NB this is different from the idea that there are many other career paths that would be high-impact to stay in indefinitely. I think this is also true, but at least if one has a narrow focus on the long-term future I feel less sure if there are ‘easy wins’ left here.
(An underlying disagreement here might be: Is this feasible, or are we just too much bottlenecked by something like what Carrick Flynn has called ‘disentanglement’. Very crudely, I tend to agree that we’re bottlenecked by disentanglement but that there are still some improvements we can make along the above lines. A more substantive underlying question might be how important domain knowledge and domain-specific skills are for being able to do disentanglement, where my impression is that I place an unusually high value on them whereas other EAs are closer to ‘the most important thing is to hang out with other EAs and absorb the epistemic norms, results, and culture’.)

Max_Daniel 11 Mar 2019 15:00 UTC
13 points
0 ∶ 0
in reply to: Howie_Lempel’s comment on: Getting People Excited About More EA Careers: A New Community Building Challenge
Of our top rated plan changes, only 25% involve people working at EA orgs
For what it’s worth, given how few EA orgs there are in relation to the number of highly dedicated EAs and how large the world outside of EA is (e.g. in terms of institutions/orgs that work in important areas or are reasonably good at teaching important skills), 25% actually strikes me as a high figure. Even if this was right, there might be good reasons for the figure being that high, e.g. it’s natural and doesn’t necessarily reflect any mistake that 80K knows more about which careers at EA orgs are high-impact, can do a better job at finding people for them etc. However, I would be surprised if as the EA movement becomes more mature the optimal proportion was as high.
(I didn’t read your comment as explicitly agreeing or disagreeing with anything in the above paragraph, just wanted to share my intuitive reaction.)
Thank you for your comments here, they’ve helped me understand 80K’s current thinking on the issue raised by the OP.

Max_Daniel 11 Mar 2019 15:11 UTC
5 points
0 ∶ 0
in reply to: GeorgeBridgwater’s comment on: Getting People Excited About More EA Careers: A New Community Building Challenge
Thanks for sharing, I suspect this might be somewhat common. I’ve speculated about a related cause in another comment.

Max_Daniel

Why s-risks are the worst ex­is­ten­tial risks, and how to pre­vent them

When should EAs al­lo­cate fund­ing ran­domly? An in­con­clu­sive liter­a­ture re­view.

Why s-risks are the worst existential risks, and how to prevent them

When should EAs allocate funding randomly? An inconclusive literature review.