This all makes sense, and it does seem that people who are launching big projects might benefit from paid emails as a norm. On the other hand, you seem unusually worried about “spamming” people by sending them things it’s pretty plausible they’d be interested in. It would be fairly easy to put at the top of your email something like “If you’re interested in doing AI forecasting, read on; otherwise feel free to ignore this email” which means the cost is something like ~10 seconds per uninterested recipient, which seems reasonable.
On a meta note, I think I felt less positively towards this post than I otherwise would have, because it felt like a call to action (which I hold to high standards) rather than an exploratory poll—e.g. I read the first few bullet points as rhetorical questions. Seems like it was just a phrasing issue; and as an exploratory poll, I think it’s interesting and I’m glad to have had the issue brought to mind :)
It’s not clear to me that we are in a mess. The only actual example you gave was a spammy corporate newsletter, which seems irrelevant.
This might look as follows: Lots of people write to senior researchers asking for feedback on papers or ideas, yet they’re mostly crackpots or uninteresting, so most stuff is not worth reading. A promising young researcher without many connections would want their feedback (and the senior researcher would want to give it!), but it simply takes too much effort to figure out that the paper is promising, so it never gets read. In fact, expecting this, the junior researcher might not even send it in the first place.
Does this happen much? Have you received feedback from people saying that this has happened to them? I expect personal networks in EA to be pretty good at connecting people—and if a young researcher is promising they can often explain why in a sentence or two (even if it’s just by name-dropping previous positions).
Currently, the signalling problem is solved by things like:
Spending lots of effort crafting interesting-sounding intros which signal that the thing is worth reading, instead of just getting to the point
Burning social capital—adding tags like “[Urgent]” or “[Important]” to the subject line
Does the latter actually happen? I’ve never seen it. Also, why is the former bad? It seems like an even better costly signal than paying money to send emails because it also produces a short description of the work which helps the recipient evaluate it. And very few people have too little time to skim a paragraph-long summary.
I think I enjoyed Diaspora more, and it seems a little more relevant to far-future considerations. What about Permutation City in particular did you like?
Meta: your last link doesn’t seem to point anywhere.
Interesting post. I wanted to write a substantive response, but ran out of energy. However, I have written previously on why I’m skeptical of the relevance of formally defined utility functions to ethics. Here’s one essay about the differences between people’s preferences and the type of “utility” that’s morally valuable. Here’s one about why there’s no good way to ground preferences in the real world. And here’s one attacking the underlying mindset that makes it tempting to model humans are agents with coherent goals.
There are two functions I’m looking for: the “archive/index” function, and the “sequences” function. The former should store as much EA content as possible (including stuff that’s not high-quality enough for us to want to direct newcomers to); it’d ideally also have enough structure to make it easily-browsable. The latter should zoom in on a specific topic or person and showcase their ideas in a way that can be easily read and digested.
https://priority.wiki/ is somewhere in between those two, in a way that seems valuable, but that doesn’t quite fit with the functions I outlined above. It doesn’t seem like it’s aiming to be an exhaustive repository of content. But the individual topic pages also don’t seem well-curated enough that I could just point someone to them and say “read all the stuff on this page to learn about the topic”. The latter might change as more work goes into it, but I’m more hopeful about the EA forum sequences feature for this purpose.
The list of syllabi on EAHub is also interesting, and fits with the sequences function, albeit only on one specific topic (introducing EA).
Are those what you were referring to, or are there other places on EAHub where (object-level) content is collected that I didn’t spot?
I was particularly reminded of this by spending twenty minutes yesterday searching for an EA blog I wanted to cite, which has somehow vanished into the aether. EDIT: never mind, found it.
Collection and curation of EA content from across the internet in a way that’s accessible to newcomers, easily searchable, and will last for decades (at least). Seems like it wouldn’t take that long to do a decent job, doesn’t require uncommon skills, and could be pretty high-value.
I would be open to paying people to do this; message me if interested.
I think Eli was asking whether your whole response was a quote, since the whole thing is in block quote format.
What’s your position on people coming for only part of the workshop? I’d be interested in attending but would rather not miss more than one day of work.
Strong +1 for the kalzumeus blog post, that was very helpful for me.
In general, stocking programmes aim at supporting commercial fisheries.
I’m a little confused by this, since it seems hugely economically inefficient to go to all the effort of raising fish, only to release them and then recapture them. Am I missing something, or is this basically a make-work program for the fishing industry?
Note that your argument here is roughly Ben Pace’s position in this post which we co-wrote. I argued against Ben’s position in the post because I thought it was too extreme, but I agree with both of you that most EAs aren’t going far enough in that direction.
Excellent post, although I think about it using a slightly different framing. How vetting-constrained granters are depends a lot on how high their standards are. In the limit of arbitrarily high standards, all the vetting in the world might not be enough. In the limit of arbitrarily low standards, no vetting is required.
If we find that there’s not enough capability to vet, that suggests that either our standards are correct and we need more vetters, or that our standards are too high and we should lower them. I don’t have much inside information, so this is mostly based on my overall worldview, but I broadly think it’s more the latter: that standards are too high, and that worrying too much about protecting EA’s reputation makes it harder for us to innovate.
I think it would be very valuable to have more granters publicly explaining how they make tradeoffs between potential risks, clear benefits, and low-probability extreme successes; if these explanations exist and I’m just not aware of them, I’d appreciate pointers.
Another startup contacted at least 4 grantmaking organisations. Three of them deferred to the fourth.
One “easy fix” would simply be to encourage grantmakers to defer to each other less. Imagine that only one venture capital fund was allowed in Silicon Valley. I claim that’s one of the worst things you could do for entrepreneurship there.
I agree that all of the things you listed are great. But note that almost all of them look like “convince already-successful people of EA ideas” rather than “talented young EAs doing exceptional things”. For the purposes of this discussion, the main question isn’t when we get the first EA senator, but whether the advice we’re giving to young EAs will make them more likely to become senators or billion-dollar donors or other cool things. And yes, there’s a strong selection bias here because obviously if you’re young, you’ve had less time to do cool things. But I still think your argument weighs only weakly against Vishal’s advocacy of what I’m tempted to call the “Silicon Valley mindset”.
So the empirical question here is something like, if more EAs steer their careers based on a Silicon Valley mindset (as opposed to an EA mindset), will the movement overall be able to do more good? Personally I think that’s true for driven, high-conscientiousness generalists, e.g. the sort of people OpenPhil hires. For other people, I guess what I advocate in the post above is sort of a middle ground between Vishal’s “go for extreme growth” and the more standard EA advice to “go for the most important cause areas”.
Is this not explained by founder effects from Less Wrong?
One other thing that I just noticed: looking at the list of 80k’s 10 priority paths found here, the first 6 (and arguably also #8: China specialist) are all roles for which the majority of existing jobs are within an EA bubble. On one hand, this shows how well the EA community has done in creating important jobs, but it also highlights my concern about us steering people away from conventionally successful careers and engagement with non-EAs.
This just seems like an unusually bad joke (as he also clarifies later). I think the phenomenon you’re talking about is real (although I’m unsure as to the extent) but wouldn’t use this as evidence.
Hi Michelle, thanks for the thoughtful reply; I’ve responded below. Please don’t feel obliged to respond in detail to my specific points if that’s not a good use of your time; writing up a more general explanation of 80k’s position might be more useful?
You’re right that I’m positive about pretty broad capital building, but I’m not sure we disagree that much here. On a scale of breadth to narrowness of career capital, consulting is at one extreme because it’s so generalist, and the other extreme is working at EA organisations or directly on EA causes straight out of university. I’m arguing against the current skew towards the latter extreme, but I’m not arguing that the former extreme is ideal. I think something like working at a top think tank (your example above) is a great first career step. (As a side note, I mention consulting twice in my post, but both times just as an illustrative example. Since this seems to have been misleading, I’ll change one of those mentions to think tanks).
However, I do think that there are only a small number of jobs which are as good on so many axes as top think tanks, and it’s usually quite difficult to get them as a new grad. Most new grads therefore face harsher tradeoffs between generality and narrowness.
More importantly, in order to help others as much as we can, we really need to both work on the world’s most pressing problems and find what inputs are most needed in order to make progress on them. While this will describe a huge range of roles in a wide variety of areas, it will still be the minority of jobs.
I guess my core argument is that in the past, EA has overfit to the jobs we thought were important at the time, both because of explicit career advice and because of implicit social pressure. So how do we avoid doing so going forward? I argue that given the social pressure which pushes people towards wanting to have a few very specific careers, it’s better to have a community default which encourages people towards a broader range of jobs, for three reasons: to ameliorate the existing social bias, to allow a wider range of people to feel like they belong in EA, and to add a little bit of “epistemic modesty”-based deference towards existing non-EA career advice. I claim that if EA as a movement had been more epistemically modest about careers 5 years ago, we’d have a) more people with useful general career capital, b) more people in things which didn’t use to be priorities, but now are, like politics, c) fewer current grads who (mistakenly/unsuccessfully) prioritised their career search specifically towards EA orgs, and maybe d) more information about a broader range of careers from people pursuing those paths. There would also have been costs to adding this epistemic modesty, of course, and I don’t have a strong opinion on whether the costs outweight the benefits, but I do think it’s worth making a case for those benefits.
We’ve updated pretty substantially away from that in favour of taking a more directed approach to your career
Looking at this post on how you’ve changed your mind, I’m not strongly convinced by the reasons you cited. Summarised:
1. If you’re focused on our top problem areas, narrow career capital in those areas is usually more useful than flexible career capital.
Unless it turns out that there’s a better form of narrow career which it would be useful to be able to shift towards (e.g. shifts in EA ideas, or unexpected doors opening as you get more senior).
2. You can get good career capital in positions with high immediate impact
I’ve argued that immediate impact is usually a fairly unimportant metric which is outweighed by the impact later on in your career.
3. Discount rates on aligned-talent are quite high in some of the priority paths, and seem to have increased, making career capital less valuable.
I am personally not very convinced by this, but I appreciate that there’s a broad range of opinions and so it’s a reasonable concern.
It still seems to be the case that organisations like the Open Philanthropy Project and GiveWell are occasionally interested in hiring people 0-2 years out of university. And while there seem to be some people to whom working at EA organisations seems more appealing than it should, there are also many people for whom it seems less appealing or cognitively available than it should. For example, while the people on this forum are likely to be very inclined to apply for jobs at EA organisations, many of the people I talk to in coaching don’t know that much about various EA organisations and why they might be good places to work.
Re OpenPhil and GiveWell wanting to hire new grads: in general I don’t place much weight on evidence of the form “organisation x thinks their own work is unusually impactful and worth the counterfactual tradeoffs”.
I agree that you have a very difficult job in trying to convey key ideas to people who are are coming from totally different positions in terms of background knowledge and experience with EA. My advice is primarily aimed at people who are already committed EAs, and who are subject to the social dynamics I discuss above—hence why this is a “community” post. I think you do amazing work in introducing a wider audience to EA ideas, especially with nuance via the podcast as you mentioned.
I quite like this idea, and think that the unilateralist’s curse is less important than others make it out to be (I’ll elaborate on this in a forum post soon).
Just wanted to quickly mention https://lets-fund.org/ as a related project, in case you hadn’t already heard of it.