Thanks for this post Jack, I found it really useful as I haven’t got round yet to reading the updated paper. This break down in the cluelessness section was a new arrangement to me. Does anyone know if this break down has been used elsewhere? If not this seems like useful progress in better defining the cluelessness objections to longtermism.
RobertHarling
Karma: 730
Thanks very much for your post! I think this a really interesting idea and it’s really useful to learn from your experience in this area.
What would you think of the concern that these types of ads would be a “low fidelity” way of spreading EA that could risk misinforming people about EA? I think from my experience community building, it’s really useful to be able to describe and discuss EA ideas in detail, and that there are risks to giving someone an incorrect view of EA. These risks include someone being critical of what they believe EA is, and spreading this critique, as well as discouraging them from getting involved when they may have done so at a later time. The risk is probably lower if someone clicks on a short ad that takes them to say effectivealtruism.com where the various ideas are carefully explained and introduced. But someone who only saw the ads and didn’t click could end up with an incorrect view of EA.
I would be interested to see discussion about what would and wouldn’t make a good online ad for EA e.g. how to intrigue people without being inaccurate or over-sensationalizing parts of EA.
There might also be an interesting balance between how much interest we want to someone to have shown in EA-related topics before advertising to them. E.g. every university student in the US is probably too wide a net, but everyone who’s searching “effective altruism” or “existential risk” are probably already on their way to EA resources without the need for an advert.
I know lots of university EA groups make use of Facebook advertising and some have found this useful to promote events. I don’t know whether Google/Youtube ads allow targeting at the level of students of a specific university?
I think I would have some worry that if external evaluations of individual grant recipients became common, this could discourage people from applying from grants in future, for fear of being negatively judged should the project not work out.
Potential grant recipients might worry that external evaluators may not have all the information about their project or the grant makers reasoning for awarding the grant. This lack of information could then lead to unfair or incorrect evaluations. This would be more a risk if it becomes common for people to write low quality evaluations that are weakly reasoned, uncharitable or don’t respect privacy. I’m unsure whether it would be easy to encourage high quality evaluations (such as your own) without also increasing the risk of low quality evaluations.The risk of discouraging grant applications would probably be greater for more speculative funds such as the Long Term Future Fund (LTFF), as it’s easier for projects to not work out and look like wasted funds to uninformed outsiders.
There could also be an opposite risk that by seeking to discourage low quality evaluations, we discourage people too much from evaluating and criticizing work. It might be useful to establish key principles for writing evaluations that enable people to right respectful and useful evaluations, even with limited knowledge or time.
I’m unsure where the right trade-off between usefully evaluating projects, and not discouraging grant applications would be. Thank you for your review of the LTFF recipients and for posting this question, I found both really interesting.
Thanks for sharing this paper, I had not heard of it before and it sounds really interesting.
Thanks for your comment Jack, that’s a really great point. I suppose that we would seek to influence AI slightly differently for each reason:
Reduce chance of unaligned/uncontrolled AI
Increase chance of useful AI
Increase chance of exactly aligned AI
e.g. you could reduce the chance of AI risk by stopping all AI development but then lose the other two benefits, or you could create a practically useful AI but not one that would guide humanity towards an optimal future. That being said I reckon in practice a lot of work to improve the development of AI would hit all 3. Though maybe if you view one reason as much more important than the others then you focus on a specific type of AI work.
Thank you very much for this post, I found it very interesting. I remember reading the original paper and feeling a bit confused by it. It’s not too fresh in my mind so I don’t feel too able to try to defend it. I appreciate you highlighting how the method they use to estimate f_l is unique and drives their main result.
A range of 0.01 to 1 for fl in your preferred model seems surprisingly high to me, though I don’t understand the Lineweaver Davis paper well enough to really comment on its result which I think your range is based on. I think they mention how their approach leaves uncertainty in n_e as to what counts as a terrestrial planet. I wonder if most estimates of any one parameter have a tendency to shift uncertainty onto other parameters, so that when combining individual estimates of each of parameter you end up with an unrealistically certain result.
Thanks for your comment athowes. I appreciate your point that I could have done more in the post to justify this “binary” of good and optimal.
Though the simulated minds scenario I described seems at first to be pretty much optimal, it could be much larger if you thought it would last for many more years. Given large enough uncertainty about future technology, maybe seeking to identify the optimal future is impossible.
I think your resources, value and efficiency model is really interesting. My intuition is that values is the limiting factor. I can believe there are pretty strong forces that mean that humanity will eventually end up optimising resources and efficiency, but less confident values will converge to the best ones over time. This probably depends on whether you think a singleton will form at some point, and then it feels like the limit is how good the values of the singleton are.
Thanks again for creating this post Neel. I can confirm I managed to write and publish my post in time!
I think without commiting to writing it here, my post would either have been made a few months later, or perhaps not been published at all.
Good v. Optimal Futures
Thanks for your comment!
I hadn’t thought to think about selection effects, thanks for pointing that out. I suppose Bostrom actually describes black balls as technologies that cause catastrophe but doesn’t set the bar as high as extinction. Then drawing a black ball doesn’t affect future populations drastically, so perhaps selection effects don’t apply?
Also, I think in The Precipice Toby Ord makes some inferences for natural extinction risk given the length of time humanity has existed for? Though I may not be remembering correctly. I think the logic was something like “Assume we’re randomly distributed amongst possible humans. If existential risk was very high, then there’d be a very small set of worlds in which humans have been around for this long, and it would be very unlikely that we’d be in such a world. Therefore it’s more likely that our estimate of existential risk is too high”. This then seems quite similar to my model of making inferences based on not having previously drawn a black ball. I don’t think I understand selection effects too well though so I appreciate any comments on this!
A toy model for technological existential risk
Commitment: I commit to writing a post on a vague idea about where most of the value of the long term future is and how sensitive it is to different values by 7pm on 11th December.
Thanks for suggesting this Neel!
Thanks for this post Akash, I found it really interesting to read. I definitely agree with your point about how friendly EAs can be when you reach out to them. I think this is something I’ve been aware of for a while, but it still takes me time to internalise and make myself more willing to reach out to people. But it’s definitely something I want to push myself to do more, and encourage other people to do. No one is going to be unhappy about someone showing an interest in their work and ideas!
This is a really interesting idea. I think I instinctively have a couple of concerns about such an idea
1) What is the benefit of such statements? Can we expect the opinion of the EA community to really carry much weight beyond relatively niche areas?
2) Can the EA community be sufficiently well defined to collect opinion? It is quite hard to work out who identifies as an EA, not least because some people are unsure themselves. I would worry any attempt to define the EA community too strictly (such as when surveying the community’s opinion) could come across as exclusionary and discourage some people from getting involved.
Thanks for your response!
I definitely see your point on the value of information to the future civilisation. The technology required to reach the moon and find the cache is likely quite different to the level required to resurrect humanity from the cache so the information could still be very valuable.
An interesting consideration may be how we value a planet being under human control vs control of this new civilisation. We may think we cannot assume that the new civilisation would be doing valuable things but that a human planet would be quite valuable. This consideration would depend a lot on your moral beliefs. If we don’t extrapolate the value of humanity to the value of this new civilisation, we could then ask whether we can extrapolate from how humanity would respond to finding the cache on the moon to how the new civilisation would respond.
Ah yes! Thanks for pointing that out!
Thanks for your comment, I found that paper really interesting and it was definitely an idea I’d not considered before.
My main two questions would be:
1) What is the main value of humanity being resurrected? - We could inherently value the preservation of humanity and it’s culture. However, my intuition would be that humanity would be resurrected in small numbers and these humans might not even have very pleasant lives if they’re being analysed or experimented on. Furthermore the resurrected humans are likely to have very little agency, being controlled by technologically superior beings. Therefore it would seem unlikely that the resurrected humans could create much value, much less achieve a grand future.
2) How valuable would information on humanity be to a civilisation that had technologically surpassed it? - The civilisation that resurrected humanity would probably be much more technologically advanced than humanity, and might even have it’s own AI as mentioned in the paper. It would then seem that it must have overcome many of the technological x-risks to reach that point, so information on humanity succumbing to one may not be that useful. It may not be prepared for certain natural x-risks that could have caused human extinction, but these seem much less likely than manmade x-risks.
Thanks again for such an interesting paper!
Thanks for your comment. “Caring about more” is quite a vague way of describing what I wanted to say. I think I was just trying to say that the risk of a true existential event from A is about 7 times greater than the risk from B (as 0.7/0.095 =~ 7.368) so it would be 7 times not 70 times?
Considering evolutionary timelines is definitely very hard because it’s such a chaotic process. I don’t have too much knowledge about evolutionary history and am hoping to research this more. I think after most human existential events, the complexity of the life that remains would be much greater than that for most of the history of the Earth. So although it took humans 4.6 billion years to evolve “from scratch”, it could take significantly less time for intelligent life to re-evolve after an existential event as a lot of the hard evolutionary work has already been done.
I could definitely believe it could take longer than 0.5 billion years for intelligent life to re-evolve, but I’d be very uncertain on that and give some credence that it could take significantly less time. For example, humanity evolved “only” 65 million years after the asteroid that caused the dinosaur extinction.
The consideration of how “inevitable” intelligence is in evolution is very interesting. One argument that high intelligence would be likely to re-emerge could be that humanity has shown it to be a very successful strategy. So it would just take one species to evolve high levels of intelligence for there to then become a large number of intelligent beings on Earth again.
(Apologies for my slow reply to your comment!)
Thanks for this interesting analysis! Do you have a link to Foster’s analysis of MindEase’s impact?
How do you think the research on MindEase’s impact compares to that of GiveWell’s top charities? Based on your description of Hildebrandt’s analysis for example, it seems less strong than e.g. the several randomized control trials supporting distributing bed nets. Do you think discounting based on this could substantially effect the cost-effectiveness? (Given how much lower Foster’s estimate of impact is though and that this is more heavily used in the overall cost-effectiveness, I would be interested to see whether this has a stronger evidence base?)