Impact Academy is a non-profit organization that enables people to become world-class leaders, thinkers, and doers who are using their careers and character to solve our most pressing problems and create the best possible future.
I also work as an impact-driven and truth-seeking coach for people who are trying to do the most good.
I’m also a medical doctor, author, and former visiting researcher (biosecurity) at Stanford.
Thanks. Hmm. The vibe I’m getting from these answers is P(extinction)>5% (which is higher than the XST you linked).
Ohh that’s great. We’re starting to do significant work in India and would be interested in knowing similar things there. Any idea of what it’d cost to run there?
Thanks for sharing. This is a very insightful piece. Im surprised that folks were more concerned about larger scale abstract risks compared to more well defined and smaller scale risks (like bias). I’m also surprised that they are this pro regulation (including a Sox months pause). Given this, I feel a bit confused that they mostly support the development of AI and I wonder what had most shaped their view.
Overall, I mildly worry that the survey led people to express more concern than they feel. Because this seems surprisingly close to my perception of the views of many existential risks “experts”. What do you think?
Would love to see this for other countries too. How feasible do you think that would be?
Thanks for writing this up and sharing. I strongly appreciate the external research evaluation initiative and was generally impressed with the apparent counterfactual impact.
Thanks for your response Ben. All of these were on my radar but thanks for sharing.
Good luck with what you’ll be working on too!
Congratulations! I’m looking forward to understanding the strategic direction you and CEA will pursue going forward. Do you have a sense of when you’d be able to share a version of these?
2.
- How do you define guided self-help? Do you mean facilitated group sessions?
- Do you have any specific papers/references that you’ve used to for those estimates?
Thanks for writing this up. We’re running an incubation pilot at Impact Academy and found this post very helpful as a reference class (for comparison in terms of success) as well as providing strategic clarity.
I’m curious, what were the best initiatives (inside and outside of EA) you came across in your search (e.g., y-combinator, charity entrepreneurship, etc.)?
Congratulations on running the pilot and getting these results!
They seem rather promising to me and above a threshold for testing more rigorously and scaling. However, a couple of questions:
When did you measure the end effects and are you planning on doing a follow-up?
How does this compare to interventions such as eight weeks therapy or coaching?
Congratulations! Very happy to see the fruit of your diligent effort.
Glad it was useful. Looking forward to seeing the course (fortunate that you had reached out to request feedback on it).
Great to see how concrete and serious the US is now. This basically means that models more powerful than GPT-4 have to be reported to the government.
Thanks for posting this!
I appreciate the overview and attempt to build conceptual clarity around this related cluster of ideas that you neatly refer to Very Positive Futures.
To respond to some of your prompts:
What are your views on the potential pros and cons of more focus on VPF? With which points do you agree or disagree? What are other arguments?
I think you summarized the pros and cons quite well but I’m just emphasizing the most compelling from my perspective.
IMO, the strongest pros are i) increased clarity on prioritization, ii) motivation/psychology (anecdotally, Martin Luther King’s speech started with “I have a dream” not “I have a nightmare” and similarly it seems that positive psychology has made some progress that the original dysfunction-oriented psychology didn’t), and iii) being able to attract a different crowd of people.
IMO, the strongest cons are i) risk of unconditional enthusiasm that might end up being a distraction from the very pressing risks and suffering.
Which term resonates most with you: Existential Hope, Positive Trajectory Change, Utopia, or perhaps a similar term not described in the post?
I quite like Existential hope. Another term that resonates is “future vision”.
MacAskill briefly talks about procedural visions which I also quite like. Finally, I’d imagine that people within the EA space would also resonate quite a bit with “Existential Security”.
I think some interesting initiatives related to VPF are:
FLI’s world-building contest.
Karnofsky’s poll of 400 Americans’ responses to 16 different scenarios and 22 different traits of utopia.
Finally, I’d like to ask—if you had to design an introductory course that includes VPF, what are the key concepts, resources, or arguments you’d want to include?
You’re most welcome Geoffrey. Glad someone like yourself find it helpful.
Having thought more about it, I think the AI safety institute might be a continuation of the UK Frontier AI Taskforce. I don’t know anything about the object-level output of the Taskforce but they’ve certainly managed to put together a great bunch of people as advisors and contributors (Yoshua Bengio, Paul Christiano, etc.). Very excited to see what comes out of this.
Thanks, Dave! Yes, we found that even the conservative estimate indicates that this can be a worthwhile investment and I feel more robustly good about folks investing in experimenting with high-quality services. That said, some initiatives have significantly higher ROI than 3.7 (our upper bound) so people will have to make individual judgment calls.
In terms of your considerations:
This seems right to me. I also want to stress that I think it’s likely that some of the people who could benefit the most from these services aren’t currently working at organizations. E.g., individual researchers who don’t have anything that resembles management and other beneficial structures typically found within organizations.
Seems true to me! However, even very senior people may benefit a lot from this. For example, many of the big tech CEOs (e.g., Eric Schmidt, Tim Cook, and Sundar Pichai) were coached by Bill Campbell and attributed significant value to that coaching.
Yes, we’re happy to support follow-ups!
Some moderately strong and reasonable statements coming from a PM. I wonder what their vision for the AI safety institute is and how the expert panel might look like.
Seems a bit misplaced to say that the institute will be the first in the world as there’s already several institutes working on this (though he could be meaning within government).
Thanks for the comment Tee!
As mentioned in the post, this approach had many flaws. This is partly because we wanted to rely on published studies on the association between various conditions (e.g., stress) and productivity loss. Most of the studies we looked at relied on self-reported absenteeism and presenteeism (loss of productivity while at work due to lower performance). This means that these estimates don’t include turnover which can indeed cause decreased organizational productivity, emotional challenges, and other costs. Overall, this might mean that this is an underestimate though there are other ways in which the estimates above might be an overestimate. Do you have a sense of how many people this might involve—e.g., is it 0.1% of staff, 1% of staff, or something else?
Thanks for sharing your thoughts and frustrations—I’m very curious to learn more about LMIC perspectives! I have the following reflections that agree with some aspects of your points while potentially disagreeing with others:
It’s important to consider how to adapt cause prioritization to the local context. E.g., there may be something to prioritizing basic infrastructure, and the urgency of problems can be very significant. However, some issues seem to be particularly big and of a global scale in a way that are surprisingly weird and hard to fully grasp (certainly took me a long time and I still have a lot to learn). These include existential risks from AI and pandemics that could affect all of us and future generations.
Using contextualized sources that communicate the core ideas and claims to a broader audience makes sense. I’m reminded of this curricula. That said, I think ideas should come first. While diversity is important, it shouldn’t be prioritized over the core ideas.
Thanks for sharing and congrats on being the first in your family to attend uni and having landed a promising job! :)
I’ll keep this in mind when I encounter people who find that their career doesn’t really fit with EA and feels like a mess.
On a personal note, when I first encountered EA, I recall thinking that many people also had very impressive CVs and felt some sense of inadequacy and imposter syndrome.
That makes sense. We might do some more strategic outreach later this year where a report like this would be relevant but for now i don’t have a clear use case in mind for this so probably better to wait. Approximately how much time would you need to run this?