I suffer strongly from the following, and I suspect many EAs do too (all numbers are to approximations to illustrate my point):
I think that AGI is coming within the next 50 years, 90% probability, with medium confidence
I think that there is a ~10% chance that development of AGI leads to catastrophic outcomes for humanity, with very low confidence
I think there is a ~50% chance that development of AGI leads to massive amounts of flourishing for humanity, with very low confidence
Increasing my confidence in points 2 & 3 seems very difficult and time consuming, as the questions at hand are exceptionally complex, and even identifying personal cruxes will be a challenge
I feel a moral obligation to prevent catastrophes and enable flourishing, where I have the influence to do so
I want to take actions that accurately reflect my values
Given the probabilities above, not taking strong, if not radical, action to try to influence the outcomes feels like a failure to embody my values, and a moral failure.
Iām still figuring out what to do about this. When youāre highly uncertain itās obviously fine to hedge against being wrong, but again, given the numbers itās hard to justify hedging all the way down to inaction.
I am trying to learn more about AI safety, but Iām not spending very much time on it currently. Iām trying to talk to others about it, but Iām not evangelising it, nor necessarily speaking with a great sense of urgency. At the moment, itās low down my de factor priority list, even though I think thereās a significant chance it changes everything I know and care about. Is part of this a lack of visceral connection to the risks and rewards? What can I do to feel like my values are in line with my actions?
Your ā90% confidence intervalā ofā¦ what, exactly? This looks like a confidence interval over the value of your own subjective probability estimate? And ā90% as the meanā ofā¦ a bunch of different guesses youāve taken at your ātrueā subjective probability? I canāt imagine why anyone would do that but I canāt think what else this could coherently meanā¦?
If I can be blunt, I suspect you might be repeating probabilistic terms without really tracking their technical meaning, as though youāre just inserting nontechnical hedges. Maybe itās worth taking the time to reread the map/āterritory stuff and then run through some calibration practice problems while thinking closely about what youāre doing. Or maybe just use nontechnical hedges more, they work perfectly well for expressing things like this.
Thanks for the feedbackāit has indeed been a long time since I did high school statistics!
I specified that the numbers I gave were āapproximations to prove my pointā is because I know that I do not have a technical statistical model in my head, and I didnāt want to pretend that was the case. Given this is a non-technical, shortform post, I thought it was clear what I meantāapologies if that wasnāt so.
Skill up and work on technical AI safety! Two good resources: 1, 2. Even if you donāt yet feel the moral urgency, skilling up in ML can put you in a better position to do technical research in the future.
Thanks for the suggestion! I have actually spent quite a lot of time thinking about thisāI had my 80k call last April and this was their advice. Iāve hesitated against doing this for a number of reasons:
Iām worried that even if I do upskill in ML, I wonāt be a good enough software engineer to land a research engineering position, so part of me wants to improve as a SWE first
At the moment Iām very busy and a marginal hour of my time is very valuable, upskilling in ML is likely 200-500 hours, at the moment I would struggle to commit to even 5 hours per week
I donāt know whether I would enjoy ML, whereas I know I somewhat enjoy at least some parts of the SWE work I currently do
Learning ML potentially narrows my career options vs learning broader skills, so itās hard to hedge
My impression is that there are a lot of people trying to do this right now, and itās not clear to me that doing so would be my comparative advantage. Perhaps carving out a different niche would be more valuable in the future.
There are probably good rebuttals to at least some of these points, and I think that is adding to my confusion. My intuition is to keep doing what Iām currently doing, rather than go try and learn ML, but maybe my intuition here is bad.
Edit: writing this comment made me realise that I ought to write a proper doc with the pros/ācons of learning ML and get feedback on it if necessary. Thanks for helping pull this useful thought out of my brain :)
I suffer strongly from the following, and I suspect many EAs do too (all numbers are to approximations to illustrate my point):
I think that AGI is coming within the next 50 years, 90% probability, with medium confidence
I think that there is a ~10% chance that development of AGI leads to catastrophic outcomes for humanity, with very low confidence
I think there is a ~50% chance that development of AGI leads to massive amounts of flourishing for humanity, with very low confidence
Increasing my confidence in points 2 & 3 seems very difficult and time consuming, as the questions at hand are exceptionally complex, and even identifying personal cruxes will be a challenge
I feel a moral obligation to prevent catastrophes and enable flourishing, where I have the influence to do so
I want to take actions that accurately reflect my values
Given the probabilities above, not taking strong, if not radical, action to try to influence the outcomes feels like a failure to embody my values, and a moral failure.
Iām still figuring out what to do about this. When youāre highly uncertain itās obviously fine to hedge against being wrong, but again, given the numbers itās hard to justify hedging all the way down to inaction.
I am trying to learn more about AI safety, but Iām not spending very much time on it currently. Iām trying to talk to others about it, but Iām not evangelising it, nor necessarily speaking with a great sense of urgency. At the moment, itās low down my de factor priority list, even though I think thereās a significant chance it changes everything I know and care about. Is part of this a lack of visceral connection to the risks and rewards? What can I do to feel like my values are in line with my actions?
...What on earth does ā90% probability, with medium confidenceā mean? Do you think itās 90% likely or not?
It means something like āmy 90% confidence interval is 80% ā 95%, with 90% as the meanā.
Your ā90% confidence intervalā ofā¦ what, exactly? This looks like a confidence interval over the value of your own subjective probability estimate? And ā90% as the meanā ofā¦ a bunch of different guesses youāve taken at your ātrueā subjective probability? I canāt imagine why anyone would do that but I canāt think what else this could coherently meanā¦?
If I can be blunt, I suspect you might be repeating probabilistic terms without really tracking their technical meaning, as though youāre just inserting nontechnical hedges. Maybe itās worth taking the time to reread the map/āterritory stuff and then run through some calibration practice problems while thinking closely about what youāre doing. Or maybe just use nontechnical hedges more, they work perfectly well for expressing things like this.
Thanks for the feedbackāit has indeed been a long time since I did high school statistics!
I specified that the numbers I gave were āapproximations to prove my pointā is because I know that I do not have a technical statistical model in my head, and I didnāt want to pretend that was the case. Given this is a non-technical, shortform post, I thought it was clear what I meantāapologies if that wasnāt so.
Skill up and work on technical AI safety! Two good resources: 1, 2. Even if you donāt yet feel the moral urgency, skilling up in ML can put you in a better position to do technical research in the future.
Thanks for the suggestion! I have actually spent quite a lot of time thinking about thisāI had my 80k call last April and this was their advice. Iāve hesitated against doing this for a number of reasons:
Iām worried that even if I do upskill in ML, I wonāt be a good enough software engineer to land a research engineering position, so part of me wants to improve as a SWE first
At the moment Iām very busy and a marginal hour of my time is very valuable, upskilling in ML is likely 200-500 hours, at the moment I would struggle to commit to even 5 hours per week
I donāt know whether I would enjoy ML, whereas I know I somewhat enjoy at least some parts of the SWE work I currently do
Learning ML potentially narrows my career options vs learning broader skills, so itās hard to hedge
My impression is that there are a lot of people trying to do this right now, and itās not clear to me that doing so would be my comparative advantage. Perhaps carving out a different niche would be more valuable in the future.
There are probably good rebuttals to at least some of these points, and I think that is adding to my confusion. My intuition is to keep doing what Iām currently doing, rather than go try and learn ML, but maybe my intuition here is bad.
Edit: writing this comment made me realise that I ought to write a proper doc with the pros/ācons of learning ML and get feedback on it if necessary. Thanks for helping pull this useful thought out of my brain :)
Thanks for sharing this. I thought point 7 was well-put, and something I relate to (and could apply to several other cause areas/āproblems)
Hey,
My favorite tool for resolving internal conflict like this (which also resonated with many people I spoke to) is āinternal double cruxā.
TL;DR: Write down what you different parts want to say (and respond) to each other (as opposed to writing a list of something-like-pros-and-cons)
If you want, we can talk and do it together, and see if it works for you
This is a good idea, thanks for the suggestion! Iāve never really tried any of the CFAR stuff but this seems like a good place to start.
Iāll give it a go over the weekend and if Iām struggling then Iāll let you know and we can do it together :)
:)