I think a common pitfall from being part of groups which appear to have better epistemics than a lot of others (i.e. EA, LW) is that being part of these groups implicitly gives a feeling of being able to let your [epistemic] guard down (e.g. to defer).
Iâve noticed this in myself recently; identifying (whether consciously or not) as being more intelligent/ârational than average Joe is actually is a surefire way for me to end up not thinking as clearly as I would have otherwise. (this is obvious in retrospect, but I think itâs pretty important to keep in mind)
I agree with a lot of what you said, and have had similar concerns. I appreciate you writing this and making it public!
Not sure what the motivation behind this post is; it would be good for you to clarify.
I think the question isnât framed very well, since a love letter doesnât make a person happy for their entire life. Clearly more QALYs or WELLBYs or whatever are preserved by running over the letters.
I like the general idea of this post, and I think this idea/âprogram is something worth experimenting with. Would love to see how it goes, if you decide to run it. That being said, I have a couple thoughts.
Weak criticism: (I expect there is probably a good rebuttal to this) We might want some selection for students who have already determined that they want to make âdoing goodâ a large part of their life. Maybe these students are more conscientious than the average individual, and this is an early signal of them being people who self reflects on their values/âthoughts/âbeliefs more. This could mean that they will perform better at careers that take a lot of critical thinking and/âor careful moral reasoning. Thatâs not to say these kinds of thinking skills cannot be learned, but they may be picked up faster and performed better by students who already exhibit some level of personal reflection at a younger age.
Stronger criticism: If students have not internalized that doing good matters to them, and would therefore not want to join the Intro EA Program, I strongly suspect they will also not be interested in a 5-week program about their purpose and life planning. My main concern here is that outreach will be difficult (but itâs easy to prove me wrong empirically, so feel free to go out and do it!)
A final thought on framing/âoverstepping: If I were a student first hearing about this program, I think I would be a little bit suspicious of the underlying motives. From a surface-level impression, I would think that the goal of the program would be for me âfind my purposeâ⌠then I would look more into who is running the program and ask myself âwho are these EA people, and why do they care about my purpose?â, after which I would quickly find out that they want me to join their organization.
I think the main concern that I want to bring up is that I think this program could easily be turned into a sort of bait-and-switch. Finding oneâs purpose and life goals is a very individual process, and I wouldnât want this process to be âhijackedâ by an EA program that directs people in a very specific direction. I.e. my concern is that the program will presented as if it is encouraging people to find their values and purpose, but in reality itâs just trying to incept them into following an EA career path.
Not sure if/âhow this can be avoided, except for being really up-front about the motivation of the program with applicants. I also might be misunderstanding something, so feel free to correct me.
You mention a few times that EV calculations are susceptible to motivated reasoning. But this conflicts with my understanding, which is that EV calculations are useful partly (largely) because they help to prevent motivated reasoning from guiding our decisions too heavily
(e.g. You can imagine a situation where charity Y performs an intervention that is more cost effective than charity X. By following on EV calculation, one might switch their donation from charity X to charity Y, despite that charity X sounds intuitively better.)
Maybe you could include some examples/âcitations of where you think this âEV motivated reasoningâ has occurred. Otherwise I find it hard to believe that EV calculations are worse than the alternative, from a âsusceptible-to-motivated-reasoningâ perspective (here, the alternative is not using EV calculations).
This post makes the case that warning shots wonât change the picture in policy much, but I could imagine a world where some warning shot makes the leading AI labs decide to focus more on safety, or agree to slow down their deployment, without policy change occurring. Maybe this could buy a couple of years time for safety researchers?
This isnât a well developed thought, just something that came to mind while reading.
Thanks for writing this! I think a lot of this is great to keep in mind for university groups.
I especially liked the âfree stuff is not actually freeâ framing. Putting a counterfactual on conference costs can be humbling, and really makes one think carefully about attending⌠if ~$5000 dollars could save a life elsewhere (say, generate 80 QALYs), then a $500 reimbursement for a trip to a conference is sacrificing 8 years of life. Not a decision to take lightly!
Do you have any more info on these âepistemic infrastructureâ projects or the people working on them? I would be super curious to look into this more.
Harrison đ¸
I think a common pitfall from being part of groups which appear to have better epistemics than a lot of others (i.e. EA, LW) is that being part of these groups implicitly gives a feeling of being able to let your [epistemic] guard down (e.g. to defer).
Iâve noticed this in myself recently; identifying (whether consciously or not) as being more intelligent/ârational than average Joe is actually is a surefire way for me to end up not thinking as clearly as I would have otherwise. (this is obvious in retrospect, but I think itâs pretty important to keep in mind)
I agree with a lot of what you said, and have had similar concerns. I appreciate you writing this and making it public!
I also was wondering this.
Huge! I am so excited to see this announcement, and wish the best of luck for AoI.
Not sure what the motivation behind this post is; it would be good for you to clarify.
I think the question isnât framed very well, since a love letter doesnât make a person happy for their entire life. Clearly more QALYs or WELLBYs or whatever are preserved by running over the letters.
I like the general idea of this post, and I think this idea/âprogram is something worth experimenting with. Would love to see how it goes, if you decide to run it. That being said, I have a couple thoughts.
Weak criticism: (I expect there is probably a good rebuttal to this) We might want some selection for students who have already determined that they want to make âdoing goodâ a large part of their life. Maybe these students are more conscientious than the average individual, and this is an early signal of them being people who self reflects on their values/âthoughts/âbeliefs more. This could mean that they will perform better at careers that take a lot of critical thinking and/âor careful moral reasoning. Thatâs not to say these kinds of thinking skills cannot be learned, but they may be picked up faster and performed better by students who already exhibit some level of personal reflection at a younger age.
Stronger criticism: If students have not internalized that doing good matters to them, and would therefore not want to join the Intro EA Program, I strongly suspect they will also not be interested in a 5-week program about their purpose and life planning. My main concern here is that outreach will be difficult (but itâs easy to prove me wrong empirically, so feel free to go out and do it!)
A final thought on framing/âoverstepping: If I were a student first hearing about this program, I think I would be a little bit suspicious of the underlying motives. From a surface-level impression, I would think that the goal of the program would be for me âfind my purposeâ⌠then I would look more into who is running the program and ask myself âwho are these EA people, and why do they care about my purpose?â, after which I would quickly find out that they want me to join their organization.
I think the main concern that I want to bring up is that I think this program could easily be turned into a sort of bait-and-switch. Finding oneâs purpose and life goals is a very individual process, and I wouldnât want this process to be âhijackedâ by an EA program that directs people in a very specific direction. I.e. my concern is that the program will presented as if it is encouraging people to find their values and purpose, but in reality itâs just trying to incept them into following an EA career path.
Not sure if/âhow this can be avoided, except for being really up-front about the motivation of the program with applicants. I also might be misunderstanding something, so feel free to correct me.
You mention a few times that EV calculations are susceptible to motivated reasoning. But this conflicts with my understanding, which is that EV calculations are useful partly (largely) because they help to prevent motivated reasoning from guiding our decisions too heavily
(e.g. You can imagine a situation where charity Y performs an intervention that is more cost effective than charity X. By following on EV calculation, one might switch their donation from charity X to charity Y, despite that charity X sounds intuitively better.)
Maybe you could include some examples/âcitations of where you think this âEV motivated reasoningâ has occurred. Otherwise I find it hard to believe that EV calculations are worse than the alternative, from a âsusceptible-to-motivated-reasoningâ perspective (here, the alternative is not using EV calculations).
+1 here; looks like this is a vestige from the previous version, and should probably be corrected
This post makes the case that warning shots wonât change the picture in policy much, but I could imagine a world where some warning shot makes the leading AI labs decide to focus more on safety, or agree to slow down their deployment, without policy change occurring. Maybe this could buy a couple of years time for safety researchers?
This isnât a well developed thought, just something that came to mind while reading.
Thanks for writing this! I think a lot of this is great to keep in mind for university groups.
I especially liked the âfree stuff is not actually freeâ framing. Putting a counterfactual on conference costs can be humbling, and really makes one think carefully about attending⌠if ~$5000 dollars could save a life elsewhere (say, generate 80 QALYs), then a $500 reimbursement for a trip to a conference is sacrificing 8 years of life. Not a decision to take lightly!
Do you have any more info on these âepistemic infrastructureâ projects or the people working on them? I would be super curious to look into this more.