FWIW, it’s not clear to me that AI alignment folks with different agendas have put less effort into (or have made less progress on) understanding the motivations for other agendas than is typical in other somewhat-analogous fields. Like, MIRI leadership and Paul have put >25 (and maybe >100, over the years?) hours into arguing about merits of their differing agendas (in person, on the web, in GDocs comments), and my impression is that central participants to those conversations (e.g. Paul, Eliezer, Nate) can pass the others’ ideological Turing tests reasonably well on a fair number of sub-questions and down 1-3 levels of “depth” (depending on the sub-question), and that might be more effort and better ITT performance than is typical for “research agenda motivation disagreements” in small niche fields that are comparable on some other dimensions.
FWIW, it’s not clear to me that AI alignment folks with different agendas have put less effort into (or have made less progress on) understanding the motivations for other agendas than is typical in other somewhat-analogous fields. Like, MIRI leadership and Paul have put >25 (and maybe >100, over the years?) hours into arguing about merits of their differing agendas (in person, on the web, in GDocs comments), and my impression is that central participants to those conversations (e.g. Paul, Eliezer, Nate) can pass the others’ ideological Turing tests reasonably well on a fair number of sub-questions and down 1-3 levels of “depth” (depending on the sub-question), and that might be more effort and better ITT performance than is typical for “research agenda motivation disagreements” in small niche fields that are comparable on some other dimensions.