Hi Lizka, WOW – Thank you for writing this. Great to see Rethink Priorities working on this. Absolutely loving the diagrams here.
I have worked in this space for a number of years mostly here, have been advocating for this cause within EA since 2016 and advised both Jess/80K and the effective institutions project on their writeups. Thought I would give some quick feedback. Let me know if it is useful.
I thought your disentanglement did a decent job. Here are a few thought I had on it.
I really like how you split IIDM into “A technical approach to IIDM” and “A value-aligning approach to IIDM.”
However I found the details of how you split it to be very confusing. It left me quite unsure what goes into what bucket. For example intuitively I would see increasing the “accuracy of governments” (i.e. aligning governments with the interests of the voters) as “value-aligning” yet you classify it as “technical”.
That said despite this, I very much agreed with the conclusion that “value-oriented IIDM makes more sense than value-neutral IIDM” and the points you made to that effect.
I didn’t quite understand what “(1) IIDM can improve our intellectual and political environment” was really getting at. My best guess is that by (1) you mean work that only indirectly leads to “(3) improved outcomes”. So value-oriented (1) would look like general value spreading. Is that correct?
I agree with “for the sake of clarity … we should generally distinguish between ‘meta EA’ work and IIDM work”. That said I think it is worth bearing in mind that on occasion the approaches might not be that different. For example I have been advising the UK government on how to asses high-impact risks which is relevant for EAs too.*
One institution can have many parts. Might be a thing to highlight if you do more disentanglement. E.g. Is a new office for future generations within a government, a new institution or improving an existing institution?
One other thought I had whilst reading.
I think it is important not to assign value to IIDM based on what is “predictable”.
For example you say “it would be extremely hard to produce candidate IIDM interventions that would have sufficiently predictable outcomes via this pathway, as the outcomes would depend on many very uncertain factors.” Predictions do matter but one of the key cases for IIDM is that it offers a solution to the unpredictable, the unknown unknows, to the uncertainty of the EA (and especially longtermist) endeavour. All the advice on dealing with high-uncertainty and things that are hard to predict suggest that interventions like IIDM are the kinds of interventions that should work – as set out by Ian David Moss here (from this piece).
Finally, at points you seemed uncertain about tractability of this work. I wanted to add that so far I have found it much much easier than I expected. Eg you say “it is possible that shifting the aims of institutions is generally very difficult or that the potential benefits from institutions is overwhelmingly bottlenecked by decision-making ability, rather than by the value-alignment of institutions’ existing aims”. (I am perhaps still confused about what you would count as shifting aims Vs decision-making ability see my point 1. above, but) my rough take on this is that I have found shifting the aims of government to be fairly easy and that there are not too many decision-making bottlenecks.
So super excited to see more EA work in this space.
* Oddly enough, despite being in EA for years, I think I have found it easier to influence the UK government to get better at risk identification work than the EA community. Not sure what to do about that. Just wanted to say that I would love to input if RP is working in this space.
I’m curious why you think aligning governments with the interests of voters is value-aligning rather than technical. I can see that being the case for autocratic regimes, but isn’t that the whole point of representative democracy?
4. When I hear “(1) IIDM can improve our intellectual and political environment”, I’m imagining something like if the concept of steelmanning becomes common in public discourse, we might expect that to indirectly lead to better decisions by key institutions.
Hi Lizka, WOW – Thank you for writing this. Great to see Rethink Priorities working on this. Absolutely loving the diagrams here.
I have worked in this space for a number of years mostly here, have been advocating for this cause within EA since 2016 and advised both Jess/80K and the effective institutions project on their writeups. Thought I would give some quick feedback. Let me know if it is useful.
I thought your disentanglement did a decent job. Here are a few thought I had on it.
I really like how you split IIDM into “A technical approach to IIDM” and “A value-aligning approach to IIDM.”
However I found the details of how you split it to be very confusing. It left me quite unsure what goes into what bucket. For example intuitively I would see increasing the “accuracy of governments” (i.e. aligning governments with the interests of the voters) as “value-aligning” yet you classify it as “technical”.
That said despite this, I very much agreed with the conclusion that “value-oriented IIDM makes more sense than value-neutral IIDM” and the points you made to that effect.
I didn’t quite understand what “(1) IIDM can improve our intellectual and political environment” was really getting at. My best guess is that by (1) you mean work that only indirectly leads to “(3) improved outcomes”. So value-oriented (1) would look like general value spreading. Is that correct?
I agree with “for the sake of clarity … we should generally distinguish between ‘meta EA’ work and IIDM work”. That said I think it is worth bearing in mind that on occasion the approaches might not be that different. For example I have been advising the UK government on how to asses high-impact risks which is relevant for EAs too.*
One institution can have many parts. Might be a thing to highlight if you do more disentanglement. E.g. Is a new office for future generations within a government, a new institution or improving an existing institution?
One other thought I had whilst reading.
I think it is important not to assign value to IIDM based on what is “predictable”.
For example you say “it would be extremely hard to produce candidate IIDM interventions that would have sufficiently predictable outcomes via this pathway, as the outcomes would depend on many very uncertain factors.” Predictions do matter but one of the key cases for IIDM is that it offers a solution to the unpredictable, the unknown unknows, to the uncertainty of the EA (and especially longtermist) endeavour. All the advice on dealing with high-uncertainty and things that are hard to predict suggest that interventions like IIDM are the kinds of interventions that should work – as set out by Ian David Moss here (from this piece).
Finally, at points you seemed uncertain about tractability of this work. I wanted to add that so far I have found it much much easier than I expected. Eg you say “it is possible that shifting the aims of institutions is generally very difficult or that the potential benefits from institutions is overwhelmingly bottlenecked by decision-making ability, rather than by the value-alignment of institutions’ existing aims”. (I am perhaps still confused about what you would count as shifting aims Vs decision-making ability see my point 1. above, but) my rough take on this is that I have found shifting the aims of government to be fairly easy and that there are not too many decision-making bottlenecks.
So super excited to see more EA work in this space.
* Oddly enough, despite being in EA for years, I think I have found it easier to influence the UK government to get better at risk identification work than the EA community. Not sure what to do about that. Just wanted to say that I would love to input if RP is working in this space.
I’m curious why you think aligning governments with the interests of voters is value-aligning rather than technical. I can see that being the case for autocratic regimes, but isn’t that the whole point of representative democracy?
(not the author)
4. When I hear “(1) IIDM can improve our intellectual and political environment”, I’m imagining something like if the concept of steelmanning becomes common in public discourse, we might expect that to indirectly lead to better decisions by key institutions.