Ok thanks I think it’s fair to call me on this (I realise the question of what Thiel actually thinks is not super interesting to me, compared to “does this critique contain inspiration for things to be aware of that I wasn’t previously really tracking”; but get that most people probably aren’t orienting similarly, and I was kind of assuming that they were when I suggested this was why it was getting sympathy).
I do think though that there’s a more nuanced point here than “trying too hard to do good can result in harm”. It’s more like “over-claiming about how to do good can result in harm”. For a caricature to make the point cleanly: suppose EA really just promoted bednets, and basically told everyone that what it meant to be good was to give more money to bednets. I think it’s easy to see how this gaining a lot of memetic influence (bednet cults; big bednet, etc.) could end up being destructive (even if bednets are great).
I think that EA is at least conceivably vulnerable to more subtle versions of the same mistake. And that that is worth being vigilant against. (Note this is only really a mistake that comes up for ideas that are so self-recommending that they lead to something like strategic movement-building around the ideas.)
Ok thanks I think it’s fair to call me on this (I realise the question of what Thiel actually thinks is not super interesting to me, compared to “does this critique contain inspiration for things to be aware of that I wasn’t previously really tracking”; but get that most people probably aren’t orienting similarly, and I was kind of assuming that they were when I suggested this was why it was getting sympathy).
I do think though that there’s a more nuanced point here than “trying too hard to do good can result in harm”. It’s more like “over-claiming about how to do good can result in harm”. For a caricature to make the point cleanly: suppose EA really just promoted bednets, and basically told everyone that what it meant to be good was to give more money to bednets. I think it’s easy to see how this gaining a lot of memetic influence (bednet cults; big bednet, etc.) could end up being destructive (even if bednets are great).
I think that EA is at least conceivably vulnerable to more subtle versions of the same mistake. And that that is worth being vigilant against. (Note this is only really a mistake that comes up for ideas that are so self-recommending that they lead to something like strategic movement-building around the ideas.)