I agree that the very strong sort of alignment you describe—with the Coherent Extrapolated Volition of humanity, or the collective interest of all sentient beings, or The Form of The Good—is probably impossible and perhaps ill-posed. Insofar as we need this sort of aligned AI for things to go as well as they possibly could, they won’t.
But I don’t see why that’s the only acceptable target. Aligning a superintelligence with the will of basically any psychologically normal human being (narrower than any realistic target except perhaps a profit-maximizer—in which case yeah, we’re doomed) would still be an ok outcome for humans: it certainly doesn’t end in paperclips. And alignment with someone even slightly inclined towards impartial benevolence probably goes much better than the status quo, especially for the extremely poor.
(Animals are at much more risk here, but their current situation is also much worse: I’m extremely uncertain how a far richer world would treat factory farming)
I think this is conflating the preferences of women in general with those of women from a particular class and cultural milieu[1]: highly educated urban professional-managerial class liberals[2] in the developed world.
Broad-spectrum appeals to (what are perceived as) their tastes and interests (as opposed to measures targeted at the interests of women qua women, like attempts to reduce sexual harassment):
may or may not improve EA’s gender ratio: my best guess is a very low confidence “yes, but not by much”
will likely make EA less welcoming to people who haven’t been acculturated (typically through western university education) into PMC norms and tastes.
will almost certainly make EA less welcoming to people to whom PMC culture is hostile or patronizing: in particular Christians and blue-collar laborers (who together constitute a majority of the US population), and local elites outside major urban areas (who control a significant fraction of its wealth).
This is not necessarily a bad choice. As something of a “liberal elite” myself, while I think the dominance of highly educated liberals within EA is both annoying and an epistemic weakness, the unfortunate fact of the matter is that EA is not a mass movement or likely to become one. Accepting a certain degree of parochialism in exchange for money and influence is probably worth it, and the PMC is probably the least-misaligned of all the patrons available.
But it is a dangerous choice, and not one that can be easily reversed. We shouldn’t sleepwalk into it.
It used to be fashionable to worry about EA spiraling into quasi-religious extremism. Personally, I think a descent into characterless opportunism is far, far more likely. We’ll take one more tiny step towards alignment with our chosen power base, cross some critical threshold from “involvement in EA looks good in contexts X Y and Z” to “involvement in EA is a good general-purpose elite resume builder”, and that’ll be that.
Or more precisely: the socially-normative views of that class and culture, which are often but not always the actual views of the majority.
AKA “coastal elites”, “stuff white people like”, “high cultural capital”, “the brahmin left” - there are no non-controversial labels for them (ok, us) but I know you know who I’m talking about.