I thought this comment was valuable and it’s also a concern I have.
It makes me wonder if some of the “original EA norms”, like donating a substantial proportion of income or becoming vegan, might still be quite important to build trust, even as they seem less important in the grand scheme of things (mostly, the increase in the proportion of people believing in longtermism). This post makes a case for signalling.
It also seems to increase the importance of vetting people in somewhat creative ways. For instance, did they demonstrate altruistic things before they knew there was lots of money in EA? I know EAs who spent a lot of their childhoods volunteering, told their families to stop giving them birthday presents and instead donate to charities, became vegan at a young age at their own initiative, were interested in utilitarianism very young, adopted certain prosocial beliefs their communities didn’t have, etc. When somebody did such things long before it was “cool” or they knew there was anything in it for them, this demonstrates something, even if they didn’t become involved with EA until it might help their self-interest. At least until we have Silicon Valley parents making sure their children do all the maximally effective things starting at age 8.
It’s kind of useful to consider an example, and the only example I can really give on the EA forum is myself. I went to one of my first EA events partially because I wanted a job, but I didn’t know that there was so much money in EA until I was somewhat involved (also this was Fall 2019, so there was somewhat less money). I did some of the things I mentioned above when I was a kid (or at least, so I claim on the EA forum)! Would I trust me immediately if I met me? Eh, a bit but not a lot, partially because I’m one of the hundreds of undergrads somewhere near AI safety technical research and not (e.g.) an animal welfare person. It would be significantly easier if I’d gotten involved in 2015 and harder if I’d gotten involved in 2021.
Part of what this means is that we can’t rely on trust so much anymore. We have to rely on cold, hard, accomplishments. It’s harder, it’s more work, it feels less warm and fuzzy, but it seems necessary in this second phase. This means we have to be better about evaluating accomplishments in ways that don’t rely on social proof. I think this is easier in some fields (e.g. earning to give, distributing bednets) than others (e.g. policy), but we should try in all fields.
How bad is it to fund someone untrustworthy? Obviously if they take the money and run, that would be a total loss, but I doubt that’s a particularly common occurrence (you can only do it once, and would completely shatter social reputation, so even unethical people don’t tend to do that). A more common failure mode would seem to be apathy, where once funded not much gets done, because the person doesn’t really care about the problem. However, if something gets done instead of nothing at all, then that would probably be (a fairly weak) net positive. The reason why that’s normally negative is due to that money then not being used in a more cost-effective manner, but if our primary problem is spending enough money in the first place, that may not be much of an issue at all.
I think it’s easier than it might seem to do something net negative even ignoring opportunity cost. For example, actively compete with some other better project, interfere with politics or policy incorrectly, create a negative culture shift in the overall ecosystem, etc.
Besides, I don’t think the attitude that our primary problem is spending down the money is prudent. This is putting the cart before the horse, and as Habryka said might lead to people asking “how can I spend money quick?” rather than “how can I ambitiously do good?” EA certainly has a lot of money, but I think people underestimate how fast $50 billion can disappear if it’s mismanaged (see, for an extreme example, Enron).
Thomas—excellent reply, and good points. I’ve written a bit about virtue signaling, and agree that there are good forms (reliable, predictive) and bad forms (cheap talk, deceptive, misguided) of virtue signaling.
I also agree that EA could be more creative and broad-minded about what kinds of virtue signaling are likely to be helpful in predictive future integrity, dedication, and constructiveness in EA. Historically, a lot of EA signaling has involved living frugally, being vegan, being a good house-mate in an EA shared house, collaborating well on EA projects, getting lots of upvotes on EA Forum, etc. Assessing those signals accurately requires a lot of first-hand or second-hand knowledge, which can be hard to do at scale, as the EA movement grows.
As EA grows in scale and becomes more diverse in terms of background (e.g. recruits more established professionals from other fields, not just recent college grads), we may need to get savvier about domain-specific virtue signals, e.g. how do medical researchers vs geopolitical security experts vs defense attorneys vs bioethicists vs blockchain developers show their true colors?
The very tricky trade-off, IMHO, is that often the most reliable virtue signals in terms of predicting personality traits (honesty, humility, conscientiousness, kindness) are often the least efficient in terms of actually accomplishing real-world good. For example, defense attorneys who do a lot of pro bono work doing appeals for death row inmates might be showing genuine dedication and altruism—but this might be among the least effective uses of their time in achieving criminal justice reform. So, do we want the super-trustworthy but scope-insensitive lawyers involved in EA, or the slightly less virtue-signaling but more rational and scope-sensitive lawyers?
That seems like a real dilemma. Traditionally, EA has solved it mostly by expecting a fair amount of private personality-signaling (e.g. being a conscientious vegan house-mate) plus a lot of public, hyper-rational, scope-sensitive analysis and discussion.
I thought this comment was valuable and it’s also a concern I have.
It makes me wonder if some of the “original EA norms”, like donating a substantial proportion of income or becoming vegan, might still be quite important to build trust, even as they seem less important in the grand scheme of things (mostly, the increase in the proportion of people believing in longtermism). This post makes a case for signalling.
It also seems to increase the importance of vetting people in somewhat creative ways. For instance, did they demonstrate altruistic things before they knew there was lots of money in EA? I know EAs who spent a lot of their childhoods volunteering, told their families to stop giving them birthday presents and instead donate to charities, became vegan at a young age at their own initiative, were interested in utilitarianism very young, adopted certain prosocial beliefs their communities didn’t have, etc. When somebody did such things long before it was “cool” or they knew there was anything in it for them, this demonstrates something, even if they didn’t become involved with EA until it might help their self-interest. At least until we have Silicon Valley parents making sure their children do all the maximally effective things starting at age 8.
It’s kind of useful to consider an example, and the only example I can really give on the EA forum is myself. I went to one of my first EA events partially because I wanted a job, but I didn’t know that there was so much money in EA until I was somewhat involved (also this was Fall 2019, so there was somewhat less money). I did some of the things I mentioned above when I was a kid (or at least, so I claim on the EA forum)! Would I trust me immediately if I met me? Eh, a bit but not a lot, partially because I’m one of the hundreds of undergrads somewhere near AI safety technical research and not (e.g.) an animal welfare person. It would be significantly easier if I’d gotten involved in 2015 and harder if I’d gotten involved in 2021.
Part of what this means is that we can’t rely on trust so much anymore. We have to rely on cold, hard, accomplishments. It’s harder, it’s more work, it feels less warm and fuzzy, but it seems necessary in this second phase. This means we have to be better about evaluating accomplishments in ways that don’t rely on social proof. I think this is easier in some fields (e.g. earning to give, distributing bednets) than others (e.g. policy), but we should try in all fields.
How bad is it to fund someone untrustworthy? Obviously if they take the money and run, that would be a total loss, but I doubt that’s a particularly common occurrence (you can only do it once, and would completely shatter social reputation, so even unethical people don’t tend to do that). A more common failure mode would seem to be apathy, where once funded not much gets done, because the person doesn’t really care about the problem. However, if something gets done instead of nothing at all, then that would probably be (a fairly weak) net positive. The reason why that’s normally negative is due to that money then not being used in a more cost-effective manner, but if our primary problem is spending enough money in the first place, that may not be much of an issue at all.
I think it’s easier than it might seem to do something net negative even ignoring opportunity cost. For example, actively compete with some other better project, interfere with politics or policy incorrectly, create a negative culture shift in the overall ecosystem, etc.
Besides, I don’t think the attitude that our primary problem is spending down the money is prudent. This is putting the cart before the horse, and as Habryka said might lead to people asking “how can I spend money quick?” rather than “how can I ambitiously do good?” EA certainly has a lot of money, but I think people underestimate how fast $50 billion can disappear if it’s mismanaged (see, for an extreme example, Enron).
That’s a fair point, thank you for bringing that up :)
Thomas—excellent reply, and good points. I’ve written a bit about virtue signaling, and agree that there are good forms (reliable, predictive) and bad forms (cheap talk, deceptive, misguided) of virtue signaling.
I also agree that EA could be more creative and broad-minded about what kinds of virtue signaling are likely to be helpful in predictive future integrity, dedication, and constructiveness in EA. Historically, a lot of EA signaling has involved living frugally, being vegan, being a good house-mate in an EA shared house, collaborating well on EA projects, getting lots of upvotes on EA Forum, etc. Assessing those signals accurately requires a lot of first-hand or second-hand knowledge, which can be hard to do at scale, as the EA movement grows.
As EA grows in scale and becomes more diverse in terms of background (e.g. recruits more established professionals from other fields, not just recent college grads), we may need to get savvier about domain-specific virtue signals, e.g. how do medical researchers vs geopolitical security experts vs defense attorneys vs bioethicists vs blockchain developers show their true colors?
The very tricky trade-off, IMHO, is that often the most reliable virtue signals in terms of predicting personality traits (honesty, humility, conscientiousness, kindness) are often the least efficient in terms of actually accomplishing real-world good. For example, defense attorneys who do a lot of pro bono work doing appeals for death row inmates might be showing genuine dedication and altruism—but this might be among the least effective uses of their time in achieving criminal justice reform. So, do we want the super-trustworthy but scope-insensitive lawyers involved in EA, or the slightly less virtue-signaling but more rational and scope-sensitive lawyers?
That seems like a real dilemma. Traditionally, EA has solved it mostly by expecting a fair amount of private personality-signaling (e.g. being a conscientious vegan house-mate) plus a lot of public, hyper-rational, scope-sensitive analysis and discussion.