I was surprised how slim the reviews were of the billionaires given the hours spent on it TBH. I think we do a disservice oversimplifying the impact (or lack thereof of billionaires) and not hold them to a stricter standard. Below are some examples of things I would have mentioned.
Bill Gates has tried to keep the IP of COVID vaccines from being released to manufacturers of countries like India. I don’t think the case of COVID IP is open and shut, but it’s quite possible that Gates is contributing to lots of unecessary COVID deaths.
Elon Musk did a great service by making electric vehicles trendy, but he is also really hurting the climate by working against public transit and pushing car ownership. To the extent we produce new vehicles, they should be electric. But addressing climate change means reducing car ownership/car production not maintaining or growing the status quo—building electric cars is still significantly harmful to the environment.
You mention Elon’s allged intention to make Twitter a better public good which has often been brought up in terms of free speech. But you don’t mention Elon has an anti-free-speech track record of going after (legitimate) critics of him and trying to get them deplatformed. We should be concerned how he might actually lead Twitter.
Jeff Bezos’ Amazon unecessarily pushes consumerism which is quite harmful for the environment, as you point out with Arnault. Amazon would still be thriving without trying to manipulate people’s psychology to get them to buy more things. This is also arguably bad for the consumer if they make purchases that end up being of little value.
For what it’s worth, Gates broke a lot of IP law in Microsoft’s origins and didn’t become philanthropic until Microsoft was under investigation for being a monopoly.
It’s omissions like these that make me think this post contributes more so to bad discourse about billionaires than a good discourse.
Don’t take it personally. I think you could resolve this stuff by being more holistic in the review. Or if this is just an idea you are playing around with, then I would leave off the grades.
I think I disagree with most of the points on your comment, which is kind of surprising. Though I do think that they are interesting in a red-teaming kind of way.
I think we do a disservice oversimplifying the impact (or lack thereof of billionaires)
I agree that the impact is going to be multifaceted, but I think that most of it will be dominated by the first few factors. More specifically, I think that impact is probably something like lognormally distributed, and that an estimate which only takes into account the first few components fundamentally makes sense
Bill Gates has tried to keep the IP of COVID vaccines from being released to manufacturers of countries like India. I don’t think the case of COVID IP is open and shut, but it’s quite possible that Gates is contributing to lots of unecessary COVID deaths.
It seems very likely to me that expropriating IP leads to less innovation in the next disaster.
Elon Musk did a great service by making electric vehicles trendy, but he is also really hurting the climate by working against public transit and pushing car ownership. To the extent we produce new vehicles, they should be electric. But addressing climate change means reducing car ownership/car production not maintaining or growing the status quo—building electric cars is still significantly harmful to the environment.
I’m not sure whether this point stands when you consider what the counterfactual would have been. I think that the most likely counterfactual is that people would have been producing and buying normal cars, whereas now not only Tesla but also other major manufacturers are producing electric ones. I probably agree that electric cars aren’t as good if the energy comes from coal plants instead of from renewables. But pushing for electric vehicles still enables advances in renewables to cash out into more renewable vehicles, so I think that his Shapley value is high.
Maybe there is something I’m missing? What do you think would have happened in the absence of Tesla? Or maybe you think that lead/lithium batteries are just very harmful? That is something I know less about.
You mention Elon’s allged intention to make Twitter a better public good which has often been brought up in terms of free speech. But you don’t mention Elon has an anti-free-speech track record of going after (legitimate) critics of him and trying to get them deplatformed. We should be concerned how he might actually lead Twitter.
Not convinced, but hard to articulate why.
Jeff Bezos’ Amazon unecessarily pushes consumerism
I’d think that this effect is mild in comparison with the efficiency gains.
Gates broke a lot of IP law in Microsoft’s origins
I don’t really care about this in comparison with his philanthropy and the Giving Pledge.
So overall I think that your points are good in a red-teaming kind of sense, but that they ultimately don’t switch the impacts around. E.g., if you consider grades as indicative of the order, it would surprise me if these considerations moved that around.
You mention Elon’s allged intention to make Twitter a better public good which has often been brought up in terms of free speech. But you don’t mention Elon has an anti-free-speech track record of going after (legitimate) critics of him and trying to get them deplatformed. We should be concerned how he might actually lead Twitter.
Are you predicting that Twitter will be worse along the free speech dimension if Musk takes control of it? If so, what metrics would you use to operationalize this prediction?
I see this post as part of a project to attempt “very brief/rough” evaluations of several topics, instead of spending a lot of time on any one.
Of course, if readers can’t tell a “very brief/rough” analysis from a “very thorough” analysis, I guess then we just can’t post the former.
In this case, the word “brief” was in the title, and the post/comments are so short, I would have hoped that readers would understand that the brevity of the analysis.
We’ll try harder to make this fact more prominent in future posts. Also, if others reading this have thoughts on ways to highlight the depth/breath of an analysis, I’d be quite curious.
For me this isn’t counterproductive because it’s “brief.” It’s counterproductive because even in being brief it still misses important considerations. Take a look at the issues that I highlight in my earlier comment. I did all of that without any research. I typed it out during a long metro ride. For me this wasn’t “brief,” it was half-baked, and it contributes to problematic framing around billionaires that doesn’t hold them to higher moral standards. (In my book, Bill Gates shouldn’t get higher than a B+ and the rest of the Western billionaires are C or lower).
Unless the author had minimal familiarity with billionaires, I am having trouble seeing how we arrived at these assessments after 20 hours of research. Maybe elaborating on the research process would illuminate why there are blind spots.
As answered in your comment above, I think that the downside of this is that you don’t stop to consider whether your points switch the conclusion, or what the magnitude of the problems you point out is, and so your red-team ends up being less valuable than it could have been.
I don’t see where the harm is coming from to be honest.
I doubt someone’s going to be like “I was originally hesitant to work at OpenAI, but after seeing Nuño’s “B” rating for Elon Musk, I now think working for him is a good idea.” Or “I was unsure of where to donate to, but Warren Buffet’s “A” rating means I should probably donate to him instead.”
I think the harm from posting things with flimsy methodology and get a lot of upvotes/uncritical comments is something like “lower epistemic rigour on the forum in general”, rather than this article in particular causing a great deal of harm. I think the impact of this article whether positive or negative is likely to be small.
I think Nuno acted in a permissable way. I only mentioned the expected EV likely being negative because of how much attention the post was getting + how straightforward Nuno generally is.
For what it’s worth, Nuno and I were both expecting this post to get a lot less attention. Maybe 30 karma or so (for myself). I think a lot of the interest is mainly due to the topic.
Seems like a signal that much more rigorous work here would be read.
Thanks for linking! I agree with your points. In some situations my evaluation is pretty flimsy but I have to make a decision anyways, so the evaluation still seems worth doing and using.
I might distinguish between doing an evaluation and publishing the full evaluation. If you’re testing a new evaluation method and you notice it’s giving bad results, maybe you want to just post “I tried this evaluation method and it gave bad results,” or post your evaluation but with a disclaimer that the results are clearly wrong and you hope it will help other people improve their methods.
I think you might be more optimistic than me about other people’s ability to update away from an incorrect evaluation. I’ve found it very difficult for me to update away from the first thing I read on a topic even if it’s later shown to be clearly wrong. I subconsciously have a much higher bar for later evaluations than the first one I read. That’s part of why I try to point out when evaluations aren’t very rigorous—I need to remind myself when I shouldn’t update much on something and when I should.
I commented on the blog, but will reproduce my comment here too since maybe not everyone will click the link.
I suspect that some of the negative reaction was from a combination of (1) the methodology being flimsy and (2) the subject matter being somewhat controversial. Out there in the non-EA world, billionaires are extremely controversial. I suspect some of the people who read your post thought there’s a cost in seeming to approve of billionaires and as a consequence that, if we do want to state our approval of them, we should better be sure that we’re right.
Thanks! Your explanation makes a lot of sense. I think ~20h of work is reasonable for an exploratory Forum post (the majority of my posts have taken less time[1]). Re lots of uncritical comments, I do think my comment (right now the top comment) is fairly critical: the implication of “I think this ranking neglects to track the most important factor” is pretty damning in terms of decision-relevancy. But it’s possible my words were too sugarcoated.
Thanks Linch, I think the amount of time /effort probably depends on the topic, and is unfortunately very difficult to make much progress on “ranking ten people’s entire life impact” with 20 hours. For many posts, 2 hours of effort would suffice, it just depends on the topic! But I’m certain reasonable people can disagree here
I also think the tone of the comments has shifted since I posted this, and ironically the number of people who agree with my comments might suggest that I was wrong to be concerned.
I don’t think (1) is a problem, if anything I think the world is far too skewed in its tendency to judge powerful people based on personal virtues and foibles, and not enough on judging them based on their impact (in either direction).
E.g. all the comments about Zuckerberg sounding robotic and very few about him donating money to random charities, far more comments about Elon’s various Twitter spats and few about either space or OpenAI, various public figures lambasted for breaking lockdowns (rather than utterly poor pandemic preparedness and response), the whole Andrew Cuomo thing, etc.
Well, I disagree on that one. In my view it’s harmful to “give a free pass” to obviously bad behaviours by powerful people because of other things they do, both in terms of justice in general and of sending a message to society that morality is optional.
E.g. many people fetishizing Musk over his space ideas or Twitter personae, and ignoring his awful labour practices or his sexual harassment and what appears to me to be predatory behaviour.
Edit to add: I wouldn’t put “Zuckerberg sounding robotic” in this category, but I would “various public figures breaking lockdowns”. I think you greatly underestimate the importance of leading by example.
Your methodology looks pretty flimsy but it looks like other EAs are taking it seriously
I was surprised how slim the reviews were of the billionaires given the hours spent on it TBH. I think we do a disservice oversimplifying the impact (or lack thereof of billionaires) and not hold them to a stricter standard. Below are some examples of things I would have mentioned.
Bill Gates has tried to keep the IP of COVID vaccines from being released to manufacturers of countries like India. I don’t think the case of COVID IP is open and shut, but it’s quite possible that Gates is contributing to lots of unecessary COVID deaths.
Elon Musk did a great service by making electric vehicles trendy, but he is also really hurting the climate by working against public transit and pushing car ownership. To the extent we produce new vehicles, they should be electric. But addressing climate change means reducing car ownership/car production not maintaining or growing the status quo—building electric cars is still significantly harmful to the environment.
You mention Elon’s allged intention to make Twitter a better public good which has often been brought up in terms of free speech. But you don’t mention Elon has an anti-free-speech track record of going after (legitimate) critics of him and trying to get them deplatformed. We should be concerned how he might actually lead Twitter.
Jeff Bezos’ Amazon unecessarily pushes consumerism which is quite harmful for the environment, as you point out with Arnault. Amazon would still be thriving without trying to manipulate people’s psychology to get them to buy more things. This is also arguably bad for the consumer if they make purchases that end up being of little value.
For what it’s worth, Gates broke a lot of IP law in Microsoft’s origins and didn’t become philanthropic until Microsoft was under investigation for being a monopoly.
It’s omissions like these that make me think this post contributes more so to bad discourse about billionaires than a good discourse.
Don’t take it personally. I think you could resolve this stuff by being more holistic in the review. Or if this is just an idea you are playing around with, then I would leave off the grades.
I think I disagree with most of the points on your comment, which is kind of surprising. Though I do think that they are interesting in a red-teaming kind of way.
I agree that the impact is going to be multifaceted, but I think that most of it will be dominated by the first few factors. More specifically, I think that impact is probably something like lognormally distributed, and that an estimate which only takes into account the first few components fundamentally makes sense
It seems very likely to me that expropriating IP leads to less innovation in the next disaster.
I’m not sure whether this point stands when you consider what the counterfactual would have been. I think that the most likely counterfactual is that people would have been producing and buying normal cars, whereas now not only Tesla but also other major manufacturers are producing electric ones. I probably agree that electric cars aren’t as good if the energy comes from coal plants instead of from renewables. But pushing for electric vehicles still enables advances in renewables to cash out into more renewable vehicles, so I think that his Shapley value is high.
Maybe there is something I’m missing? What do you think would have happened in the absence of Tesla? Or maybe you think that lead/lithium batteries are just very harmful? That is something I know less about.
Not convinced, but hard to articulate why.
I’d think that this effect is mild in comparison with the efficiency gains.
I don’t really care about this in comparison with his philanthropy and the Giving Pledge.
So overall I think that your points are good in a red-teaming kind of sense, but that they ultimately don’t switch the impacts around. E.g., if you consider grades as indicative of the order, it would surprise me if these considerations moved that around.
Are you predicting that Twitter will be worse along the free speech dimension if Musk takes control of it? If so, what metrics would you use to operationalize this prediction?
I see this post as part of a project to attempt “very brief/rough” evaluations of several topics, instead of spending a lot of time on any one.
Of course, if readers can’t tell a “very brief/rough” analysis from a “very thorough” analysis, I guess then we just can’t post the former.
In this case, the word “brief” was in the title, and the post/comments are so short, I would have hoped that readers would understand that the brevity of the analysis.
We’ll try harder to make this fact more prominent in future posts. Also, if others reading this have thoughts on ways to highlight the depth/breath of an analysis, I’d be quite curious.
For me this isn’t counterproductive because it’s “brief.” It’s counterproductive because even in being brief it still misses important considerations. Take a look at the issues that I highlight in my earlier comment. I did all of that without any research. I typed it out during a long metro ride. For me this wasn’t “brief,” it was half-baked, and it contributes to problematic framing around billionaires that doesn’t hold them to higher moral standards. (In my book, Bill Gates shouldn’t get higher than a B+ and the rest of the Western billionaires are C or lower).
Unless the author had minimal familiarity with billionaires, I am having trouble seeing how we arrived at these assessments after 20 hours of research. Maybe elaborating on the research process would illuminate why there are blind spots.
As answered in your comment above, I think that the downside of this is that you don’t stop to consider whether your points switch the conclusion, or what the magnitude of the problems you point out is, and so your red-team ends up being less valuable than it could have been.
How are you getting the impression that other EAs are taking it seriously, or seriously enough for it to cause harm?
I don’t see where the harm is coming from to be honest.
I doubt someone’s going to be like “I was originally hesitant to work at OpenAI, but after seeing Nuño’s “B” rating for Elon Musk, I now think working for him is a good idea.” Or “I was unsure of where to donate to, but Warren Buffet’s “A” rating means I should probably donate to him instead.”
I think the harm from posting things with flimsy methodology and get a lot of upvotes/uncritical comments is something like “lower epistemic rigour on the forum in general”, rather than this article in particular causing a great deal of harm. I think the impact of this article whether positive or negative is likely to be small.
I think Nuno acted in a permissable way. I only mentioned the expected EV likely being negative because of how much attention the post was getting + how straightforward Nuno generally is.
For what it’s worth, Nuno and I were both expecting this post to get a lot less attention. Maybe 30 karma or so (for myself). I think a lot of the interest is mainly due to the topic.
Seems like a signal that much more rigorous work here would be read.
Hey Kirsten (& others), I’ve briefly written my thoughts about when flimsier evaluations are worth it. I would be curious to get your thoughts <https://nunosempere.com/blog/2022/10/27/are-flimsy-evaluations-worth-it/> before I either post it to or reference in the EA forum.
Thanks for linking! I agree with your points. In some situations my evaluation is pretty flimsy but I have to make a decision anyways, so the evaluation still seems worth doing and using.
I might distinguish between doing an evaluation and publishing the full evaluation. If you’re testing a new evaluation method and you notice it’s giving bad results, maybe you want to just post “I tried this evaluation method and it gave bad results,” or post your evaluation but with a disclaimer that the results are clearly wrong and you hope it will help other people improve their methods.
I think you might be more optimistic than me about other people’s ability to update away from an incorrect evaluation. I’ve found it very difficult for me to update away from the first thing I read on a topic even if it’s later shown to be clearly wrong. I subconsciously have a much higher bar for later evaluations than the first one I read. That’s part of why I try to point out when evaluations aren’t very rigorous—I need to remind myself when I shouldn’t update much on something and when I should.
Thanks Kirsten, these are good points.
I commented on the blog, but will reproduce my comment here too since maybe not everyone will click the link.
I suspect that some of the negative reaction was from a combination of (1) the methodology being flimsy and (2) the subject matter being somewhat controversial. Out there in the non-EA world, billionaires are extremely controversial. I suspect some of the people who read your post thought there’s a cost in seeming to approve of billionaires and as a consequence that, if we do want to state our approval of them, we should better be sure that we’re right.
Thanks! Your explanation makes a lot of sense. I think ~20h of work is reasonable for an exploratory Forum post (the majority of my posts have taken less time[1]). Re lots of uncritical comments, I do think my comment (right now the top comment) is fairly critical: the implication of “I think this ranking neglects to track the most important factor” is pretty damning in terms of decision-relevancy. But it’s possible my words were too sugarcoated.
I would guess this post took 15-20h, and this post took ~5-8, including reading the primary source
Thanks Linch, I think the amount of time /effort probably depends on the topic, and is unfortunately very difficult to make much progress on “ranking ten people’s entire life impact” with 20 hours. For many posts, 2 hours of effort would suffice, it just depends on the topic! But I’m certain reasonable people can disagree here
I also think the tone of the comments has shifted since I posted this, and ironically the number of people who agree with my comments might suggest that I was wrong to be concerned.
I can see some, although I admit I too am skeptical of the harm here:
People become aware of good things about [powerful person] without realising that person is also a douchebag in many ways
People want to bring about a world without billionaires slightly less (or more, depending on which of these you think is bad)
I don’t think (1) is a problem, if anything I think the world is far too skewed in its tendency to judge powerful people based on personal virtues and foibles, and not enough on judging them based on their impact (in either direction).
E.g. all the comments about Zuckerberg sounding robotic and very few about him donating money to random charities, far more comments about Elon’s various Twitter spats and few about either space or OpenAI, various public figures lambasted for breaking lockdowns (rather than utterly poor pandemic preparedness and response), the whole Andrew Cuomo thing, etc.
Well, I disagree on that one. In my view it’s harmful to “give a free pass” to obviously bad behaviours by powerful people because of other things they do, both in terms of justice in general and of sending a message to society that morality is optional.
E.g. many people fetishizing Musk over his space ideas or Twitter personae, and ignoring his awful labour practices or his sexual harassment and what appears to me to be predatory behaviour.
Edit to add: I wouldn’t put “Zuckerberg sounding robotic” in this category, but I would “various public figures breaking lockdowns”. I think you greatly underestimate the importance of leading by example.
I don’t understand why “we should hold powerful figures accountable for the impact of their actions” translates to “morality is optional.”
From their comments
Not that convinced, but I’ve added a caveat at the top & pointed out the estimated number of hours it took me to write this.
Cool, thanks :)