The odd thing is, I would use HPMOR as a model of how to do it right. Its main failing is that it fails to make it clear that the protagonist isn’t a perfect propaganda figure, and shouldn’t be emulated—that the audience ends up thinking that the protagonist is making giant mistakes throughout the story because the author thinks those are the correct decisions, not because he’s an eleven-year-old in over his head. But the author agrees with him enough that if you aren’t paying careful attention, he comes off as an arrogant jerk the author endorses, instead of a person with many virtues and the vice of being an arrogant jerk.
(Clarification: I tried to read HPMOR twice and disliked it before I fell in love the third time. I think it is genuinely Great Art, but flawed in the way most Great Art is flawed—that of aiming for being twice as good as the best art that previously existed and ending up being an uneven mix of 160% as good and 60% as good, which may well make you throw the book at the wall during the 60% parts.)
But that side-note aside: I agree that making all EA-style figures into saints is a risk. People may well get turned off at being preached to; I know I do. Again, see my comment on “Blue Bird and Black Bird.” But...
If you’ve ever read Scott Alexander’s Unsong—I don’t know if you have—the central figure in the story is the Comet King. He isn’t the hero; the hero is a not-very-interesting nebbish stuck in the world the Comet King made. But the Comet King is EA, is really presented as purely good, and is a genuinely psychologically fascinating character. Every page he’s on blazes with light and life and joy, and yet he isn’t a Mary Sue, because by the time the main story starts, he has already lost. The book is about the aftermath of a perfectly good hero failing to save the world, the book is unquestionably pro-effective-altruist, and where the book fails, it isn’t because it’s being too pro-effective-altruist, it’s because the protagonist if the main story is really kind of dull and uninteresting.
So I think it can work. As good examples of other fiction that work despite being ideological, I’d recommend Bujold’s “Shards of Honor,” Terry Pratchett’s “Night Watch,” and, as I said, “The Lord of the Rings.”
But although I I think there’s problems with EA-heroes if badly written, I also think there’s an equal and equivalent problem with EA-style villains, even if they are well-written. It can work I agree, artistically speaking, to put a viewpoint you sympathize with but disagree with in the mouth of your villain, then make him take it to evil ends. That’s something I’ve written and it’s something I enjoy writing, because it allows you to have a strong ideological conflict between two good ends while still having a hero, and that has a lot of potential to work well.
But I’m not wholly comfortable with it, and this is why:
I think our subconscious or semi-conscious mind has bins labeled “traits of villains” and “traits of heroes,” and when we see something in real life that we are used to thinking of as always being labeled in fiction with “trait of villain” or “trait of heroes,” we apply that label in real life. When an artwork gives ‘uses reason and logic to try to maximize good’ as a trait of villains, especially in a culture in which everyone else is also using it as a trait of villain, it reinforces that as being part of the ‘villain trait’ box, since it’s usually (not in your case!) paralleled with a conservative, chauvinistic anti-intellectual badass hero. I think this will genuinely cause people to immediately round off, in real life, ‘tries to use reason and logic to maximize good’ as something that will go horribly wrong in real life, and ‘is a badass cowboy who plays by his own rules but accomplishes good things anyway because he’s a FUNDAMENTALLY GOOD PERSON’ as being something that will go right in real life. In this case, if you want me to say I have less respect for people than you do, that’s entirely possible; I don’t think people rationally feel that they do this, I don’t think people who slow down to think do this, but I think that people who operate on automatic do this; that they treat the Terminator movies as a model for how AI will go wrong in real life, and I think that this has bad consequences if everyone is saying the same thing about how ‘trying to maximize good’ will go horribly wrong.
Now, if you believe that EA will go horribly wrong in real life, if you really think that Engels ought to be treated as an early member of EA and as a model of how badly EA will go wrong in the future, it makes sense to write that. But I don’t. And from your comments, I don’t think you do.
Now, again, this isn’t a reason why I think you should never do the ‘charismatic villain arguing for an underappreciated cause’ bit. Plenty of good stories have done it. I’ve done it. But if it isn’t a cause you disapprove of, and most of the other writers writing about the cause have the same take you have on it (and if it’s a thriving cause and the take is one of the first ones you thought of, they probably will) you’re (a) doing something that is individually totally reasonable, and (b) contributing to an unreasonable aggregate. And that’s not something I really approve of, and it’s not something you… ex ante ought to expect members of the subculture to approve of? Or subsidize? Like, Larks’ second point wasn’t one I said explicitly, because it’s attributing malice where malice isn’t the most likely option, but I totally thought it before the obvious realization kicked in that it was statistically unlikely.
These thoughts are super helpful for understanding where you’re coming from, so thank you!! I really appreciate you taking the time to write them all out—my thoughts will be much shorter because I don’t have much to add, not because they weren’t thought-provoking and interesting!I
think we have somewhat different beliefs about what makes the speaker’s actions wrong—I think for me it lands very far to one side of the “clearly evil” to “clearly good” trolley problem spectrum and it’s wrongness is a) very clear to me, and b) very hard for me to pin down a reason for: I don’t really find any of the answers I can think of satisfying (including the ones in the works you mentioned—the fact that plans can fail and change the planner in unforeseen ways is a beautiful and important observation, but in this case the plan more-or-less succeeds and it still feels evil to me.) I find this combination fascinating, but I can see how this comes across rather differently if you don’t share this dissatisfaction, which it sounds like is less universal than I believed.
Unsong sounds like a very interesting piece of writing, I will have to check it out!
I’m glad you aren’t offended! I get easily worried that I might be saying things in an offensive manner and I appreciate you reassuring me that I didn’t! I am always very happy to write long and elaborate reviews of fiction and I am glad you appreciated it.
And I would agree that the protagonist is evil (indeed, he admits he is evil—he’s quite clear that he enjoyed what he did) and also took a set of actions which may have had net-positive utility. I don’t think we know that it did; it’s possible that some vague combination of making people distrust EA-style arguments, imposing costs on people both directly (his victims) and indirectly (court costs, prison costs, stress to everyone vaguely associated, costs of additional security precautions taken because his existence is evidence of the world being less safe than you thought) and so forth and so on made it net-negative.
But I will confidently deny that he was in an epistemic position to expect his actions would be positive, let alone the optimal decision. I could theoretically imagine a world in which this was false, and he genuinely did have the knowledge required for his actions to actually be both ex post and ex ante optimal, but I don’t actually think I can actually imagine a world in which I was in the epistemic state of knowing that he knew his actions were ex post and ex ante optimal; my mental state in such a world would be sufficiently different that I’m not sure I’d be the same person. So I’m really quite comfortable condemning him, though I’ll admit I’d vote for life imprisonment instead of execution.
And Unsong is very interesting! It doesn’t always succeed at what it’s doing, as I mentioned I find the protagonist kind of boring, but it’s trying such fascinating things and it succeeds sufficiently often to be worth reading.
Reply-edit for clarification to expand my response to one of your points: I think it is worth, in a lot of situations, judging based on “should it have worked,” instead of “did it work.” That your model predicted it shouldn’t work and it did work is evidence your model is seriously flawed, just to be clear, I’m not arguing we should completely throw out the experiment and just go with our previous model, but, also, we shouldn’t say “the one guy who won the lottery was right and everyone else was wrong,” because everyone who bought a ticket had the same chance of winning, and ex ante the lottery was a losing bet for all of them.
(Unless the lottery was crooked but that’s a side note.)
So, even if it worked, I still think the protagonist’s motive was unreasonable; even if it worked, I don’t feel it should have worked, statistically speaking, as opposed to him getting immediately spotted, arrested, and spending the next five years of his life in jail in which he can do no good at all. Or someone’s angry brother taking a shot at him with a firearm, causing him to die instantly after he’d donated only $8000 to Givewell’s top charities, as opposed to if he’d peacefully sat back and worked a high-paying job he would have donated $800,000 over the course of his life. Or someone successfully suing to get all the donated money back as a class-action suit, causing the Against Malaria Foundation to go bankrupt because it already spent it all on bed nets and couldn’t get a refund. Not that all of those are equally likely, but there are a lot of ways for his kind of plan to fail at levels of badness approaching these, and if they fail this way he definitely killed people, and I don’t find the assumption that he knew none of them would happen very persuasive.
The odd thing is, I would use HPMOR as a model of how to do it right. Its main failing is that it fails to make it clear that the protagonist isn’t a perfect propaganda figure, and shouldn’t be emulated—that the audience ends up thinking that the protagonist is making giant mistakes throughout the story because the author thinks those are the correct decisions, not because he’s an eleven-year-old in over his head. But the author agrees with him enough that if you aren’t paying careful attention, he comes off as an arrogant jerk the author endorses, instead of a person with many virtues and the vice of being an arrogant jerk.
(Clarification: I tried to read HPMOR twice and disliked it before I fell in love the third time. I think it is genuinely Great Art, but flawed in the way most Great Art is flawed—that of aiming for being twice as good as the best art that previously existed and ending up being an uneven mix of 160% as good and 60% as good, which may well make you throw the book at the wall during the 60% parts.)
But that side-note aside: I agree that making all EA-style figures into saints is a risk. People may well get turned off at being preached to; I know I do. Again, see my comment on “Blue Bird and Black Bird.” But...
If you’ve ever read Scott Alexander’s Unsong—I don’t know if you have—the central figure in the story is the Comet King. He isn’t the hero; the hero is a not-very-interesting nebbish stuck in the world the Comet King made. But the Comet King is EA, is really presented as purely good, and is a genuinely psychologically fascinating character. Every page he’s on blazes with light and life and joy, and yet he isn’t a Mary Sue, because by the time the main story starts, he has already lost. The book is about the aftermath of a perfectly good hero failing to save the world, the book is unquestionably pro-effective-altruist, and where the book fails, it isn’t because it’s being too pro-effective-altruist, it’s because the protagonist if the main story is really kind of dull and uninteresting.
So I think it can work. As good examples of other fiction that work despite being ideological, I’d recommend Bujold’s “Shards of Honor,” Terry Pratchett’s “Night Watch,” and, as I said, “The Lord of the Rings.”
But although I I think there’s problems with EA-heroes if badly written, I also think there’s an equal and equivalent problem with EA-style villains, even if they are well-written. It can work I agree, artistically speaking, to put a viewpoint you sympathize with but disagree with in the mouth of your villain, then make him take it to evil ends. That’s something I’ve written and it’s something I enjoy writing, because it allows you to have a strong ideological conflict between two good ends while still having a hero, and that has a lot of potential to work well.
But I’m not wholly comfortable with it, and this is why:
I think our subconscious or semi-conscious mind has bins labeled “traits of villains” and “traits of heroes,” and when we see something in real life that we are used to thinking of as always being labeled in fiction with “trait of villain” or “trait of heroes,” we apply that label in real life. When an artwork gives ‘uses reason and logic to try to maximize good’ as a trait of villains, especially in a culture in which everyone else is also using it as a trait of villain, it reinforces that as being part of the ‘villain trait’ box, since it’s usually (not in your case!) paralleled with a conservative, chauvinistic anti-intellectual badass hero. I think this will genuinely cause people to immediately round off, in real life, ‘tries to use reason and logic to maximize good’ as something that will go horribly wrong in real life, and ‘is a badass cowboy who plays by his own rules but accomplishes good things anyway because he’s a FUNDAMENTALLY GOOD PERSON’ as being something that will go right in real life. In this case, if you want me to say I have less respect for people than you do, that’s entirely possible; I don’t think people rationally feel that they do this, I don’t think people who slow down to think do this, but I think that people who operate on automatic do this; that they treat the Terminator movies as a model for how AI will go wrong in real life, and I think that this has bad consequences if everyone is saying the same thing about how ‘trying to maximize good’ will go horribly wrong.
Now, if you believe that EA will go horribly wrong in real life, if you really think that Engels ought to be treated as an early member of EA and as a model of how badly EA will go wrong in the future, it makes sense to write that. But I don’t. And from your comments, I don’t think you do.
Now, again, this isn’t a reason why I think you should never do the ‘charismatic villain arguing for an underappreciated cause’ bit. Plenty of good stories have done it. I’ve done it. But if it isn’t a cause you disapprove of, and most of the other writers writing about the cause have the same take you have on it (and if it’s a thriving cause and the take is one of the first ones you thought of, they probably will) you’re (a) doing something that is individually totally reasonable, and (b) contributing to an unreasonable aggregate. And that’s not something I really approve of, and it’s not something you… ex ante ought to expect members of the subculture to approve of? Or subsidize? Like, Larks’ second point wasn’t one I said explicitly, because it’s attributing malice where malice isn’t the most likely option, but I totally thought it before the obvious realization kicked in that it was statistically unlikely.
These thoughts are super helpful for understanding where you’re coming from, so thank you!! I really appreciate you taking the time to write them all out—my thoughts will be much shorter because I don’t have much to add, not because they weren’t thought-provoking and interesting!I
think we have somewhat different beliefs about what makes the speaker’s actions wrong—I think for me it lands very far to one side of the “clearly evil” to “clearly good” trolley problem spectrum and it’s wrongness is a) very clear to me, and b) very hard for me to pin down a reason for: I don’t really find any of the answers I can think of satisfying (including the ones in the works you mentioned—the fact that plans can fail and change the planner in unforeseen ways is a beautiful and important observation, but in this case the plan more-or-less succeeds and it still feels evil to me.) I find this combination fascinating, but I can see how this comes across rather differently if you don’t share this dissatisfaction, which it sounds like is less universal than I believed.
Unsong sounds like a very interesting piece of writing, I will have to check it out!
I’m glad you aren’t offended! I get easily worried that I might be saying things in an offensive manner and I appreciate you reassuring me that I didn’t! I am always very happy to write long and elaborate reviews of fiction and I am glad you appreciated it.
And I would agree that the protagonist is evil (indeed, he admits he is evil—he’s quite clear that he enjoyed what he did) and also took a set of actions which may have had net-positive utility. I don’t think we know that it did; it’s possible that some vague combination of making people distrust EA-style arguments, imposing costs on people both directly (his victims) and indirectly (court costs, prison costs, stress to everyone vaguely associated, costs of additional security precautions taken because his existence is evidence of the world being less safe than you thought) and so forth and so on made it net-negative.
But I will confidently deny that he was in an epistemic position to expect his actions would be positive, let alone the optimal decision. I could theoretically imagine a world in which this was false, and he genuinely did have the knowledge required for his actions to actually be both ex post and ex ante optimal, but I don’t actually think I can actually imagine a world in which I was in the epistemic state of knowing that he knew his actions were ex post and ex ante optimal; my mental state in such a world would be sufficiently different that I’m not sure I’d be the same person. So I’m really quite comfortable condemning him, though I’ll admit I’d vote for life imprisonment instead of execution.
And Unsong is very interesting! It doesn’t always succeed at what it’s doing, as I mentioned I find the protagonist kind of boring, but it’s trying such fascinating things and it succeeds sufficiently often to be worth reading.
Reply-edit for clarification to expand my response to one of your points: I think it is worth, in a lot of situations, judging based on “should it have worked,” instead of “did it work.” That your model predicted it shouldn’t work and it did work is evidence your model is seriously flawed, just to be clear, I’m not arguing we should completely throw out the experiment and just go with our previous model, but, also, we shouldn’t say “the one guy who won the lottery was right and everyone else was wrong,” because everyone who bought a ticket had the same chance of winning, and ex ante the lottery was a losing bet for all of them.
(Unless the lottery was crooked but that’s a side note.)
So, even if it worked, I still think the protagonist’s motive was unreasonable; even if it worked, I don’t feel it should have worked, statistically speaking, as opposed to him getting immediately spotted, arrested, and spending the next five years of his life in jail in which he can do no good at all. Or someone’s angry brother taking a shot at him with a firearm, causing him to die instantly after he’d donated only $8000 to Givewell’s top charities, as opposed to if he’d peacefully sat back and worked a high-paying job he would have donated $800,000 over the course of his life. Or someone successfully suing to get all the donated money back as a class-action suit, causing the Against Malaria Foundation to go bankrupt because it already spent it all on bed nets and couldn’t get a refund. Not that all of those are equally likely, but there are a lot of ways for his kind of plan to fail at levels of badness approaching these, and if they fail this way he definitely killed people, and I don’t find the assumption that he knew none of them would happen very persuasive.