I finally got over the trivial inconvenience of creating an account just to upvote this.
This sort of thing seems like precisely the right approach from an explore/exploit (multi-armed bandit) perspective. In fact, while most criticism of EA seems to center around the “effective” part, it seems like the “altruism” part may be the weaker target. (What if the specific subset of goals EAs look to maximize aren’t what we’d endorse upon reflection?)
So, this sort of research could lead to entirely new branches of EA, which wouldn’t exist counterfactually. It could also help EAs make informed decisions reguarding how to weight different types of suffering (like QALYs and DALYs), or human suffering against animal suffering, or X units of happiness against Y units of suffering, or even between entirely different types of qualia.
However, even more broadly, I don’t see qualia themselves as being the only thing of value. It seems like they are the reward/punishment signals that prod us into developing certain values and not others. As a preference utilitarian and not a hedonistic utilitarian, it’s those values I’m most interested in, and not qualia per se, unless on reflection we decide to place terminal value on the qualia themselves.
That said, understanding qualia seems like at least a first step in understanding and improving value generation, if it occurs via reinforcement learning. And, even if we only value some nonzero fraction of qulia upon reflection, tiling the universe with that is still hugely preferable to tiling the universe with paperclips or whatever. So, it could offer a plan B for some existential threats, with flexible timing in the event of an arms race.
I’d certainly like to encourage more work in this space, as long as we can approach it with rigor.
Traditionally, both consciousness research and ethics research have been black holes—smart people go in, but nothing comes out. But I think that’s changing, as a few paths to systematically approach these topics open up. But by default, these paths won’t get taken, because there are no institutions pushing them.
I finally got over the trivial inconvenience of creating an account just to upvote this.
This sort of thing seems like precisely the right approach from an explore/exploit (multi-armed bandit) perspective. In fact, while most criticism of EA seems to center around the “effective” part, it seems like the “altruism” part may be the weaker target. (What if the specific subset of goals EAs look to maximize aren’t what we’d endorse upon reflection?)
So, this sort of research could lead to entirely new branches of EA, which wouldn’t exist counterfactually. It could also help EAs make informed decisions reguarding how to weight different types of suffering (like QALYs and DALYs), or human suffering against animal suffering, or X units of happiness against Y units of suffering, or even between entirely different types of qualia.
However, even more broadly, I don’t see qualia themselves as being the only thing of value. It seems like they are the reward/punishment signals that prod us into developing certain values and not others. As a preference utilitarian and not a hedonistic utilitarian, it’s those values I’m most interested in, and not qualia per se, unless on reflection we decide to place terminal value on the qualia themselves.
That said, understanding qualia seems like at least a first step in understanding and improving value generation, if it occurs via reinforcement learning. And, even if we only value some nonzero fraction of qulia upon reflection, tiling the universe with that is still hugely preferable to tiling the universe with paperclips or whatever. So, it could offer a plan B for some existential threats, with flexible timing in the event of an arms race.
Thank you!
I’d certainly like to encourage more work in this space, as long as we can approach it with rigor.
Traditionally, both consciousness research and ethics research have been black holes—smart people go in, but nothing comes out. But I think that’s changing, as a few paths to systematically approach these topics open up. But by default, these paths won’t get taken, because there are no institutions pushing them.
Re: preference-based ethics vs qualia-based ethics, here’s a piece by Andres Gomez Emilsson that makes the case that although we think we’re optimizing preferences, often we’re actually optimizing qualia: https://qualiacomputing.com/2016/11/19/the-tyranny-of-the-intentional-object/