Epistemologist specialized in the difficulties of alignment and how to solve AI X-Risks. Currently at Conjecture.
Blogging at For Methods.
adamShimi
Karma: 238
Thanks for this thoughtful analysis! I must admit that I never really considered the welfare of snails as an issue, maybe because I am french and thus am culturally used to eating them.
One thing I wanted to confirm anecdotally is the consummation of snails in France. Even if snails with parsley butter is a classic french dish, it is eaten quite rarely (only for celebrations or Christmas and new year’s eve diners). And I know many people that don’t eat snails because they find it disgusting, even though they saw people eating them all their lives (similar to oysters in a sense).
As for what should be done, your case for the non-tractability and non-priority of snails welfare is pretty convincing. I still take from this post that undue pain (with or without sentience) is caused in snails, even from the position where it is okay to eat animals (My current position, which is in reassessing). I was quite horrified by the slime part.
The geometric intuition underlying this post already proves useful for me!
Yesterday, while discussing with a friend why I want to change my research topic to AI Safety instead of what I currently do (distributed computing), my first intuition was that AI safety aims at shaping the future, while distributed computing is relatively agnostic about it. But a far better intuition comes when considering the vector along the current trajectory in state space, starting at the current position of the world, and whose direction and length capture the trajectory and the speed at which we follow it.
From this perspective, the difference between distributed computing/hardware/cloud computing research and AI safety research is obvious in terms of vector operations:
The former amounts to positive scaling of the vector, and thus makes us go along our current trajectory faster.
While the latter amounts to rotations (and maybe scaling, but it is a bit less relevant), which allows us to change our trajectory.
And since I am not sure we are heading in the right direction, I prefer to be able to change the trajectory (at least potentially).
That’s a great criterion! We might be able to find some weird counter-example, but it solves all of my issues. Because intellectual work/knowledge might be a part of all actions, but it isn’t necessary on the main causal path.
I think this might actually deserve its own post.
Thanks for the in-depth answer!
Let’s take your concrete example about democracy. If I understand correctly, you separate the progress towards democracy into:
discovering/creating the concept of democracy, learning it, spreading the concept itself, which is under the differential intellectual progress.
convince people to implement democracy, do the fieldwork for implementing it, which is at least partially under the differential progress.
But the thing is, I don’t have a clear criterion for distinguishing the two. My first ideas were:
differential intellectual progress is about any interaction where the relevant knowledge of some participant increases (in the democracy example, learning the idea is relevant, learning that the teeth of your philosophy teacher are slightly ajar is not). And then differential progress is about any interaction making headway towards a change in the world (in the example the implementation of democracy). But I cannot think of a situation where no one learns anything relevant to the situation at hand. That is, for these definitions, differential progress is differential intellectual progress.
Another idea is that differential intellectual progress is about all the work needed for making rational agents implement a change in the world, while differential progress is about all the work needed for making humans implement a change in the world. Here the two are clearly different. My issue there stems with the word intellectual: in this case Amos and Tversky’s work, and pretty much all of behavioral economics, is not intellectual.
Does any of these two criteria feel right to you?
I did find this post clear and useful; it will be my main recommendation if I want to explain this concept to someone else.
I also really like your proposition of “potential information hazards”, as at that point in the post, I was wondering if all basic research should be considered information hazards, which would make the whole concept rather vacuous. Maybe one way to address the potential information hazards is to try to quantify how removed are they from potential concrete risks?
Anyway, I’m looking forward to the next posts on dealing with these potential information hazards.
Thanks a lot for this summary post! I did not know of these concepts, and I feel they are indeed very useful for thinking about these issues.
I do have some trouble with the distinction between intellectual progress and progress though. During my first reading, I felt like all the topics mentioned in the progress section were actually about intellectual progress.
Now, rereading the post, I think I see a distinction, but I have trouble crystallizing it in concrete terms. Is the difference about creating ideas and implementing them? But then it feels very reminiscent of the whole fundamental/applied research distinction, which gets blurry very fast. And even the applications of ideas and solutions requires a whole lot of intellectual work.
Maybe the issue is with the word intellectual? I get that you’re not the one choosing the terms, but maybe something like fundamental progress or abstraction progress or theoretical progress would be more fitting? Or did I miss some other difference?
As a tool for existential risk research, I feel like the graphical representation will indeed be useful in crystallizing the differences in hypotheses between researchers. It might even serves as a self-assessing tool, for checking quickly some of the consequences of one’s own view.
But beyond the trajectories (and maybe specific distances), are you planning on representing the other elements you mention? Like the uncertainty or the speed along trajectories? I feel like the more details about an approach can be integrated into a simple graphical representation, the more this tool will serve to disentangle disagreement between researchers.
Thanks a lot for this podcast! I liked the summary you provided, and I think it is great to see people struggling to make sense of a lot of complex information on a topic, almost in direct. Given that you repeat multiple times that neither of you is an expert on the subject, I think this podcast is a net positive: it gives information while encouraging the listeners to go look for themselves.
Another great point: the criticism of the meme about overreacting. While listening to the beginning, when you said that there was no reason to panic, I wanted to object that preparing for possible catastrophes is as important, if not more important, to do before they are obviously here. But the discussion of the meme clarified this point, and I thought it was great.
Thanks for the effort in summarizing and synthesizing this tangle of notions! Notably, I learned about axiology, and I am very glad I did.
One potential addition to the discussion of decision theory might be the use of “normative”, “descriptive” and “prescriptive” within decision theory itself, which is slightly different. To quote the Decision Theory FAQ on Less Wrong:
Because that was one way I think about these words, I got confused by your use of “prescriptive”, even though you used it correctly in this context.