Buck- for an internal counterpoint you may want to discuss QRI’s research with Vaniver. We had a good chat about what we’re doing at the Boston SSC meetup, and Romeo attended a MIRI retreat earlier in the summer and had some good conversations with him there also.
To put a bit of a point on this, I find the “crank philosophy” frame a bit questionable if you’re using only thin-slice outside view and not following what we’re doing. Probably, one could use similar heuristics to pattern-match MIRI as “crank philosophy” also (probably, many people have already done exactly this to MIRI, unfortunately).
FWIW I agree with Buck’s criticisms of the Symmetry Theory of Valence (both content and meta) and also think that some other ideas QRI are interested in are interesting. Our conversation on the road trip was (I think) my introduction to Connectome Specific Harmonic Waves (CSHW), for example, and that seemed promising to think about.
I vaguely recall us managing to operationalize a disagreement, let me see if I can reconstruct it:
A ‘multiple drive’ system, like PCT’s hierarchical control system, has an easy time explaining independent desires and different flavors of discomfort. (If one both has a ‘hunger’ control system and a ‘thirst’ control system, one can easily track whether one is hungry, thirsty, both, or neither.) A ‘single drive’ system, like expected utility theories more generally, has a somewhat more difficult time explaining independent desires and different flavors of discomfort, since you only have the one ‘utilon’ number.
But this is mostly because we’re looking at different parts of the system as the ‘value’. If I have a vector of ‘control errors’, I get the nice multidimensional property. If I have a utility function that’s a function of a vector, the gradient of that function will be a vector that gives me the same nice multidimensional property.
CSHW gives us a way to turn the brain into a graph and then the graph activations into energies in different harmonics. Then we can look at an energy distribution and figure out how consonant or dissonant it is. This gives us the potentially nice property that ‘gradients are easy’, where if ‘perfect harmony’ (= all consonant energy) corresponds to the ‘0 error’ case in PCT, being hungry will look like missing some consonant energy or having some dissonant energy.
Here we get the observational predictions: for PCT, ‘hunger’ and ‘thirst’ and whatever other drives just need to be wire voltages somewhere, but for QRI’s theory as I understand it, they need to be harmonic energies with particular numerical properties (such that they are consonant or dissonant as expected to make STV work out).
Of course, it could be the case that there are localized harmonics in the connectome, such that we get basically the same vector represented in the energy distribution, and don’t have a good way to distinguish between them.
On that note, I remember we also talked about the general difficulty of distinguishing between theories in this space; for example, my current view is that Friston-style predictive coding approaches and PCT-style hierarchical control approaches end up predicting very similar brain architecture, and the difference is ‘what seems natural’ or ‘which underlying theory gets more credit.’ (Is it the case that the brain is trying to be Bayesian, or the brain is trying to be homeostatic, and embedded Bayesianism empirically performs well at that task?) I expect a similar thing could be true here, where whether symmetry is the target or the byproduct is unclear, but in such cases I normally find myself reaching for ‘byproduct’. It’s easy to see how evolution could want to build homeostatic systems, and harder to see how evolution could want to build Bayesian systems; I think a similar story goes through for symmetry and brains.
This makes me more sympathetic to something like “symmetry will turn out to be a marker for something important and good” (like, say, ‘focus’) than something like “symmetry is definitionally what feeling good is.”
I think this is a great description. “What happens if we seek out symmetry gradients in brain networks, but STV isn’t true?” is something we’ve considered, and determining ground-truth is definitely tricky. I refer to this scenario as the “Symmetry Theory of Homeostatic Regulation”—https://opentheory.net/2017/05/why-we-seek-out-pleasure-the-symmetry-theory-of-homeostatic-regulation/ (mostly worth looking at the title image, no need to read the post)
I’m (hopefully) about a week away from releasing an update to some of the things we discussed in Boston, basically a unification of Friston/Carhart-Harris’s work on FEP/REBUS with Atasoy’s work on CSHW—will be glad to get your thoughts when it’s posted.
Oh, an additional detail that I think was part of that conversation: there’s only really one way to have a ‘0-error’ state in a hierarchical controls framework, but there are potentially many consonant energy distributions that are dissonant with each other. Whether or not that’s true, and whether each is individually positive valence, will be interesting to find out.
(If I had to guess, I would guess the different mutually-dissonant internally-consonant distributions correspond to things like ‘moods’, in a way that means they’re not really value but are somewhat close, and also that they exist. The thing that seems vaguely in this style are differing brain waves during different cycles of sleep, but I don’t know if those have clear waking analogs, or what they look like in the CSHW picture.)
I think that MIRI looks kind of crankish from the outside, and this should indeed make people initially more skeptical of us. I think that we have a few other external markers of legitimacy now, such as the fact that MIRI people were thinking and writing about AI safety from the early 2000s and many smart people have now been persuaded that this is indeed an issue to be concerned with. (It’s not totally obvious to me that these markers of legitimacy mean that anyone should take us seriously on the question “what AI safety research is promising”.) When I first ran across MIRI, I was kind of skeptical because of the signs of crankery; I updated towards them substantially because I found their arguments and ideas compelling, and people whose judgement I respected also found them compelling.
I think that the signs of crankery in QRI are somewhat worse than 2008 MIRI’s signs of crankery.
I also think that I’m somewhat qualified to assess QRI’s work (as someone who’s spent ~100 paid hours thinking about philosophy of mind in the last few years), and when I look at it, I think it looks pretty crankish and wrong.
QRI is tackling a very difficult problem, as is MIRI. It took many, many years for MIRI to gather external markers of legitimacy. My inside view is that QRI is on the path of gaining said markers; for people paying attention to what we’re doing, I think there’s enough of a vector right now to judge us positively. I think these markers will be obvious from the ‘outside view’ within a short number of years.
But even without these markers, I’d poke at your position from a couple angles:
I. Object-level criticism is best
First, I don’t see evidence you’ve engaged with our work beyond very simple pattern-matching. You note that “I also think that I’m somewhat qualified to assess QRI’s work (as someone who’s spent ~100 paid hours thinking about philosophy of mind in the last few years), and when I look at it, I think it looks pretty crankish and wrong.” But *what* looks wrong? Obviously doing something new will pattern-match to crankish, regardless of whether it is crankish, so in terms of your rationale-as-stated, I don’t put too much stock in your pattern detection (and perhaps you shouldn’t either). If we want to avoid accidentally falling into (1) ‘negative-sum status attack’ interactions, and/or (2) hypercriticism of any fundamentally new thing, neither of which is good for QRI, for MIRI, or for community epistemology, object-level criticisms (and having calibrated distaste for low-information criticisms) seem pretty necessary.
Also, we do a lot more things than just philosophy, and we try to keep our assumptions about the Symmetry Theory of Valence separate from our neuroscience—STV can be wrong and our neuroscience can still be correct/useful. That said, empirically the neuroscience often does ‘lead back to’ STV.
I’d also suggest that the current state of philosophy, and especially philosophy of mind and ethics, is very dismal. I give my causal reasons for this here: https://opentheory.net/2017/10/rescuing-philosophy/ - I’m not sure if you’re anchored to existing theories in philosophy of mind being reasonable or not.
II. What’s the alternative?
If there’s one piece I would suggest engaging with, it’s my post arguing against functionalism. I think your comments presuppose functionalism is reasonable and/or the only possible approach, and the efforts QRI is putting into building an alternative are certainly wasted. I strongly disagree with this; as I noted in my Facebook reply,
>Philosophically speaking, people put forth analytic functionalism as a theory of consciousness (and implicitly a theory of valence?), but I don’t think it works *qua* a theory of consciousness (or ethics or value or valence), as I lay out here: https://forum.effectivealtruism.org/.../why-i-think-the...-- This is more-or-less an answer to some of Brian Tomasik’s (very courageous) work, and to sum up my understanding I don’t think anyone has made or seems likely to make ‘near mode’ progress, e.g. especially of the sort that would be helpful for AI safety, under the assumption of analytic functionalism.
I always find in-person interactions more amicable & high-bandwidth—I’ll be back in the Bay early December, so if you want to give this piece a careful read and sit down to discuss it I’d be glad to join you. I think it could have significant implications for some of MIRI’s work.
Buck- for an internal counterpoint you may want to discuss QRI’s research with Vaniver. We had a good chat about what we’re doing at the Boston SSC meetup, and Romeo attended a MIRI retreat earlier in the summer and had some good conversations with him there also.
To put a bit of a point on this, I find the “crank philosophy” frame a bit questionable if you’re using only thin-slice outside view and not following what we’re doing. Probably, one could use similar heuristics to pattern-match MIRI as “crank philosophy” also (probably, many people have already done exactly this to MIRI, unfortunately).
FWIW I agree with Buck’s criticisms of the Symmetry Theory of Valence (both content and meta) and also think that some other ideas QRI are interested in are interesting. Our conversation on the road trip was (I think) my introduction to Connectome Specific Harmonic Waves (CSHW), for example, and that seemed promising to think about.
I vaguely recall us managing to operationalize a disagreement, let me see if I can reconstruct it:
Of course, it could be the case that there are localized harmonics in the connectome, such that we get basically the same vector represented in the energy distribution, and don’t have a good way to distinguish between them.
On that note, I remember we also talked about the general difficulty of distinguishing between theories in this space; for example, my current view is that Friston-style predictive coding approaches and PCT-style hierarchical control approaches end up predicting very similar brain architecture, and the difference is ‘what seems natural’ or ‘which underlying theory gets more credit.’ (Is it the case that the brain is trying to be Bayesian, or the brain is trying to be homeostatic, and embedded Bayesianism empirically performs well at that task?) I expect a similar thing could be true here, where whether symmetry is the target or the byproduct is unclear, but in such cases I normally find myself reaching for ‘byproduct’. It’s easy to see how evolution could want to build homeostatic systems, and harder to see how evolution could want to build Bayesian systems; I think a similar story goes through for symmetry and brains.
This makes me more sympathetic to something like “symmetry will turn out to be a marker for something important and good” (like, say, ‘focus’) than something like “symmetry is definitionally what feeling good is.”
I think this is a great description. “What happens if we seek out symmetry gradients in brain networks, but STV isn’t true?” is something we’ve considered, and determining ground-truth is definitely tricky. I refer to this scenario as the “Symmetry Theory of Homeostatic Regulation”—https://opentheory.net/2017/05/why-we-seek-out-pleasure-the-symmetry-theory-of-homeostatic-regulation/ (mostly worth looking at the title image, no need to read the post)
I’m (hopefully) about a week away from releasing an update to some of the things we discussed in Boston, basically a unification of Friston/Carhart-Harris’s work on FEP/REBUS with Atasoy’s work on CSHW—will be glad to get your thoughts when it’s posted.
Oh, an additional detail that I think was part of that conversation: there’s only really one way to have a ‘0-error’ state in a hierarchical controls framework, but there are potentially many consonant energy distributions that are dissonant with each other. Whether or not that’s true, and whether each is individually positive valence, will be interesting to find out.
(If I had to guess, I would guess the different mutually-dissonant internally-consonant distributions correspond to things like ‘moods’, in a way that means they’re not really value but are somewhat close, and also that they exist. The thing that seems vaguely in this style are differing brain waves during different cycles of sleep, but I don’t know if those have clear waking analogs, or what they look like in the CSHW picture.)
Most things that look crankish are crankish.
I think that MIRI looks kind of crankish from the outside, and this should indeed make people initially more skeptical of us. I think that we have a few other external markers of legitimacy now, such as the fact that MIRI people were thinking and writing about AI safety from the early 2000s and many smart people have now been persuaded that this is indeed an issue to be concerned with. (It’s not totally obvious to me that these markers of legitimacy mean that anyone should take us seriously on the question “what AI safety research is promising”.) When I first ran across MIRI, I was kind of skeptical because of the signs of crankery; I updated towards them substantially because I found their arguments and ideas compelling, and people whose judgement I respected also found them compelling.
I think that the signs of crankery in QRI are somewhat worse than 2008 MIRI’s signs of crankery.
I also think that I’m somewhat qualified to assess QRI’s work (as someone who’s spent ~100 paid hours thinking about philosophy of mind in the last few years), and when I look at it, I think it looks pretty crankish and wrong.
QRI is tackling a very difficult problem, as is MIRI. It took many, many years for MIRI to gather external markers of legitimacy. My inside view is that QRI is on the path of gaining said markers; for people paying attention to what we’re doing, I think there’s enough of a vector right now to judge us positively. I think these markers will be obvious from the ‘outside view’ within a short number of years.
But even without these markers, I’d poke at your position from a couple angles:
I. Object-level criticism is best
First, I don’t see evidence you’ve engaged with our work beyond very simple pattern-matching. You note that “I also think that I’m somewhat qualified to assess QRI’s work (as someone who’s spent ~100 paid hours thinking about philosophy of mind in the last few years), and when I look at it, I think it looks pretty crankish and wrong.” But *what* looks wrong? Obviously doing something new will pattern-match to crankish, regardless of whether it is crankish, so in terms of your rationale-as-stated, I don’t put too much stock in your pattern detection (and perhaps you shouldn’t either). If we want to avoid accidentally falling into (1) ‘negative-sum status attack’ interactions, and/or (2) hypercriticism of any fundamentally new thing, neither of which is good for QRI, for MIRI, or for community epistemology, object-level criticisms (and having calibrated distaste for low-information criticisms) seem pretty necessary.
Also, we do a lot more things than just philosophy, and we try to keep our assumptions about the Symmetry Theory of Valence separate from our neuroscience—STV can be wrong and our neuroscience can still be correct/useful. That said, empirically the neuroscience often does ‘lead back to’ STV.
Some things I’d offer for critique:
https://opentheory.net/2018/08/a-future-for-neuroscience/#
https://opentheory.net/2018/12/the-neuroscience-of-meditation/
https://www.qualiaresearchinstitute.org/research-lineages
(you can also watch our introductory video for context, and perhaps a ‘marker of legitimacy’, although it makes very few claims https://www.youtube.com/watch?v=HetKzjOJoy8 )
I’d also suggest that the current state of philosophy, and especially philosophy of mind and ethics, is very dismal. I give my causal reasons for this here: https://opentheory.net/2017/10/rescuing-philosophy/ - I’m not sure if you’re anchored to existing theories in philosophy of mind being reasonable or not.
II. What’s the alternative?
If there’s one piece I would suggest engaging with, it’s my post arguing against functionalism. I think your comments presuppose functionalism is reasonable and/or the only possible approach, and the efforts QRI is putting into building an alternative are certainly wasted. I strongly disagree with this; as I noted in my Facebook reply,
>Philosophically speaking, people put forth analytic functionalism as a theory of consciousness (and implicitly a theory of valence?), but I don’t think it works *qua* a theory of consciousness (or ethics or value or valence), as I lay out here: https://forum.effectivealtruism.org/.../why-i-think-the...-- This is more-or-less an answer to some of Brian Tomasik’s (very courageous) work, and to sum up my understanding I don’t think anyone has made or seems likely to make ‘near mode’ progress, e.g. especially of the sort that would be helpful for AI safety, under the assumption of analytic functionalism.
https://forum.effectivealtruism.org/posts/FfJ4rMTJAB3tnY5De/why-i-think-the-foundational-research-institute-should#6Lrwqcdx86DJ9sXmw
----------
I always find in-person interactions more amicable & high-bandwidth—I’ll be back in the Bay early December, so if you want to give this piece a careful read and sit down to discuss it I’d be glad to join you. I think it could have significant implications for some of MIRI’s work.
cf. Jeff Kaufman on MIRI circa 2003: https://www.jefftk.com/p/yudkowsky-and-miri