I feel pretty skeptical of their work and their judgement.
I am very unpersuaded by their Symmetry Theory of Valence, which I think is summarized by “Given a mathematical object isomorphic to the qualia of a system, the mathematical property which corresponds to how pleasant it is to be that system is that object’s symmetry“.
I think of valence as the kind of thing which is probably encoded into human brains by a bunch of complicated interconnected mechanisms rather than by something which seems simple from the perspective of an fMRI-equipped observer, so I feel very skeptical of this. Even if it was true about human brains, I’d be extremely surprised if the only possible way to build a conscious goal-directed learning system involved some kind of symmetrical property in the brain state, so this would feel like a weird contingent fact about humans rather than something general about consciousness.
And I’m skeptical of their judgement for reasons like the following. Michael Johnson, the ED of QRI, wrote:
I’ve written four pieces of philosophy I consider profound. [...]
The first, Principia Qualia, took me roughly six years of obsessive work. [...] But if there is a future science of consciousness, I’m very confident it’ll follow in the footsteps of this work. The ‘crown jewel’ of PQ is the Symmetry Theory of Valence (STV), a first-principles account of exactly how pain and pleasure work [...].
I give STV roughly an 80% chance of being true. If it is true, it will change the world, and hopefully usher in a new science and era of emotional well-being (more on that later). Pending certain research partnerships, it’s looking hopeful that we’ll either validate or falsify STV within a year, which means some very big developments could be on the horizon.
Given how skeptical I am of the theory, I feel very negatively towards someone being 80% confident of it being true and saying “if true, it will change the world”. I offered to bet with Mike Johnson about their predictions not coming true, but didn’t put in the effort to operationalize the disagreement and bet with him. If someone wanted to propose some operationalizations I’d be potentially willing to bet thousands of dollars on this; for example I’d be willing to bet $10k at even odds that STV does not “usher in a new science and era of emotional well-being”, or that the future science of consciousness doesn’t particularly follow in the footsteps of Principia Qualia.
I feel more confident in my negative assessment because I think there’s a known human failure mode where you’re interested in psychedelic drugs and consciousness and you end up making simple theories that you feel very persuaded by and which don’t seem persuasive to anyone else.
Overall I think of their work as crank philosophy and I’d love to place money on my skeptical predictions, though I also think they’re nice people and all that.
For the record, the QRI folks know much more neuroscience than I do, and it also gives me pause that Scott Alexander sometimes says nice things about their research.
Buck- for an internal counterpoint you may want to discuss QRI’s research with Vaniver. We had a good chat about what we’re doing at the Boston SSC meetup, and Romeo attended a MIRI retreat earlier in the summer and had some good conversations with him there also.
To put a bit of a point on this, I find the “crank philosophy” frame a bit questionable if you’re using only thin-slice outside view and not following what we’re doing. Probably, one could use similar heuristics to pattern-match MIRI as “crank philosophy” also (probably, many people have already done exactly this to MIRI, unfortunately).
FWIW I agree with Buck’s criticisms of the Symmetry Theory of Valence (both content and meta) and also think that some other ideas QRI are interested in are interesting. Our conversation on the road trip was (I think) my introduction to Connectome Specific Harmonic Waves (CSHW), for example, and that seemed promising to think about.
I vaguely recall us managing to operationalize a disagreement, let me see if I can reconstruct it:
A ‘multiple drive’ system, like PCT’s hierarchical control system, has an easy time explaining independent desires and different flavors of discomfort. (If one both has a ‘hunger’ control system and a ‘thirst’ control system, one can easily track whether one is hungry, thirsty, both, or neither.) A ‘single drive’ system, like expected utility theories more generally, has a somewhat more difficult time explaining independent desires and different flavors of discomfort, since you only have the one ‘utilon’ number.
But this is mostly because we’re looking at different parts of the system as the ‘value’. If I have a vector of ‘control errors’, I get the nice multidimensional property. If I have a utility function that’s a function of a vector, the gradient of that function will be a vector that gives me the same nice multidimensional property.
CSHW gives us a way to turn the brain into a graph and then the graph activations into energies in different harmonics. Then we can look at an energy distribution and figure out how consonant or dissonant it is. This gives us the potentially nice property that ‘gradients are easy’, where if ‘perfect harmony’ (= all consonant energy) corresponds to the ‘0 error’ case in PCT, being hungry will look like missing some consonant energy or having some dissonant energy.
Here we get the observational predictions: for PCT, ‘hunger’ and ‘thirst’ and whatever other drives just need to be wire voltages somewhere, but for QRI’s theory as I understand it, they need to be harmonic energies with particular numerical properties (such that they are consonant or dissonant as expected to make STV work out).
Of course, it could be the case that there are localized harmonics in the connectome, such that we get basically the same vector represented in the energy distribution, and don’t have a good way to distinguish between them.
On that note, I remember we also talked about the general difficulty of distinguishing between theories in this space; for example, my current view is that Friston-style predictive coding approaches and PCT-style hierarchical control approaches end up predicting very similar brain architecture, and the difference is ‘what seems natural’ or ‘which underlying theory gets more credit.’ (Is it the case that the brain is trying to be Bayesian, or the brain is trying to be homeostatic, and embedded Bayesianism empirically performs well at that task?) I expect a similar thing could be true here, where whether symmetry is the target or the byproduct is unclear, but in such cases I normally find myself reaching for ‘byproduct’. It’s easy to see how evolution could want to build homeostatic systems, and harder to see how evolution could want to build Bayesian systems; I think a similar story goes through for symmetry and brains.
This makes me more sympathetic to something like “symmetry will turn out to be a marker for something important and good” (like, say, ‘focus’) than something like “symmetry is definitionally what feeling good is.”
I think this is a great description. “What happens if we seek out symmetry gradients in brain networks, but STV isn’t true?” is something we’ve considered, and determining ground-truth is definitely tricky. I refer to this scenario as the “Symmetry Theory of Homeostatic Regulation”—https://opentheory.net/2017/05/why-we-seek-out-pleasure-the-symmetry-theory-of-homeostatic-regulation/ (mostly worth looking at the title image, no need to read the post)
I’m (hopefully) about a week away from releasing an update to some of the things we discussed in Boston, basically a unification of Friston/Carhart-Harris’s work on FEP/REBUS with Atasoy’s work on CSHW—will be glad to get your thoughts when it’s posted.
Oh, an additional detail that I think was part of that conversation: there’s only really one way to have a ‘0-error’ state in a hierarchical controls framework, but there are potentially many consonant energy distributions that are dissonant with each other. Whether or not that’s true, and whether each is individually positive valence, will be interesting to find out.
(If I had to guess, I would guess the different mutually-dissonant internally-consonant distributions correspond to things like ‘moods’, in a way that means they’re not really value but are somewhat close, and also that they exist. The thing that seems vaguely in this style are differing brain waves during different cycles of sleep, but I don’t know if those have clear waking analogs, or what they look like in the CSHW picture.)
I think that MIRI looks kind of crankish from the outside, and this should indeed make people initially more skeptical of us. I think that we have a few other external markers of legitimacy now, such as the fact that MIRI people were thinking and writing about AI safety from the early 2000s and many smart people have now been persuaded that this is indeed an issue to be concerned with. (It’s not totally obvious to me that these markers of legitimacy mean that anyone should take us seriously on the question “what AI safety research is promising”.) When I first ran across MIRI, I was kind of skeptical because of the signs of crankery; I updated towards them substantially because I found their arguments and ideas compelling, and people whose judgement I respected also found them compelling.
I think that the signs of crankery in QRI are somewhat worse than 2008 MIRI’s signs of crankery.
I also think that I’m somewhat qualified to assess QRI’s work (as someone who’s spent ~100 paid hours thinking about philosophy of mind in the last few years), and when I look at it, I think it looks pretty crankish and wrong.
QRI is tackling a very difficult problem, as is MIRI. It took many, many years for MIRI to gather external markers of legitimacy. My inside view is that QRI is on the path of gaining said markers; for people paying attention to what we’re doing, I think there’s enough of a vector right now to judge us positively. I think these markers will be obvious from the ‘outside view’ within a short number of years.
But even without these markers, I’d poke at your position from a couple angles:
I. Object-level criticism is best
First, I don’t see evidence you’ve engaged with our work beyond very simple pattern-matching. You note that “I also think that I’m somewhat qualified to assess QRI’s work (as someone who’s spent ~100 paid hours thinking about philosophy of mind in the last few years), and when I look at it, I think it looks pretty crankish and wrong.” But *what* looks wrong? Obviously doing something new will pattern-match to crankish, regardless of whether it is crankish, so in terms of your rationale-as-stated, I don’t put too much stock in your pattern detection (and perhaps you shouldn’t either). If we want to avoid accidentally falling into (1) ‘negative-sum status attack’ interactions, and/or (2) hypercriticism of any fundamentally new thing, neither of which is good for QRI, for MIRI, or for community epistemology, object-level criticisms (and having calibrated distaste for low-information criticisms) seem pretty necessary.
Also, we do a lot more things than just philosophy, and we try to keep our assumptions about the Symmetry Theory of Valence separate from our neuroscience—STV can be wrong and our neuroscience can still be correct/useful. That said, empirically the neuroscience often does ‘lead back to’ STV.
I’d also suggest that the current state of philosophy, and especially philosophy of mind and ethics, is very dismal. I give my causal reasons for this here: https://opentheory.net/2017/10/rescuing-philosophy/ - I’m not sure if you’re anchored to existing theories in philosophy of mind being reasonable or not.
II. What’s the alternative?
If there’s one piece I would suggest engaging with, it’s my post arguing against functionalism. I think your comments presuppose functionalism is reasonable and/or the only possible approach, and the efforts QRI is putting into building an alternative are certainly wasted. I strongly disagree with this; as I noted in my Facebook reply,
>Philosophically speaking, people put forth analytic functionalism as a theory of consciousness (and implicitly a theory of valence?), but I don’t think it works *qua* a theory of consciousness (or ethics or value or valence), as I lay out here: https://forum.effectivealtruism.org/.../why-i-think-the...-- This is more-or-less an answer to some of Brian Tomasik’s (very courageous) work, and to sum up my understanding I don’t think anyone has made or seems likely to make ‘near mode’ progress, e.g. especially of the sort that would be helpful for AI safety, under the assumption of analytic functionalism.
I always find in-person interactions more amicable & high-bandwidth—I’ll be back in the Bay early December, so if you want to give this piece a careful read and sit down to discuss it I’d be glad to join you. I think it could have significant implications for some of MIRI’s work.
For a fuller context, here is my reply to Buck’s skepticism about the 80% number during our back-and-forth on Facebook—as a specific comment, the number is loosely held, more of a conversation-starter than anything else. As a general comment I’m skeptical of publicly passing judgment on my judgment based on one offhand (and unanswered- it was not engaged with) comment on Facebook. Happy to discuss details in a context we’ll actually talk to each other. :)
--------------my reply from the Facebook thread a few weeks back--------------
I think the probability question is an interesting one—one frame is asking what is the leading alternative to STV?
At its core, STV assumes that if we have a mathematical representation of an experience, the symmetry of this object will correspond to how pleasant the experience is. The latest addition to this (what we’re calling ‘CDNS’) assumes that consonance under Selen Atasoy’s harmonic analysis of brain activity (connectome-specific harmonic waves, CSHW) is a good proxy for this in humans. This makes relatively clear predictions across all human states and could fairly easily be extended to non-human animals, including insects (anything we can infer a connectome for, and the energy distribution for the harmonics of the connectome). So generally speaking we should be able to gather a clear signal as to whether the evidence points this way or not (pending resources to gather this data- we’re on a shoestring budget).
Empirically speaking, the competition doesn’t seem very strong. As I understand it, currently the gold standard for estimating self-reports of emotional valence via fMRI uses regional activity correlations, and explains ~16% of the variance. Based on informal internal estimations looking at coherence within EEG bands during peak states, I’d expect us to do muuuuch better.
Philosophically speaking, people put forth analytic functionalism as a theory of consciousness (and implicitly a theory of valence?), but I don’t think it works *qua* a theory of consciousness (or ethics or value or valence), as I lay out here: https://forum.effectivealtruism.org/.../why-i-think-the...-- This is more-or-less an answer to some of Brian Tomasik’s (very courageous) work, and to sum up my understanding I don’t think anyone has made or seems likely to make ‘near mode’ progress, e.g. especially of the sort that would be helpful for AI safety, under the assumption of analytic functionalism.
So in short, I think STV is perhaps the only option that is well-enough laid out, philosophically and empirically, to even be tested, to even be falsifiable. That doesn’t mean it’s true, but my prior is it’s ridiculously worthwhile to try to falsify, and it seems to me a massive failure of the EA and x-risk scene that resources are not being shifted toward this sort of inquiry. The 80% I gave was perhaps a bit glib, but to dig a little, I’d say I’d give at least an 80% chance of ‘Qualia Formalism’ being true, and given that, a 95% chance of STV being true, and a 70% chance of CDNS+CSHW being a good proxy for the mathematical symmetry of human experiences.
An obvious thing we’re lacking is resources; a non-obvious thing we’re lacking is good critics. If you find me too confident I’d be glad to hear why. :)
I’m having a hard time understanding whether everything below the dotted lines is something you just wrote, or a full quote from an old thread. The first time I read it I thought the former, and on reread think the latter. Might you be able to make it more explicit at the top of your comment?
We’re pretty up-front about our empirical predictions; if critics would like to publicly bet against us we’d welcome this, as long as it doesn’t take much time away from our research. If you figure out a bet we’ll decide whether to accept it or reject it, and if we reject it we’ll aim to concisely explain why.
Mike, while I appreciate the empirical predictions of the symmetry theory of valence, I have a deeper problem with QRI philosophy, and it makes me skeptical even if the predictions come to bear.
In physics, there are two distinctions we can make about our theories:
Disputes over what we predict will happen.
Disputes over the interpretation of experimental results.
The classic Many Worlds vs. Copenhagen is a dispute of the second kind, at least until someone can create an experiment which distinguishes the two. Another example of the second type of dispute is special relativity vs. Lorentz ether theory.
Typically, philosophers of science and most people who follow Lesswrong philosophy, will say that the way to resolve disputes of the second kind is to find out which interpretation is simplest. That’s one reason why most people follow Einstein’s special relativity over the Lorentz ether theory.
However, simplicity of an interpretation is often hard to measure. It’s made more complicated for two reasons,
First, there’s no formal way of measuring simplicity even in principle in a way that is language independent.
Second, there are ontological disputes about what type of theories we are even allowing to be under consideration.
The first case is usually not a big deal because we mostly can agree on the right language to frame our theories. The second case, however, plays a deep role in why I consider QRI philosophy to be likely incorrect.
Take, for example, the old dispute over whether physics is discrete or continuous. If you apply standard Solomonoff induction, then you will axiomatically assign 0 probability to physics being continuous.
It is in this sense that QRI philosophy takes an ontological step that I consider unjustified. In particular, QRI assumes that there simply is an ontologically primitive consciousness-stuff that exists. That is, it takes it as elementary that qualia exist, and then reasons about them as if they are first class objects in our ontology.
I have already talked to you in person why I reject this line of reasoning. I think that an illusionist perspective is adequate to explain our beliefs in why we believe in consciousness, without making any reference to consciousness as an ontological primitive. Furthermore, my basic ontological assumption is that physical entities, such as electrons, have mathematical properties, but not mental properties.
The idea that electrons can have both mathematical and mental properties (ie. panpsychism) is something I consider to be little more than property dualism, and has the same known issues as every property dualist theory that I have been acquainted with.
I hope that clears some things up about why I disagree with QRI philosophy. However, I definitely wouldn’t describe you as practicing crank philosophy, as that term is both loaded, and empirically false. I know you care a lot about critical reflection, debate, and standard scientific virtues, which immediately makes you unable to be a “crank” in my opinion.
Thanks Matthew! I agree issues of epistemology and metaphysics get very sticky very quickly when speaking of consciousness.
My basic approach is ‘never argue metaphysics when you can argue physics’—the core strategy we have for ‘proving’ we can mathematically model qualia is to make better and more elegant predictions using our frameworks, with predicting pain/pleasure from fMRI data as the pilot project.
One way to frame this is that at various points in time, it was completely reasonable to be a skeptic about modeling things like lightning, static, magnetic lodestones, and such, mathematically. This is true to an extent even after Faraday and Maxwell formalized things. But over time, with more and more unusual predictions and fantastic inventions built around electromagnetic theory, it became less reasonable to be skeptical of such.
My metaphysical arguments are in my ‘Against Functionalism’ piece, and to date I don’t believe any commenters have addressed my core claims:
But, I think metaphysical arguments change distressingly few peoples’ minds. Experiments and especially technology changes peoples’ minds. So that’s what our limited optimization energy is pointed at right now.
Thanks Matthew! I agree issues of epistemology and metaphysics get very sticky very quickly when speaking of consciousness.
Agreed :).
My basic approach is ‘never argue metaphysics when you can argue physics’
My main claim was that by only arguing physics, I will never agree upon your theory because your theory assumes the existence of elementary stuff that I don’t believe in. Therefore, I don’t understand how this really helps.
Would you be prepared say the same about many worlds vs consciousness causes collapse theories? (Let’s assume that we have no experimental data which distinguishes the two theories).
One way to frame this is that at various points in time, it was completely reasonable to be a skeptic about modeling things like lightning, static, magnetic lodestones, and such, mathematically.
The problem with the analogy to magnetism and electricity is that fails to match the pattern of my argument. In order to incorporate magnetism into our mathematical theory of physics, we merely added more mathematical parts. In this, I see a fundamental difference between the approach you take and the approach taken by physicists when they admit the existence of new forces, or particles.
In particular, your theory of consciousness does not just do the equivalent of add a new force, or mathematical law that governs matter, or re-orient the geometry of the universe. It also posits that there is a dualism in physical stuff: that is, that matter can be identified as having both mathematical and mental properties.
Even if your theory did result in new predictions, I fail to see why I can’t just leave out the mental interpretation of it, and keep the mathematical bits for myself.
To put it another way, if you are saying that symmetry can be shown to be the same as valence, then I feel I can always provide an alternative explanation that leaves out valence as a first-class object in our ontology. If you are merely saying that symmetry is definitionally equivalent to valence, then your theory is vacuous because I can just delete that interpretation from my mathematical theory and emerge with equivalent predictions about the world.
And in practice, I would probably do so, because symmetry is not the kind of thing I think about when I worry about suffering.
I think metaphysical arguments change distressingly few peoples’ minds. Experiments and especially technology changes peoples’ minds. So that’s what our limited optimization energy is pointed at right now.
I agree that if you had made predictions that classical neuroscientists all agreed would never occur, and then proved them all wrong, then that would be striking evidence that I had made an error somewhere in my argument. But as it stands, I’m not convinced by your analogy to magnetism, or your strict approach towards talking about predictions rather than metaphysics.
(I may one day reply to your critique of FRI, as I see it as similarly flawed. But it is simply too long to get into right now.)
I think we actually mostly agree: QRI doesn’t ‘need’ you to believe qualia are real, that symmetry in some formalism of qualia corresponds to pleasure, that there is any formalism about qualia to be found at all. If we find some cool predictions, you can strip out any mention of qualia from them, and use them within the functionalism frame. As you say, the existence of some cool predictions won’t force you to update your metaphysics (your understanding of which things are ontologically ‘first class objects’).
But- you won’t be able to copy our generator by doing that, the thing that created those novel predictions, and I think that’s significant, and gets into questions of elegance metrics and philosophy of science.
I actually think the electromagnetism analogy is a good one: skepticism is always defensible, and in 1600, 1700, 1800, 1862, and 2018, people could be skeptical of whether there’s ‘deep unifying structure’ behind these things we call static, lightning, magnetism, shocks, and so on. But it was much more reasonable to be skeptical in 1600 than in 1862 (the year Maxwell’s Equations were published), and more reasonable in 1862 than it was in 2018 (the era of the iPhone).
Whether there is ‘deep structure’ in qualia is of course an open question in 2019. I might suggest STV is equivalent to a very early draft of Maxwell’s Equations: not a full systematization of qualia, but something that can be tested and built on in order to get there. And one that potentially ties together many disparate observations into a unified frame, and offers novel / falsifiable predictions (which seem incredibly worth trying to falsify!)
I’d definitely push back on the frame of dualism, although this might be a terminology nitpick: my preferred frame here is monism: https://opentheory.net/2019/06/taking-monism-seriously/ - and perhaps this somewhat addresses your objection that ‘QRI posits the existence of too many things’.
But- you won’t be able to copy our generator by doing that, the thing that created those novel predictions
I would think this might be our crux (other than perhaps the existence of qualia themselves). I imagine any predictions you produce can be adequately captured in a mathematical framework that makes no reference to qualia as ontologically primitive. And if I had such a framework, then I would have access to the generator, full stop. Adding qualia doesn’t make the generator any better—it just adds unnecessary mental stuff that isn’t actually doing anything for the theory.
I am not super confident in anything I said here, although that’s mostly because I have an outside view that tells me consciousness is hard to get right. My inside view tells me that I am probably correct, because I just don’t see how positing mental stuff that’s separate from mathematical law can add anything whatsoever to a physical theory.
I’m happy to talk more about this some day, perhaps in person. :)
I’m a community moderator at Metaculus and am generally interested in creating more EA-relevant questions. Are your predictions explicitly listed somewhere? It would be great to add at least some of them to the site.
Hey Pablo! I think Andres has a few up on Metaculus; I just posted QRI’s latest piece of neuroscience here, which has a bunch of predictions (though I haven’t separated them out from the text):
I think it would be worthwhile to separate these out from the text, and (especially) to generate predictions that are crisp, distinctive, and can be resolved in the near term. The QRI questions on metaculus are admirably crisp (and fairly near term), but not distinctive (they are about whether certain drugs will be licensed for certain conditions—or whether evidence will emerge supporting drug X for condition Y, which offer very limited evidence for QRI’s wider account ‘either way’).
This is somewhat more promising from your most recent post:
I’d expect to see substantially less energy in low-frequency CSHWs [Connectome-Specific Harmonic Waves] after trauma, and substantially more energy in low-frequency CSHWs during both therapeutic psychedelic use (e.g. MDMA therapy) and during psychological integration work.
This is crisp, plausibly distinctive, yet resolving this requires a lot of neuroimaging work which (presumably) won’t be conducted anytime soon. In the interim, there isn’t much to persuade a sceptical prior.
I see this and appreciate it; the problem is that I want to bet on something like “your overall theory is wrong”, but I don’t know enough neuroscience to know whether the claims you’re making are things that are probably true for reasons unrelated to your overall theory. If you could find someone who I trusted who knew neuroscience and who thought your predictions seemed unlikely, then I’d bet with them against you.
We’ve looked for someone from the community to do a solid ‘adversarial review’ of our work, but we haven’t found anyone that feels qualified to do so and that we trust to do a good job, aside from Scott, and he’s not available at this time. If anyone comes to mind do let me know!
Do you have any thoughts on Qualia Research Institute?
I feel pretty skeptical of their work and their judgement.
I am very unpersuaded by their Symmetry Theory of Valence, which I think is summarized by “Given a mathematical object isomorphic to the qualia of a system, the mathematical property which corresponds to how pleasant it is to be that system is that object’s symmetry“.
I think of valence as the kind of thing which is probably encoded into human brains by a bunch of complicated interconnected mechanisms rather than by something which seems simple from the perspective of an fMRI-equipped observer, so I feel very skeptical of this. Even if it was true about human brains, I’d be extremely surprised if the only possible way to build a conscious goal-directed learning system involved some kind of symmetrical property in the brain state, so this would feel like a weird contingent fact about humans rather than something general about consciousness.
And I’m skeptical of their judgement for reasons like the following. Michael Johnson, the ED of QRI, wrote:
Given how skeptical I am of the theory, I feel very negatively towards someone being 80% confident of it being true and saying “if true, it will change the world”. I offered to bet with Mike Johnson about their predictions not coming true, but didn’t put in the effort to operationalize the disagreement and bet with him. If someone wanted to propose some operationalizations I’d be potentially willing to bet thousands of dollars on this; for example I’d be willing to bet $10k at even odds that STV does not “usher in a new science and era of emotional well-being”, or that the future science of consciousness doesn’t particularly follow in the footsteps of Principia Qualia.
I feel more confident in my negative assessment because I think there’s a known human failure mode where you’re interested in psychedelic drugs and consciousness and you end up making simple theories that you feel very persuaded by and which don’t seem persuasive to anyone else.
Overall I think of their work as crank philosophy and I’d love to place money on my skeptical predictions, though I also think they’re nice people and all that.
For the record, the QRI folks know much more neuroscience than I do, and it also gives me pause that Scott Alexander sometimes says nice things about their research.
Buck- for an internal counterpoint you may want to discuss QRI’s research with Vaniver. We had a good chat about what we’re doing at the Boston SSC meetup, and Romeo attended a MIRI retreat earlier in the summer and had some good conversations with him there also.
To put a bit of a point on this, I find the “crank philosophy” frame a bit questionable if you’re using only thin-slice outside view and not following what we’re doing. Probably, one could use similar heuristics to pattern-match MIRI as “crank philosophy” also (probably, many people have already done exactly this to MIRI, unfortunately).
FWIW I agree with Buck’s criticisms of the Symmetry Theory of Valence (both content and meta) and also think that some other ideas QRI are interested in are interesting. Our conversation on the road trip was (I think) my introduction to Connectome Specific Harmonic Waves (CSHW), for example, and that seemed promising to think about.
I vaguely recall us managing to operationalize a disagreement, let me see if I can reconstruct it:
Of course, it could be the case that there are localized harmonics in the connectome, such that we get basically the same vector represented in the energy distribution, and don’t have a good way to distinguish between them.
On that note, I remember we also talked about the general difficulty of distinguishing between theories in this space; for example, my current view is that Friston-style predictive coding approaches and PCT-style hierarchical control approaches end up predicting very similar brain architecture, and the difference is ‘what seems natural’ or ‘which underlying theory gets more credit.’ (Is it the case that the brain is trying to be Bayesian, or the brain is trying to be homeostatic, and embedded Bayesianism empirically performs well at that task?) I expect a similar thing could be true here, where whether symmetry is the target or the byproduct is unclear, but in such cases I normally find myself reaching for ‘byproduct’. It’s easy to see how evolution could want to build homeostatic systems, and harder to see how evolution could want to build Bayesian systems; I think a similar story goes through for symmetry and brains.
This makes me more sympathetic to something like “symmetry will turn out to be a marker for something important and good” (like, say, ‘focus’) than something like “symmetry is definitionally what feeling good is.”
I think this is a great description. “What happens if we seek out symmetry gradients in brain networks, but STV isn’t true?” is something we’ve considered, and determining ground-truth is definitely tricky. I refer to this scenario as the “Symmetry Theory of Homeostatic Regulation”—https://opentheory.net/2017/05/why-we-seek-out-pleasure-the-symmetry-theory-of-homeostatic-regulation/ (mostly worth looking at the title image, no need to read the post)
I’m (hopefully) about a week away from releasing an update to some of the things we discussed in Boston, basically a unification of Friston/Carhart-Harris’s work on FEP/REBUS with Atasoy’s work on CSHW—will be glad to get your thoughts when it’s posted.
Oh, an additional detail that I think was part of that conversation: there’s only really one way to have a ‘0-error’ state in a hierarchical controls framework, but there are potentially many consonant energy distributions that are dissonant with each other. Whether or not that’s true, and whether each is individually positive valence, will be interesting to find out.
(If I had to guess, I would guess the different mutually-dissonant internally-consonant distributions correspond to things like ‘moods’, in a way that means they’re not really value but are somewhat close, and also that they exist. The thing that seems vaguely in this style are differing brain waves during different cycles of sleep, but I don’t know if those have clear waking analogs, or what they look like in the CSHW picture.)
Most things that look crankish are crankish.
I think that MIRI looks kind of crankish from the outside, and this should indeed make people initially more skeptical of us. I think that we have a few other external markers of legitimacy now, such as the fact that MIRI people were thinking and writing about AI safety from the early 2000s and many smart people have now been persuaded that this is indeed an issue to be concerned with. (It’s not totally obvious to me that these markers of legitimacy mean that anyone should take us seriously on the question “what AI safety research is promising”.) When I first ran across MIRI, I was kind of skeptical because of the signs of crankery; I updated towards them substantially because I found their arguments and ideas compelling, and people whose judgement I respected also found them compelling.
I think that the signs of crankery in QRI are somewhat worse than 2008 MIRI’s signs of crankery.
I also think that I’m somewhat qualified to assess QRI’s work (as someone who’s spent ~100 paid hours thinking about philosophy of mind in the last few years), and when I look at it, I think it looks pretty crankish and wrong.
QRI is tackling a very difficult problem, as is MIRI. It took many, many years for MIRI to gather external markers of legitimacy. My inside view is that QRI is on the path of gaining said markers; for people paying attention to what we’re doing, I think there’s enough of a vector right now to judge us positively. I think these markers will be obvious from the ‘outside view’ within a short number of years.
But even without these markers, I’d poke at your position from a couple angles:
I. Object-level criticism is best
First, I don’t see evidence you’ve engaged with our work beyond very simple pattern-matching. You note that “I also think that I’m somewhat qualified to assess QRI’s work (as someone who’s spent ~100 paid hours thinking about philosophy of mind in the last few years), and when I look at it, I think it looks pretty crankish and wrong.” But *what* looks wrong? Obviously doing something new will pattern-match to crankish, regardless of whether it is crankish, so in terms of your rationale-as-stated, I don’t put too much stock in your pattern detection (and perhaps you shouldn’t either). If we want to avoid accidentally falling into (1) ‘negative-sum status attack’ interactions, and/or (2) hypercriticism of any fundamentally new thing, neither of which is good for QRI, for MIRI, or for community epistemology, object-level criticisms (and having calibrated distaste for low-information criticisms) seem pretty necessary.
Also, we do a lot more things than just philosophy, and we try to keep our assumptions about the Symmetry Theory of Valence separate from our neuroscience—STV can be wrong and our neuroscience can still be correct/useful. That said, empirically the neuroscience often does ‘lead back to’ STV.
Some things I’d offer for critique:
https://opentheory.net/2018/08/a-future-for-neuroscience/#
https://opentheory.net/2018/12/the-neuroscience-of-meditation/
https://www.qualiaresearchinstitute.org/research-lineages
(you can also watch our introductory video for context, and perhaps a ‘marker of legitimacy’, although it makes very few claims https://www.youtube.com/watch?v=HetKzjOJoy8 )
I’d also suggest that the current state of philosophy, and especially philosophy of mind and ethics, is very dismal. I give my causal reasons for this here: https://opentheory.net/2017/10/rescuing-philosophy/ - I’m not sure if you’re anchored to existing theories in philosophy of mind being reasonable or not.
II. What’s the alternative?
If there’s one piece I would suggest engaging with, it’s my post arguing against functionalism. I think your comments presuppose functionalism is reasonable and/or the only possible approach, and the efforts QRI is putting into building an alternative are certainly wasted. I strongly disagree with this; as I noted in my Facebook reply,
>Philosophically speaking, people put forth analytic functionalism as a theory of consciousness (and implicitly a theory of valence?), but I don’t think it works *qua* a theory of consciousness (or ethics or value or valence), as I lay out here: https://forum.effectivealtruism.org/.../why-i-think-the...-- This is more-or-less an answer to some of Brian Tomasik’s (very courageous) work, and to sum up my understanding I don’t think anyone has made or seems likely to make ‘near mode’ progress, e.g. especially of the sort that would be helpful for AI safety, under the assumption of analytic functionalism.
https://forum.effectivealtruism.org/posts/FfJ4rMTJAB3tnY5De/why-i-think-the-foundational-research-institute-should#6Lrwqcdx86DJ9sXmw
----------
I always find in-person interactions more amicable & high-bandwidth—I’ll be back in the Bay early December, so if you want to give this piece a careful read and sit down to discuss it I’d be glad to join you. I think it could have significant implications for some of MIRI’s work.
cf. Jeff Kaufman on MIRI circa 2003: https://www.jefftk.com/p/yudkowsky-and-miri
For a fuller context, here is my reply to Buck’s skepticism about the 80% number during our back-and-forth on Facebook—as a specific comment, the number is loosely held, more of a conversation-starter than anything else. As a general comment I’m skeptical of publicly passing judgment on my judgment based on one offhand (and unanswered- it was not engaged with) comment on Facebook. Happy to discuss details in a context we’ll actually talk to each other. :)
--------------my reply from the Facebook thread a few weeks back--------------
I think the probability question is an interesting one—one frame is asking what is the leading alternative to STV?
At its core, STV assumes that if we have a mathematical representation of an experience, the symmetry of this object will correspond to how pleasant the experience is. The latest addition to this (what we’re calling ‘CDNS’) assumes that consonance under Selen Atasoy’s harmonic analysis of brain activity (connectome-specific harmonic waves, CSHW) is a good proxy for this in humans. This makes relatively clear predictions across all human states and could fairly easily be extended to non-human animals, including insects (anything we can infer a connectome for, and the energy distribution for the harmonics of the connectome). So generally speaking we should be able to gather a clear signal as to whether the evidence points this way or not (pending resources to gather this data- we’re on a shoestring budget).
Empirically speaking, the competition doesn’t seem very strong. As I understand it, currently the gold standard for estimating self-reports of emotional valence via fMRI uses regional activity correlations, and explains ~16% of the variance. Based on informal internal estimations looking at coherence within EEG bands during peak states, I’d expect us to do muuuuch better.
Philosophically speaking, people put forth analytic functionalism as a theory of consciousness (and implicitly a theory of valence?), but I don’t think it works *qua* a theory of consciousness (or ethics or value or valence), as I lay out here: https://forum.effectivealtruism.org/.../why-i-think-the...-- This is more-or-less an answer to some of Brian Tomasik’s (very courageous) work, and to sum up my understanding I don’t think anyone has made or seems likely to make ‘near mode’ progress, e.g. especially of the sort that would be helpful for AI safety, under the assumption of analytic functionalism.
So in short, I think STV is perhaps the only option that is well-enough laid out, philosophically and empirically, to even be tested, to even be falsifiable. That doesn’t mean it’s true, but my prior is it’s ridiculously worthwhile to try to falsify, and it seems to me a massive failure of the EA and x-risk scene that resources are not being shifted toward this sort of inquiry. The 80% I gave was perhaps a bit glib, but to dig a little, I’d say I’d give at least an 80% chance of ‘Qualia Formalism’ being true, and given that, a 95% chance of STV being true, and a 70% chance of CDNS+CSHW being a good proxy for the mathematical symmetry of human experiences.
An obvious thing we’re lacking is resources; a non-obvious thing we’re lacking is good critics. If you find me too confident I’d be glad to hear why. :)
Resources:
Principia Qualia: https://opentheory.net/PrincipiaQualia.pdf(exploratory arguments for formalism and STV laid out)
Against Functionalism: https://forum.effectivealtruism.org/.../why-i-think-the...
(an evaluation of what analytic functionalism actually gives us)
Quantifying Bliss: https://qualiacomputing.com/.../quantifying-bliss-talk.../
(Andres Gomez Emilsson’s combination of STV plus Selen Atasoy’s CSHW, which forms the new synthesis we’re working from)
A Future for Neuroscience: https://opentheory.net/2018/08/a-future-for-neuroscience/#
(more on CSHW)
Happy to chat more in-depth about details.
I’m having a hard time understanding whether everything below the dotted lines is something you just wrote, or a full quote from an old thread. The first time I read it I thought the former, and on reread think the latter. Might you be able to make it more explicit at the top of your comment?
Thanks, added.
We’re pretty up-front about our empirical predictions; if critics would like to publicly bet against us we’d welcome this, as long as it doesn’t take much time away from our research. If you figure out a bet we’ll decide whether to accept it or reject it, and if we reject it we’ll aim to concisely explain why.
Mike, while I appreciate the empirical predictions of the symmetry theory of valence, I have a deeper problem with QRI philosophy, and it makes me skeptical even if the predictions come to bear.
In physics, there are two distinctions we can make about our theories:
Disputes over what we predict will happen.
Disputes over the interpretation of experimental results.
The classic Many Worlds vs. Copenhagen is a dispute of the second kind, at least until someone can create an experiment which distinguishes the two. Another example of the second type of dispute is special relativity vs. Lorentz ether theory.
Typically, philosophers of science and most people who follow Lesswrong philosophy, will say that the way to resolve disputes of the second kind is to find out which interpretation is simplest. That’s one reason why most people follow Einstein’s special relativity over the Lorentz ether theory.
However, simplicity of an interpretation is often hard to measure. It’s made more complicated for two reasons,
First, there’s no formal way of measuring simplicity even in principle in a way that is language independent.
Second, there are ontological disputes about what type of theories we are even allowing to be under consideration.
The first case is usually not a big deal because we mostly can agree on the right language to frame our theories. The second case, however, plays a deep role in why I consider QRI philosophy to be likely incorrect.
Take, for example, the old dispute over whether physics is discrete or continuous. If you apply standard Solomonoff induction, then you will axiomatically assign 0 probability to physics being continuous.
It is in this sense that QRI philosophy takes an ontological step that I consider unjustified. In particular, QRI assumes that there simply is an ontologically primitive consciousness-stuff that exists. That is, it takes it as elementary that qualia exist, and then reasons about them as if they are first class objects in our ontology.
I have already talked to you in person why I reject this line of reasoning. I think that an illusionist perspective is adequate to explain our beliefs in why we believe in consciousness, without making any reference to consciousness as an ontological primitive. Furthermore, my basic ontological assumption is that physical entities, such as electrons, have mathematical properties, but not mental properties.
The idea that electrons can have both mathematical and mental properties (ie. panpsychism) is something I consider to be little more than property dualism, and has the same known issues as every property dualist theory that I have been acquainted with.
I hope that clears some things up about why I disagree with QRI philosophy. However, I definitely wouldn’t describe you as practicing crank philosophy, as that term is both loaded, and empirically false. I know you care a lot about critical reflection, debate, and standard scientific virtues, which immediately makes you unable to be a “crank” in my opinion.
Thanks Matthew! I agree issues of epistemology and metaphysics get very sticky very quickly when speaking of consciousness.
My basic approach is ‘never argue metaphysics when you can argue physics’—the core strategy we have for ‘proving’ we can mathematically model qualia is to make better and more elegant predictions using our frameworks, with predicting pain/pleasure from fMRI data as the pilot project.
One way to frame this is that at various points in time, it was completely reasonable to be a skeptic about modeling things like lightning, static, magnetic lodestones, and such, mathematically. This is true to an extent even after Faraday and Maxwell formalized things. But over time, with more and more unusual predictions and fantastic inventions built around electromagnetic theory, it became less reasonable to be skeptical of such.
My metaphysical arguments are in my ‘Against Functionalism’ piece, and to date I don’t believe any commenters have addressed my core claims:
https://forum.effectivealtruism.org/posts/FfJ4rMTJAB3tnY5De/why-i-think-the-foundational-research-institute-should#6Lrwqcdx86DJ9sXmw
But, I think metaphysical arguments change distressingly few peoples’ minds. Experiments and especially technology changes peoples’ minds. So that’s what our limited optimization energy is pointed at right now.
Agreed :).
My main claim was that by only arguing physics, I will never agree upon your theory because your theory assumes the existence of elementary stuff that I don’t believe in. Therefore, I don’t understand how this really helps.
Would you be prepared say the same about many worlds vs consciousness causes collapse theories? (Let’s assume that we have no experimental data which distinguishes the two theories).
The problem with the analogy to magnetism and electricity is that fails to match the pattern of my argument. In order to incorporate magnetism into our mathematical theory of physics, we merely added more mathematical parts. In this, I see a fundamental difference between the approach you take and the approach taken by physicists when they admit the existence of new forces, or particles.
In particular, your theory of consciousness does not just do the equivalent of add a new force, or mathematical law that governs matter, or re-orient the geometry of the universe. It also posits that there is a dualism in physical stuff: that is, that matter can be identified as having both mathematical and mental properties.
Even if your theory did result in new predictions, I fail to see why I can’t just leave out the mental interpretation of it, and keep the mathematical bits for myself.
To put it another way, if you are saying that symmetry can be shown to be the same as valence, then I feel I can always provide an alternative explanation that leaves out valence as a first-class object in our ontology. If you are merely saying that symmetry is definitionally equivalent to valence, then your theory is vacuous because I can just delete that interpretation from my mathematical theory and emerge with equivalent predictions about the world.
And in practice, I would probably do so, because symmetry is not the kind of thing I think about when I worry about suffering.
I agree that if you had made predictions that classical neuroscientists all agreed would never occur, and then proved them all wrong, then that would be striking evidence that I had made an error somewhere in my argument. But as it stands, I’m not convinced by your analogy to magnetism, or your strict approach towards talking about predictions rather than metaphysics.
(I may one day reply to your critique of FRI, as I see it as similarly flawed. But it is simply too long to get into right now.)
I think we actually mostly agree: QRI doesn’t ‘need’ you to believe qualia are real, that symmetry in some formalism of qualia corresponds to pleasure, that there is any formalism about qualia to be found at all. If we find some cool predictions, you can strip out any mention of qualia from them, and use them within the functionalism frame. As you say, the existence of some cool predictions won’t force you to update your metaphysics (your understanding of which things are ontologically ‘first class objects’).
But- you won’t be able to copy our generator by doing that, the thing that created those novel predictions, and I think that’s significant, and gets into questions of elegance metrics and philosophy of science.
I actually think the electromagnetism analogy is a good one: skepticism is always defensible, and in 1600, 1700, 1800, 1862, and 2018, people could be skeptical of whether there’s ‘deep unifying structure’ behind these things we call static, lightning, magnetism, shocks, and so on. But it was much more reasonable to be skeptical in 1600 than in 1862 (the year Maxwell’s Equations were published), and more reasonable in 1862 than it was in 2018 (the era of the iPhone).
Whether there is ‘deep structure’ in qualia is of course an open question in 2019. I might suggest STV is equivalent to a very early draft of Maxwell’s Equations: not a full systematization of qualia, but something that can be tested and built on in order to get there. And one that potentially ties together many disparate observations into a unified frame, and offers novel / falsifiable predictions (which seem incredibly worth trying to falsify!)
I’d definitely push back on the frame of dualism, although this might be a terminology nitpick: my preferred frame here is monism: https://opentheory.net/2019/06/taking-monism-seriously/ - and perhaps this somewhat addresses your objection that ‘QRI posits the existence of too many things’.
I would think this might be our crux (other than perhaps the existence of qualia themselves). I imagine any predictions you produce can be adequately captured in a mathematical framework that makes no reference to qualia as ontologically primitive. And if I had such a framework, then I would have access to the generator, full stop. Adding qualia doesn’t make the generator any better—it just adds unnecessary mental stuff that isn’t actually doing anything for the theory.
I am not super confident in anything I said here, although that’s mostly because I have an outside view that tells me consciousness is hard to get right. My inside view tells me that I am probably correct, because I just don’t see how positing mental stuff that’s separate from mathematical law can add anything whatsoever to a physical theory.
I’m happy to talk more about this some day, perhaps in person. :)
Hey Mike,
I’m a community moderator at Metaculus and am generally interested in creating more EA-relevant questions. Are your predictions explicitly listed somewhere? It would be great to add at least some of them to the site.
Hey Pablo! I think Andres has a few up on Metaculus; I just posted QRI’s latest piece of neuroscience here, which has a bunch of predictions (though I haven’t separated them out from the text):
https://opentheory.net/2019/11/neural-annealing-toward-a-neural-theory-of-everything/
I think it would be worthwhile to separate these out from the text, and (especially) to generate predictions that are crisp, distinctive, and can be resolved in the near term. The QRI questions on metaculus are admirably crisp (and fairly near term), but not distinctive (they are about whether certain drugs will be licensed for certain conditions—or whether evidence will emerge supporting drug X for condition Y, which offer very limited evidence for QRI’s wider account ‘either way’).
This is somewhat more promising from your most recent post:
This is crisp, plausibly distinctive, yet resolving this requires a lot of neuroimaging work which (presumably) won’t be conducted anytime soon. In the interim, there isn’t much to persuade a sceptical prior.
I see this and appreciate it; the problem is that I want to bet on something like “your overall theory is wrong”, but I don’t know enough neuroscience to know whether the claims you’re making are things that are probably true for reasons unrelated to your overall theory. If you could find someone who I trusted who knew neuroscience and who thought your predictions seemed unlikely, then I’d bet with them against you.
We’ve looked for someone from the community to do a solid ‘adversarial review’ of our work, but we haven’t found anyone that feels qualified to do so and that we trust to do a good job, aside from Scott, and he’s not available at this time. If anyone comes to mind do let me know!
See also this recent Qualia Computing post about the orthogonality thesis. (Qualia Computing is the blog of QRI’s research director.)