MikeJohnson

Karma: 637

MikeJohnson Nov 16, 2018, 2:07 AM
8 points
0 ∶ 0
on: Wireheading as a Possible Contributor to Civilizational Decline
Hi Alexey,
Glad to see the thought that went into this post. I’m in agreement that this is a real danger and potentially an x-risk. I’d recommend the following links from QRI:
- A review of what affective neuroscience knows about valence—essentially, affective neuroscience is very confused about what pleasure is, and thus is confused about how to talk about wireheading.
- Wireheading Done Right—there are better and worse ways to do hedonic recalibration. Andres describes a scenario that might be the best we can hope for.
- A Future for Neuroscience—one way to cut through the confusion plaguing affective neuroscience (and thus avoid some ‘dirty wireheading’ scenarios) is to find a good algorithm-level description of pain and pleasure. I think QRI has found one.
- An interview with Adam Ford I did on the topic—here’s the relevant excerpt:
A device that could temporarily cause extreme positive or negative valence on demand would immediately change the world.
First, it would validate valence realism in a very visceral way. I’d say it would be the strongest philosophical argument ever made.
Second, it would obviously have huge economic & ethical uses.
Third, I agree that being able to induce strong positive & negative valence on demand could help align different schools of utilitarianism. Nothing would focus philosophical arguments about the discount rate between pleasure & suffering more than a (consensual!) quick blast of pure suffering followed by a quick blast of pure pleasure. Similarly, a lot of people live their lives in a rather numb state. Giving them a visceral sense that ‘life can be more than this’ could give them ‘skin in the game’.
Fourth, it could mess a lot of things up. Obviously, being able to cause extreme suffering could be abused, but being able to cause extreme pleasure on-demand could lead to bad outcomes too. You (Andres) have written about wireheading before, and I agree with the game-theoretic concerns involved. I would also say that being able to cause extreme pleasure in others could be used in adversarial ways. More generally, human culture is valuable and fragile; things that could substantially disrupt it should be approached carefully.
A friend of mine was describing how in the 70s, the emerging field of genetic engineering held the Asilomar Conference on Recombinant DNA to discuss how the field should self-regulate. The next year, these guidelines were adopted by the NIH wholesale as the basis for binding regulation, and other fields (such as AI safety!) have attempted to follow the same model. So the culture around technologies may reflect a strong “founder effect”, and we should be on the lookout for a good, forward-looking set of principles for how valence technology should work.
One principle that seems to make sense is to not publicly post ‘actionable’ equations, pseudocode, or code for how one could generate suffering with current computing resources (if this is indeed possible). Another principle is to focus resources on positive, eusocial applications only, insofar as that’s possible– I’m especially concerned about addiction, and bad actors ‘weaponizing’ this sort of research. Another would be to be on guard against entryism, or people who want to co-opt valence research for political ends.
All of this is pretty straightforward, but it would be good to work it out a bit more formally, look at the successes and failures of other research communities, and so on.
A question I find very interesting is whether valence research is socially disruptive or socially stabilizing by default. I think we should try very hard to make it a socially stabilizing force. One way to think about this is in terms of existential risk. It’s a little weird to say, but I think the fact that so many people are jaded, or feel hopeless, is a big existential risk, because they feel like they have very little to lose. So they don’t really care what happens to the world, because they don’t have good qualia to look forward to, no real ‘skin in the game’. If valence tech could give people a visceral, ‘felt sense’ of wonder and possibility, I think the world could become a much safer place, because more people would viscerally care about AI safety, avoiding nuclear war, and so on.
Finally, one thing that I think doesn’t make much sense is handing off the ethical issues to professional bioethicists and expecting them to be able to help much. Speaking as a philosopher, I don’t think bioethics itself has healthy community & research norms (maybe bioethics needs some bioethicsethicists…). And in general, I think especially when issues are particularly complex or technical, I think the best type of research norms comes from within a community.
Generally speaking, I think we should be taking this topic a lot more seriously, and thinking about specific plans for how to make hedonic recalibration happen in pro-social, pro-human ways, rather than the (default?) cyberpunk dystopia route.

MikeJohnson Sep 18, 2018, 8:00 PM
2 points
0 ∶ 0
on: Is it better to be a wild rat or a factory farmed cow? A systematic method for comparing animal welfare.
Glad to see work on this.

It seems to me there are two questions here: (1) what are the average effects of different environments (e.g. wilderness; factory farm) on animal well-being? (2) what is the average hedonic well-being of different species?

It feels like you’re attempting to find a method that will give the combined score for any given animal. But maybe it’d be best to focus on each individually. Some of the methods you mentioned (e.g. cortisol levels, behavior anomalies, self-narcotization) seem fairly solid for addressing (1), if you had more data. What’s the biggest hurdle to gathering more data? Can you think of any clever ways to gather lots of data cheaply? Basically it seems really useful to try to build an intra-species hedonic comparison first, and worry about inter-species comparisons later.

That said—on inter-species comparisons, I don’t think any of the methods you mention are likely to give a good answer to (2), especially as none deal directly with brain activity. It’s possible (although I don’t know for sure) that some of QRI’s work is relevant here- essentially, we have a method (‘CDNS’) that could be adapted to estimate the degree to which a given connectome is naturally ‘tuned’ toward harmony or dissonance. This would face many of the same data & validation challenges you mention for other proxy measures, but essentially I’m skeptical that it’s possible to address (2) without something like what QRI is doing, that actually looks at brain activity and doesn’t rely on hard-coded assumptions about things that could be species-specific and are probably leaky anyway (e.g., brain region X is associated with pain).

If it checks out, this could give a rough inter-species comparison of natural hedonic set-points between literally any two connectomes—cows, chickens, rats, grasshoppers, mosquitos, humans. Probably not an end-all-be-all, but a useful tool in the toolbox. More on our ‘CDNS’ method.

MikeJohnson May 23, 2018, 2:49 PM
21 points
1 ∶ 0
on: Ineffective entrepreneurship: post-mortem of Hippo, the happiness app that never quite was
I admire many things about this story, not least being your willingness to try & fail, and your brutal honesty in sharing what lessons you learned (or didn’t learn).

There are many bad things about silicon valley, but one thing I think it gets right is giving partial credit for failed moonshots. Thank you for Really Trying to make this happen.

MikeJohnson May 23, 2018, 2:38 PM
3 points
0 ∶ 0
on: 1. What Is Moral Realism?
Thanks for putting this out there. I like how you list the two versions of moral realism you find coherent, and especially that you list what would convince you of each.

My intuition here is the first option is the case, but also that instead of speaking about moral realism we should talk about qualia formalism. I.e., whether consciousness is real enough such that it can be spoken about in crisp formal terms, seems prior to whether morality is real in that same sense. I’ve written about this here, and spoke about this in the intro of my TSC2018 talk.

Whether qualia formalism is true seems an empirical question; if it is, we should be able to make novel and falsifiable predictions with it. This seems like a third option for moving forward, in addition to your other two.

MikeJohnson Mar 2, 2018, 5:15 AM
0 points
0 ∶ 0
in reply to: itaibn’s comment on: Why I prioritize moral circle expansion over artificial intelligence alignment
EA forum threads auto-hide so I’m not too worried about clutter.

I don’t think you’re fully accounting for the difference in my two models of meaning. And, I think the objections you raise to consciousness being well-defined would also apply to physics being well-defined, so your arguments seem to prove too much.

To attempt to address your specific question, I find the hypothesis that ‘qualia (and emotional valence) are well-defined across all arrangements of matter’ convincing because (1) it seems to me the alternative is not coherent (as I noted in the piece on computationalism I linked for you) and (2) it seems generative and to lead to novel and plausible predictions I think will be proven true (as noted in the linked piece on quantifying bliss and also in Principia Qualia).

All the details and sub arguments can be found in those links.

Will be traveling until Tuesday; probably with spotty internet access until then.

MikeJohnson Feb 28, 2018, 6:05 PM
0 points
0 ∶ 0
in reply to: itaibn’s comment on: Why I prioritize moral circle expansion over artificial intelligence alignment
This is an important point and seems to hinge on the notion of reference, or the question of how language works in different contexts. The following may or may not be new to you, but trying to be explicit here helps me think through the argument.

Mostly, words gain meaning from contextual embedding- i.e. they’re meaningful as nodes in a larger network. Wittgenstein observed that often, philosophical confusion stems from taking a perfectly good word and trying to use it outside its natural remit. His famous example is the question, “what time is it on the sun?”. As you note, maybe notions about emotional valence are similar- trying to ‘universalize’ valence may be like trying to universalize time-zones, an improper move.

But there’s another notable theory of meaning, where parts of language gain meaning through deep structural correspondence with reality. Much of physics fits this description, for instance, and it’s not a type error to universalize the notion of the electromagnetic force (or electroweak force, or whatever the fundamental unification turns out to be). I am essentially asserting that qualia is like this- that we can find universal principles for qualia that are equally and exactly true in humans, dogs, dinosaurs, aliens, conscious AIs, etc. When I note I’m a physicalist, I intend to inherit many of the semantic properties of physics, how meaning in physics ‘works’.

I suspect all conscious experiences have an emotional valence, in much the same way all particles have a charge or spin. I.e. it’s well-defined across all physical possibilities.

MikeJohnson Feb 26, 2018, 3:35 AM
1 point
0 ∶ 0
in reply to: itaibn’s comment on: Why I prioritize moral circle expansion over artificial intelligence alignment
Thanks, this is helpful. My general position on your two questions is indeed “Yes/No”.

The question of ‘what are reality’s natural kinds?’ is admittedly complex and there’s always room for skepticism. That said, I’d suggest the following alternatives to your framing:
- Whether the existence of qualia itself is ‘crisp’ seems prior to whether pain/pleasure are. I call this the ‘real problem’ of consciousness.
- I’m generally a little uneasy with discussing pain/pleasure in technically precise contexts- I prefer ‘emotional valence’.
- Another reframe to consider is to disregard talk about pain/pleasure, and instead focus on whether value is well-defined on physical systems (i.e. the subject of Tegmark’s worry here). Conflation of emotional valence & moral value can then be split off as a subargument.
Generally speaking, I think if one accepts that it’s possible in principle to talk about qualia in a way that ‘carves reality at the joints’, it’s not much of a stretch to assume that emotional valence is one such natural kind (arguably the ‘c. elegans of qualia’). I don’t think we’re logically forced to assume this, but I think it’s prima facie plausible, and paired with some of our other work it gives us a handhold for approaching qualia in a scientific/predictive/falsifiable way.

Essentially, QRI has used this approach to bootstrap the world’s first method for quantifying emotional valence in humans from first principles, based on fMRI scans. (It also should work for most non-human animals; it’s just harder to validate in that case.) We haven’t yet done the legwork on connecting future empirical results here back to the computationalism vs physicalism debate, but it’s on our list.

TL;DR: If consciousness is a ‘crisp’ thing with discoverable structure, we should be able to build/predict useful things with this that cannot be built/predicted otherwise, similar to how discovering the structure of electromagnetism let us build/predict useful things we could not have otherwise. This is probably the best route to solve these metaphysical disagreements.

MikeJohnson Feb 25, 2018, 8:46 PM
0 points
0 ∶ 0
in reply to: itaibn’s comment on: Why I prioritize moral circle expansion over artificial intelligence alignment
It seems to me your #2 and #4 still imply computationalism and/or are speaking about a straw man version of physicalism. Different physical theories will address your CPT reversal objection differently, but it seems pretty trivial to me.

If I understood you correctly, physicalism as a statement about consciousness is primary a negative statement, “the computational behavior of a system is not sufficient to determine what sort of conscious activity occurs there”, which doesn’t by itself tell you what sort of conscious activity occurs.

I would generally agree, but would personally phrase this differently; rather, as noted here, there is no objective fact-of-the-matter as to what the ‘computational behavior’ of a system is. I.e., no way to objectively derive what computations a physical system is performing. In terms of a positive statement about physicalism & qualia, I’m assuming something on the order of dual-aspect monism / neutral monism. And yes insofar as a formal theory of consciousness which has broad predictive power would depart from folk intuition, I’d definitely go with the formal theory.

MikeJohnson Feb 25, 2018, 5:30 AM
1 point
0 ∶ 0
in reply to: itaibn’s comment on: Why I prioritize moral circle expansion over artificial intelligence alignment
Possibly the biggest unknown in ethics is whether bits matter, or whether atoms matter.

If you assume bits matter, then I think this naturally leads into a concept cluster where speaking about utility functions, preference satisfaction, complexity of value, etc, makes sense. You also get a lot of weird unresolved thought-experiments like homomorphic encryption.

If you assume atoms matter, I think this subtly but unavoidably leads to a very different concept cluster—qualia turns out to be a natural kind instead of a leaky reification, for instance. Talking about the ‘unity of value thesis’ makes more sense than talking about the ‘complexity of value thesis’.

TL;DR: I think you’re right that if we assume computationalism/functionalism is true, then pleasure and suffering are inherently ill-defined, not crisp. They do seem well-definable if we assume physicalism is true, though.

MikeJohnson Feb 22, 2018, 2:56 AM
6 points
1 ∶ 0
in reply to: Daniel_Eth’s comment on: Why I prioritize moral circle expansion over artificial intelligence alignment

Second, I’d imagine that a mature science of consciousness would increase MCE significantly. Many people don’t think animals are conscious, and almost no one thinks anything besides animals can be conscious. How would we even know if an AI was conscious, and if so, if it was experiencing joy or suffering? The only way would be if we develop theories of consciousness that we have high confidence in. But right now we’re very limited in studying consciousness, because our tools at interfacing with the brain are crude. Advanced neurotechnologies could change that—they could allow us to potentially test hypotheses about consciousness. Again, developing these technologies would be a technical problem.

I think that’s right. Specifically, I would advocate consciousness research as a foundation for principled moral circle expansion. I.e., if we do consciousness research correctly, the equations themselves will tell us how conscious insects are, whether algorithms can suffer, how much moral weight we should give animals, and so on.

On the other hand, if there is no fact of the matter as to what is conscious, we’re headed toward a very weird, very contentious future of conflicting/incompatible moral circles, with no ‘ground truth’ or shared principles to arbitrate disputes.

Edit: I’d also like to thank Jacy for posting this- I find it a notable contribution to the space, and clearly a product of a lot of hard work and deep thought.

MikeJohnson Feb 18, 2018, 7:54 PM
3 points
0 ∶ 0
on: On funding medical research
Thanks for this writeup. I found it thoughtful and compelling.

My understanding is that ME is real, serious, and understudied/underfunded. Perhaps the core reason it’s understudied is that it’s unclear where to start, physiologically speaking—there’s a lot of ontological uncertainty in terms of what this thing is that sometimes cripples people.

This is sometimes solvable by throwing resources at the problem, and sometimes not.

It might be helpful to survey some other diseases that followed a similar trajectory (mysterious crippling conditions that later resolved into known diseases with known causes and known treatments) and see if there are any general lessons to learn. My expectation here is that often, what makes a mystery disease ‘make sense’ is a new method that gives a novel/fresh window into physiology. Celiac disease could be an interesting case study: it was hugely mysterious (and hugely underdiagnosed) until (1) we got a decent IgG screen, and (2) we started to understand how gut permeability works.

I’d also suggest that you may be a little too cynical about alternative medicine; there’s a huge amount of snake-oil there, but alternative medicine is also highly heterogeneous, exploring a lot of the possibility space. There will be a lot of bs, but there often are some pearls as well. Mainstream medicine is also not particularly known for immediately finding these pearls and synthesizing them back into the medical literature, so I think it’s also plausible that a viable way to make progress on this problem is to survey what alt-med thinks it knows about ME, filter the bs out, and see if there’s anything left that can help mainstream medicine understand what ME is and what general class of treatments might help.

MikeJohnson Feb 18, 2018, 7:57 AM
2 points
0 ∶ 0
on: Lessons for Building Up a Cause
This is really great info—I’ll probably blog about this on the QRI website at some point. (Thanks!)

Another interesting resource along these lines is Luke M’s set of case studies about early field growth: I particularly enjoyed his notes on how bioethicists shaped medicine. https://www.openphilanthropy.org/research/history-of-philanthropy/some-case-studies-early-field-growth

MikeJohnson Jan 20, 2018, 1:02 AM
4 points
0 ∶ 0
on: “Just take the expected value” – a possible reply to concerns about cluelessness
I’m late to the party, but I’ve really enjoyed this series of posts. Thanks for writing.

MikeJohnson Jan 17, 2018, 9:58 PM
2 points
0 ∶ 0
in reply to: Brian_Tomasik’s comment on: Why I think the Foundational Research Institute should rethink its approach
Aaronson’s “Is ‘information is physical’ contentful?” also seems relevant to this discussion (though I’m not sure exactly how to apply his arguments):

But we should’ve learned by now to doubt this sort of argument. There’s no general principle, in our universe, saying that you can hide as many bits as you want in a physical object, without those bits influencing the object’s observable properties. On the contrary, in case after case, our laws of physics seem to be intolerant of “wallflower bits,” which hide in a corner without talking to anyone. If a bit is there, the laws of physics want it to affect other nearby bits and be affected by them in turn. … In summary, our laws of physics are structured in such a way that even pure information often has “nowhere to hide”: if the bits are there at all in the abstract machinery of the world, then they’re forced to pipe up and have a measurable effect. And this is not a tautology, but comes about only because of nontrivial facts about special and general relativity, quantum mechanics, quantum field theory, and thermodynamics. And this is what I think people should mean when they say “information is physical.”

https://www.scottaaronson.com/blog/?p=3327

MikeJohnson Nov 20, 2017, 9:42 PM
1 point
0 ∶ 0
in reply to: RomeoStevens’s comment on: Mental Health Shallow Review
I would definitely endorse these.

MikeJohnson Nov 20, 2017, 8:46 PM
6 points
0 ∶ 0
on: Mental Health Shallow Review
This is a wonderful overview. I especially appreciated the notes about possible biases in each study.

My expectation is that the “mental health tech” field is also worth keeping an eye on, although it’s often characterized by big claims and not a lot of supporting data. I’m cautiously optimistic that an app like UpLift (Spencer Greenberg et. al) might be able to improve upon existing self-administered CBT options.

There have also been a lot of promising developments in neuroscience and ‘applied philosophy of mind’, and if there are ways of turning these into technology, it seems plausible we could start to see some “10x results”. Better ways to understand what’s going on in brains will lead to better tools to fix them when they break.

The two paradigms I find most intriguing here are
- the predictive coding / free energy paradigm (primary work by Karl Friston, Anil K. Seth, Andy Clark; for a nice summary see SSC’s book review of Surfing Uncertainty and ‘toward a predictive theory of depression’ - also, Adam Safron is an EA who really knows his stuff here, and would be a good person to talk to about how predictive coding models could help inform mental health interventions)
- the connectome-specific harmonic wave paradigm (primary work by Selen Atasoy; for a nice summary see this video&transcript—this has informed much of QRI’s thinking about mental health)
I’d also love to survey other peoples’ intuitions on what neuroscience work they think could lead to a ’10x breakthrough’ in mental health tech.

MikeJohnson Nov 4, 2017, 8:22 PM
0 points
0 ∶ 0
in reply to: Gregory Lewis🔸’s comment on: Anti-tribalism and positive mental health as high-value cause areas

I prefer to keep discussion on the object level

I’m not seeing object-level arguments against mental health as an EA cause area. We have made some object-level arguments for, and I’m working on a longer-form description of what QRI plans in this space. Look for more object-level work and meta-level organizing over the coming months.

I’d welcome object-level feedback on our approaches. It didn’t seem like your comments above were feedback-focused, but rather they seemed motivated by a belief that this was not “a good direction for EA energy to go relative to the other major ones.” I can’t rule that out at this point. But I don’t like seeing a community member just dishing out relatively content-free dismissiveness on people at a relatively early stage in trying to build something new. If you don’t see any good interventions here, and don’t think we’ll figure out any good interventions, it seems much better to just let us fail, rather than actively try to pour cold water on us. If we’re on the verge of using lots of community resources on something that you know to be unworkable, please pour the cold water. But if your argument boils down to “this seems like a bad idea, but I can’t give any object-level reasons, but I really want people to know I think this is a bad idea” then I’m not sure what value this interaction can produce.

But, that said, I’d also like to apologize if I’ve come on too strong in this back-and-forth, or if you feel I’ve maligned your motives. I think you seem smart, honest, invested in doing good as you see it, and are obviously willing to speak your mind. I would love to channel this into making our ideas better! In trying to do something new, there’s approximately a 100% chance we’ll make a lot of mistakes. I’d like to enlist your help in figuring out where the mistakes are and better alternatives. Or, if you’d rather preemptively write off mental health as a cause area, that’s your prerogative. But we’re in this tent together, and although all the evidence I have suggests we have significantly different (perhaps downright dissonant) cognitive styles, perhaps we can still find some moral trade.

Best wishes, Mike

MikeJohnson Nov 2, 2017, 9:42 PM
5 points
1 ∶ 0
on: Multiverse-wide cooperation in a nutshell
This is a very clear description of some cool ideas. Thanks to you and Caspar for doing this!

MikeJohnson Oct 28, 2017, 3:02 PM
1 point
0 ∶ 0
in reply to: Gregory Lewis🔸’s comment on: Anti-tribalism and positive mental health as high-value cause areas
Hi Gregory,

We have never interacted before this, at least to my knowledge, and I worry that you may be bringing some external baggage into this interaction (perhaps some poor experience with some cryonics enthusiast...). I find your “let’s shut this down before it competes for resources” attitude very puzzling and aggressive, especially since you show zero evidence that you understand what I’m actually attempting to do or gather support for on the object-level. Very possibly we’d disagree on that too, which is fine, but I’m reading your responses as preemptively closed and uncharitable (perhaps veering toward ‘aggressively hostile’) toward anything that might ‘rock the EA boat’ as you see it.

I don’t think this is good for EA, and I don’t think it’s working off a reasonable model of the expected value of a new cause area. I.e., you seem to be implying the expected cause area would be at best zero, but more probably negative, due to zero-sum dynamics. On the other hand, I think a successful new cause area would more realistically draw in or internally generate at least as many resources as it would consume, and probably much more—my intuition is that at the upper bound we may be looking at something as synergistic as a factorial relationship (with three causes, the total ‘EA pie’ might be 321=6; with four causes the total ‘EA pie’ might be 432*1=24). More realistically, perhaps 4+3+2+1 instead of 3+2+1. This could be and probably is very wrong—but at the same time I think it’s more accurate than a zero-sum model.

At any rate, I’m skeptical that we can turn this discussion into something that will generate value to either of us or to EA, so unless you have any specific things you’d like to discuss or clarify, I’m going to leave things here. Feel free to PM me questions.

MikeJohnson Oct 27, 2017, 11:03 PM
1 point
0 ∶ 0
in reply to: Gregory Lewis🔸’s comment on: Anti-tribalism and positive mental health as high-value cause areas
I worry that you’re also using a fully-general argument here, one that would also apply to established EA cause areas.

This stands out at me in particular:

Naturally I don’t mind if enthusiasts pick some area and give it a go, but appeals to make it a ‘new cause area’ based on these speculative bets look premature by my lights: better to pick winners based on which of the disparate fields shows the greatest progress, such that one forecasts similar marginal returns to the ‘big three’.

There’s a lot here that I’d challenge. E.g., (1) I think you’re implicitly overstating how good the marginal returns on the ‘big three’ actually are, (2) you seem to be doubling down on the notion that “saving lives is better than improving lives” or that “the current calculus of EA does and should lean toward reduction of mortality, not improving well-being”, which I challenged above, (3) I don’t think your analogy between cryonics (which, for the record, I’m skeptical on as an EA cause area) and e.g., Enthea’s collation of research on psilocybin seems very solid.

I would also push back on how dismissive “Naturally I don’t mind if enthusiasts pick some area and give it a go, but appeals to make it a ‘new cause area’ based on these speculative bets look premature by my lights” sounds. Enthusiasts are the ones that create new cause areas. We wouldn’t have any cause areas, save for those ‘silly enthusiasts’. Perhaps I’m misreading your intended tone, however.