My latest tragic belief is that in order to improve my ability to think (so as to help others more competently) I ought to gradually isolate myself from all sources of misaligned social motivation. And that nearly all my social motivation is misaligned relative to the motivations I can (learn to) generate within myself. So I aim to extinguish all communication before the year ends (with exception for Maria).
I’m posting this comment in order to redirect some of this social motivation into the project of isolation itself. Well, that, plus I notice that part of my motivation comes from wanting to realify an interesting narrative about myself; and partly in order to publicify an excuse for why I’ve ceased (and aim to cease more) writing/communicating.
Hi Emrik, I wanted to flag the outside view is that stopping all communication with other people seems like a very bad idea. If I understand right from your link, Maria is a spirit-animal rather than another person in the usual sense of the word.
My best guess is that isolation will not improve your ability to help others, but will create a complete echo chamber that won’t be good for your wellbeing or your ability to help others.
(I think there can be some variations on this that make sense. Like I have a family member who can only write science fiction when he’s been away from people for several days, so for a long time he structured his life to be pretty isolated in order to write books. But he was still in touch with friends and family at intervals.)
It sounds like you’re in a difficult place, and I really hope you’re able to find other people you trust to help you work out how you want to approach things. [Edited a bit since I think my tone was off, thanks quila for identifying some of that]
I predict this comment would lead to, in the one it’s replying to, feeling misunderstood. This mostly comes from imagining how I would feel.
To me, this comment reads like it’s written to be convincing or agreeable or norm-affirming to other EA forum readers, but not intended to truly help Emrik.* (Note: I can’t know your inner intent, so this is only a description of how it seems to me.)
*In case it’s not clear why I would form that perception, it might be helpful for me to try to point to some examples of elements in your reply that contribute to it seeming this way to me. If so, I should first explain why pointing to examples is partially fraught, though.
When reading your reply, I didn’t update towards what I wrote at the start only upon observing each specific example. My perception comes from the text as a whole, smaller-scale examples are just the only way I know to try to communicate about this.
(Had some other caveats, but I think they simplify into the above)
Okay, the examples:
If I understand right from your link, “Maria” is a “spirit-animal”
This is technically true: Emrik’s link describes their tulpa as having a spirit-animal identity. But for an average reader, who hasn’t checked the link (and so doesn’t know this refers to a tulpa), and who probably also doesn’t know what a tulpa is, they just see that Emrik believes they can talk to a spirit animal named Maria. This probably connotes something not physically possible (e.g., an ghost-like animal-shaped entity, like those in art, by one’s side in the physical world). This could cause readers to negatively update on Emrik’s epistemics or possibly sanity.
edit: after reading Emrik’s linked post, I think this is more nuanced than what I wrote above, because in a sense it would be true to say they believe there’s a spirit being with them. They do describe intentionally believing with part of their mind that there is a Maria present, but they’re aware they’re doing this. This is described in the section, “Self-fulfilling fixed-point beliefs”. I think it was a mistake by me to imply ‘tulpa’ is somehow ontologically separate from techniques like these. The relevant distinction is instead something like, ‘partial and intentional’ vs ‘full and unintentional’ beliefs.
I wanted to flag the outside view [...]
I don’t think this would be present if the response were meant to help develop Emrik’s inside view.
If I were in Emrik’s position, I would find it most helpful for others engage with me by trying to understand why I believe what I wrote, and either discussing that, or telling me about unrelated reasons it’s bad that are applicable to my mind and that I’m probably not aware of. For the latter, it would be best if there were some back-and-forth so the other person can develop an understanding of which of their reasons could be applicable to my mind (because there is a lot of diversity between minds).
There are several aspects of this project which seem admirable to me. The general goal of helping others, and taking that seriously. Trying to protect your ability to think clearly. Acting on your own inside views, in order to better learn about the world.
Reading this, however, I feel alarmed, for a few reasons:
The plan, on its face, not seeming to hang together so well
Why are you pursuing isolation gradually? If the aim is (eventually) to help others, is it not important to keep that in sight directly, and view time out of contact with others as a cost (which may sometimes be worth paying)
Moreover, even for the learning phase I would have guessed that the thing you wanted to learn was how to think clearly in the presence of society. And that could certainly involve some taking time in isolation, but I would expect that periodic isolation and reengagement would give you better ability to train the muscles than a long period of complete isolation
A view that communication with others, while it can have costs for thinking, also has large benefits
I feel that I am smarter by having access to a large exo-brain consisting of other-people-that-I-can-consult-on-things
As well as helping me by giving me a stream of ideas I can passively consume (which wouldn’t require communication from me), they react to my ideas in ways that are helpful for me in identifying which parts are something special, and where I’m missing something
I’m sure that there are sometimes social distortions on my thinking that accompany this, but it seems to me that the benefits outweigh the costs
Moreover, there is a spectrum of ways of engaging, and if I were more paranoid about social distortions I could restrict myself to just those engagements which are most purely idea-focused, and which give minimal opportunity for social incentives, in order to get the highest benefit:cost ratio
It seems to me (noting of course that your circumstances might be different, or I might just be wrong) that the lowest-hanging fruit here will have benefits that very very clearly outweigh the costs
A worry that even if you are mistaken about this being a good course of action, it may not be self-correcting
e.g. I’m concerned that you’ve been operating in a status quo baseline X, which is not working well, for reasons. Now you’re going to move to an isolationist Y. You may observe that Y > X, and decide that you were correct to do Y, and keep on doing it—all while missing a non-isolationist Z which would have been >> Y.
I feel moved to ask whether you have a (good) therapist?
If you said that you were isolating, except for regular check-ins with a therapist, I’d feel significantly less alarmed (not zero alarm, but more of a sense that this would be a good precaution which might catch some of the times when it would otherwise fail to be self-correcting)
I imagine that a therapist would be less problematic than most communication for your ability to think, since they wouldn’t have a social agenda in the interactions
Actually I think that this whole topic pattern-matches to places where a therapist is unusually likely to be helpful-for-thinking
It’s gnarly and about one’s own internal cognition
An anchoring perspective can help to hold various poles, and to keep track of things, as well as to actively create small social motivations in precise directions that you mutually agree on
It’s unusually easy to have blind spots about one’s own cognition, and no natural self-correction mechanism unless you talk things through with someone external
Many people (including you, if my read is correct) find it socially costly and inaccessible to ask friends for help with this stuff
Even if friends did offer help, there would be concerns that they would have various social motivations, which could themselves be distortions on your thinking
Whereas a therapist should (largely) dodge these issues, by being in the role of professionally trying to help you (to do whatever things are important for you)
This might of course be wrong, but FWIW my strong recommendation would be (if you haven’t already) to try to find someone who works well for you in this role
My particular claim is that given your particular position as described here, there’s reason to think there’s a decent chance (>20%) of a very large benefit (IDK, >50% increase in your ability to self-actualize?), and this is well worth investing in as a serious experiment if you haven’t already
(to be transparent, like other people commenting my prior is that you cutting yourself off from communication seems sad and probably a bad idea, but I’m interested in hearing more about what exactly your opinions are before I argue against it)
and partly in order to publicify an excuse for why I’ve ceased (and aim to cease more) writing/communicating
I see. I’ve found our communication valuable, and it also makes me a little sad because I only have a few people to infrequently communicate with about alignment. But that would be a selfish (or ‘your-inside-view discrediting’) reason to advise against it.
I do endorse an underlying meta-strategy: I think it’s valuable for some of us—those of us who are so naturally inclined—to try some odd research/thinking-optimizing-strategy that, if it works, could be enough of a benefit to push at least that one researcher above the bar of ‘capable of making serious progress on the core problems’.
One motivating intuition: if an artificial neural network were consistently not solving some specific problem, we’d probably try to improve or change that ANN somehow or otherwise solve it with a ‘different’ one. Humans, by default, have a large measure of similarity to each other. Throwing more intelligent humans at the alignment problem may not work, if one believes it hasn’t worked so far.
In such a situation, we’d instead want to try to ‘diverge’ something like our ‘creative/generative algorithm’, in hopes that at least one (and hopefully more) of us will become something capable of making serious progress.
Social caveat: To me this is logically orthogonal, but I imagine it might be complicated for others to figure out when to be concerned for someone, and when to write it off as them doing this.
(My intuition): More ‘contextualness’ could help, e.g trying to ask some questions to assess someone’s state.
I don’t usually think about ‘what community norms should be’
Seems like a cry for help. In particular, instead of “isolating [yourself] from all sources of misaligned social motivation” you might be ″isolating yourself from all ways of realizing that you are falsifying your own preferences″.
It also seems dumb because it’s not a particularly corrigible action.
Do you have people you can reach out though? Reading through your forum posts some of the projects you have are cool. Any collaborators which you can reach out to? Or are you already pretty isolated?
My latest tragic belief is that in order to improve my ability to think (so as to help others more competently) I ought to gradually isolate myself from all sources of misaligned social motivation. And that nearly all my social motivation is misaligned relative to the motivations I can (learn to) generate within myself. So I aim to extinguish all communication before the year ends (with exception for Maria).
I’m posting this comment in order to redirect some of this social motivation into the project of isolation itself. Well, that, plus I notice that part of my motivation comes from wanting to realify an interesting narrative about myself; and partly in order to publicify an excuse for why I’ve ceased (and aim to cease more) writing/communicating.
Hi Emrik,
I wanted to flag the outside view is that stopping all communication with other people seems like a very bad idea. If I understand right from your link, Maria is a spirit-animal rather than another person in the usual sense of the word.
My best guess is that isolation will not improve your ability to help others, but will create a complete echo chamber that won’t be good for your wellbeing or your ability to help others.
(I think there can be some variations on this that make sense. Like I have a family member who can only write science fiction when he’s been away from people for several days, so for a long time he structured his life to be pretty isolated in order to write books. But he was still in touch with friends and family at intervals.)
It sounds like you’re in a difficult place, and I really hope you’re able to find other people you trust to help you work out how you want to approach things.
[Edited a bit since I think my tone was off, thanks quila for identifying some of that]
I predict this comment would lead to, in the one it’s replying to, feeling misunderstood. This mostly comes from imagining how I would feel.
To me, this comment reads like it’s written to be convincing or agreeable or norm-affirming to other EA forum readers, but not intended to truly help Emrik.*
(Note: I can’t know your inner intent, so this is only a description of how it seems to me.)
*In case it’s not clear why I would form that perception, it might be helpful for me to try to point to some examples of elements in your reply that contribute to it seeming this way to me. If so, I should first explain why pointing to examples is partially fraught, though.
When reading your reply, I didn’t update towards what I wrote at the start only upon observing each specific example. My perception comes from the text as a whole, smaller-scale examples are just the only way I know to try to communicate about this.
(Had some other caveats, but I think they simplify into the above)
Okay, the examples:
This is technically true: Emrik’s link describes their tulpa as having a spirit-animal identity. But for an average reader, who hasn’t checked the link (and so doesn’t know this refers to a tulpa), and who probably also doesn’t know what a tulpa is, they just see that Emrik believes they can talk to a spirit animal named Maria. This probably connotes something not physically possible (e.g., an ghost-like animal-shaped entity, like those in art, by one’s side in the physical world). This could cause readers to negatively update on Emrik’s epistemics or possibly sanity.
edit: after reading Emrik’s linked post, I think this is more nuanced than what I wrote above, because in a sense it would be true to say they believe there’s a spirit being with them. They do describe intentionally believing with part of their mind that there is a Maria present, but they’re aware they’re doing this. This is described in the section, “Self-fulfilling fixed-point beliefs”. I think it was a mistake by me to imply ‘tulpa’ is somehow ontologically separate from techniques like these. The relevant distinction is instead something like, ‘partial and intentional’ vs ‘full and unintentional’ beliefs.
I don’t think this would be present if the response were meant to help develop Emrik’s inside view.
If I were in Emrik’s position, I would find it most helpful for others engage with me by trying to understand why I believe what I wrote, and either discussing that, or telling me about unrelated reasons it’s bad that are applicable to my mind and that I’m probably not aware of. For the latter, it would be best if there were some back-and-forth so the other person can develop an understanding of which of their reasons could be applicable to my mind (because there is a lot of diversity between minds).
Also, you’re always welcome at the EA peer support group.
I’m glad you’re alive. I wasn’t sure what happened to you, and was worried.
Hey Emrik --
There are several aspects of this project which seem admirable to me. The general goal of helping others, and taking that seriously. Trying to protect your ability to think clearly. Acting on your own inside views, in order to better learn about the world.
Reading this, however, I feel alarmed, for a few reasons:
The plan, on its face, not seeming to hang together so well
Why are you pursuing isolation gradually? If the aim is (eventually) to help others, is it not important to keep that in sight directly, and view time out of contact with others as a cost (which may sometimes be worth paying)
Moreover, even for the learning phase I would have guessed that the thing you wanted to learn was how to think clearly in the presence of society. And that could certainly involve some taking time in isolation, but I would expect that periodic isolation and reengagement would give you better ability to train the muscles than a long period of complete isolation
A view that communication with others, while it can have costs for thinking, also has large benefits
I feel that I am smarter by having access to a large exo-brain consisting of other-people-that-I-can-consult-on-things
As well as helping me by giving me a stream of ideas I can passively consume (which wouldn’t require communication from me), they react to my ideas in ways that are helpful for me in identifying which parts are something special, and where I’m missing something
I’m sure that there are sometimes social distortions on my thinking that accompany this, but it seems to me that the benefits outweigh the costs
Moreover, there is a spectrum of ways of engaging, and if I were more paranoid about social distortions I could restrict myself to just those engagements which are most purely idea-focused, and which give minimal opportunity for social incentives, in order to get the highest benefit:cost ratio
It seems to me (noting of course that your circumstances might be different, or I might just be wrong) that the lowest-hanging fruit here will have benefits that very very clearly outweigh the costs
A worry that even if you are mistaken about this being a good course of action, it may not be self-correcting
e.g. I’m concerned that you’ve been operating in a status quo baseline X, which is not working well, for reasons. Now you’re going to move to an isolationist Y. You may observe that Y > X, and decide that you were correct to do Y, and keep on doing it—all while missing a non-isolationist Z which would have been >> Y.
I feel moved to ask whether you have a (good) therapist?
If you said that you were isolating, except for regular check-ins with a therapist, I’d feel significantly less alarmed (not zero alarm, but more of a sense that this would be a good precaution which might catch some of the times when it would otherwise fail to be self-correcting)
I imagine that a therapist would be less problematic than most communication for your ability to think, since they wouldn’t have a social agenda in the interactions
Actually I think that this whole topic pattern-matches to places where a therapist is unusually likely to be helpful-for-thinking
It’s gnarly and about one’s own internal cognition
An anchoring perspective can help to hold various poles, and to keep track of things, as well as to actively create small social motivations in precise directions that you mutually agree on
It’s unusually easy to have blind spots about one’s own cognition, and no natural self-correction mechanism unless you talk things through with someone external
Many people (including you, if my read is correct) find it socially costly and inaccessible to ask friends for help with this stuff
Even if friends did offer help, there would be concerns that they would have various social motivations, which could themselves be distortions on your thinking
Whereas a therapist should (largely) dodge these issues, by being in the role of professionally trying to help you (to do whatever things are important for you)
This might of course be wrong, but FWIW my strong recommendation would be (if you haven’t already) to try to find someone who works well for you in this role
My particular claim is that given your particular position as described here, there’s reason to think there’s a decent chance (>20%) of a very large benefit (IDK, >50% increase in your ability to self-actualize?), and this is well worth investing in as a serious experiment if you haven’t already
Obligatory link to Lynette’s post on finding a therapist
In any case, good luck with things!
I’m interested in what ‘social motivations’ means here and why you think nearly all of your social motivation is misaligned.
(to be transparent, like other people commenting my prior is that you cutting yourself off from communication seems sad and probably a bad idea, but I’m interested in hearing more about what exactly your opinions are before I argue against it)
the post they linked is pretty impressive! i’m reading it now.
#2 here touches on this
i interpret ‘purely impressing maria’ as implying ‘purely good under my inside view, which maria shares with me’
I see. I’ve found our communication valuable, and it also makes me a little sad because I only have a few people to infrequently communicate with about alignment. But that would be a selfish (or ‘your-inside-view discrediting’) reason to advise against it.
I do endorse an underlying meta-strategy: I think it’s valuable for some of us—those of us who are so naturally inclined—to try some odd research/thinking-optimizing-strategy that, if it works, could be enough of a benefit to push at least that one researcher above the bar of ‘capable of making serious progress on the core problems’.
One motivating intuition: if an artificial neural network were consistently not solving some specific problem, we’d probably try to improve or change that ANN somehow or otherwise solve it with a ‘different’ one. Humans, by default, have a large measure of similarity to each other. Throwing more intelligent humans at the alignment problem may not work, if one believes it hasn’t worked so far.
In such a situation, we’d instead want to try to ‘diverge’ something like our ‘creative/generative algorithm’, in hopes that at least one (and hopefully more) of us will become something capable of making serious progress.
Social caveat: To me this is logically orthogonal, but I imagine it might be complicated for others to figure out when to be concerned for someone, and when to write it off as them doing this.
(My intuition): More ‘contextualness’ could help, e.g trying to ask some questions to assess someone’s state.
I don’t usually think about ‘what community norms should be’
Seems like a cry for help. In particular, instead of “isolating [yourself] from all sources of misaligned social motivation” you might be ″isolating yourself from all ways of realizing that you are falsifying your own preferences″.
It also seems dumb because it’s not a particularly corrigible action.
Do you have people you can reach out though? Reading through your forum posts some of the projects you have are cool. Any collaborators which you can reach out to? Or are you already pretty isolated?