Kaj_Sotala

• Kaj_Sotala 7 Jan 2024 9:01 UTC

  7 points

  0 ∶ 0

  in reply to: Mitchell Laughlin’s comment on: Effec­tive al­tru­ism as the most ex­cit­ing cause in the world

  Thanks for noticing that! These days I would always use “their” in this kind of context, but I guess I didn’t yet have that habit back in 2014. Edited.

Most peo­ple should prob­a­bly feel safe most of the time

• Kaj_Sotala 25 Apr 2023 17:03 UTC

  10 points

  0 ∶ 0

  on: EA might sys­tem­at­i­cally gen­er­ate a scarcity mind­set that pro­duces low-in­tegrity actors

  (Upvoted.)

  Some of the attempts within EA to solve this seem to be to push even more towards just being a professional network. I think that’s dangerously wrong, because it doesn’t remove the informal networks and their power. It just makes access to them harder, and people more desperate to get in.

  Somewhat relevant counterpoint:

  For everyone to have the opportunity to be involved in a given group and to participate in its activities the structure must be explicit, not implicit. The rules of decision-making must be open and available to everyone, and this can happen only if they are formalized. This is not to say that formalization of a structure of a group will destroy the informal structure. It usually doesn’t. But it does hinder the informal structure from having predominant control and make available some means of attacking it if the people involved are not at least responsible to the needs of the group at large. [...]

  … an elite refers to a small group of people who have power over a larger group of which they are part, usually without direct responsibility to that larger group, and often without their knowledge or consent. [...] Elites are nothing more, and nothing less, than groups of friends who also happen to participate in the same political activities. They would probably maintain their friendship whether or not they were involved in political activities; they would probably be involved in political activities whether or not they maintained their friendships. It is the coincidence of these two phenomena which creates elites in any group and makes them so difficult to break.

  These friendship groups function as networks of communication outside any regular channels for such communication that may have been set up by a group. If no channels are set up, they function as the only networks of communication. [...]

  Some groups, depending on their size, may have more than one such informal communications network. [...] In a Structured group, two or more such friendship networks usually compete with each other for formal power. This is often the healthiest situation, as the other members are in a position to arbitrate between the two competitors for power and thus to make demands on those to whom they give their temporary allegiance.

  What links here?

  • Severin's comment on EA might sys­tem­at­i­cally gen­er­ate a scarcity mind­set that pro­duces low-in­tegrity actors by Severin (25 Apr 2023 22:17 UTC; 1 point)

Here, have a calm­ness video

• Kaj_Sotala 23 Nov 2022 20:29 UTC

  3 points

  0 ∶ 0

  in reply to: WilliamKiely’s comment on: Meta AI an­nounces Cicero: Hu­man-Level Di­plo­macy play (with di­alogue)

  gwern on /​r/​machinelearning:

  There’s no comparison to prior full-press Diplomacy agents, but if I’m reading the prior-work cites right, this is because basically none of them work—not only do they not beat humans, they apparently don’t even always improve over themselves playing the game as if it was no-press Diplomacy (ie not using dialogue at all). That gives an idea how big a jump this is for full-press Diplomacy.

• Kaj_Sotala 22 Nov 2022 19:15 UTC

  6 points

  0 ∶ 0

  in reply to: WilliamKiely’s comment on: Meta AI an­nounces Cicero: Hu­man-Level Di­plo­macy play (with di­alogue)

  There’s a commentated video by someone who plays as the only human in an otherwise all-Cicero game, which at least makes it seem like the dialogue is doing a lot.

• Kaj_Sotala 22 Nov 2022 18:22 UTC

  8 points

  1 ∶ 0

  on: Meta AI an­nounces Cicero: Hu­man-Level Di­plo­macy play (with di­alogue)

  Worth noting that this was “Blitz” Diplomacy with only five-minute negotiation rounds. Still very impressive though.

• Kaj_Sotala 20 Oct 2022 16:58 UTC

  5 points

  2 ∶ 1

  on: The her­i­ta­bil­ity of hu­man val­ues: A be­hav­ior ge­netic cri­tique of Shard Theory

  Some behavioral traits such as general intelligence show very high heritability – over 0.70 – in adults, which is about as heritable as human height.

  I’m very confused about what numbers such as this mean in practice, since the most natural interpretation (“70% of the trait is genetically determined”) is wrong, but there aren’t very many clear explanations of what the correct interpretation is. When I tried asking this on LW, the top-voted answer was that it’s a number that’s mostly useful if you’re doing animal breeding, but probably not useful for much else.

  You mention a lot of heritability numbers, could you clarify what it is that we’re intended to infer from them? (It seems to me that the main thing we can infer from heritability numbers is that if a trait has heritability above zero, then there’s some genetic influence on it, but since you mention some traits having “very high” heritability, I presume that you find there to be some other information too.)

• Kaj_Sotala 19 Sep 2022 7:15 UTC

  7 points

  1 ∶ 0

  in reply to: That's Confidential’s comment on: Many ther­apy schools work with in­ner mul­ti­plic­ity (not just IFS)

  I’m not sure if there is any reason that should be strongly persuasive to a disinterested third party, at the moment. I think the current evidence is more on the level of “anecdotally, it seems like a lot of rationalists and EAs get something out of things like IFS”.

  But given that one can try out a few sessions of a therapy and see whether you seem to be getting anything out of it, that seems to be okay? Anecdotal evidence isn’t enough to strongly show that one should definitely do a particular kind of therapy. But it can be enough to elevate a therapy to the level of things that might be worth giving a shot to see if anything comes out of it.

• Kaj_Sotala 18 Sep 2022 20:18 UTC

  8 points

  2 ∶ 1

  in reply to: That's Confidential’s comment on: Many ther­apy schools work with in­ner mul­ti­plic­ity (not just IFS)

  If they are better, why haven’t they been more widely adopted by mainstream medicine?

  Part of it is that the effectiveness of therapy is often hard and slow to study, so it’s hard to get unambiguous evidence of one being better than another. E.g. many therapists, even if working within a particular school of therapy, will apply it in an idiosyncratic style that draws upon their entire life/​job experience and also knowledge of any other therapy styles they might have. That makes it hard to verify the extent to which all the therapists in the study really are doing the same thing.

  My impression is that CBT got popular in part because it can be administered in a relatively standardized form, making it easy to study. But that means that it’s not necessarily any better than the other therapies, it’s just easier to get legible-seeming evidence on.

  Another issue is that therapy can often take a long time, and lots of the quantitative measures (e.g. depression questionnaires, measures of well-being) used for measuring its effects are relatively crude and imprecise and it can be hard to know exactly what one should even be measuring.

  So overall it can just be relatively hard to compare the effectiveness of therapies, and particular ones get more popular more slowly, e.g. by therapist A hearing that therapist B has been doing a new thing that seems to get better results that A does and then getting curious about it. Or by word of mouth through therapy clients when a new approach is found that seems to work better than conventional therapies, as has been happening with IFS.

• Kaj_Sotala 12 Jul 2022 20:34 UTC

  2 points

  0 ∶ 0

  in reply to: CounterBlunder’s comment on: Wheel­ing and deal­ing: An in­ter­nal bar­gain­ing ap­proach to moral uncertainty

  Oh wow, that is a really great paper! Thank you very much for linking it.

• Kaj_Sotala 9 Jul 2022 21:51 UTC

  8 points

  0 ∶ 0

  in reply to: howdoyousay?’s comment on: Doom Circles

  why anyone would choose one big ritual like ‘Doom circles’ instead of just purposefully inculcating a culture of opennes to giving /​ receiving critique that is supportive and can help others?

  These don’t sound mutually exclusive to me; you can have a formal ritual about something and also practice doing some of the related skills on a more regular basis.

  That said, for many people, it would be emotionally challenging if they needed to be ready to receive criticism all the time. A doom circle is something where you can receive feedback at such a time when you have emotionally prepared for it, and then go back to receiving less criticism normally.

  It might be better if everyone was capable of always receiving it, but it’s also good to have options for people who aren’t quite there yet. A doom circle is a thing that can be done in less than an hour, whereas skill-building is much more of a long-term project.

  low-context feedback is often not helpful;

  That’s true, but also: since people generally know that low-context feedback can be unhelpful, they might hold back offering any such feedback, even when it would be useful! Having an explicit context for offering the kind of critical feedback that you know might be incorrect gives people the chance to receive even the kinds of impressions that they otherwise wouldn’t have the chance to hear.

  feedback is ultimately just an opinion; you should be able to take and also discard it.

  Yes, in any well-run doom circle, this exact thing would be emphasized in the beginning (levin mentioned this in their comment and it was probably done in the beginning of the circles I’ve participated in, as well; I don’t remember what the exact preliminaries were, but it certainly matches my general sense of the spirit of the circle).

• Kaj_Sotala 9 Jul 2022 21:17 UTC

  2 points

  0 ∶ 0

  in reply to: Amy Labenz’s comment on: Doom Circles

  That’s correct, most of the people in the circle (including the person with the wizard line, I think) I’d only met a couple of days before.

• Kaj_Sotala 8 Jul 2022 22:47 UTC

  4 points

  1 ∶ 0

  in reply to: Amy Labenz’s comment on: Doom Circles

  That’s great, thank you :D

• Kaj_Sotala 8 Jul 2022 21:33 UTC

  2 points

  0 ∶ 0

  in reply to: Amy Labenz’s comment on: Doom Circles

  Yes please :D

• Kaj_Sotala 8 Jul 2022 21:28 UTC

  4 points

  0 ∶ 0

  in reply to: Jakub Stencel’s comment on: Doom Circles

  It’s my impression that in writing workshops where people bring their writing to be criticized, it’s also a common rule that the writers are not allowed to respond to the feedback. I believe the rule exists exactly because of what you say: because another person’s feedback may be off or biased for a variety of reasons. If there was a discussion about it, the recipient of the feedback might get defensive and want to explain why the feedback was flawed. That would risk the conversation taking an unpleasant tone and also any correct feedback not being properly heard.

  When the rule is instead “you are required to listen to the feedback, but it’s then totally up to you what you do with it and can choose to ignore it as totally deluded if you wish”, that gives people the full license to do so. Unfair-feeling criticism won’t put them in a position where they have to choose between defending themselves and losing face—not defending yourself won’t make anyone lose face since nobody is allowed to defend themselves. That helps make it easier to consider whether some of the unfair-seeming criticism might actually have a point. And on the other hand, if the criticism really was unfair, then someone defending themselves might give the other people the temptation to argue against the defense in order for them to save face and justify their criticism as correct… and someone trying to argue to you that no really, you really are doomed (or “your writing really is bad”), would totally break any atmosphere of mutual kindness and support.

• Kaj_Sotala 8 Jul 2022 21:13 UTC

  13 points

  0 ∶ 0

  in reply to: KMF’s comment on: Doom Circles

  I would like to encourage environments in EA which are mutually supportive and kind.

  For what it’s worth, my experience of Doom Circles is that they felt explicitly supportive and kind. It felt like other people were willing to pay a social cost to give me honest feedback in a way that they would otherwise feel hesitant to do, and I appreciated them doing that for me.

  The incentives are to provide harsh critique to appear novel and insightful of the other person.

  I wouldn’t say that there’s an incentive to be harsh, since while you are providing feedback, others will also be providing feedback to you; so if you’re overly harsh to others, you might get the same in return. In my experience at least, the shared vulnerability created an incentive to be kind but honest.

• Kaj_Sotala 8 Jul 2022 21:08 UTC

  33 points

  0 ∶ 0

  on: Doom Circles

  I feel like this post is lacking an explanation of what’s good about this practice, so I’ll share my experiences.

  I think I’ve attended a couple of Doom Circles (weirdly, my memory claims that there would have been one at my 2014 CFAR workshop but the post says they were first done is 2015, so maybe I’m mixing that up with some later event). I’ve usually thought of myself as pretty thin-skinned (much less these days but much more back then), but my recollection is that the experience was clearly positive.

  It’s very rare to actually get critical feedback that you can trust to be both honest and well-intentioned. The fact that it was purely opt-in and with an intimate atmosphere of people who would also hear their doom in turn made it feel safe and like there was a definite feeling of bonding with the others, in a “we’re all doing a thing that might be slightly unpleasant but we also trust each other to be able to hear it and are making ourselves vulnerable both in hearing and speaking the doom and at the end we’ll have gone through a slightly challenging experience together” kind of vibe. It definitely felt to me like a privilege to be able to participate.

  I’ve forgotten most of the dooms I personally got, but I recall one I got from a doom circle in my 2018 CFAR mentorship training. I’d had a bunch of social anxiety when I’d initially gotten to the training and then it had gradually subsided over time. I think someone then referenced that and said something along the lines of “you’re being too tame when you could be more powerful, you should lean into weirdness more and be like a cackling mad wizard”. I can’t say I’d be totally sure about what they meant, but there was something about that which stayed with me and which I’ve occasionally thought about. Maybe something like, even though being a cackling mad wizard isn’t quite the type of identity that’d be my thing, it felt significant to hear that someone thinks that I do have more power than I’m letting myself believe, and that I could play the role of a cackling mad wizard if I chose to.

  That’s another thing that can also make Doom Circles positive. Because the doom you can see facing someone else can also be a thing like their own potential that they’re not letting themselves see and are thus letting go to waste. Normally, it would still be weird to state that kind of a thing aloud. But if you take the opportunity to say it in a Doom Circle, it can actually become something that ends up landing as a compliment that they’ll think about for years afterwards—“huh, would I really have it in me to be a cackling mad wizard?”

• Kaj_Sotala 1 Mar 2022 8:55 UTC

  1 point

  0 ∶ 0

  in reply to: David_Althaus’s comment on: What psy­cholog­i­cal traits pre­dict in­ter­est in effec­tive al­tru­ism?

  Huh! That’s surprising.

• Kaj_Sotala 27 Feb 2022 7:56 UTC

  7 points

  0 ∶ 0

  in reply to: Gordon Seidoh Worley’s comment on: What psy­cholog­i­cal traits pre­dict in­ter­est in effec­tive al­tru­ism?

  Fair. In that case this seems like a necessary prerequisite result for doing that deeper investigation, though, so valuable in that respect.