So, Nonlinear-affiliated people are here in the comments disagreeing, promising proof that important claims in the post are false. I fully expect that Nonlinear’s response, and much of the discussion, will be predictably shoved down the throat of my attention, so I’m not too worried about missing the rebuttals, if rebuttals are in fact coming.
But there’s a hard-won lesson I’ve learned by digging into conflicts like this one, which I want to highlight, which I think makes this post valuable even if some of the stories turn out to be importantly false:
If a story is false, the fact that the story was told, and who told it, is valuable information. Sometimes it’s significantly more valuable than if the story was true. You can’t untangle a web of lies by trying to prevent anyone from saying things that have falsehoods embedded in them. You can untangle a web of lies by promoting a norm of maximizing the available information, including indirect information like who said what.
Think of the game Werewolf, as an analogy. Some moves are Villager strategies, and some moves are Werewolf strategies, in the sense that, if you notice someone using the strategy, you should make a Bayesian update in the direction of thinking the person using that strategy is a Villager or is a Werewolf.
It sounds like you’re claiming something like “all information is valuable information, because even if the information is false you’ve learned something (e.g. that the source is untrustworthy)”. I think this is too strong of a claim. Trying to figure out what’s true amidst lots of falsehoods is very difficult and takes time. Most people in real life aren’t playing perfect Werewolf with a complex Bayesian model that encompasses all hypotheses. Quite the opposite, from what I’ve seen both in myself and others, our natural tendency is to quickly collapse on one hypothesis and then interpret everything through that lens (confirmation bias). I think this is what’s happening with a lot of the reactions to this post, and I don’t think it’s valuable.
If you’d instead said “this post is valuable if you view it as a game of Werewolf, keep your hypothesis space open and update as new evidence comes in” then I’d be more in agreement. I think this is still a very difficult task though, and I’d rather that Ben had waited for Nonlinear’s counter-evidence and taken that that into consideration instead of forcing us to play Werewolf with his post. (Basically, I’m suggesting that Ben does the hard job of playing Werewolf for us. This is explicitly not what he did, as he himself says in his disclaimer of explicitly seeking out anti-nonlinear evidence.)
Disclaimer: I am friends with Kat and know some of the counter-evidence.
Repeating myself from when I first saw this comment:
As I mentioned to you before, I suspect werewolf/mafia/avalon is a pretty bad analogy for how to suss out the trustworthiness of people irl:
in games, the number of werewolves etc is often fixed and known to all players ahead of time; irl a lot of the difficulty is figuring out whether (and how many) terminally bad actors exist, vs honest misunderstandings, vs generically suss people.
random spurious accusations with zero factual backing are usually considered town/vanilla/arthurian moves in werewolf games; irl this breeds chaos and is a classic DARVO tactic.
In games, the set of both disputed and uncontested facts are discrete and often small; this is much less the case irl.
in games, bad guys have a heavy incentive to be uncorrelated (and especially to be seen as being uncorrelated); irl there are very few worlds where regularly agreeing with the now-known-to-be-bad-actors is a positive update on your innocence.
In games, the set of actions available to both good and bad actors are well-defined and often known in advance; irl does not have this luxury.
etc
All these points, but especially the second one, means that people should be very hesitant to generalize hard-won lessons about macrolevel social dynamics from social deception games to real life.
So, Nonlinear-affiliated people are here in the comments disagreeing, promising proof that important claims in the post are false. I fully expect that Nonlinear’s response, and much of the discussion, will be predictably shoved down the throat of my attention, so I’m not too worried about missing the rebuttals, if rebuttals are in fact coming.
But there’s a hard-won lesson I’ve learned by digging into conflicts like this one, which I want to highlight, which I think makes this post valuable even if some of the stories turn out to be importantly false:
If a story is false, the fact that the story was told, and who told it, is valuable information. Sometimes it’s significantly more valuable than if the story was true. You can’t untangle a web of lies by trying to prevent anyone from saying things that have falsehoods embedded in them. You can untangle a web of lies by promoting a norm of maximizing the available information, including indirect information like who said what.
Think of the game Werewolf, as an analogy. Some moves are Villager strategies, and some moves are Werewolf strategies, in the sense that, if you notice someone using the strategy, you should make a Bayesian update in the direction of thinking the person using that strategy is a Villager or is a Werewolf.
It sounds like you’re claiming something like “all information is valuable information, because even if the information is false you’ve learned something (e.g. that the source is untrustworthy)”. I think this is too strong of a claim. Trying to figure out what’s true amidst lots of falsehoods is very difficult and takes time. Most people in real life aren’t playing perfect Werewolf with a complex Bayesian model that encompasses all hypotheses. Quite the opposite, from what I’ve seen both in myself and others, our natural tendency is to quickly collapse on one hypothesis and then interpret everything through that lens (confirmation bias). I think this is what’s happening with a lot of the reactions to this post, and I don’t think it’s valuable.
If you’d instead said “this post is valuable if you view it as a game of Werewolf, keep your hypothesis space open and update as new evidence comes in” then I’d be more in agreement. I think this is still a very difficult task though, and I’d rather that Ben had waited for Nonlinear’s counter-evidence and taken that that into consideration instead of forcing us to play Werewolf with his post. (Basically, I’m suggesting that Ben does the hard job of playing Werewolf for us. This is explicitly not what he did, as he himself says in his disclaimer of explicitly seeking out anti-nonlinear evidence.)
Disclaimer: I am friends with Kat and know some of the counter-evidence.
Great point.
Repeating myself from when I first saw this comment: