(3) this post then saying people should defer to authority.
Taken in isolation, these are somewhat concerning. Taken together, they start to look frightening—of the flavour, “join our elite society, see the truth, obey your leaders”.
I am pretty sure anyone reading this will agree that this is not how we want EA either to be or to be perceived to be. However, things do seem to be moving in that direction, and I don’t think this post helped—sorry, Owen, I am sure you wrote it with the best of intentions. But the road to hell, pavements, etc.
I am concerned about some of the long-termism push but didn’t get that vibe from this post, as an alternate perspective
Edit: wow why is Michael getting downvoted though, wtf? different people can have different impressions of the tone of a written piece of work, it’s not harmful to point it out
Edit: wow why is Michael getting downvoted though, wtf?
Perhaps people didn’t like the cult-ish comparison? But criticising someone for saying they are feeling something is cult-ish is, um, well, pretty cult-ish...
Or perhaps it’s people who can’t properly distinguish between “criticising because you care and want to improve something” and “criticising to be mean” and mistakenly assume I’m doing the latter (despite my strenuous attempts to make it clear I am doing the former).
I sort of guess the second thing?; although I never downvoted at least I felt a little defensive and negative about “tone-deaf, indeed chilling” and didn’t upvote despite having found your comment useful!
I don’t think we should only downvote harmful things, we should instead look at the amount of karma and use our votes to push the score to the value we think the post should be at.
I downvoted the comment because:
Saying things like ”… obviously push an agenda....” And “I’m pretty sure anyone reading this… ” Has persuasiony vibes which I don’t like.
Saying “this post says people should defer to authority” is a bit of a straw/weak man and isn’t very charitable.
Using votes to push towards the score we think it should be at sounds worse than just individually voting according to some thresholds of how good/helpful/whatever a post needs to be? I’m worried about zero sum (so really negative sum because of the effort) attempts to move karma around where different people are pushing in different ways, where it’s hard to know how to interpret the results, compared to people straightforwardly voting without regard to others’ votes.
At least, if we should be voting to push things towards our best guess I think the karma system should be reformed to something that plays nice with that—e.g. each individual gives their preferred score, and the displayed karma is the median.
(I think that the pushing towards a score thing wasn’t a crux in downvoting, I think there are lots of reasons to downvote things that aren’t harmful as outlined in the ‘how to use the form post/moderator guidelines’)
I think that karma is supposed to be a proxy for the relative value that a post provides.
I’m not sure what you mean by zero-sum here, but I would have thought that the control system type approach is better as the steady-state values will be pushed towards the mean of what users see as the true value of the post. I think that this score + total number of votes is quite easy to interpret.
The everyone voting independently thing performs poorly when some posts have much more views than others (so it seems to be tracking something more like how many people saw it and liked it rather than is the post high quality).
I think I misunderstand your concern, but the control system approach seems, on the surface to be much better to me, but I am keen to find the crux here, if there is one.
So, my immediate reaction is that I can feel that kind of concern, but I think the “see the truth, obey your leaders” is exactly the kind of dynamic I’m worried about! & then I’m trying to help avoid it by helping to disambiguate between epistemic deferring and deferring to authority (because conflating them is where I think a lot of the damage comes from).
So then I’m wondering if I’ve made some bad branding decisions (e.g. should I have used a different term for what I called “deferring to authority”? It’s meant to evoke that someone has authority in a particular domain, not some kind of general purpose authority, and not that they know a lot), or if I’m failing to frame my positions correctly? I guess at least a bit of the latter, since it sounds like you read my post as saying people should defer more? Which definitely wasn’t something I intended to say (I’m confused; I’d like to see more deferring of some types and less of other types; I guess overall I’d be into a bit less deferring but not confident enough about that that I’d want to make it a headline).
(fwiw I upvoted this post, because I thought it raised a lot of interesting points that are worth discussing despite disagreeing some bits).
In sum: I think your post sometimes lacks specificity which makes people think you’re talking more generally than (I suspect) you are.
Who exactly you’re proposing doesn’t buy into the agenda—this is left vague in your post. Are you envisioning 20% of people? 50%? What kinds of roles are these folks in? Is it only junior level non-technical roles or even mid-managers doing direct work?
Those details matter because I think I’d be fine with e.g. junior ops people at an AI org not fully buying the specific research agenda of that org, but I’m not sure about the other roles here.
Who do you count as the EA community or movement? I think if we are thinking big tent EA where you have people with the needed skills for the movement but not necessarily a deep understanding of EA, I’m more sympathetic to this argument. But if we’re thinking core community EA where many people are doing things like community building or for whom EA is a bit part of their lives, I feel much more uncomfortable with people deferring to authority—perhaps I feel particularly uncomfortable with people in the meta space deferring to authority.
I vibe with the sentiment “particularly uncomfortable with people in the meta space deferring to authority”, but I think it’s too strong. e.g. I think it’s valuable for people to be able to organize big events, and delegate tasks among the event organizers.
Maybe I’m more like “I feel particularly uncomfortable with people in the meta space deferring without high bandwidth, and without explicit understanding that that’s what they’re doing”.
I think the important thing with delegation which Howie pointed out, is that there is a social contract in the example you gave of event organising between the volunteer / volunteer manager or employer / contractor where I’d expect that in the process of choosing to sign up for this job, the person makes a decision based on their own thinking (or epistemic deference) to contribute to this event—I think this is what you mean by high bandwidth?
If so, I feel in agreement with the statement: “I feel particularly uncomfortable with people in the meta space delegating choice without high bandwidth, and without explicit understanding that that’s what they’re doing”
I’m fine with junior ops people at an AI org being not really atall bought into the specific research agenda.
I’m fine with senior technical people not being fully bought in—in the sense that maybe they think if it were up to them a different agenda would be slightly higher value, or that they’d go about things a slightly different way. I think we should expect that people have slightly different takes, and don’t get the luxury of ironing all of those differences out, and that’s pretty healthy. (Of course I like them having a go at discussing differences of opinion, but I don’t think failure to resolve a difference means that the they need to adopt the party line or go find a different org.)
That makes sense, and feels mostly in line with what I would imagine.
Maybe this is a small point (since there will be many more junior than senior roles in the long run) : I feel like the senior group would likely join an org for many other reasons than deference to authority (e.g. not wanting to found an org themselves, wanting to work with particular people they feel they could get a good work environment from, or because of epistemic deference). It seems like in practice those would be much stronger motivating reasons than authority, and I’m having a hard time picturing someone doing this in practice.
Okay, well, just to report that what you said by way of clarification was reassuring but not what I picked up originally from your post! I agree with Vaidehi below that an issue was a lack of specificity, which led to me reading it as a pretty general comment.
Reading your other comments, it seems what you’re getting at is a distinction between trusting someone is right without understanding why vs just following their instructions. I agree that there’s something there: to e.g. run an organisation, it’s sometimes impractical or unnecessary to convince someone of your entire worldview vs just ask them to do something.
FWIW, what I see lots of in EA, worries me, and I was hoping your post would be about, is that people defer so strongly to community leaders that they refuse to even engage with object-level arguments against whatever it is that community leaders believe. To draw from a personal example, quite often when I talk about measuring wellbeing, people will listen and then say something to the effect of “what you say seems plausible, I can’t think of any objections, but I’m going to defer to GiveWell anyway”. Deferring may have a time and a place, but presumably we don’t want deference to this extent.
“Deferring to experts” might be a less loaded term. Also defining what experts are especially for a lot of EA fields that are newer and less well established could help.
“Deferring to experts” carries the wrong meaning, I think? At least to me that sounds more like epistemic deferring.
An alternative to “deferring to authority” a couple of people have suggested to me is “delegating”, which I sort of like (although maybe it’s confusing if one of the paradigm examples is delegating to your boss).
[Writing in a personal capacity, etc.]
I found this post tone-deaf, indeed chilling, when I read it, in light of the current dynamics of the EA movement. I think its the combination of:
(1) lots of money appearing in EA (with the recognition this might be a big problem for optics and epistemics and there are already ‘bad omens’)
(2) the central bits of EA seeming to obviously push an agenda (EA being ‘just longtermism’ now, with CEA’s CEO, Max Dalton, indicating their content will be “70-80% longtermism”; CEA’s Julia Wise is suggesting people shouldn’t talk to high net worths themselves, but should funnel them towards LongView)
(3) this post then saying people should defer to authority.
Taken in isolation, these are somewhat concerning. Taken together, they start to look frightening—of the flavour, “join our elite society, see the truth, obey your leaders”.
I am pretty sure anyone reading this will agree that this is not how we want EA either to be or to be perceived to be. However, things do seem to be moving in that direction, and I don’t think this post helped—sorry, Owen, I am sure you wrote it with the best of intentions. But the road to hell, pavements, etc.
I am concerned about some of the long-termism push but didn’t get that vibe from this post, as an alternate perspective
Edit: wow why is Michael getting downvoted though, wtf? different people can have different impressions of the tone of a written piece of work, it’s not harmful to point it out
Perhaps people didn’t like the cult-ish comparison? But criticising someone for saying they are feeling something is cult-ish is, um, well, pretty cult-ish...
Or perhaps it’s people who can’t properly distinguish between “criticising because you care and want to improve something” and “criticising to be mean” and mistakenly assume I’m doing the latter (despite my strenuous attempts to make it clear I am doing the former).
I sort of guess the second thing?; although I never downvoted at least I felt a little defensive and negative about “tone-deaf, indeed chilling” and didn’t upvote despite having found your comment useful!
(I’ve now noticed the discrepancy and upvoted it)
I don’t think we should only downvote harmful things, we should instead look at the amount of karma and use our votes to push the score to the value we think the post should be at.
I downvoted the comment because:
Saying things like ”… obviously push an agenda....” And “I’m pretty sure anyone reading this… ” Has persuasiony vibes which I don’t like.
Saying “this post says people should defer to authority” is a bit of a straw/weak man and isn’t very charitable.
Using votes to push towards the score we think it should be at sounds worse than just individually voting according to some thresholds of how good/helpful/whatever a post needs to be? I’m worried about zero sum (so really negative sum because of the effort) attempts to move karma around where different people are pushing in different ways, where it’s hard to know how to interpret the results, compared to people straightforwardly voting without regard to others’ votes.
At least, if we should be voting to push things towards our best guess I think the karma system should be reformed to something that plays nice with that—e.g. each individual gives their preferred score, and the displayed karma is the median.
(I think that the pushing towards a score thing wasn’t a crux in downvoting, I think there are lots of reasons to downvote things that aren’t harmful as outlined in the ‘how to use the form post/moderator guidelines’)
I think that karma is supposed to be a proxy for the relative value that a post provides.
I’m not sure what you mean by zero-sum here, but I would have thought that the control system type approach is better as the steady-state values will be pushed towards the mean of what users see as the true value of the post. I think that this score + total number of votes is quite easy to interpret.
The everyone voting independently thing performs poorly when some posts have much more views than others (so it seems to be tracking something more like how many people saw it and liked it rather than is the post high quality).
I think I misunderstand your concern, but the control system approach seems, on the surface to be much better to me, but I am keen to find the crux here, if there is one.
Interesting, thanks.
So, my immediate reaction is that I can feel that kind of concern, but I think the “see the truth, obey your leaders” is exactly the kind of dynamic I’m worried about! & then I’m trying to help avoid it by helping to disambiguate between epistemic deferring and deferring to authority (because conflating them is where I think a lot of the damage comes from).
So then I’m wondering if I’ve made some bad branding decisions (e.g. should I have used a different term for what I called “deferring to authority”? It’s meant to evoke that someone has authority in a particular domain, not some kind of general purpose authority, and not that they know a lot), or if I’m failing to frame my positions correctly? I guess at least a bit of the latter, since it sounds like you read my post as saying people should defer more? Which definitely wasn’t something I intended to say (I’m confused; I’d like to see more deferring of some types and less of other types; I guess overall I’d be into a bit less deferring but not confident enough about that that I’d want to make it a headline).
(fwiw I upvoted this post, because I thought it raised a lot of interesting points that are worth discussing despite disagreeing some bits).
In sum: I think your post sometimes lacks specificity which makes people think you’re talking more generally than (I suspect) you are.
Who exactly you’re proposing doesn’t buy into the agenda—this is left vague in your post. Are you envisioning 20% of people? 50%? What kinds of roles are these folks in? Is it only junior level non-technical roles or even mid-managers doing direct work?
Those details matter because I think I’d be fine with e.g. junior ops people at an AI org not fully buying the specific research agenda of that org, but I’m not sure about the other roles here.
Who do you count as the EA community or movement? I think if we are thinking big tent EA where you have people with the needed skills for the movement but not necessarily a deep understanding of EA, I’m more sympathetic to this argument. But if we’re thinking core community EA where many people are doing things like community building or for whom EA is a bit part of their lives, I feel much more uncomfortable with people deferring to authority—perhaps I feel particularly uncomfortable with people in the meta space deferring to authority.
I vibe with the sentiment “particularly uncomfortable with people in the meta space deferring to authority”, but I think it’s too strong. e.g. I think it’s valuable for people to be able to organize big events, and delegate tasks among the event organizers.
Maybe I’m more like “I feel particularly uncomfortable with people in the meta space deferring without high bandwidth, and without explicit understanding that that’s what they’re doing”.
I think the important thing with delegation which Howie pointed out, is that there is a social contract in the example you gave of event organising between the volunteer / volunteer manager or employer / contractor where I’d expect that in the process of choosing to sign up for this job, the person makes a decision based on their own thinking (or epistemic deference) to contribute to this event—I think this is what you mean by high bandwidth?
If so, I feel in agreement with the statement: “I feel particularly uncomfortable with people in the meta space delegating choice without high bandwidth, and without explicit understanding that that’s what they’re doing”
I’m fine with junior ops people at an AI org being not really at all bought into the specific research agenda.
I’m fine with senior technical people not being fully bought in—in the sense that maybe they think if it were up to them a different agenda would be slightly higher value, or that they’d go about things a slightly different way. I think we should expect that people have slightly different takes, and don’t get the luxury of ironing all of those differences out, and that’s pretty healthy. (Of course I like them having a go at discussing differences of opinion, but I don’t think failure to resolve a difference means that the they need to adopt the party line or go find a different org.)
That makes sense, and feels mostly in line with what I would imagine.
Maybe this is a small point (since there will be many more junior than senior roles in the long run) : I feel like the senior group would likely join an org for many other reasons than deference to authority (e.g. not wanting to found an org themselves, wanting to work with particular people they feel they could get a good work environment from, or because of epistemic deference). It seems like in practice those would be much stronger motivating reasons than authority, and I’m having a hard time picturing someone doing this in practice.
Okay, well, just to report that what you said by way of clarification was reassuring but not what I picked up originally from your post! I agree with Vaidehi below that an issue was a lack of specificity, which led to me reading it as a pretty general comment.
Reading your other comments, it seems what you’re getting at is a distinction between trusting someone is right without understanding why vs just following their instructions. I agree that there’s something there: to e.g. run an organisation, it’s sometimes impractical or unnecessary to convince someone of your entire worldview vs just ask them to do something.
FWIW, what I see lots of in EA, worries me, and I was hoping your post would be about, is that people defer so strongly to community leaders that they refuse to even engage with object-level arguments against whatever it is that community leaders believe. To draw from a personal example, quite often when I talk about measuring wellbeing, people will listen and then say something to the effect of “what you say seems plausible, I can’t think of any objections, but I’m going to defer to GiveWell anyway”. Deferring may have a time and a place, but presumably we don’t want deference to this extent.
“Deferring to experts” might be a less loaded term. Also defining what experts are especially for a lot of EA fields that are newer and less well established could help.
“Deferring to experts” carries the wrong meaning, I think? At least to me that sounds more like epistemic deferring.
An alternative to “deferring to authority” a couple of people have suggested to me is “delegating”, which I sort of like (although maybe it’s confusing if one of the paradigm examples is delegating to your boss).
In light of the other discussions, delegating choice seems better than deferring to experts.