For previous grant-making decisions, I have been interested in the question about the extent to which more radical tactics can move the Overton Window and make things easier for more moderate activists. Based on fairly quick Google Scholar searches, I haven’t found anything, so I’m glad you’ve investigated this.
I haven’t read your piece fully, so sorry if I’m asking things which are answered in your post.
92% awareness rate seems awfully high. Did the survey include anything to test whether people who said that they had heard of Just Stop Oil really had? E.g. by testing whether they knew that JSO had performed radical tactics? Surveys have a strong tendency to elicit answers which agree with the question, so if the question were worded as “Have you heard of …” people are more prone to say yes. They may also want to appear more knowledgeable and worldly aware. I’m worried that the survey might be finding “willingness to appear aware of environmental topics leads to support for Friends of the Earth” and not “being aware of JSO leads to support for Friends of the Earth”.
Did your work analyse the people who hadn’t heard of JSO before the actions and then had heard of them after the actions? That might be interesting.
I appreciate this comment is difficult (impossible?) to overcome in a survey, but I’m worried that to the extent the effect is real, it might capture attitudes/appearances not actions. Let’s assume that people who have heard about JSO decide they want to position themselves as more moderate than them but still climate-friendly. Are they positioning themselves this way in how they want to come across, or the actual actions they take? For some people (especially politicians/decision-makers) I could imagine that the performative nature of JSO-style protests might (sub-consciously?) underline to them that this is all about media theatre, and not about genuinely taking action or making change. Surveys are really good at capturing the “how they want to come across” perspective, and not the “actual actions they take” perspective.
Hi Sanjay, thanks for the good questions! Since you said you were looking, the most relevant bit of research to this topic is by Simpson et al. (2022) who did experimental research on the radical flank effect, both for radical tactics and a radical agenda. Interestingly, they find that a radical agenda had no effect, but radical tactics did. On your questions:
On the 92% figure, we weren’t explicit about this but yes, we also estimate that this is over-inflated vs reality for the reasons you said (people do weird things on surveys). In both this survey and a previous one for Animal Rebellion, we asked decoy questions about awareness of fictional organisations, and elicited awareness values of around 60% and 20% respectively, which is obviously very weird. We’re not exactly sure how we’re going to use this information to adjust our estimates, so feedback welcome. I wasn’t too concerned about that particular result as I actually think that value is the least interesting of all of our findings, and don’t think it’s particularly informative for understanding the actual impact JSO had. I’ll make an edit in the document though to be more explicit that this is likely overinflated so good point!
We hadn’t done that particular analysis (for people who went from 0 awareness to >0 awareness) but we can definitely do so! I’ll put that on the list of additional analyses to do (or consider doing).
This is definitely a good point and like you say, a limitation of surveys as we can generally only gather people’s stated intentions, rather than their actions. There seems to be a good amount of research that supports increased identification with a movement leads to increased willingness to join and support the movement (E.g. financially) so I think the link isn’t super tenuous.
Re the 92%, I agree that it is intrinsically not that interesting. I was flagging it because I was worried that some of the 92% (maybe lots? maybe most?) didn’t really understand that JSO espoused radical tactics. So the survey might be finding “willingness to appear aware of environmental topics leads to support for Friends of the Earth” and not “being aware of JSO leads to support for Friends of the Earth”. In terms of how to account for this, I would have suggested another question in the survey which asks people who said that they have heard of JSO to indicate what JSO is most associated with, and one of the answers is that they are associated with radical tactics, and others are plausible-sounding but false. I appreciate this might be difficult now—the survey is already done. And you might want to draw a line under this and move on. But if you did want to get to the bottom of this concern, you could replicate the survey (probably a fair bit more cheaply than by using YouGov—just code up the survey yourself, which is very easy with GuidedTrack, and then get sample with something like prolific.co)
people who went from 0 awareness to >0 awareness: would be interesting to see if you get round to doing that
I know I raised the question about the distinction between stated intentions and actions, but I should acknowledge that it might not matter. Example: if the intended theory of change is protest --> change in public attitudes --> people change their energy provider; in this case actions are important. Another example: if the intended theory of change is protest --> change in public attitudes --> politicians believing that the public wants change, so the politicians make a change happen; in this case, it actually doesn’t matter whether the public gains enough conviction to change their behaviour—just enough conviction that they answer more positively in a survey is actually sufficient.
Ah I’ve understood you now! I don’t see what you said being a problem as we explicitly put a sentence description in the survey for both JSO and Friends of the Earth explaining a tiny bit about who they were (e.g. they use radical tactics) without anchoring respondents too much. So I’m quite confident that respondents would have known JSO was utilising more disruptive tactics relative to Friends of the Earth. In addition, given how much media coverage JSO got during this week (and for previous actions) I would be quite surprised if a good proportion of people in the UK (50%+?) hadn’t already seen some reference to JSO in the context of arrest, police or disruptive protest.
Agreed about the stated intentions vs action point. I think increasing support for a policy can be a useful public opinion signal, but JSO probably also wants increased desire to take action on climate change (e.g. sign a petition, attend protest, etc), which we didn’t find a change in.
For previous grant-making decisions, I have been interested in the question about the extent to which more radical tactics can move the Overton Window and make things easier for more moderate activists. Based on fairly quick Google Scholar searches, I haven’t found anything, so I’m glad you’ve investigated this.
I haven’t read your piece fully, so sorry if I’m asking things which are answered in your post.
92% awareness rate seems awfully high. Did the survey include anything to test whether people who said that they had heard of Just Stop Oil really had? E.g. by testing whether they knew that JSO had performed radical tactics? Surveys have a strong tendency to elicit answers which agree with the question, so if the question were worded as “Have you heard of …” people are more prone to say yes. They may also want to appear more knowledgeable and worldly aware. I’m worried that the survey might be finding “willingness to appear aware of environmental topics leads to support for Friends of the Earth” and not “being aware of JSO leads to support for Friends of the Earth”.
Did your work analyse the people who hadn’t heard of JSO before the actions and then had heard of them after the actions? That might be interesting.
I appreciate this comment is difficult (impossible?) to overcome in a survey, but I’m worried that to the extent the effect is real, it might capture attitudes/appearances not actions. Let’s assume that people who have heard about JSO decide they want to position themselves as more moderate than them but still climate-friendly. Are they positioning themselves this way in how they want to come across, or the actual actions they take? For some people (especially politicians/decision-makers) I could imagine that the performative nature of JSO-style protests might (sub-consciously?) underline to them that this is all about media theatre, and not about genuinely taking action or making change. Surveys are really good at capturing the “how they want to come across” perspective, and not the “actual actions they take” perspective.
Hi Sanjay, thanks for the good questions! Since you said you were looking, the most relevant bit of research to this topic is by Simpson et al. (2022) who did experimental research on the radical flank effect, both for radical tactics and a radical agenda. Interestingly, they find that a radical agenda had no effect, but radical tactics did. On your questions:
On the 92% figure, we weren’t explicit about this but yes, we also estimate that this is over-inflated vs reality for the reasons you said (people do weird things on surveys). In both this survey and a previous one for Animal Rebellion, we asked decoy questions about awareness of fictional organisations, and elicited awareness values of around 60% and 20% respectively, which is obviously very weird. We’re not exactly sure how we’re going to use this information to adjust our estimates, so feedback welcome. I wasn’t too concerned about that particular result as I actually think that value is the least interesting of all of our findings, and don’t think it’s particularly informative for understanding the actual impact JSO had. I’ll make an edit in the document though to be more explicit that this is likely overinflated so good point!
We hadn’t done that particular analysis (for people who went from 0 awareness to >0 awareness) but we can definitely do so! I’ll put that on the list of additional analyses to do (or consider doing).
This is definitely a good point and like you say, a limitation of surveys as we can generally only gather people’s stated intentions, rather than their actions. There seems to be a good amount of research that supports increased identification with a movement leads to increased willingness to join and support the movement (E.g. financially) so I think the link isn’t super tenuous.
Thanks for pointing me to Simpson et al 2022.
Re the 92%, I agree that it is intrinsically not that interesting. I was flagging it because I was worried that some of the 92% (maybe lots? maybe most?) didn’t really understand that JSO espoused radical tactics. So the survey might be finding “willingness to appear aware of environmental topics leads to support for Friends of the Earth” and not “being aware of JSO leads to support for Friends of the Earth”. In terms of how to account for this, I would have suggested another question in the survey which asks people who said that they have heard of JSO to indicate what JSO is most associated with, and one of the answers is that they are associated with radical tactics, and others are plausible-sounding but false. I appreciate this might be difficult now—the survey is already done. And you might want to draw a line under this and move on. But if you did want to get to the bottom of this concern, you could replicate the survey (probably a fair bit more cheaply than by using YouGov—just code up the survey yourself, which is very easy with GuidedTrack, and then get sample with something like prolific.co)
people who went from 0 awareness to >0 awareness: would be interesting to see if you get round to doing that
I know I raised the question about the distinction between stated intentions and actions, but I should acknowledge that it might not matter. Example: if the intended theory of change is protest --> change in public attitudes --> people change their energy provider; in this case actions are important. Another example: if the intended theory of change is protest --> change in public attitudes --> politicians believing that the public wants change, so the politicians make a change happen; in this case, it actually doesn’t matter whether the public gains enough conviction to change their behaviour—just enough conviction that they answer more positively in a survey is actually sufficient.
Ah I’ve understood you now! I don’t see what you said being a problem as we explicitly put a sentence description in the survey for both JSO and Friends of the Earth explaining a tiny bit about who they were (e.g. they use radical tactics) without anchoring respondents too much. So I’m quite confident that respondents would have known JSO was utilising more disruptive tactics relative to Friends of the Earth. In addition, given how much media coverage JSO got during this week (and for previous actions) I would be quite surprised if a good proportion of people in the UK (50%+?) hadn’t already seen some reference to JSO in the context of arrest, police or disruptive protest.
Agreed about the stated intentions vs action point. I think increasing support for a policy can be a useful public opinion signal, but JSO probably also wants increased desire to take action on climate change (e.g. sign a petition, attend protest, etc), which we didn’t find a change in.