Hi Sanjay, thanks for the good questions! Since you said you were looking, the most relevant bit of research to this topic is by Simpson et al. (2022) who did experimental research on the radical flank effect, both for radical tactics and a radical agenda. Interestingly, they find that a radical agenda had no effect, but radical tactics did. On your questions:
On the 92% figure, we weren’t explicit about this but yes, we also estimate that this is over-inflated vs reality for the reasons you said (people do weird things on surveys). In both this survey and a previous one for Animal Rebellion, we asked decoy questions about awareness of fictional organisations, and elicited awareness values of around 60% and 20% respectively, which is obviously very weird. We’re not exactly sure how we’re going to use this information to adjust our estimates, so feedback welcome. I wasn’t too concerned about that particular result as I actually think that value is the least interesting of all of our findings, and don’t think it’s particularly informative for understanding the actual impact JSO had. I’ll make an edit in the document though to be more explicit that this is likely overinflated so good point!
We hadn’t done that particular analysis (for people who went from 0 awareness to >0 awareness) but we can definitely do so! I’ll put that on the list of additional analyses to do (or consider doing).
This is definitely a good point and like you say, a limitation of surveys as we can generally only gather people’s stated intentions, rather than their actions. There seems to be a good amount of research that supports increased identification with a movement leads to increased willingness to join and support the movement (E.g. financially) so I think the link isn’t super tenuous.
Re the 92%, I agree that it is intrinsically not that interesting. I was flagging it because I was worried that some of the 92% (maybe lots? maybe most?) didn’t really understand that JSO espoused radical tactics. So the survey might be finding “willingness to appear aware of environmental topics leads to support for Friends of the Earth” and not “being aware of JSO leads to support for Friends of the Earth”. In terms of how to account for this, I would have suggested another question in the survey which asks people who said that they have heard of JSO to indicate what JSO is most associated with, and one of the answers is that they are associated with radical tactics, and others are plausible-sounding but false. I appreciate this might be difficult now—the survey is already done. And you might want to draw a line under this and move on. But if you did want to get to the bottom of this concern, you could replicate the survey (probably a fair bit more cheaply than by using YouGov—just code up the survey yourself, which is very easy with GuidedTrack, and then get sample with something like prolific.co)
people who went from 0 awareness to >0 awareness: would be interesting to see if you get round to doing that
I know I raised the question about the distinction between stated intentions and actions, but I should acknowledge that it might not matter. Example: if the intended theory of change is protest --> change in public attitudes --> people change their energy provider; in this case actions are important. Another example: if the intended theory of change is protest --> change in public attitudes --> politicians believing that the public wants change, so the politicians make a change happen; in this case, it actually doesn’t matter whether the public gains enough conviction to change their behaviour—just enough conviction that they answer more positively in a survey is actually sufficient.
Ah I’ve understood you now! I don’t see what you said being a problem as we explicitly put a sentence description in the survey for both JSO and Friends of the Earth explaining a tiny bit about who they were (e.g. they use radical tactics) without anchoring respondents too much. So I’m quite confident that respondents would have known JSO was utilising more disruptive tactics relative to Friends of the Earth. In addition, given how much media coverage JSO got during this week (and for previous actions) I would be quite surprised if a good proportion of people in the UK (50%+?) hadn’t already seen some reference to JSO in the context of arrest, police or disruptive protest.
Agreed about the stated intentions vs action point. I think increasing support for a policy can be a useful public opinion signal, but JSO probably also wants increased desire to take action on climate change (e.g. sign a petition, attend protest, etc), which we didn’t find a change in.
Hi Sanjay, thanks for the good questions! Since you said you were looking, the most relevant bit of research to this topic is by Simpson et al. (2022) who did experimental research on the radical flank effect, both for radical tactics and a radical agenda. Interestingly, they find that a radical agenda had no effect, but radical tactics did. On your questions:
On the 92% figure, we weren’t explicit about this but yes, we also estimate that this is over-inflated vs reality for the reasons you said (people do weird things on surveys). In both this survey and a previous one for Animal Rebellion, we asked decoy questions about awareness of fictional organisations, and elicited awareness values of around 60% and 20% respectively, which is obviously very weird. We’re not exactly sure how we’re going to use this information to adjust our estimates, so feedback welcome. I wasn’t too concerned about that particular result as I actually think that value is the least interesting of all of our findings, and don’t think it’s particularly informative for understanding the actual impact JSO had. I’ll make an edit in the document though to be more explicit that this is likely overinflated so good point!
We hadn’t done that particular analysis (for people who went from 0 awareness to >0 awareness) but we can definitely do so! I’ll put that on the list of additional analyses to do (or consider doing).
This is definitely a good point and like you say, a limitation of surveys as we can generally only gather people’s stated intentions, rather than their actions. There seems to be a good amount of research that supports increased identification with a movement leads to increased willingness to join and support the movement (E.g. financially) so I think the link isn’t super tenuous.
Thanks for pointing me to Simpson et al 2022.
Re the 92%, I agree that it is intrinsically not that interesting. I was flagging it because I was worried that some of the 92% (maybe lots? maybe most?) didn’t really understand that JSO espoused radical tactics. So the survey might be finding “willingness to appear aware of environmental topics leads to support for Friends of the Earth” and not “being aware of JSO leads to support for Friends of the Earth”. In terms of how to account for this, I would have suggested another question in the survey which asks people who said that they have heard of JSO to indicate what JSO is most associated with, and one of the answers is that they are associated with radical tactics, and others are plausible-sounding but false. I appreciate this might be difficult now—the survey is already done. And you might want to draw a line under this and move on. But if you did want to get to the bottom of this concern, you could replicate the survey (probably a fair bit more cheaply than by using YouGov—just code up the survey yourself, which is very easy with GuidedTrack, and then get sample with something like prolific.co)
people who went from 0 awareness to >0 awareness: would be interesting to see if you get round to doing that
I know I raised the question about the distinction between stated intentions and actions, but I should acknowledge that it might not matter. Example: if the intended theory of change is protest --> change in public attitudes --> people change their energy provider; in this case actions are important. Another example: if the intended theory of change is protest --> change in public attitudes --> politicians believing that the public wants change, so the politicians make a change happen; in this case, it actually doesn’t matter whether the public gains enough conviction to change their behaviour—just enough conviction that they answer more positively in a survey is actually sufficient.
Ah I’ve understood you now! I don’t see what you said being a problem as we explicitly put a sentence description in the survey for both JSO and Friends of the Earth explaining a tiny bit about who they were (e.g. they use radical tactics) without anchoring respondents too much. So I’m quite confident that respondents would have known JSO was utilising more disruptive tactics relative to Friends of the Earth. In addition, given how much media coverage JSO got during this week (and for previous actions) I would be quite surprised if a good proportion of people in the UK (50%+?) hadn’t already seen some reference to JSO in the context of arrest, police or disruptive protest.
Agreed about the stated intentions vs action point. I think increasing support for a policy can be a useful public opinion signal, but JSO probably also wants increased desire to take action on climate change (e.g. sign a petition, attend protest, etc), which we didn’t find a change in.