Thanks for your thorough comment! Yeah I was shooting for about 60 participants, but due to time constraints and this being a pilot study I only ended up with 44, so even more underpowered.
Intuitively I would expect a larger effect size, given that I don’t consider the manipulation to be particularly subtle; but yes, it was much subtler than it could have been. This is something I will definitely explore more if I continue this project; for example, adding visuals and a manipulation check might do a better job of making the manipulation salient. I would like to have a manipulation check like “What is the difference between average and highly cost-effective charities?” And then set it up so that participants who get it wrong have to try again.
The fact that Donation Change differed significantly between Info groups does support that second main hypothesis, suggesting that CE info affects effective donations. This result, however, is not novel. So yes, the effect you picked up on is probably real – but this study was underpowered to detect it at a level of p<.05 (or even marginal significance).
In terms of CE info being ineffective, I’m thinking mainly about interest in EA – to which there really seems to be nothing going on, “There was no significant difference between the Info (M = 32.52, SD = 5.92) and No Info (M = 33.12, SD = 4.01) conditions, F(1, 40) = .118, p = .733, ηp2 = .003.” There isn’t even a trend in the expected direction. This was most important to me because, as far as I know, there is no previous empirical evidence to suggest that CE info affects interest in EA. It’s also more relevant to me as somebody running an EA group and trying to generate interest from people outside the group.
Thanks for your thorough comment! Yeah I was shooting for about 60 participants, but due to time constraints and this being a pilot study I only ended up with 44, so even more underpowered.
Intuitively I would expect a larger effect size, given that I don’t consider the manipulation to be particularly subtle; but yes, it was much subtler than it could have been. This is something I will definitely explore more if I continue this project; for example, adding visuals and a manipulation check might do a better job of making the manipulation salient. I would like to have a manipulation check like “What is the difference between average and highly cost-effective charities?” And then set it up so that participants who get it wrong have to try again.
The fact that Donation Change differed significantly between Info groups does support that second main hypothesis, suggesting that CE info affects effective donations. This result, however, is not novel. So yes, the effect you picked up on is probably real – but this study was underpowered to detect it at a level of p<.05 (or even marginal significance).
In terms of CE info being ineffective, I’m thinking mainly about interest in EA – to which there really seems to be nothing going on, “There was no significant difference between the Info (M = 32.52, SD = 5.92) and No Info (M = 33.12, SD = 4.01) conditions, F(1, 40) = .118, p = .733, ηp2 = .003.” There isn’t even a trend in the expected direction. This was most important to me because, as far as I know, there is no previous empirical evidence to suggest that CE info affects interest in EA. It’s also more relevant to me as somebody running an EA group and trying to generate interest from people outside the group.
Thanks again for your comment! Edit: Here’s the previous study suggesting CE info influences effective donations: http://journal.sjdm.org/20/200504/jdm200504.pdf