MMathur comments on Environmental & health appeals are the most effective vegan outreach strategies

[MMathur]

Hi Seth,

Thanks so much for the thoughtful and interesting response, and I’m honored to hear that the 2021 papers helped lead into this. Cumulative science at work!

1. I fully agree. Our study was at best comparing a measure with presumably less social desirability bias to one with presumably more, and lacked any gold-standard benchmark. In any case, it was also only one particular intervention and setting. I think your proposed exercise of coding attitude and intention measures for each study would be very valuable. A while back, we had tossed around some similar ideas in my lab. I’d be happy to chat offline about how we could try to help support you in this project, if that would be helpful.

2. Makes sense.

3. For binary outcomes, yes, I think your analog to delta is reasonable. Often these proportion-involving estimates are not normal across studies, but that’s easy enough to deal with using robust meta-analysis or log-transforms, etc. I guess you approximated the variance of this estimate with the delta method or similar, which makes sense. For continuous outcomes, this actually was the case I was referring to (a binary treatment X and continuous outcome Y), since that is the setting where the d-to-r conversion I cited holds. Below is an MWE in R, and please do let me know if I’ve misinterpreted what you were proposing. I hope not to give the impression of harping on a very minor point – again, I found your analysis very thoughtful and rigorous throughout; I’m just indulging a personal interest in effect-size conversions.

Thanks again, Seth!

Maya

library(dplyr)

# sample size

N = 10^5

# population parameter

delta = .3

# assume same SD conditional on X=0 and X=1 so that Glass = Cohen

sd.within = .5

# E[Y | X=0] and E[Y | X=1]

m0 = .5

m1 = m0 + delta*sd.within

# generate data

d = data.frame( X = c( rep(0, N/​2), rep(1, N/​2) ) )

d$Y[ d$X == 0 ] = rnorm(n = sum(d$X == 0), mean = m0, sd = sd.within )

d$Y[ d$X == 1 ] = rnorm(n = sum(d$X == 1), mean = m1, sd = sd.within )

# sanity check

d %>% group_by(X) %>%

summarise( mean(Y) )

hist(d$Y)

# check the conversion

r = cor(d$Y, d$X)

(2*r) /​ sqrt(1 - r^2); delta

# proportion of variance explained

r^2

# percent reduction in E[Y] itself

# note this is not 1-1 with delta; change m0 and this will change

100*( abs(m1 - m0) /​ m0 )

[Seth Ariel Green]

You are 100% right about this, my mistake. First, I read your first comment too fast (I placed ‘binary’ on the wrong side of the equation, as you noticed), and second, I think that the original paragraph confuses percentage change with percentile change. I removed the section.

I still want the final draft to present some intuitive, drawing-on-stats-that-we-learned-in-HS way to put standardized mean effect sizes into impact estimate terms, but I think we need to think more about this.

Thanks for engaging! FWIW I ran through your code and everything makes sense to me

[MMathur]

No worries. Effect-size conversions are very confusing. Thanks for doing this important project and for the exchange!