jonathonsmith

TLYCS Pam­phlet Pilot Results

MFA Ad Study Tar­get­ing Former Vegetarians

• jonathonsmith 17 Dec 2015 18:35 UTC

  6 points

  0 ∶ 0

  on: Effec­tive Altru­ism Outreach win­ter fundraiser

  A quick note on what encouraged me to donate to EAO.

  I navigate robotic spacecraft to destinations in deep space at JPL. If you’re trying to get somewhere like Jupiter or Saturn, the most important course corrections you can make are right after launch. We always have a crack team of analysts closely monitoring a spacecraft just after it leaves Earth, because the energy required to change the spacecraft’s heading grows exponentially with time; point in the wrong direction too long and the mission is lost.

  EA is moving really, really fast, and small adjustments to its development now are likely to have huge consequences down the road. With EAO, we have a team of talented people focused on nothing but making sure it’s heading in the right direction. They are doing a lot of really impressive, concrete work (like book promotion, EAG, VIP outreach etc), but I think the greatest value in keeping them well funded is to have a vigilant eye watching for obstacles and helping navigate them at this very important, early stage of the movement.

TLYCS Pam­phlet­ing Pilot Program

TLYCS Pam­phlet­ing Pilot Plan

• jonathonsmith 11 May 2016 3:01 UTC

  3 points

  0 ∶ 0

  on: MFA Ad Study Tar­get­ing Former Vegetarians

  Lot of great points here and angles to follow up on.

  I do want to re-assert that enormous 80% recidivism rate, and how strong these (very valid) counter-narratives would have to be to defeat our original assumptions. Consider:

  • A huge number of people would have to actively curate their Facebook profiles to give current-vegetarians the majority in these groups.

  • A curiously large number of current-vegetarians would have to click on an ad that claims to “help you eat vegetarian again” and order a “Vegetarian Starter Guide” to make the majority of the respondents non-recidivists.

  Again, there have been a lot of great points raised and I’m tempted now to fund a follow up implementing some of these fixes. At the same time, I think it’s important to step back and think about how strong these influences would have to be to overcome that monster 80%.

• jonathonsmith 10 May 2016 15:55 UTC

  3 points

  0 ∶ 0

  in reply to: redmoonsoaring’s comment on: MFA Ad Study Tar­get­ing Former Vegetarians

  It’s definitely possible that some current vegetarians might have requested the the Vegetarian Starter Guide. A follow up study could probably parse out this variable by having a simple required question for obtaining the VSG (along with their email) asking if they are currently veg, former veg and interested in trying again, or never been vegetarian at all.

• jonathonsmith 24 Oct 2015 22:45 UTC

  3 points

  0 ∶ 0

  on: Don’t sweat diet?

  I find these discussions of moral offsets somewhat disturbing, re: Tom_Davidson’s third point. Can we host a dog fighting ring at EA Global next year as long as half the buy-in goes to the Humane League? Can we get trafficked children to cook our food as long as we give a nice plump sum to SCI?

  I think the analysis is fine, and it’s good to know the real impact of certain actions (like going vegan). But then to take it a step further and say, well, I can just skip acting morally in this case and offset that with a donation seems to miss the mark. How far are we willing to go, as a community, down this road, and where do we draw the line?

The term “Ve­gan” needs to evolve

• jonathonsmith 28 May 2016 18:02 UTC

  2 points

  0 ∶ 0

  on: More Thoughts (and Anal­y­sis) on the Mercy For An­i­mals On­line Ads Study

  Vegan Outreach ran its first annual Leafletting Effectiveness Survey (LES) last fall and we had a dismal response rate as well (around 2%). We were offering $5 incentives for people to take a 2-part survey, where Part 1 was filled out immediately and then an email was sent out two months later to complete Part 2 and claim their gift card. We’ve been running small response rate studies since then to figure out what kind of incentives we need to hit our targets, but we’re seeing significant variation based on what city /​ state we’re operating in. This is making it really difficult to find one incentive level to rule them all.

  I wonder if you’ve looked at the geographical distribution of where your 2% came from? And do you have any theories why your actual response rate differed from your pilot response rate?

• jonathonsmith 10 May 2016 1:35 UTC

  2 points

  0 ∶ 0

  in reply to: Dawn Drescher’s comment on: MFA Ad Study Tar­get­ing Former Vegetarians

  Hey guys, sorry I didn’t realize this had posted. I was still editing the draft and just now finished cleaning it up only to find it went live last night (damn!). Here are some of the numbers (extracted from an email Alan sent me summarizing the results):

  “To get one person in the US to pledge to go vegetarian and order a VSG it usually takes $5-$8 but in this trial we were able to get VSG orders for an average of only $2.65. This means that 2-3x more people ordered a VSG than if we were to target the general US population with the same amount of advertising dollars...Below are the detailed results of the ads that we ran. The two most important stats for us are (1) 756 people ordered a VSG and were subscribed to our 12-part email series about eating veg, and (2) 157,512 people saw one of our pro-veg ads in their Facebook Newsfeed...

  • 157,512 people saw one of our pro-veg ads

  • 1,571 people clicked on the ad and read more about eating vegetarian on our landing page

  • 756 people ordered a Veg Starter Guide and were subscribed to our 12-part email series about eating veg

  • 941 people liked MFA on Facebook and will now see our posts about veg eating

  • 286 people shared one of the ads reaching even more people”

• jonathonsmith 27 Jul 2015 18:34 UTC

  2 points

  0 ∶ 0

  in reply to: Michelle_Hutchinson’s comment on: TLYCS Pam­phlet Pilot Results

  Yeah, one of the main “unmeasurables” of the pilot was how much more of an impression a pamphlet, handed out by a real volunteer, can have on someone over an online add. This goes into the “touchpoint” theory, where it may take multiple exposures to an idea before someone bites; and maybe getting a pamphlet is a much stronger “touchpoint” than seeing an online add. I personally think that getting handed a pamphlet by a non-paid volunteer is a really powerful thing; and that’s the reason that (again personally) I think that it would be great to see someone in the EA movement give a pamphleting program another shot, along the lines I sketched in the lessons learned (branded on EA, including multiple organizations, with a stronger, repeated ask).

  We ran cost numbers including volunteer time and pamphlet development expenses, but I decided to ignore those for the final writeup, mainly because; the way we envisioned the program, the leaflets would be handed out on a volunteer basis by college students not paid TLYCS staff; and the cost of developing the pamphlets would amortize to a minimal effect over the lifetime of an ongoing pamphleting program. It’s definitely a debatable decision though.

• jonathonsmith 6 Feb 2015 5:17 UTC

  2 points

  0 ∶ 0

  in reply to: jonathonsmith’s comment on: TLYCS Pam­phlet­ing Pilot Plan

  Actually it does appear you can hire a statistician like a lawyer or accountant, I’ll be damned lol. I just typed “statistical consultant” into Google and got like a million hits. I would love a personal recommendation if you have one though.

• jonathonsmith 28 May 2016 17:38 UTC

  1 point

  0 ∶ 0

  in reply to: zdgroff’s comment on: More Thoughts (and Anal­y­sis) on the Mercy For An­i­mals On­line Ads Study

  As I’ve said elsewhere, I’m skeptical that the approach to take is to do more such RCTs. I worry about us having to spend extremely large sums of money for such things.

  It’s probably a good idea to consider the global amount of money being spent on an AR intervention when evaluating the cost to investigate it. Like how much money is being spent across the different AR orgs on FB ads? If a proper study costs $200K and there is only $500K a year being spent globally, then it’s hard to see the value proposition. If the total being spent annually is $20M, then a full fledged RCT is probably in order.

  Does anyone know of estimates of how much the AR movement as a whole is investing in different interventions? This might help prioritize which interventions to study first and how much to pay for those studies.

• jonathonsmith 12 Sep 2015 17:26 UTC

  1 point

  0 ∶ 0

  on: A Defense of Normality

  Great article Eric. I had similar thoughts at EA Global this year. The common tagline for EA that I heard repeated was “1.) Find out how to do the most good, 2.) Do it”, and while I agree with this sentiment, I would add a couple un-sexy modifiers; “1.) Find out how to do the most good with the time and money you have decided to donate, 2.) Do it.”

  I really appreciate and look up to the people who are all-in and looking to maximize the good they can do with their lives. One of the reasons I like to go to EA events is to have my own altruistic scale recalibrated. The ten percent of money and time I donate make me pretty much a saint in my friend group, but just on the lower end of decent in the EA community. And I see this as a good thing, and I think the EA community needs these torchbearers to help motivate and remind everyone that we could be doing more.

  Ultimately, I see the future of EA along the lines that Beth Barnes described in her (I think amazing) TED Talk (https://​​www.youtube.com/​​watch?v=LtWINl3C_7s). There is a central hub of leaders keeping the philosophical flame, running excellent organizations, etc, and then a much larger group that live their everyday lives as teacher and engineers and barista’s, but that donate a minimally decent amount (10%?) to help attack whatever the most pressing problem in the world happens to be, indefinitely. I think this is a vision that is realistic and achievable, that scales well from the small community we have today to the population at large, and that would just totally transform the world into a much better place.

• jonathonsmith 27 Jul 2015 18:19 UTC

  1 point

  0 ∶ 0

  in reply to: zdgroff’s comment on: TLYCS Pam­phlet Pilot Results

  Unfortunately, we could only track website visitors as far as clicking a “donate” button on TLYCS “Where To Donate” page (http://​​www.thelifeyoucansave.org/​​Where-to-Donate). After clicking the button they are directed to the individual charity website, so we don’t have visibility into if they actually followed through with making a donation, or how much that donation was.

  That being said, we only had one visitor that plausibly came from the pamphlets that clicked a donate button; if we had paid closer to Vegan Outreach prices for the pamphlets ($0.07), the cost of the 3500 pamphlets would have been about $245, and so the donation would have had to be on that level to get parity. Would definitely have been an interesting data point to see if /​ how much our clicker actually donated.

• jonathonsmith 6 Feb 2015 4:54 UTC

  1 point

  0 ∶ 0

  in reply to: Bernadette_Young’s comment on: TLYCS Pam­phlet­ing Pilot Plan

  Thanks Bernadette! Other people have suggested consulting a statistician, but it’s not been clear to me precisely what she is supposed to do. I went through some lengths to be as specific as possible in our plan about what our data is, what we expect to see, and how we plan to calculate our success criteria (e.g. creating dummy numbers and producing working code that runs through our calculations). Can you maybe poke some holes in our approach so that I get a better idea of what a statistician would be bringing to the table?

  Also, do you know how one goes about finding a statistician? I assume there aren’t just statisticians around for hire, like lawyers or accountants. This came up in a previous thread and someone mentioned Statistics Without Borders and Charity Science, but it didn’t seem like these organizations offered this as a service, I would just be cold-calling and asking for their help. If you have someone particular in mind who would be qualified to do this, I’d love to get their contact info; at least then I could get a cost estimate to take back to TLYCS.

• jonathonsmith 9 Jan 2015 17:48 UTC

  1 point

  0 ∶ 0

  in reply to: Owen Cotton-Barratt’s comment on: TLYCS Pam­phlet­ing Pilot Program

  This is a really good point. Yeah, the scaling model is to have local TLYCS chapters organizing volunteers to do this as a regular, rolling semester activity. I hadn’t really considered myself a confounding variable in this sense, because I’m definitely not a master pamphleteer. I’m an engineer by trade, and if this program takes off, I’ll eventually just be another volunteer in the LA area that helps hand out leaflets occasionally. We’re also thinking about splitting crews on Friday distribution days—so I would have a crew that hits up two universities, and there would be another volunteer crew hitting up two different campuses. Any thoughts on this?

• jonathonsmith 9 Jan 2015 17:38 UTC

  1 point

  0 ∶ 0

  in reply to: Tom_Ash’s comment on: TLYCS Pam­phlet­ing Pilot Program

  That is definitely the intention. We are really hoping that the data we gather will be useful to other orgs considering a similar program, which was part of the motivation for posting up here ahead of time to get feedback.

• jonathonsmith 11 May 2016 2:12 UTC

  0 points

  0 ∶ 0

  in reply to: zdgroff’s comment on: MFA Ad Study Tar­get­ing Former Vegetarians

  I think I follow you here. Facebook’s ad engine is more likely to target people that have liked several of our selected groups /​ terms. Liking multiple groups indicates stronger identification with being veg, so these people are more likely to still be veg, as opposed to people who liked one but not the others. Is that right?