A slight issue here might be something of what I see as a category error—a ‘treatment resistant’ mental illness is usually not considered a distinct mental illness. For example, treatment resistant depression is depression, against which multiple treatments have been tried and have failed. There is a lot of traditional mental health research funding on severe mental illness, and a lot of funding is also geared towards trying to understand why some treatments don’t work for some people (i.e., some people’s mental illnesses resist treatment). So in this sense, I just don’t think it is neglected.
Jamie E
Karma: 1,996
Pulse Wave 2: Cause prioritization of the US public
First Embrace is a great name choice
I just think this example is misapplying the Shapley values/approach—in a case in which a payment is locked in, there is nothing in the Shapley approach saying you would treat it as if it is not locked in. As a matter of principle the Shapley values are order invariant, but that is not to say that when computing Shapley values you should imagine an alternative reality in which everything is order invariant when in fact it is not. As described I think the Shapley values in this case naturally argue for the second actor to favor getting the 15 units by completing the payment for P.
Here is how I would think this situation would be computed using Shapley values:
Whereas if player 2 were to aim for just going for their single better option, it would be:
So if player 2 seeks to maximise their Shapley value, they do in fact choose the option you think that they should.
So I think it is correct that one should consider if commitments are locked in etc., but I don’t think this is a critique of the Shapley approach, but rather that what is described as a ‘simple application of the Shapley values’ is just an incorrect application.
The cleaned data set was very nice to have access to and clear—the only thing that wasn’t clear to me was whether any exclusions were actually applied on the basis of the attention checks, and what the correct answers to the attention question were, but this may be in your documentation already, I just had a fairly quick look and downloaded the csv and got going.
Thanks for the thorough response -
indeed if the goal is about doing something to shift Chipotle/similar chains then the chicken reduction angle is unlikely to be persuasive.
Is there any way of finding out from real data whether people who literally wouldn’t go to chipotle started doing so when sofritas became a thing?
Fair enough
Agree with this. I could imagine that over the scale of something like Chipotle, it could be that satisfying the ‘every so often’ plant-based purchase, as opposed to meat, of reducetarians could be impactful and affect how much meat gets purchased overall, but it’s far from clear and not something we’d be powered to detect with all but the most elaborate experiments, most likely
That’s a good point, I actually had not considered people ordering online somehow...to the extent that the study was intending to represent an online experience then yes I consider it more ecologically than I first perceived it to be
Those studies don’t really convince me as I don’t think it’s possible to actually change what people perceive to be real norms (or basically their schemas of how the world is), or what their friends are saying and getting excited about, or the media is reporting on, which percolates organically and affects ones worldview, with small experimental manipulations, so to this extent I think the sorts of stuff I’m talking about are very hard experimentally. Even with an online menu I think people still arrive there having already been influenced by all sorts of things, I’m not referring to explicit advertising that would in the menu or in the store specifically. [edit: but I don’t think it is necessary to discuss further]
Thanks for sharing the data for the paper. Based on a fairly quick re-analysis, it appears that a decline in selection of chicken can be detected—but as you note, choice of alternative red meats tends to increase slightly, and some of the increase in selecting the new plant-based meat is driven by fewer people selecting the other veg-based options.
A few considerations as a result:
Could it still be good to reduce purchasing of chicken, given the relatively poorer conditions of chickens and greater animals per kg of meat slughtered to get chicken vs. beef?
Given some people who seemingly would have eaten the chicken switch to beef when a chicken alternative is on the menu, might some moral self deception be at play? People say they’ll eat veggie if only there is a replacement for X. When there is a replacement for X, they can’t continue eating X in good conscience but switch to Y, for which they can then say they would eat plant-based if only there were a plant-based Y alternative
I wonder if one could see if chicken increases when an explicit replacement for beef and steak were made (or is sofritas already explicitly intended as a replacement for one of these?)
I wonder if meat might ultimately decline if there were yet more plant-based explicit alternatives—what do you think?
It would be particularly interesting to know the ‘readiness to change’ or ‘intention/sympathy’ towards vegetarianism—I might expect more sympathetic/genuinely ready to change people really would try the alternative out as opposed to just switching to beef
Finally, what are your thoughts on the ecological validity of the experiment? Naturally, people can’t see or smell the options, as you note in the paper, but the experiment also takes place in the absence of repeat visits, advertising, and hearsay about how X or Y tastes really good, has to be tried etc. I’m not totally surprised that just adding an option to the end of a virtual menu doesn’t greatly shift behavior, but that also seems a slightly reductionistic model of what plant-based advocates might expect would happen in such a situation, which might be more to do with norm shifting, decreasing difficulty enabling people to switch over time etc. I hope that does not come across as overly critical, because I’m also really pleased to see a rigorous attempt to put ideas to the test and challenge assumptions. [edit: I haven’t thoroughly read you paper, so apologies if you already discuss this]
I can’t access to look at the specific studies but unfortunately most exercise science is like nutrition science generally (very hard to control, considerable participant variability) but with even smaller samples/statistical power, and insufficiently long periods to observe effects. They also rarely target vegetarian or vegans, so this combination is even less known about. The 1.6-2g/kg/day target is reasonable and supported, but there is an interesting body of research about BCAAs and in particular mechanistic work on the ‘leucine threshold’ for stimulating muscle protein synthesis, with suggestions to bump up leucine when taking plant-based proteins for maximal triggering of MPS. I’m not 100% convinced such work is correct, but enough to implement elements of it. Whether it is completely proven or not my concern raised above was that many plant-based advocates might not be aware that there is even something to consider beyond just protein or macros.
Happy to discuss further but cognizant of hijacking the discussion with something not actually related to the interesting report the authors provided!
Sure, I’m not making a strong claim that they consistently actually are giving smaller portions across stores but in my experience they often do. It may also have to do with how sofritas does not heap well on the spoon, relative to mounds of chicken or beef bits. The sofritas tray is also often almost empty, and maybe they don’t refill it so much due to not wanting it to end up going to waste—I’ve had multiple times where they just scrape some last bits out when they should refill it and get a proper spoonful. But apologies if this is turning into a chipotle review thread!
I agree that people have somewhat extreme beliefs about how much protein they need, but it’s going to be hard to tip that trend. I think plant-based advocates are also often ignorant though with respect to the optimal amino acid profiles of protein for fitness/muscle building purposes. BCAAs, for example, are often much lower than in animal proteins, even if the protein number is high. When people wade in but don’t know these things, it can invalidate their overall point.
[edit: The above relates to spaces in which I think the ‘more protein = better’ trend evolved from, but protein listed on everything has taken on a bit of a mind of its own. I’d be interested to know actually whether now there is just a generic perception that protein = healthy, in which case other healthiness factors could be somewhat fungible—just not for the sorts of people I was thinking of]
I’d be interested in research, or conducting research even, on people’s perceptions of these kinds of things if there were interest in it.
Yes I think Matt’s point is a specific example of what you might find if you split out (parcellate) the different forms of cost and assess how each is affected by the intervention. I was thinking, for example, that court or police costs are not divided by three by reducing the duration of the abuse by 3x, because they aren’t dealing with one third of the perpetrator.
I agree with Chris’s point here—I always go for sofritas but it’s very frustrating, they literally give you less than what you can see them serving with the meat options.
The protein ratio is also very important. For me—and I am confident for other consumers too—the protein is not fungible with other macros or nutritional content. One is trying to add protein to the meal, not to generically be ‘healthy’. I don’t think that pointing out the good fiber content, or increasing it, helps at all in this dimension.
US public attitudes towards artificial intelligence (Wave 2 of Pulse)
I think your assumptions around the costs and their even distributions over three years would need to be refined a bit more to work out more precisely the areas in which the proposed intervention would or would not reduce costs. To my reading of the research you cite here, the costs include things such as prevention programs and and police, court, and prison costs in dealing with the perpetrator etc., as opposed to just costs experienced by the victim (which I think you are right are also not just linearly experienced over the years). I therefore think it would be useful parcellate out more specifically which aspects of the costs you think would be reduced.
Thanks for sharing, this seems important and I’m surprised to hear that the pessimistic estimates of costs from these previous reports are not actually predictions
Hi Arthur—just to be clear I wouldn’t call any part of your post ‘lazy’!
Certainly a university group wouldn’t be actively speaking to every person on campus. We may just need to be very precise about what we mean, and perhaps even a single dimension of active vs. passive might not cut it. For example I’d say that flyering or mailing lists are more active forms of outreach/exposure than the type of exposure I’m imagining often happens in the general population (like just seeing a news article that mentions EA, possibly disparagingly), and they also offer a route to participation for those interested. Some other dimensions would likely be whether the context of the exposure presents EA as positive or negative, and whether it is coming from the EA community or simply something about EA.
We don’t have a direct assessment of ‘uptake rates’ although maybe, at least for the US, we could look at making something that at least sheds some light on it (e.g., looking at % hearing of EA in US adult demographics, vs. US respondent demographic % in the EA survey—though I think you are right that the EA survey definitely does not capture everyone and we don’t strictly know the composition of the ‘EA community’, unfortunately, which may limit the value of doing this). Of possible interest is that we observed in the most recent Pulse results that ‘elite’ universities in the US (~top 20 ranked) had higher awareness of EA than other universities. David may have more insight into other findings/data from the EA survey that would be of relevance in terms of outreach/exposure types.
Thanks Arthur for sharing your work on this, and backing it up with relevant numbers and data. Internally, we’ve used an app I developed to help try to get a handle on how levels of awareness in the population might translate to engaged members of the community and vice versa—you can take a look and see if it is in any way helpful in working with some of the numbers/estimates you make in this post.
It might also interest you to know that we have a more recent, more detailed and, we think, more accurate estimate of the number of people who’ve heard of EA in the US, which you can find here (though the app I’ve linked still uses the older numbers you’re referring to, as we made it around the same time that report was released).
I think your overall assessment is right that we shouldn’t expect a massive conversion rate for people merely hearing of EA to becoming highly engaged members of the EA community, and this can temper possible feelings of inadequacy among community builders. That being said, it’s important not to conflate ‘hearing of EA’ with ‘EA outreach’, and also to keep in mind the individual and social context in which outreach or hearing of EA is taking place.
While the percentage of the population who’ve potentially heard of EA is already rather low, the percentage who’ve been ‘reached out to’ is going to be lower still. Demographic features associated with liking EA are higher amongst university students than the general population. In addition, something like a ‘stages of change’ model likely applies, in which many people attending university may be actively seeking something to dedicate themselves to, or at least actively seeking group or club membership, and are therefore more responsive than gen pop. Finally, the sort of exposure to EA that might occur as a result of a university group outreach/presence on campus and fairs is likely very different and more active than potentially passive exposure that might occur more broadly. These things come together to make university outreach quite different from a random member of the population just coming across EA and deciding to get involved.
I mention these things because even considering—as you rightly do—the ability of those doing the outreach or people at university having more time or resources, the starting conversion rate of .0068% might plausibly make university outreach seem somewhat hopeless or useless—I’d think you can substantially beat those odds derived just from the gen pop!
[the screenshot is just an example of some of the kinds of output from the app, not substantively anything to consider in terms of the numbers]
Sure, that’s a nice/other way of looking at the relationship that is informative, we might include this in the final report. You can see for the permissive assessment its the majority of those with a negative view being aware of FTX.
I think it’s important to note that this doesn’t mean that ‘if more people who currently hold positive views of EA were aware of FTX then they would have negative views’ - because it’s possible that for some or many of those negative views, perhaps the only or primary way in which they’ve been exposed to the idea of EA is through the lens of FTX.
Thanks, yes that works now
Wave 2 of Pulse: Awareness and perceptions of Effective Altruism
‘even without taking into account other possible benefits of unconditional cash transfers (such as consumption gains)’
Presumably we’ll know from the study when it comes out, which will be exciting, but if mother’s are spending the money on services related to making their pregnancy/delivery safer, will they also receive the same amount of consumption gains? - I’d think these more general gains would be lower unless the pregnancy related costs are a lot less than the value of the transfer.
Hi Vasco, it’s an interesting though not simple question, as we’re currently in the process of looking into how to understand possible impacts of research projects in general (or rather, specific projects/pieces of research, but not currently specifically/just for Pulse).
Each time we’ve ran Pulse, we’ve had a handful of specific requests from impact-focused funders or charities for items to include and received positive feedback about the value of the information for them when we’ve done so. To some degree when we do that, we are trusting their judgment as to the value of what they wanted to know about, not explicitly modelling it.
I’ve also seen some Pulse findings in wider discussions in some more and some less formal outlets over time, which suggests some interest and traction—this has perhaps been more the case with some of the AI-related findings. I think providing such information so that people are better informed in what they’re discussing is valuable.
The general awareness findings were, I believe, something of an update for people/orgs in understanding how much reach EA actually has so far (including at RP), and I believe such information—naturally amongst many other pieces—has been at least discussed when talking about field building for EA or for different particular causes, and possible communication strategies. It could be that the continued polling there is less of an update each time, but for a lot of the information I think that the value can increase as we track opinions and awareness over time, such that some of the knowledge and possibly action relevant information we gain will be realised by understanding trends and how certain events or developments do—or surprisingly do not—affect public perceptions and concerns over different causes.
A less tangible impact that I think could arise over time is a generally better sense amongst decisions makers of what and how people outside of the ‘EA community’ actually think about the things people closer to the community care about. Whilst EA isn’t for everyone, I still think it is important to have a good mental model and ideally an empirically informed understanding of where the general public is on various issues, and contributing to that understanding and awareness is also important.