Hi Paul, thanks for checking the analysis so closely! (And apologies for the slow reply; I’ve been gathering some more information.)
But wouldn’t Impossible be a comparison for ground beef, not for steak? Am I misunderstanding something here?
This is a good point and I’ve now confirmed with the authors that the steak was cubed, rather than minced or ground, so indeed not likely directly comparable to Impossible ground beef. I’ll be making some updates to the paper accordingly. Thank you!
The build-your-own-entree bar offers shredded beef, which while also not the same, might be a more similar comparison. Unfortunately, I wasn’t able to get more granular data at this time to test whether that was more readily displaced. Overall, despite these caveats on taste, lots of plant-based meat was still sold, so it was “good enough” in some sense, but there was still potentially little resulting displacement of beef (although maybe somewhat more of chicken).
I don’t think it’s really equivalent on taste
Yes, I’m not entirely certain Impossible meat is equivalent in taste to animal-based ground beef. However, I do find the evidence I cite in the second paragraph of this section somewhat compelling.
If I’m understanding you correctly then 22% of the people previously eating steak burritos switched to Impossible burritos
I’m not sure where you’re getting this exact figure, but I don’t put much credence in it. Instead, I’d refer to estimates in Fig 3, which range from 0.3 to 4.0 percentage points of beef displacement, after accounting for behavior at the control sites and/or spillover effects. That is compared to an 5.0 or 11.4 pp increase in Impossible meal sales, respectively.
Furthermore, it’s important to keep in mind “the study employed several co-interventions designed to reduce meat consumption (Malan, 2020). These included environmental education, low carbon footprint labels on menus, and an advertising campaign to promote the new product, all of which have some evidence demonstrating their effectiveness.” So the effect is likely not entirely attributable to the Impossible meat.
even if you did have taste equivalence here I wouldn’t expect people’s decisions to be perfectly informed by that fact
I agree and discuss this issue some in the Taste section. In short, this is part of why I think informed taste tests would be more relevant than blind: in naturalistic settings, it is possible that people would report not liking the taste of PBM even though it passes a blind taste test. So I think this accurately reflects what we should expect in practice.
in your first deployment
In this case, ~32% of (surveyed) participants answered yes to “Have you tried the Impossible™ burger, Beyond Meat™, or similar products anywhere other than [the intervention site]?” (Table 19) Note the study was conducted in the Fall of 2019, right after the summer Impossible launched in Burger King. Furthermore, the study covers 10 weeks and 71% of participants who ever selected the Impossible product at the intervention site went on to select it again. So I wouldn’t think of this as a first deployment: many students were familiar with these sorts of products already and took multiple opportunities to try the product over two and a half months. (These figures are based on a (small) survey of 200 participants, 96 of whom self-reported ever selecting Impossible products at the intervention site.)
Yes, I’m not entirely certain Impossible meat is equivalent in taste to animal-based ground beef. However, I do find the evidence I cite in the second paragraph of this section somewhat compelling.
Are you referring to the blind taste test? It seems like that’s the only direct evidence on this question.
It doesn’t look like the preparations are necessarily analogous. At a minimum the plant burger had 6x more salt. All burgers were served with a “pinch” of salt but it’s hard to know what that means, and in any case the plant burger probably ended up at least 2x as salty.[1] You note this as a complicating factor, but salt has a huge impact on taste and it seems to me like it can easily dominate the results of a 2-3 bite taste test between vaguely comparable foods.
I also have no idea at all how good or bad the comparison burger was. Food varies a lot. (It’s kind of coincidental the salt happened to show up in the nutrition information—otherwise I wouldn’t even be able to make this concrete criticism). It seems really hard to draw conclusions about taste competitiveness of a meat substitute from this kind of n=1 study, beyond saying that you are in the same vague zone.
Have you compared these foods yourself? I eat both of them regularly. Taste competitiveness seemed plausible the first time I ate impossible ground beef, but at this point the difference feels obviously large. I seriously doubt that the typical omnivore would consider them equivalent after eating them a few times.
Overall, despite these caveats on taste, lots of plant-based meat was still sold, so it was “good enough” in some sense, but there was still potentially little resulting displacement of beef (although maybe somewhat more of chicken).
My conclusion would be: plant substitutes are good enough that some people will eat them, but bad enough that some people won’t. They are better than some foods and worse than others.
It feels like you are simultaneously arguing that high uptake is a sign that taste is “good enough,” and that low uptake is a sign that “good enough” taste isn’t sufficient to replace meat. I don’t think you can have it both ways, it’s not like there is a “good enough” threshold where sales jump up to the same level as if you had competitive taste. Better taste just continuously helps with sales.
I agree and discuss this issue some in the Taste section. In short, this is part of why I think informed taste tests would be more relevant than blind: in naturalistic settings, it is possible that people would report not liking the taste of PBM even though it passes a blind taste test. So I think this accurately reflects what we should expect in practice.
I disagree. Right now I think that plant-based meat substitutes have a reputation as tasting worse than meat largely because they actually taste worse. People also have memories of disliking previous plant-based substitutes they tried. In the past the gap was even larger and there is inertia in both of these.
If you had taste competitive substitutes, then I think their reputation and perception would likely improve over time. That might be wrong, but I don’t see any evidence here against the common-sense story.
The plant burger had about 330mg vs 66mg of salt. If a “pinch” is 200mg then it would end up exactly 2x as salty. But hard to know exactly what a pinch means, and also it matters if you cook salt into the beef or put a pinch on top, and so on.
Yes. The Sogari blind taste test is indeed affected by saltiness; it also includes an informed taste test similarly effected (but again finding Impossible and animal-based meat tied for first). There is a second blind taste test cited immediately thereafter (Chicken and Burger Alternatives, 2018), although salt levels were not reported.
Have you compared these foods yourself?
No, I haven’t.
It seems really hard to draw conclusions about taste competitiveness of a meat substitute from this kind of n=1 study, beyond saying that you are in the same vague zone.
I agree, food is varied and such comparisons are hard—that’s part of why I argue we should do more taste tests! Can you clarify what you mean by an N of 1 study, as usually this refers to a study with a single participants, but Sogari indeed had many participants. If you’re suggesting comparison against multiple burgers, this gets a bit tricky since one has to decide which burger you actually want to be equivalent to, if that’s your goal.
I disagree. Right now I think that plant-based meat substitutes have a reputation as tasting worse than meat largely because they actually taste worse.
Can you clarify what specifically you disagree with here? I don’t think I especially disagree with anything you wrote that follows from here. Instead, I think it’s indeed perception of taste that matters for the impact of PBM and we can likely best measure that perception with informed, rather than blind, taste tests. Overall, as I write, I think actually operationalizing a taste test to identify whether “taste competitiveness” is obtained is non-trivial. The literature so far neglects such operationalizations. What do you have in mind as an ideal experiment to conduct to measure taste competitiveness?
Hi Paul, thanks for checking the analysis so closely! (And apologies for the slow reply; I’ve been gathering some more information.)
This is a good point and I’ve now confirmed with the authors that the steak was cubed, rather than minced or ground, so indeed not likely directly comparable to Impossible ground beef. I’ll be making some updates to the paper accordingly. Thank you!
The build-your-own-entree bar offers shredded beef, which while also not the same, might be a more similar comparison. Unfortunately, I wasn’t able to get more granular data at this time to test whether that was more readily displaced. Overall, despite these caveats on taste, lots of plant-based meat was still sold, so it was “good enough” in some sense, but there was still potentially little resulting displacement of beef (although maybe somewhat more of chicken).
Yes, I’m not entirely certain Impossible meat is equivalent in taste to animal-based ground beef. However, I do find the evidence I cite in the second paragraph of this section somewhat compelling.
I’m not sure where you’re getting this exact figure, but I don’t put much credence in it. Instead, I’d refer to estimates in Fig 3, which range from 0.3 to 4.0 percentage points of beef displacement, after accounting for behavior at the control sites and/or spillover effects. That is compared to an 5.0 or 11.4 pp increase in Impossible meal sales, respectively.
Furthermore, it’s important to keep in mind “the study employed several co-interventions designed to reduce meat consumption (Malan, 2020). These included environmental education, low carbon footprint labels on menus, and an advertising campaign to promote the new product, all of which have some evidence demonstrating their effectiveness.” So the effect is likely not entirely attributable to the Impossible meat.
I agree and discuss this issue some in the Taste section. In short, this is part of why I think informed taste tests would be more relevant than blind: in naturalistic settings, it is possible that people would report not liking the taste of PBM even though it passes a blind taste test. So I think this accurately reflects what we should expect in practice.
In this case, ~32% of (surveyed) participants answered yes to “Have you tried the Impossible™ burger, Beyond Meat™, or similar products anywhere other than [the intervention site]?” (Table 19) Note the study was conducted in the Fall of 2019, right after the summer Impossible launched in Burger King. Furthermore, the study covers 10 weeks and 71% of participants who ever selected the Impossible product at the intervention site went on to select it again. So I wouldn’t think of this as a first deployment: many students were familiar with these sorts of products already and took multiple opportunities to try the product over two and a half months. (These figures are based on a (small) survey of 200 participants, 96 of whom self-reported ever selecting Impossible products at the intervention site.)
Are you referring to the blind taste test? It seems like that’s the only direct evidence on this question.
It doesn’t look like the preparations are necessarily analogous. At a minimum the plant burger had 6x more salt. All burgers were served with a “pinch” of salt but it’s hard to know what that means, and in any case the plant burger probably ended up at least 2x as salty.[1] You note this as a complicating factor, but salt has a huge impact on taste and it seems to me like it can easily dominate the results of a 2-3 bite taste test between vaguely comparable foods.
I also have no idea at all how good or bad the comparison burger was. Food varies a lot. (It’s kind of coincidental the salt happened to show up in the nutrition information—otherwise I wouldn’t even be able to make this concrete criticism). It seems really hard to draw conclusions about taste competitiveness of a meat substitute from this kind of n=1 study, beyond saying that you are in the same vague zone.
Have you compared these foods yourself? I eat both of them regularly. Taste competitiveness seemed plausible the first time I ate impossible ground beef, but at this point the difference feels obviously large. I seriously doubt that the typical omnivore would consider them equivalent after eating them a few times.
My conclusion would be: plant substitutes are good enough that some people will eat them, but bad enough that some people won’t. They are better than some foods and worse than others.
It feels like you are simultaneously arguing that high uptake is a sign that taste is “good enough,” and that low uptake is a sign that “good enough” taste isn’t sufficient to replace meat. I don’t think you can have it both ways, it’s not like there is a “good enough” threshold where sales jump up to the same level as if you had competitive taste. Better taste just continuously helps with sales.
I disagree. Right now I think that plant-based meat substitutes have a reputation as tasting worse than meat largely because they actually taste worse. People also have memories of disliking previous plant-based substitutes they tried. In the past the gap was even larger and there is inertia in both of these.
If you had taste competitive substitutes, then I think their reputation and perception would likely improve over time. That might be wrong, but I don’t see any evidence here against the common-sense story.
The plant burger had about 330mg vs 66mg of salt. If a “pinch” is 200mg then it would end up exactly 2x as salty. But hard to know exactly what a pinch means, and also it matters if you cook salt into the beef or put a pinch on top, and so on.
Yes. The Sogari blind taste test is indeed affected by saltiness; it also includes an informed taste test similarly effected (but again finding Impossible and animal-based meat tied for first). There is a second blind taste test cited immediately thereafter (Chicken and Burger Alternatives, 2018), although salt levels were not reported.
No, I haven’t.
I agree, food is varied and such comparisons are hard—that’s part of why I argue we should do more taste tests! Can you clarify what you mean by an N of 1 study, as usually this refers to a study with a single participants, but Sogari indeed had many participants. If you’re suggesting comparison against multiple burgers, this gets a bit tricky since one has to decide which burger you actually want to be equivalent to, if that’s your goal.
Can you clarify what specifically you disagree with here? I don’t think I especially disagree with anything you wrote that follows from here. Instead, I think it’s indeed perception of taste that matters for the impact of PBM and we can likely best measure that perception with informed, rather than blind, taste tests. Overall, as I write, I think actually operationalizing a taste test to identify whether “taste competitiveness” is obtained is non-trivial. The literature so far neglects such operationalizations. What do you have in mind as an ideal experiment to conduct to measure taste competitiveness?