In general, as an economist friend put it, âChanging options is a very strong intervention, like mechanically there should be an effect.â So I would expect a new meat optionâBBQ chicken or whateverâto attract customers. But you are right, we donât know that. On the other hand, our question was whether adding a chicken analogue would attract customers away from meat-based options, so whether a meat option would have also attracted customers is not really apropos of our estimand. It might help put our results in context, but itâs not the theoretical quantity weâre after. And thereâs a lot to be said for keeping a study focused. Another thing manipulated means either a smaller sample per treatment arm or a more expensive experiment. Always we are triaging.
That would have been a fine thing to check, but in the online ordering context we were trying to simulate, you also view the options without necessarily âtaking the time to read the small grey on white text,â so if they miss the new option, thatâs experimental realism. Also, as Lewis points out, we have reason to think our numbers are broadly in line with what people are actually ordering, which is some evidence that people were actually reading. However it might be interesting to do a follow-up where someone actively promotes the new PMA, which restaurants sometimes do.
I appreciate the thoughtful reply. My view hasnât changed to much, but I have updated towards more uncertainty.
Regards (2) however, I think the text being subtle is the opposite of âexperimental realismâ as when businesses introduce a new product they often give it a graphic treatment that highlights it and explain its value to customers.
Also, regarding Lewisâ comment, I donât know how much a figure from 2015, a decade ago, when there was far less familiarity and knowledge of PMA can be regarded as converging evidence for your outcome.
Hi Dorsal, good questions:
In general, as an economist friend put it, âChanging options is a very strong intervention, like mechanically there should be an effect.â So I would expect a new meat optionâBBQ chicken or whateverâto attract customers. But you are right, we donât know that. On the other hand, our question was whether adding a chicken analogue would attract customers away from meat-based options, so whether a meat option would have also attracted customers is not really apropos of our estimand. It might help put our results in context, but itâs not the theoretical quantity weâre after. And thereâs a lot to be said for keeping a study focused. Another thing manipulated means either a smaller sample per treatment arm or a more expensive experiment. Always we are triaging.
That would have been a fine thing to check, but in the online ordering context we were trying to simulate, you also view the options without necessarily âtaking the time to read the small grey on white text,â so if they miss the new option, thatâs experimental realism. Also, as Lewis points out, we have reason to think our numbers are broadly in line with what people are actually ordering, which is some evidence that people were actually reading. However it might be interesting to do a follow-up where someone actively promotes the new PMA, which restaurants sometimes do.
I appreciate the thoughtful reply. My view hasnât changed to much, but I have updated towards more uncertainty.
Regards (2) however, I think the text being subtle is the opposite of âexperimental realismâ as when businesses introduce a new product they often give it a graphic treatment that highlights it and explain its value to customers.
Also, regarding Lewisâ comment, I donât know how much a figure from 2015, a decade ago, when there was far less familiarity and knowledge of PMA can be regarded as converging evidence for your outcome.
Excited to more research in this area! : )