I buy two of your examples: in the case of masks, it seems clear now that the experts were wrong before, and in âFirst doses firstâ, you present some new evidence that the priors were right.
On nutrition and lockdowns, you havenât convinced me that the point of view youâre defending isnât the one that deference would arrive at anyway: it seems to me like the expert consensus is that lockdowns work and most nutritional fads are ignorable.
On minimum wage and alcohol during pregnancy, youâve presented a conflict between evidence and priors, but I donât feel like you resolved the conflict: someone who believed the evidence proved the priors wrong wonât find anything in your examples to change their minds. For drinking during pregnancy, Iâm not even really convinced there is a conflict: I suspect the heart of the matter is what people mean by âsafeâ, what risks or harms are small enough to be ignored.
I think in general there are for sure some cases where priors should be given more weight than theyâre currently afforded. But it also seems like there are often cases where intuitions are bad, where âitâs more complicated than thatâ tends to dominate, where there are always more considerations or open uncertainties than one can adequately navigate on priors alone. I donât think this post helps me understand how to distinguish between those cases.
Hello, my argument was that there are certain groups of experts you can ignore or put less weight on because they have the wrong epistemology. I agree that the median expert might have got some of these cases right. (Iâm not sure thatâs true in the case of nutrition however)
The point in all these cases re priors is that one should have a very strong prior, which will not be shifted much by flawed empirical research. One should have a strong prior that the efficacy of the vaccine wonât drop off massively for the over 65s even before this is studied.
One can see the priors vs evidence case for the minimum wage more formally using Bayes theorem. Suppose my prior that minimum wages reduce demand for labour is 98%, which is reasonable. I then learn that one observational study has found that they have no effect on demand for labour. Given the flaws in empirical research, letâs say there is a 30% chance of a study finding no effect conditional on there being an effect. Given this, we might put a symmetrical probability on a study finding no effect conditional on there being no effect - a 70% chance of a null result if minimum wages in fact have no effect.
Then my posterior is = (.3*.98)/â(.3*98+.7*.02) = 95.5%
So I am still very sure that minimum wages have no effect even if there is one study showing the contrary. FWIW, my reading of the evidence is that most studies do find an effect on demand for labour, so after assimilating it all, one would probably end up where oneâs prior was. This is why the value of information of research into the minimum wage is so low.
On drinking in pregnancy, I donât think this is driven by peopleâs view of acceptable risk, but rather by a myopic empiricist view of the world. Osterâs book is the go-to for data-driven parents and she claims that small amounts of alcohol has no effect, not that it has a small effect but is worth the risk. (Incidentally, the latter claim is also clearly falseâit obviously isnât worth the risk.)
On your final point, I donât think one can or should aim to give an account of whether relying on theory or common sense is always the right thing to do. I have highlighted some examples where failure to rely on theory and evidence from other domains leads people astray. Epistemology is complicated and this insight may of course not be true in all domains. For a comprehensive account of how to approach cases such as these, one cannot say much more than that the true theory of epistemology is Bayesianism and to apply that properly you need to be apprised of all of the relevant information in different fields.
Bluntly I think a prior of 98% is extremely unreasonable. I think that someone who had thoroughly studied the theory, all credible counterarguments against it, had long discussions about it with experts who disagreed, etc. could reasonably come to a belief that strong. An amateur who has undertaken a simplistic study of the basic elements of the situation canât IMO reasonably conclude that all the rest of that thought and debate would have a <2% chance of changing their mind.
Even in an extremely empirically grounded and verifiable theory like physics, for much of the history of the field, the dominant theoretical framework has had significant omissions or blind spots that would occasionally lead to faulty results when applied to areas that were previously unknown. Economic theory is much less reliable. I think youâre correct to highlight that economic data can be unreliable too, and itâs certainly true that many people overestimate the size of Bayesian updates based on shaky data, and should perhaps stick to their priors more. But letâs not kid ourselves about how good our cutting edge of theoretical understanding is in fields like economics and medicine â and letâs not kid ourselves that nonspecialist amateurs can reach even that level of accuracy.
This is maybe getting too bogged down in the object-level. The general point is that if you have a confident prior, you are not going to update on uncertain observational evidence very much. My argument in the main post is that ignoring your prior entirely is clearly not correct and that is driving a lot of the mistaken opinions I outline.
Tangentially, I stand by my position on the object-levelâI actually think that 98% is too low! For any randomly selected good I can think of, I would expect a price floor to reduce demand for it in >99% of cases. Common sense aside⊠The only theoretical reason this might not be true is if the market for labour is monopsonistic. That is just obviously not the case. There is also evidence from the immigration literature which suggests that native wages are barely affected by a massive influx of low skilled labour, which implies a near horizontal demand curve. There is also the point that if you are slightly Keynesian you think that involuntary employment is caused by the failure of wages to adjust downward; legally forbidding them from doing this must cause unemployment.
What makes you believe the market for labor isnât monopsonistic?
To me it seems pretty plausible that the labor market is full of minor monopsonies. For example I prefer to work at a store closer rather than farther away from me which would give the local store some market power on my labor.
Or maybe I prefer to work at the only coffee place in my town as opposed to the only tea place due to my interest coffee.
Iâve strong upvoted Benâs points, and would add a couple of concerns: * I donât know how in any particular situation one would usefully separate the object-level from the general principle. What heuristic would I follow to judge how far to defer to experts on banana growers in Honduras on the subject of banana-related politics? * The less pure a science gets (using https://ââxkcd.com/ââ435/ââ as a guide), the less we should be inclined to trust its authorities, but the less we should also be inclined to trust our own judgementâthe relevant factors grow at a huge rate
So sticking to the object level and the eg of minimum wage, I would not update on a study that much, but strong agree with Ben that 98% is far too confident, since when you say âthe only theoretical reasonâ, you presumably mean âas determined by other social science theoryâ.
(In this particular case, it seems like youâre conflating the (simple and intuitive to me as well fwiw) individual effect of having to pay a higher wage reducing the desirability of hiring someone with the much more complex and much less intuitive claim that higher wages in general would reduce number of jobs in generalâwhich is the sort of distinction that an expert in the field seems more likely to be able to draw.)
So my instinct is that Bayesians should only strongly disagree with experts in particular cases where they can link their disagreement to particular claims the experts have made that seem demonstrably wrong on Bayesian lights.
As I mention in the post, itâs not just theory and common sense, but also evidence from other domains. If the demand curve for labour low skilled labour is vertical, then it is all but impossible that a massive influx of Cuban workers during the Mariel boatlift had close to zero effect on native US wages. Nevertheless, that is what the evidence suggests.
I am happy to be told of other theoretical explanations of why minimum wages donât reduce demand for labour. The ones I am aware of in the literature are monopsonistic buyer of labour (clearly not the case), or one could give up on the view that firms are systematically profit-seeking (also doesnât seem true).
The claims that are wrong are the ones I highlight in the post, viz that the empirical evidence is all that matters when forming believe about the minimum wage. Most empirical research isnât that good and cannot distinguish signal from noise when there are small treatment effects e.g. the Card and Krueger research that started the whole debate off got their data by telephoning fast food restaurants.
I think economics is especially prone to this kind of âItâs more complicated than thatâ issue. The idea that firms will reduce employment in response to higher minimum wage places a great deal of faith in the efficiency of markets. There are plenty of ways that an increased cost of labor wouldnât lead to lower demand: there might be resistance to firing employees because of social pressures; institutions may simply remain stuck in thinking that a certain number of people are necessary to do all the work that needs to be done, and be resistant to changing those attitudes. Consider the phenomenon of âbullshit jobsâ. If youâve ever worked in an office, in the public or private sector, youâve probably noticed that a huge number of employees seem to do little of substance. Even as someone who worked a minimum wage job in a department store, many of my co-workers seemed to do little if anything of use, and yet no effort was made to ensure that everyone was being productive.
If anything, I would argue that the idea that markets are, by default, perfectly efficient (or close to perfectly efficient) goes against the lived experience of me and the people I know, and my prior is to disbelieve arguments predicated on it, unless there is some specific evidence or particularly good reason to think that the market would be very efficient in a specific case (such as securities trading, where there are a huge number of smart, well-qualified people working very hard to exploit any inefficiencies to make absurd amounts of money)
I buy two of your examples: in the case of masks, it seems clear now that the experts were wrong before, and in âFirst doses firstâ, you present some new evidence that the priors were right.
On nutrition and lockdowns, you havenât convinced me that the point of view youâre defending isnât the one that deference would arrive at anyway: it seems to me like the expert consensus is that lockdowns work and most nutritional fads are ignorable.
On minimum wage and alcohol during pregnancy, youâve presented a conflict between evidence and priors, but I donât feel like you resolved the conflict: someone who believed the evidence proved the priors wrong wonât find anything in your examples to change their minds. For drinking during pregnancy, Iâm not even really convinced there is a conflict: I suspect the heart of the matter is what people mean by âsafeâ, what risks or harms are small enough to be ignored.
I think in general there are for sure some cases where priors should be given more weight than theyâre currently afforded. But it also seems like there are often cases where intuitions are bad, where âitâs more complicated than thatâ tends to dominate, where there are always more considerations or open uncertainties than one can adequately navigate on priors alone. I donât think this post helps me understand how to distinguish between those cases.
Hello, my argument was that there are certain groups of experts you can ignore or put less weight on because they have the wrong epistemology. I agree that the median expert might have got some of these cases right. (Iâm not sure thatâs true in the case of nutrition however)
The point in all these cases re priors is that one should have a very strong prior, which will not be shifted much by flawed empirical research. One should have a strong prior that the efficacy of the vaccine wonât drop off massively for the over 65s even before this is studied.
One can see the priors vs evidence case for the minimum wage more formally using Bayes theorem. Suppose my prior that minimum wages reduce demand for labour is 98%, which is reasonable. I then learn that one observational study has found that they have no effect on demand for labour. Given the flaws in empirical research, letâs say there is a 30% chance of a study finding no effect conditional on there being an effect. Given this, we might put a symmetrical probability on a study finding no effect conditional on there being no effect - a 70% chance of a null result if minimum wages in fact have no effect.
Then my posterior is = (.3*.98)/â(.3*98+.7*.02) = 95.5%
So I am still very sure that minimum wages have no effect even if there is one study showing the contrary. FWIW, my reading of the evidence is that most studies do find an effect on demand for labour, so after assimilating it all, one would probably end up where oneâs prior was. This is why the value of information of research into the minimum wage is so low.
On drinking in pregnancy, I donât think this is driven by peopleâs view of acceptable risk, but rather by a myopic empiricist view of the world. Osterâs book is the go-to for data-driven parents and she claims that small amounts of alcohol has no effect, not that it has a small effect but is worth the risk. (Incidentally, the latter claim is also clearly falseâit obviously isnât worth the risk.)
On your final point, I donât think one can or should aim to give an account of whether relying on theory or common sense is always the right thing to do. I have highlighted some examples where failure to rely on theory and evidence from other domains leads people astray. Epistemology is complicated and this insight may of course not be true in all domains. For a comprehensive account of how to approach cases such as these, one cannot say much more than that the true theory of epistemology is Bayesianism and to apply that properly you need to be apprised of all of the relevant information in different fields.
Bluntly I think a prior of 98% is extremely unreasonable. I think that someone who had thoroughly studied the theory, all credible counterarguments against it, had long discussions about it with experts who disagreed, etc. could reasonably come to a belief that strong. An amateur who has undertaken a simplistic study of the basic elements of the situation canât IMO reasonably conclude that all the rest of that thought and debate would have a <2% chance of changing their mind.
Even in an extremely empirically grounded and verifiable theory like physics, for much of the history of the field, the dominant theoretical framework has had significant omissions or blind spots that would occasionally lead to faulty results when applied to areas that were previously unknown. Economic theory is much less reliable. I think youâre correct to highlight that economic data can be unreliable too, and itâs certainly true that many people overestimate the size of Bayesian updates based on shaky data, and should perhaps stick to their priors more. But letâs not kid ourselves about how good our cutting edge of theoretical understanding is in fields like economics and medicine â and letâs not kid ourselves that nonspecialist amateurs can reach even that level of accuracy.
This is maybe getting too bogged down in the object-level. The general point is that if you have a confident prior, you are not going to update on uncertain observational evidence very much. My argument in the main post is that ignoring your prior entirely is clearly not correct and that is driving a lot of the mistaken opinions I outline.
Tangentially, I stand by my position on the object-levelâI actually think that 98% is too low! For any randomly selected good I can think of, I would expect a price floor to reduce demand for it in >99% of cases. Common sense aside⊠The only theoretical reason this might not be true is if the market for labour is monopsonistic. That is just obviously not the case. There is also evidence from the immigration literature which suggests that native wages are barely affected by a massive influx of low skilled labour, which implies a near horizontal demand curve. There is also the point that if you are slightly Keynesian you think that involuntary employment is caused by the failure of wages to adjust downward; legally forbidding them from doing this must cause unemployment.
What makes you believe the market for labor isnât monopsonistic?
To me it seems pretty plausible that the labor market is full of minor monopsonies. For example I prefer to work at a store closer rather than farther away from me which would give the local store some market power on my labor.
Or maybe I prefer to work at the only coffee place in my town as opposed to the only tea place due to my interest coffee.
Iâve strong upvoted Benâs points, and would add a couple of concerns:
* I donât know how in any particular situation one would usefully separate the object-level from the general principle. What heuristic would I follow to judge how far to defer to experts on banana growers in Honduras on the subject of banana-related politics?
* The less pure a science gets (using https://ââxkcd.com/ââ435/ââ as a guide), the less we should be inclined to trust its authorities, but the less we should also be inclined to trust our own judgementâthe relevant factors grow at a huge rate
So sticking to the object level and the eg of minimum wage, I would not update on a study that much, but strong agree with Ben that 98% is far too confident, since when you say âthe only theoretical reasonâ, you presumably mean âas determined by other social science theoryâ.
(In this particular case, it seems like youâre conflating the (simple and intuitive to me as well fwiw) individual effect of having to pay a higher wage reducing the desirability of hiring someone with the much more complex and much less intuitive claim that higher wages in general would reduce number of jobs in generalâwhich is the sort of distinction that an expert in the field seems more likely to be able to draw.)
So my instinct is that Bayesians should only strongly disagree with experts in particular cases where they can link their disagreement to particular claims the experts have made that seem demonstrably wrong on Bayesian lights.
As I mention in the post, itâs not just theory and common sense, but also evidence from other domains. If the demand curve for labour low skilled labour is vertical, then it is all but impossible that a massive influx of Cuban workers during the Mariel boatlift had close to zero effect on native US wages. Nevertheless, that is what the evidence suggests.
I am happy to be told of other theoretical explanations of why minimum wages donât reduce demand for labour. The ones I am aware of in the literature are monopsonistic buyer of labour (clearly not the case), or one could give up on the view that firms are systematically profit-seeking (also doesnât seem true).
The claims that are wrong are the ones I highlight in the post, viz that the empirical evidence is all that matters when forming believe about the minimum wage. Most empirical research isnât that good and cannot distinguish signal from noise when there are small treatment effects e.g. the Card and Krueger research that started the whole debate off got their data by telephoning fast food restaurants.
I think economics is especially prone to this kind of âItâs more complicated than thatâ issue. The idea that firms will reduce employment in response to higher minimum wage places a great deal of faith in the efficiency of markets. There are plenty of ways that an increased cost of labor wouldnât lead to lower demand: there might be resistance to firing employees because of social pressures; institutions may simply remain stuck in thinking that a certain number of people are necessary to do all the work that needs to be done, and be resistant to changing those attitudes. Consider the phenomenon of âbullshit jobsâ. If youâve ever worked in an office, in the public or private sector, youâve probably noticed that a huge number of employees seem to do little of substance. Even as someone who worked a minimum wage job in a department store, many of my co-workers seemed to do little if anything of use, and yet no effort was made to ensure that everyone was being productive.
If anything, I would argue that the idea that markets are, by default, perfectly efficient (or close to perfectly efficient) goes against the lived experience of me and the people I know, and my prior is to disbelieve arguments predicated on it, unless there is some specific evidence or particularly good reason to think that the market would be very efficient in a specific case (such as securities trading, where there are a huge number of smart, well-qualified people working very hard to exploit any inefficiencies to make absurd amounts of money)