If that is the view, I am unsure what the bank of Japan example is meant to motivate.
The example is confounded by the fact that Eliezer reports a lot of outside-view information to make the determination the BoJ is making a bad call. The judgement (and object level argument) which he endorses originally came from econ bloggers (I gather profs like Sumner) who Eliezer endorses due to their good track record. In addition he reports the argument the econ bloggers make does make object level sense.
Yet modest approaches can get the same answer without conceding the object-level evidence is dispositive. If the bank of Japan is debunked as an authority (for whatever reason), then in a dispute of ‘them versus economists with a good empirical track record.’, the outside view favours the latter’s determination for standard reasons (it might caution one should look more widely across economic expertise, but bracket this). It also plausibly allows one to assert confidence in the particular used to make the determination the BoJ makes a bad call.
So I think I’d have made a similar judgement to Eliezer in this case whether or not I had any ‘object level’ evidence to go on: if I didn’t know (or couldn’t understand) the argument Sumner et al. used, I’d still conclude they’re likely right.
It seems one needs to look for cases where ‘outside’ and ‘inside’ diverge. So maybe something like, “Eliezer judged from his personal knowledge of economics the BoJ was making a bad call (without inspiration from any plausible epistemic authority), and was right to back himself ‘over’ the BoJ.”
That would be a case where someone would disagree this is the right approach. If all I had to go on was my argument and knowledge of the BoJs policy (e.g., I couldn’t consult economists or econbloggers or whatever), then I suggest one should think that the incentives of the BoJ are probably at least somewhat better than orthogonal on expectation, and probably better correlated than an argument made by an amateur economist. If it transpired the argument was actually right, modesty’s failure in a single case is not much of a strike against it, at least without some track record beyond this single case..
If that is the view, I am unsure what the bank of Japan example is meant to motivate.
The example is confounded by the fact that Eliezer reports a lot of outside-view information to make the determination the BoJ is making a bad call. The judgement (and object level argument) which he endorses originally came from econ bloggers (I gather profs like Sumner) who Eliezer endorses due to their good track record. In addition he reports the argument the econ bloggers make does make object level sense.
Yet modest approaches can get the same answer without conceding the object-level evidence is dispositive. If the bank of Japan is debunked as an authority (for whatever reason), then in a dispute of ‘them versus economists with a good empirical track record.’, the outside view favours the latter’s determination for standard reasons (it might caution one should look more widely across economic expertise, but bracket this). It also plausibly allows one to assert confidence in the particular used to make the determination the BoJ makes a bad call.
So I think I’d have made a similar judgement to Eliezer in this case whether or not I had any ‘object level’ evidence to go on: if I didn’t know (or couldn’t understand) the argument Sumner et al. used, I’d still conclude they’re likely right.
It seems one needs to look for cases where ‘outside’ and ‘inside’ diverge. So maybe something like, “Eliezer judged from his personal knowledge of economics the BoJ was making a bad call (without inspiration from any plausible epistemic authority), and was right to back himself ‘over’ the BoJ.”
That would be a case where someone would disagree this is the right approach. If all I had to go on was my argument and knowledge of the BoJs policy (e.g., I couldn’t consult economists or econbloggers or whatever), then I suggest one should think that the incentives of the BoJ are probably at least somewhat better than orthogonal on expectation, and probably better correlated than an argument made by an amateur economist. If it transpired the argument was actually right, modesty’s failure in a single case is not much of a strike against it, at least without some track record beyond this single case..