For example, these estimates can also be viewed as forecasts of what I would estimate if I spent 100x as much time on this, or as forecasts of what a really good system would output.
Agreed, but thatās still different from forecasting the impact of a project that hasnāt happened yet, and the difference intuitively seems like it might be meaningful for our purposes. I.e., itās not immediately obvious that methods and intuitions that work well for the sort of estimation/āforecasting done in this post would also work well for forecasting the impact of a project that hasnāt happened yet.
One could likewise say that itās not obvious that methods and intuitions that work well for forecasting how Iāll do in job applications would also work well for forecasting GDP growth in developing countries. So I guess my point was more fundamentally about the potential significance of the domain being different, rather than whether the thing can be seen as a type of forecasting or not.
So it sounds like youāre thinking that the sort of thing done in this post would be āa way to calibrate oneās intuitions/āforecasts (with the hope being that thereāll be transfer between calibration when estimating the impact of past projects and calibration when forecasting the impact of future projects)ā?
That does seem totally plausible to me; it just adds a step to the argument.
(I guess Iām also more generally interested in the question of how well forecasting accuracy and calibration transfers across domainsāthough at the same time I havenāt made the effort to look into it at all...)
Yes, I expect the intuitions and for estimation to generalize/āhelp a great deal with the forecasting step, and though I agree that this might not be intuitively obvious. I understand that estimation and forecasting seem like different categories, but I donāt expect that to be a significant hurdle in practice.
Agreed, but thatās still different from forecasting the impact of a project that hasnāt happened yet, and the difference intuitively seems like it might be meaningful for our purposes. I.e., itās not immediately obvious that methods and intuitions that work well for the sort of estimation/āforecasting done in this post would also work well for forecasting the impact of a project that hasnāt happened yet.
One could likewise say that itās not obvious that methods and intuitions that work well for forecasting how Iāll do in job applications would also work well for forecasting GDP growth in developing countries. So I guess my point was more fundamentally about the potential significance of the domain being different, rather than whether the thing can be seen as a type of forecasting or not.
So it sounds like youāre thinking that the sort of thing done in this post would be āa way to calibrate oneās intuitions/āforecasts (with the hope being that thereāll be transfer between calibration when estimating the impact of past projects and calibration when forecasting the impact of future projects)ā?
That does seem totally plausible to me; it just adds a step to the argument.
(I guess Iām also more generally interested in the question of how well forecasting accuracy and calibration transfers across domainsāthough at the same time I havenāt made the effort to look into it at all...)
Yes, I expect the intuitions and for estimation to generalize/āhelp a great deal with the forecasting step, and though I agree that this might not be intuitively obvious. I understand that estimation and forecasting seem like different categories, but I donāt expect that to be a significant hurdle in practice.