The research I’ve reviewed broadly supports this impression. For example:
Rieber (2004) lists “training for calibration feedback” as his first recommendation for improving calibration, and summarizes a number of studies indicating both short- and long-term improvements on calibration.4In particular, decades ago, Royal Dutch Shell began to provide calibration for their geologists, who are now (reportedly) quite well-calibrated when forecasting which sites will produce oil.5
Since 2001, Hubbard Decision Research trained over 1,000 people across a variety of industries. Analyzing the data from these participants, Doug Hubbard reports that 80% of people achieve perfect calibration (on trivia questions) after just a few hours of training. He also claims that, according to his data and at least one controlled (but not randomized) trial, this training predicts subsequent real-world forecasting success.
First bullet: I read citation #4 and it describes improvement in a lab with like domain (e.g. trivia) not across domains (e.g. trivia ⇒ world events) as far as I could tell. The Shell example is also within domain.
The second bullet is the same info shared in Hubbard’s book, not a controlled trial and he doesn’t provide the underlying data.
Unfortunately, I don’t think any of this info is very persuasive for answering the question about cross-domain applicability.
This seems like a good place to look for studies:
Thanks for the reply.
First bullet: I read citation #4 and it describes improvement in a lab with like domain (e.g. trivia) not across domains (e.g. trivia ⇒ world events) as far as I could tell. The Shell example is also within domain.
The second bullet is the same info shared in Hubbard’s book, not a controlled trial and he doesn’t provide the underlying data.
Unfortunately, I don’t think any of this info is very persuasive for answering the question about cross-domain applicability.