I just tried out the judgment calibration test with my Munich students, with 25 relatively difficult-to-assess statements.
Main finding: Roughly half score better and half score worse than they would have done if they had just uniformly answered “50%”. I guess that this indicates that the test was slightly too difficult. Notably, I had included many statements about orders of magnitude (e.g. energy released by the sun in a year, or time scales), and those seem challenging.
But the best students had a mean square deviation of the estimate from the truth value of about 16%, which I guess is quite good.
Great that you like this testing idea, Siebe!
I just tried out the judgment calibration test with my Munich students, with 25 relatively difficult-to-assess statements.
Main finding: Roughly half score better and half score worse than they would have done if they had just uniformly answered “50%”. I guess that this indicates that the test was slightly too difficult. Notably, I had included many statements about orders of magnitude (e.g. energy released by the sun in a year, or time scales), and those seem challenging.
But the best students had a mean square deviation of the estimate from the truth value of about 16%, which I guess is quite good.