In light of your post, I’ve adjusted towards somewhat more often writing continuous rather than binary questions, since I’m now a bit less worried about the fact that that might tend to reduce the number of unique predictors. Just wanted to check if you think that that’s an appropriate takeaway from your post? (It’s probably hard to say without knowing what I mean by the eminently vague term “somewhat”, but maybe you’d have something useful to say anyway.)
A related question: I had previously been concerned about the possibility that the number of questions on Metaculus will rise faster than the number of active predictors rise, and I had thought I should see this like a tragedy of the commons situation and that I should be wary of “overgrazing” myself by too liberally creating questions. (E.g., maybe I should avoid posting the least-useful-seeming 25% of the question ideas that seem worth my time to create.) This post makes me think I should be moderately less concerned about that, e.g. should adjust a bit towards feeling fine about creating many questions and about other people doing so. Do you think that that’s an appropriate takeaway?
Possibly, but I would not be confident in that. One plausible reason that questions with low numbers of predictors do surprisingly well is that the site’s power users are the ones predicting on them, and they are generally pretty good predictors.
I would not be very concerned about asking too many questions though, until this is closer to being a problem than it currently is.
In light of your post, I’ve adjusted towards somewhat more often writing continuous rather than binary questions, since I’m now a bit less worried about the fact that that might tend to reduce the number of unique predictors. Just wanted to check if you think that that’s an appropriate takeaway from your post? (It’s probably hard to say without knowing what I mean by the eminently vague term “somewhat”, but maybe you’d have something useful to say anyway.)
Yeah I think that’s a reasonable update if there’s not a near equivalent binary question which gets you the information that you really want.
I would be less hesitant to write continuous questions than I was before for sure.
Cool, thanks.
A related question: I had previously been concerned about the possibility that the number of questions on Metaculus will rise faster than the number of active predictors rise, and I had thought I should see this like a tragedy of the commons situation and that I should be wary of “overgrazing” myself by too liberally creating questions. (E.g., maybe I should avoid posting the least-useful-seeming 25% of the question ideas that seem worth my time to create.) This post makes me think I should be moderately less concerned about that, e.g. should adjust a bit towards feeling fine about creating many questions and about other people doing so. Do you think that that’s an appropriate takeaway?
Possibly, but I would not be confident in that. One plausible reason that questions with low numbers of predictors do surprisingly well is that the site’s power users are the ones predicting on them, and they are generally pretty good predictors.
I would not be very concerned about asking too many questions though, until this is closer to being a problem than it currently is.