What do you think would occur if you added in the 1st or 2nd most upvoted, recent comments in the GPT-3 description, following the question?
I think it might make the difference on some questions with high forecaster volume, but might detract from the accuracy on questions with lower forecaster volume.
If the comments include a prediction my guess is that GPT would often make the same prediction and thus become much more accurate. Not because it learned to predict things but because there’s probably a strong correlation between the community prediction and the most upvoted comments prediction.
If the goal is to give GPT more context than just the title of the question, then you could include the descriptions for each question as well, but when I tried this I got worse results (fewer legible predictions).
What do you think would occur if you added in the 1st or 2nd most upvoted, recent comments in the GPT-3 description, following the question?
I think it might make the difference on some questions with high forecaster volume, but might detract from the accuracy on questions with lower forecaster volume.
If the comments include a prediction my guess is that GPT would often make the same prediction and thus become much more accurate. Not because it learned to predict things but because there’s probably a strong correlation between the community prediction and the most upvoted comments prediction.
If the goal is to give GPT more context than just the title of the question, then you could include the descriptions for each question as well, but when I tried this I got worse results (fewer legible predictions).