This is really awesome! Along the things that Hauke mentioned around scientometrics, I’d love to figure out a native integration for predicting different kinds of metrics for new research papers. Then other scientists browsing on Arxiv can quickly submit their own thoughts on the quality and accuracy of different aspects of each paper, as a more quantitative and public way of delivering feedback to the authors.
A quick sketch: On every new paper submission, we automatically create a markets for:
“How many citations will this paper have?”
“Will this paper have successfully replicated in 1 year?”
“Will this paper be retracted in the next 6 months?”
Along with letting the author set up markets on any key claims made within the paper, or the sources the paper depends on
Manifold would be happy to provide the technical expertise/integration for this; we’ve previously explored this space with the folks behind Research.bet, which I would highly encourage reaching out to as well.
This is indeed one of the “wow” features I was considering, but I didn’t think it through as much as you obviously have (really nice!).
(and also ways it could cause harm by accident, consider talking to me before you launch it very widely on something like “all research”?)
Anyway my current opinion is that I’d be very happy to integrate with Manifold for doing something like this, though please also note I am not a decision maker.
If you don’t mind, I’m going to add a screenshot from this cool website you linked, in favor of people who might not click through :)
Regarding the screenshot—could you explain what these graphs give us? Compared, for example, to “Is the paper scientifically sound?” or “Will the result replicate?”
This is really awesome! Along the things that Hauke mentioned around scientometrics, I’d love to figure out a native integration for predicting different kinds of metrics for new research papers. Then other scientists browsing on Arxiv can quickly submit their own thoughts on the quality and accuracy of different aspects of each paper, as a more quantitative and public way of delivering feedback to the authors.
A quick sketch: On every new paper submission, we automatically create a markets for:
“How many citations will this paper have?”
“Will this paper have successfully replicated in 1 year?”
“Will this paper be retracted in the next 6 months?”
Along with letting the author set up markets on any key claims made within the paper, or the sources the paper depends on
Manifold would be happy to provide the technical expertise/integration for this; we’ve previously explored this space with the folks behind Research.bet, which I would highly encourage reaching out to as well.
Thank you!
This is indeed one of the “wow” features I was considering, but I didn’t think it through as much as you obviously have (really nice!).
(and also ways it could cause harm by accident, consider talking to me before you launch it very widely on something like “all research”?)
Anyway my current opinion is that I’d be very happy to integrate with Manifold for doing something like this, though please also note I am not a decision maker.
If you don’t mind, I’m going to add a screenshot from this cool website you linked, in favor of people who might not click through :)
Regarding the screenshot—could you explain what these graphs give us? Compared, for example, to “Is the paper scientifically sound?” or “Will the result replicate?”