Another argument against is that, for actually directly improving the accuracy of some piece of work, it’s probably more effective to pay people who are already know to be good at relevant work to do reviewing / red-teaming prior to publication
Yeah I think this is key. I’m much more optimistic about getting trainees to do this being a good training intervention than a “directly improve research quality” intervention. There are some related arguments why you want to pay people who are either a) already good at the relevant work or b) specialized reviewers/red-teamers
paying people to criticize your work would risk creating a weird power dynamic, and more experienced reviewers would be better at navigating this
For example, trainees may be afraid of criticizing you too harshly.
Also, if the critique is in fact bad, you may be placed in a somewhat awkward position when deciding whether to publish/publicize it.
Yeah I think this is key. I’m much more optimistic about getting trainees to do this being a good training intervention than a “directly improve research quality” intervention. There are some related arguments why you want to pay people who are either a) already good at the relevant work or b) specialized reviewers/red-teamers
paying people to criticize your work would risk creating a weird power dynamic, and more experienced reviewers would be better at navigating this
For example, trainees may be afraid of criticizing you too harshly.
Also, if the critique is in fact bad, you may be placed in a somewhat awkward position when deciding whether to publish/publicize it.