Good idea! I just read through the study preprint last night and was puzzling over how you could get a good grasp on how much of an impact the wikipedia edits are making.
Another option would be to see if there was a model that took in ~half of the PAW edits in it’s training corpus, and test it on all the claims. Then you could see roughly how much of a difference having the info on wikipedia specifically made. Might still need to control for prominence outside wikipedia though.
Good idea! I just read through the study preprint last night and was puzzling over how you could get a good grasp on how much of an impact the wikipedia edits are making.
Another option would be to see if there was a model that took in ~half of the PAW edits in it’s training corpus, and test it on all the claims. Then you could see roughly how much of a difference having the info on wikipedia specifically made. Might still need to control for prominence outside wikipedia though.