This looks like a genuinely useful experiment. I’m especially curious whether making the process public leads to better decisions, rather than just more transparent ones.
One thing I’d watch for is reviewers anchoring on the first few comments or on recognizable names. It might be interesting to collect people’s initial views privately, then compare them with how they update after reading the discussion.
You’d end up learning quite a lot about when public deliberation actually improves grantmaking, and when it just creates herding.
Thanks, Ethan! I agree. We even thought about making the private reviews / scores hidden from each other, but decided not to this round. Maybe in the future!
This looks like a genuinely useful experiment. I’m especially curious whether making the process public leads to better decisions, rather than just more transparent ones.
One thing I’d watch for is reviewers anchoring on the first few comments or on recognizable names. It might be interesting to collect people’s initial views privately, then compare them with how they update after reading the discussion.
You’d end up learning quite a lot about when public deliberation actually improves grantmaking, and when it just creates herding.
Thanks, Ethan! I agree. We even thought about making the private reviews / scores hidden from each other, but decided not to this round. Maybe in the future!