There’s a lot of room between publishing more than ~1 paragraph and “publishing their internal analyses.” I didn’t read Vasco as suggesting publication of the full analyses.
Assertion 4 -- “The costs for Open Phil to reduce the error rate of analyses, would not be worth the benefits”—seems to be doing a lot of work in your model here. But it seems to be based on assumptions about the nature and magnitude of errors that would be detected. If a number of errors were material (in the sense that correcting them would have changed the grant/no grant decision, or would have seriously changed the funding level), I don’t think it would take many errors for assertion 4 to be incorrect.
Moreover, if an error were found in—e.g., a five-paragraph summary of a grant rationale—the odds of the identified error being material / important would seem higher than the average error found in (say) a 30-page writeup. Presumably the facts and conclusions that made the short writeup would be ~the more important ones.
There’s a lot of room between publishing more than ~1 paragraph and “publishing their internal analyses.” I didn’t read Vasco as suggesting publication of the full analyses.
Assertion 4 -- “The costs for Open Phil to reduce the error rate of analyses, would not be worth the benefits”—seems to be doing a lot of work in your model here. But it seems to be based on assumptions about the nature and magnitude of errors that would be detected. If a number of errors were material (in the sense that correcting them would have changed the grant/no grant decision, or would have seriously changed the funding level), I don’t think it would take many errors for assertion 4 to be incorrect.
Moreover, if an error were found in—e.g., a five-paragraph summary of a grant rationale—the odds of the identified error being material / important would seem higher than the average error found in (say) a 30-page writeup. Presumably the facts and conclusions that made the short writeup would be ~the more important ones.