I definitely agree that the ACE numbers should not be taken literally, as there are fundamental problems in the underlying studies from which those numbers are derived.
If the point estimate or expected value given the uncertainty and reasonable priors would be a lot lower, then the reporting should reflect that, e.g. “this methodology says $X per animal prevented from existing, but different members of our team have expected values of $X, $2X, $100X, and $5000X when taking into account priors, other evidence, regression to the reference class, winner’s curse, etc.”
GiveWell publishes a cost-effectiveness spreadsheet at giving season where various GiveWell staff members input their own estimates in charity comparison.
If the point estimate or expected value given the uncertainty and reasonable priors would be a lot lower, then the reporting should reflect that, e.g. “this methodology says $X per animal prevented from existing, but different members of our team have expected values of $X, $2X, $100X, and $5000X when taking into account priors, other evidence, regression to the reference class, winner’s curse, etc.”
GiveWell publishes a cost-effectiveness spreadsheet at giving season where various GiveWell staff members input their own estimates in charity comparison.
I agree this would be cool for ACE to do. I don’t know what the values of N*X are for other staff and board members, though.