I agree that more public evaluations of things like CEA programs and programs that they fund, would be really valuable. I’m sure people at CEA would agree too.
I don’t think CEA staff, or at least CEA leadership, actually agrees that public evaluations are “really valuable”. CEA has repeatedly deprioritized public evaluations, even after public commitments to conduct them (e.g. CBG, EA Grants, Pareto Fellowship). And Max has been pretty clear that he prioritizes accountability to board members and funders for CEA rather than to the public or EA community, and that he thinks public evaluations generally aren’t worth the cost to produce them (especially because CEA is hesitant to publicly criticize people/orgs and that is often where the most the most useful information can be gleaned.) So my sense is that CEA thinks public evaluations would be valuable in some abstract sense where there aren’t any costs to conducting them, but not in a practical sense that incorporates the tradeoffs that always exist in the real world.
We might be quibbling a bit over what “really valuable” means. I agree that CEA definitely could have prioritized these higher, and likely would have if they cared about it much more.
I think they would be happy to have evaluations done if they were very inexpensive or free, for whatever that’s worth. This is much better than with many orgs, who would try to oppose evaluations even if they are free; but perhaps it is suboptimal.
I think only doing something if it’s free/inexpensive is almost the opposite of thinking something is ‘really valuable’, so that’s far from a quibble (almost the opposite because, as you point out, actively being against something is the actual opposite).
Ozzie, I think you and I agree on CEA’s stance about public evaluations (not actively opposed to them, but mainly interested only if they are free or very inexpensive to execute). My interpretation of that position is largely in line with Rebecca’s though.
I don’t think CEA staff, or at least CEA leadership, actually agrees that public evaluations are “really valuable”. CEA has repeatedly deprioritized public evaluations, even after public commitments to conduct them (e.g. CBG, EA Grants, Pareto Fellowship). And Max has been pretty clear that he prioritizes accountability to board members and funders for CEA rather than to the public or EA community, and that he thinks public evaluations generally aren’t worth the cost to produce them (especially because CEA is hesitant to publicly criticize people/orgs and that is often where the most the most useful information can be gleaned.) So my sense is that CEA thinks public evaluations would be valuable in some abstract sense where there aren’t any costs to conducting them, but not in a practical sense that incorporates the tradeoffs that always exist in the real world.
We might be quibbling a bit over what “really valuable” means. I agree that CEA definitely could have prioritized these higher, and likely would have if they cared about it much more.
I think they would be happy to have evaluations done if they were very inexpensive or free, for whatever that’s worth. This is much better than with many orgs, who would try to oppose evaluations even if they are free; but perhaps it is suboptimal.
I think only doing something if it’s free/inexpensive is almost the opposite of thinking something is ‘really valuable’, so that’s far from a quibble (almost the opposite because, as you point out, actively being against something is the actual opposite).
Ozzie, I think you and I agree on CEA’s stance about public evaluations (not actively opposed to them, but mainly interested only if they are free or very inexpensive to execute). My interpretation of that position is largely in line with Rebecca’s though.