I think people take this into account but not enough or something? I strongly suspect when evaluating research many people have a vague, and not sufficiently precise, sense of both the numerator and denominator, and their vague intuitions aren’t sufficiently linear. I know I do this myself unless it’s a grant I’m actively investigating.
This is easiest to notice in research because it’s both a) a large fraction of (non-global health and development) EA output and b) very gnarly. But I don’t think research is unusually gnarly in terms of EA outputs or grants, advocacy, comms, etc have similar issues.
I think people take this into account but not enough or something? I strongly suspect when evaluating research many people have a vague, and not sufficiently precise, sense of both the numerator and denominator, and their vague intuitions aren’t sufficiently linear. I know I do this myself unless it’s a grant I’m actively investigating.
This is easiest to notice in research because it’s both a) a large fraction of (non-global health and development) EA output and b) very gnarly. But I don’t think research is unusually gnarly in terms of EA outputs or grants, advocacy, comms, etc have similar issues.