As a positive example, 80,000 Hours does relatively extensive impact evaluations. The most obvious limitation is that they have to guess whether any career changes are actually improvements, but I don’t see how to fix that—determining the EV of even a single person’s career is an extremely hard problem. IIRC they’ve done some quasi-experiments but I couldn’t find them from quickly skimming their impact evaluations.
As a positive example, 80,000 Hours does relatively extensive impact evaluations. The most obvious limitation is that they have to guess whether any career changes are actually improvements, but I don’t see how to fix that—determining the EV of even a single person’s career is an extremely hard problem. IIRC they’ve done some quasi-experiments but I couldn’t find them from quickly skimming their impact evaluations.