Great to see more advocacy and advocacy evaluation-related content on the EA Forum! Sharing a few things that might be of interest to you / others
Founders Pledge has a great document on evaluating policy organisations that puts forward some interesting considerations on evaluating the counterfactual impact of an org e.g.
“We gather evidence from independent referees and written sources to confirm or disconfirm the charity’s own account of their impact. Below is a hierarchy of testimony evidence, ranked from the most to least desirable.
1. Well-informed people with incentives to downplay the role played by the organisation (e.g. a rival organisation)
2. Well-informed people with no incentive to mislead about the role played by the organisation (e.g. Government bureaucrats or politicians who were directly involved in the policy change”
3. Well-informed people who have incentives to mislead about the role played by the organisation (e.g. An organisation’s campaign partners.)
4. People with less information on the role played by the organisation” [I made small edits to make this shorter]
Great to see more advocacy and advocacy evaluation-related content on the EA Forum! Sharing a few things that might be of interest to you / others
Founders Pledge has a great document on evaluating policy organisations that puts forward some interesting considerations on evaluating the counterfactual impact of an org e.g.
“We gather evidence from independent referees and written sources to confirm or disconfirm the charity’s own account of their impact. Below is a hierarchy of testimony evidence, ranked from the most to least desirable.
1. Well-informed people with incentives to downplay the role played by the organisation (e.g. a rival organisation)
2. Well-informed people with no incentive to mislead about the role played by the organisation (e.g. Government bureaucrats or politicians who were directly involved in the policy change”
3. Well-informed people who have incentives to mislead about the role played by the organisation (e.g. An organisation’s campaign partners.)
4. People with less information on the role played by the organisation” [I made small edits to make this shorter]
I also recommend Hear This Idea’s podcast with Steven Teles, a political scientist who wrote a great book about advocacy within the conservative legal movement and an article about the difficulty of evaluating advocacy.