“I do think this has happened to some extent with M&E, where it has been conflated with existing EA concepts like cost-effectiveness projections.”—this is an interesting observation and something that has come up in some of my own recent work, where estimating and tracking impact in expectation is perhaps mistaken as M&E as typically understood by experts in M&E. Can you share some concrete aspects of/tools from M&E you think others may be overlooking?
I also generally agree with your overall thoughs, and feel that people in EA are sometimes unwittingly unaware of entire fields that address the specific problem they believe is completely neglected.
I would say that EAs are missing large parts of M&E, including: - The formal setting of key questions / assumptions that form the basis of what you will focus on trying to answer - Creating formal monitoring frameworks (e.g. a log frame) that takes these questions / assumptions and identifies practical indicators and a method of measuring them - I think EAs don’t use the full diversity of M&E tools. In my experience we tend to over-index on surveys (vs., say, interviews, focus group discussion, or observational data) - I think considering the frequency of use of surveys, we could generally up-skill in high-quality survey design - Using a diverse set of evaluation types (EAs generally know about RCTs, but these are such a narrow slice of the available evaluation types)
In general I think we care about M&E but lack experience in the formal processes of it, especially monitoring. So application is patchy and not generally in line with best practices.
I should perhaps clarify that I am mostly talking about the non-global development side of EA. I think their norms for M&E are significantly better.
Intrac’s M&E universe is one place to see an overview of what M&E entails. I think also The Mission Motor intends to create more resources on these topics in the future :)
“I do think this has happened to some extent with M&E, where it has been conflated with existing EA concepts like cost-effectiveness projections.”—this is an interesting observation and something that has come up in some of my own recent work, where estimating and tracking impact in expectation is perhaps mistaken as M&E as typically understood by experts in M&E. Can you share some concrete aspects of/tools from M&E you think others may be overlooking?
I also generally agree with your overall thoughs, and feel that people in EA are sometimes unwittingly unaware of entire fields that address the specific problem they believe is completely neglected.
For sure!
I would say that EAs are missing large parts of M&E, including:
- The formal setting of key questions / assumptions that form the basis of what you will focus on trying to answer
- Creating formal monitoring frameworks (e.g. a log frame) that takes these questions / assumptions and identifies practical indicators and a method of measuring them
- I think EAs don’t use the full diversity of M&E tools. In my experience we tend to over-index on surveys (vs., say, interviews, focus group discussion, or observational data)
- I think considering the frequency of use of surveys, we could generally up-skill in high-quality survey design
- Using a diverse set of evaluation types (EAs generally know about RCTs, but these are such a narrow slice of the available evaluation types)
In general I think we care about M&E but lack experience in the formal processes of it, especially monitoring. So application is patchy and not generally in line with best practices.
I should perhaps clarify that I am mostly talking about the non-global development side of EA. I think their norms for M&E are significantly better.
Intrac’s M&E universe is one place to see an overview of what M&E entails. I think also The Mission Motor intends to create more resources on these topics in the future :)
Great, thanks for sharing these!