Thank you to those who had a look at this report. Our team put a lot into this as you might imagine. I’ve been anticipating some commentary in this evaluation along the lines of “this is far too complex/quantitative for a $40,000 grant recommendation.” We’d agree. We gesture at this in the “The future of Rethink Grants” section at the end of the Executive Summary.
This could have perhaps been communicated better, but my hope is that readers will come to interpret this report, and the methods employed therein, as additional tools to consider when evaluating grants. There may be occasions where evaluators might find it useful to boost their repertoire by using these methods (or something similar) to potentially make better decisions. Project leads may also get some mileage out of how much we’ve put on display here.
There are certain instances where key reasoning (see Team Strength section), or quick deferral to experts, or even a simple back-of-the-envelope (BOTEC) calculation will suffice. But as with charity evaluation, we might agree, there are circumstances where intuition and BOTECs are not enough. An example from this report that Derek mentions, the VOI calculation and CEE lead us to a more nuanced conclusion that funding decent-sized pilot was very much worth doing in our opinion, rather than fully funding it from the outset or passing over this opportunity. Our conclusions from just a BOTEC might have been different.
I think we have good reason to believe that the level of rigor displayed in this evaluation is warranted at times. And when those situations arise, we hope others will reach for this report if they’ve found it useful.
Thank you to those who had a look at this report. Our team put a lot into this as you might imagine. I’ve been anticipating some commentary in this evaluation along the lines of “this is far too complex/quantitative for a $40,000 grant recommendation.” We’d agree. We gesture at this in the “The future of Rethink Grants” section at the end of the Executive Summary.
This could have perhaps been communicated better, but my hope is that readers will come to interpret this report, and the methods employed therein, as additional tools to consider when evaluating grants. There may be occasions where evaluators might find it useful to boost their repertoire by using these methods (or something similar) to potentially make better decisions. Project leads may also get some mileage out of how much we’ve put on display here.
There are certain instances where key reasoning (see Team Strength section), or quick deferral to experts, or even a simple back-of-the-envelope (BOTEC) calculation will suffice. But as with charity evaluation, we might agree, there are circumstances where intuition and BOTECs are not enough. An example from this report that Derek mentions, the VOI calculation and CEE lead us to a more nuanced conclusion that funding decent-sized pilot was very much worth doing in our opinion, rather than fully funding it from the outset or passing over this opportunity. Our conclusions from just a BOTEC might have been different.
I think we have good reason to believe that the level of rigor displayed in this evaluation is warranted at times. And when those situations arise, we hope others will reach for this report if they’ve found it useful.