Some ideas for improving or reducing the costs of failure transparency
This is an open question. The following list is intended to be a starting point for conversation. Where possible tried to make these examples as shovel-ready as possible. It would be great to hear more ideas, or examples of successfully implemented things.
Thanks to Abi Olvera, Nathan Young, Ben Millwood, Adam Gleave & Arjun Khandelwal for many of these suggestions.
Create a range of space(s) to discuss failure of any size.
I think the explicit intention of helping the community and providing relevant information is probably important to avoid goodharting.
Questions that could help you determine how valuable this mistake is to the wider community :
How generalizable was the failure?
(trying) to separate personal faults from external factors
What projects could you have done instead of this one?
Would you do the project again? (was it worth it)
Do you think your evaluation of the project is the same someone 1) working on you with it 2) a funder 3) recipients would agree with? How might they differ?
It would be especially valuable to have high-profile members of the EA community do this, since they have relatively less status to lose and their
Note that these spaces donât have to all be public!
At EA conferences (this is more for signalling /â setting norms)
A regular /â semi-regular âFailed Projectsâ or âThings I changed my mind onâ or âevolution in my thinkingâ kind of panels at EA Globals and other conferences
Asking EA public figures questions about failure at talks
At EA conferences or local groups: Events, workshops or meet-ups for people to share their thinking, changes in their thinking and mistakes and reflect on them together, collaboratively
Create committee(s) to evaluate failed projects
For larger projects with bigger stakes, it seems valuable to invest more resources into learning from it
Interviews with stakeholders & reading relevant documents & outputs
Aim to create a neutral, fair report which creates an accurate map of the problem
It seems plausible the EAIF would fund something like this
Pay grantees to follow-up on their projects
Could funders offer to pay an additional X dollars to grantees to get them to write up reflections or takeaways from their projects, successful or not? (This is probably more valuable for people being funded to work on very different kinds of projects, and who wouldnât otherwise write themâe.g. not established organisations whoâd spend time writing an annual report anyways)
Anonymous Mistake Reporting
Have a call for anonymous reports of failures that people might not want to report publicly (either their own or others)
Some ideas for improving or reducing the costs of failure transparency
This is an open question. The following list is intended to be a starting point for conversation. Where possible tried to make these examples as shovel-ready as possible. It would be great to hear more ideas, or examples of successfully implemented things.
Thanks to Abi Olvera, Nathan Young, Ben Millwood, Adam Gleave & Arjun Khandelwal for many of these suggestions.
Create a range of space(s) to discuss failure of any size.
I think the explicit intention of helping the community and providing relevant information is probably important to avoid goodharting.
Questions that could help you determine how valuable this mistake is to the wider community :
How generalizable was the failure?
(trying) to separate personal faults from external factors
What projects could you have done instead of this one?
Would you do the project again? (was it worth it)
Do you think your evaluation of the project is the same someone 1) working on you with it 2) a funder 3) recipients would agree with? How might they differ?
It would be especially valuable to have high-profile members of the EA community do this, since they have relatively less status to lose and their
Note that these spaces donât have to all be public!
At EA conferences (this is more for signalling /â setting norms)
A regular /â semi-regular âFailed Projectsâ or âThings I changed my mind onâ or âevolution in my thinkingâ kind of panels at EA Globals and other conferences
Asking EA public figures questions about failure at talks
At EA conferences or local groups: Events, workshops or meet-ups for people to share their thinking, changes in their thinking and mistakes and reflect on them together, collaboratively
Create committee(s) to evaluate failed projects
For larger projects with bigger stakes, it seems valuable to invest more resources into learning from it
Interviews with stakeholders & reading relevant documents & outputs
Aim to create a neutral, fair report which creates an accurate map of the problem
It seems plausible the EAIF would fund something like this
Pay grantees to follow-up on their projects
Could funders offer to pay an additional X dollars to grantees to get them to write up reflections or takeaways from their projects, successful or not? (This is probably more valuable for people being funded to work on very different kinds of projects, and who wouldnât otherwise write themâe.g. not established organisations whoâd spend time writing an annual report anyways)
Anonymous Mistake Reporting
Have a call for anonymous reports of failures that people might not want to report publicly (either their own or others)