I’m confused about how you trace whether a project was responsible for an outcome.
Two examples:
Many scientific discoveries are the result of many researchers working on a problem for a long time, making it difficult to trace back whose contribution had what fractional impact towards e.g. an AI alignment breakthrough. But if we’re only paying the person who did the last bit, we’re dramatically underfunding all projects that lay important groundwork, and overfunding work near the end.
If malaria decreases in Senegal, but then two different charities claim this effect, one of which was doing mosquito gene drives and the other of which was distributing bednets. There is insufficient data to run an RCT good enough to provide high precision about the relative strength of these interventions. Do the VCs need to spend all their time trying to figure out how future RCT designs will or will not structurally favor their investment?
Thanks! I think those are important especially in situation where organizations more or less explicitly collaborate on something but then fail to include the other as owner of certificates that they issue. That could cause the other to respond by ending their collaboration.
Have you written about similar existing efforts to measure impact and pay just for that impact through SIBs / pay for performance frameworks and things of that nature? There are real world examples that would be good to investigate.
I haven’t but I’m aware of a forthcoming report by Rethink Priorities that covers prize contests. If you find more research on that – or on the similar dynamic of hopes for acquisitions incentivizing entrepreneurship – I’d be very interested!
I’m confused about how you trace whether a project was responsible for an outcome. Two examples:
Many scientific discoveries are the result of many researchers working on a problem for a long time, making it difficult to trace back whose contribution had what fractional impact towards e.g. an AI alignment breakthrough. But if we’re only paying the person who did the last bit, we’re dramatically underfunding all projects that lay important groundwork, and overfunding work near the end.
If malaria decreases in Senegal, but then two different charities claim this effect, one of which was doing mosquito gene drives and the other of which was distributing bednets. There is insufficient data to run an RCT good enough to provide high precision about the relative strength of these interventions. Do the VCs need to spend all their time trying to figure out how future RCT designs will or will not structurally favor their investment?
Thanks! I think those are important especially in situation where organizations more or less explicitly collaborate on something but then fail to include the other as owner of certificates that they issue. That could cause the other to respond by ending their collaboration.
I’ve written about similar problems in The Bulk of the Impact Iceberg and The Attribution Moloch.
Have you written about similar existing efforts to measure impact and pay just for that impact through SIBs / pay for performance frameworks and things of that nature? There are real world examples that would be good to investigate.
I haven’t but I’m aware of a forthcoming report by Rethink Priorities that covers prize contests. If you find more research on that – or on the similar dynamic of hopes for acquisitions incentivizing entrepreneurship – I’d be very interested!
Not a prize though this might be relevant: https://www.aqaix.com/case-studies/monterey-bay-stormwater-digital-master-plan