I’ve thought about this much less than you so take everything I say with a huge grain of salt, but I think I’m less worried about incentives and feedback mechanisms for the developers and more worried about incentives and feedback mechanisms for the “customers.” As you noted, the “something for nothing” angle is pretty perverse, but the real theoretical issue to me isn’t the problem with providing feedback on developers (which a customer realistically would give about things like features, UI, timeliness, and as you noted very rarely technical quality concerns), but having frank assessments of how valuable projects actually are, both ex ante and ex post.
By default if you’re offering free, high-quality labor to people, people are incentivized to exaggerate the value of their projects. Of course everybody batting for “team EA” lowers the incentive somewhat, but I would guess not enough.
Even if we imagine that this incentive does not exist (eg. because EA orgs are by stipulation always honest and deontologically opposed to exaggeration or motivated reasoning), I’d still expect having only goodwill and your own judgment (as opposed to goodwill, your own judgment, and $s) as the only check for impact to apply a strong selection effect on your customers, such that the customers who systematically overestimate the impact of (free) tech to their work will crowd out customers with more legitimate uses.
I suspect this issue to continue to exist even with paid EA work, as long as the value of your work exceeds the cost, but at least there is some sanity check on how skewed the incentives can get.
I might have misunderstood, but wouldn’t the same apply to the same customers applying for funding for tech work? Ie the people who systematically overestimate the impact of free tech to their work would systematically overestimate the impact of paid tech to it, and so inasmuch as this is a problem (and I definitely agree with treating EAs and orgs as being at least somewhat selfish/self-biased), it’s one that already exists.
If you do think an agency would create new incentives for such gaming of the system, it seems like it could still avoid them by being part of or entirely deferring to a granting agency. This seems basically like the mode in part 4, with all the upsides and downsides that would have. I’m pretty neutral on that mode vs independence, though Luke mentioned in the comments to his recent thread that he was against expanding OpenPhil’s domain that way.
I might have misunderstood, but wouldn’t the same apply to the same customers applying for funding for tech work?
I think this applies somewhat for newer orgs and for very discerning/active donors. But my guess is not as much for more established orgs and busier/less discerning funders, because a funder can in theory look at the total inputs and outputs of an EA org and consider either average and marginal cost-benefit when deciding whether to fund them, rather than do sophisticated accounting on specific purchase decisions. So at least in a retrospective evaluation, rather than a line item consideration of whether past donations specifically spent on an improved website, better HR software, external recruiters, higher quality toilet paper etc, is worth it, I can just look at an org’s reported outputs and make a broad judgment of whether it’s worth the money (or other capital) that went in, and then further decide whether I trust the org’s leadership to make good decisions with extra $s. This means that customer EA orgs may be specifically mistaken about funding software work, but unless they make up for it in efficiency gains elsewhere, the total relatively low efficiency will become obvious eventually, and thus customer orgs are incentivized to maximize cost-benefits across the org.
Whereas if the EA org made a lot of use of high-quality labor implicitly or explicitly paid for elsewhere in the movement, this is harder to trace/account for as a funder.
Re:
I’ve thought about this much less than you so take everything I say with a huge grain of salt, but I think I’m less worried about incentives and feedback mechanisms for the developers and more worried about incentives and feedback mechanisms for the “customers.” As you noted, the “something for nothing” angle is pretty perverse, but the real theoretical issue to me isn’t the problem with providing feedback on developers (which a customer realistically would give about things like features, UI, timeliness, and as you noted very rarely technical quality concerns), but having frank assessments of how valuable projects actually are, both ex ante and ex post.
By default if you’re offering free, high-quality labor to people, people are incentivized to exaggerate the value of their projects. Of course everybody batting for “team EA” lowers the incentive somewhat, but I would guess not enough.
Even if we imagine that this incentive does not exist (eg. because EA orgs are by stipulation always honest and deontologically opposed to exaggeration or motivated reasoning), I’d still expect having only goodwill and your own judgment (as opposed to goodwill, your own judgment, and $s) as the only check for impact to apply a strong selection effect on your customers, such that the customers who systematically overestimate the impact of (free) tech to their work will crowd out customers with more legitimate uses.
I suspect this issue to continue to exist even with paid EA work, as long as the value of your work exceeds the cost, but at least there is some sanity check on how skewed the incentives can get.
I might have misunderstood, but wouldn’t the same apply to the same customers applying for funding for tech work? Ie the people who systematically overestimate the impact of free tech to their work would systematically overestimate the impact of paid tech to it, and so inasmuch as this is a problem (and I definitely agree with treating EAs and orgs as being at least somewhat selfish/self-biased), it’s one that already exists.
If you do think an agency would create new incentives for such gaming of the system, it seems like it could still avoid them by being part of or entirely deferring to a granting agency. This seems basically like the mode in part 4, with all the upsides and downsides that would have. I’m pretty neutral on that mode vs independence, though Luke mentioned in the comments to his recent thread that he was against expanding OpenPhil’s domain that way.
I think this applies somewhat for newer orgs and for very discerning/active donors. But my guess is not as much for more established orgs and busier/less discerning funders, because a funder can in theory look at the total inputs and outputs of an EA org and consider either average and marginal cost-benefit when deciding whether to fund them, rather than do sophisticated accounting on specific purchase decisions. So at least in a retrospective evaluation, rather than a line item consideration of whether past donations specifically spent on an improved website, better HR software, external recruiters, higher quality toilet paper etc, is worth it, I can just look at an org’s reported outputs and make a broad judgment of whether it’s worth the money (or other capital) that went in, and then further decide whether I trust the org’s leadership to make good decisions with extra $s. This means that customer EA orgs may be specifically mistaken about funding software work, but unless they make up for it in efficiency gains elsewhere, the total relatively low efficiency will become obvious eventually, and thus customer orgs are incentivized to maximize cost-benefits across the org.
Whereas if the EA org made a lot of use of high-quality labor implicitly or explicitly paid for elsewhere in the movement, this is harder to trace/account for as a funder.