Should donor lottery winners write reports?
Summary: I was worried that donor lottery winners writing reports might have a net negative effect by discouraging entrants. After modelling the outcomes of reports, I provisionally think that writing reports is likely a good thing and should continue. This is mostly because the winner sharing their research work may improve the quality of others’ donations.
EDIT/Update: Denise Melchin points out that people may overrate lottery winners’ evaluation. This makes me less clear about the sign of reports’ effects on others’ donations, and so I’m now pretty uncertain whether reports are good or not.
NB: I discuss possible downsides of donor lottery winners writing reports. I definitely don’t want to come across as criticising winners like Adam Gleave who wrote up a report. A lot of work went into the report, and reading it gave me useful object-level information and evaluation heuristics. I just wanted to look into a possible downside that occurred to me.
Initial worry
Section summary: if people think lottery winners have to publicly justify their choices, entries to lotteries may be reduced; if lotteries are the most efficient choice for small donors, this could be bad
I am choosing to use a donor lottery as a vehicle for my donations this year. When I was trying to decide whether or not to do so, I weighed the fact that my donations could be publicly scrutinised if I won. Would I have to follow in the footsteps of Adam Gleave’s excellent 2017 report? I certainly didn’t relish the thought of doing so.
Though I ultimately decided to use a lottery, if my reaction is common, some people could be dissuaded from entering the lottery. I think that economies of scale from donations may make the lottery the best choice for small donors. If that’s the case, people being dissuaded from entering could be a big deal.
Possible upsides to reports
Section summary: reports could increase willingness to enter, induce more work from the winner and provide valuable information to other donors
Historical record
A report allows potential lottery entrants to see the sort of donations that have historically been made by winners. Is this a good reason to write a report?
On the one hand, in many situations, it shouldn’t matter what other lottery entrants would donate to (search for “Does it matter what other donor lottery participants would do with their funds?”).
Of course, the expected outcome if you don’t win the lottery does in fact affect the overall expected outcome of the lottery. Yes, it’s true that lotteries could make sense even if the expectation after losing is neutral, but they make even more sense if that expectation is positive.
In the end, I decided not to model the increased willingness to enter from a history of reports.
More work from the winner
If the winner intends to write a public report about their donations, they are probably going to do their due diligence and come up with good reasons for their donations. Of course, it’s pretty likely they’d do that even without the report, but I think it likely increases the conscientiousness of the winner.
Improving others’ donations
As I mentioned in the note at the beginning of this post, reading Adam’s report gave me information about charities and some ideas about how to evaluate charities. Sharing the information from the evaluation is in fact the main reason Adam gives in the post for writing up the report.
I think this is likely to be fairly valuable. When I model this, I focus only on the information it provides to small donors. After all, larger donors already have quite a lot of evaluative power (Adam writes that he spent about 45 hours on the process. I don’t think a marginal week of evaluation is very important for large donors)
A model
After thinking about this for a bit, I decided to make a Guesstimate model. The model is very rough, and feedback on parameter values or missing pathways from the model would be appreciated.
Despite the model’s roughness, I find the benefit from writing reports to be large enough (~300x more benefit than cost) that I’ve changed my mind and think willing winners should probably write reports on their donations. In fact, even ignoring the knock-on effect of improving others’ donations, I find writing the report to be a net benefit.
The summary from the model’s page
I’m roughly estimating the value of donor lottery winners writing up a report of where they donated.
The causal pathways to improved outcomes that I’m modelling are (1) better donations by the lottery winner given that the winner knows they’ll be writing up a report and (2) improvements to others’ donations based on the lottery winner sharing their work.
The possible negative outcome is that it contributes to an expectation that the donor lottery winner should write up a report, leading people to not want to enter the donor lottery. If the donor lottery is in general more efficient than individual donations, this displacement of funds away from the lottery reduces the amount of good done.
A big problem with this model is that it goes through estimating the total size of the EA donor pool. This really isn’t ideal, mostly because the estimate is very rough and changes would make a big difference to the overall improvement in donation.
A more conceptual problem is that I don’t model changes in the donor lottery’s size over time. I just consider the effect at current sizes. If an effect is slower growth of the donor lottery, that could be the largest contribution to the outcome of writing reports.
Evidence that would be useful
Here are the main things that would make me change my mind:
Arguments or evidence that reports may slow uptake of lotteries
Arguments or evidence against the claim that writing a report improves the quality of the winner’s donations
Better information on the total amount of donations from small donors (on its own, this could only make me unsure whether reports are good or not; it couldn’t make me think they might be bad)
My main worry about donor lottery reports is somewhat different. Usually, people seem to assign some extra credibility to a donor’s reasoning if the donation/s is/are large. This seems reasonable to me, since donors who donate large sums often have a lot more experience with making donation decisions. But donor lottery winners have much less expertise than the average person who makes large donations (and only just as much as those long-term large donors had when they made a large donation for the first time).
In sum my concern is that people will trust donor lottery winner’s evaluations of donation targets more than they should.
This is a good point, and worth highlighting in discussion of reports (especially as we get more data on the effects of winning on donation patterns). On the other hand, the average depth and quality of investigation by winners (and the access they got) does seem higher than what they would otherwise have done, whilst less than expert donors.
This seems like an important worry. I’ve updated the main post to state that I’m now unclear whether reports are good or bad (because it seems like most of the effect comes from how others’ use the information in the reports, and it’s unclear to me whether they will mostly improve or worsen their judgement).
I do think that (a) people will discount lottery winners at least a bit relative to donors of the same size and (b) it’s good to introduce input on funding evaluation from someone with errors that are (relatively) uncorrelated with major funding bodies’ errors.
You’re right that I had subtly become nervous about joining the donor lottery because “then I’d have to do all the work that Adam did”. Thanks for reminding me I don’t have to if it doesn’t seem worth the opportunity cost, and that I can just donate to whatever seems like the best opportunity given my own models :)
It may be a good thing that people that are less inclined to write up a report will be less likely to join a donor lottery.
I don’t think this is true. The probabilities and payouts are the same for any given participant, regardless of what others do, so people who are unlikely to write up a report don’t reduce the average number of reports produced by those who would.
It actually doesn’t make a difference in terms of expected value
I think it does, it just is unlikely to change it by all that much.
Imagine there are two donor lotteries, each one having had 40k donated to them, one with lots of people in the lottery you think are very thoughtful about what projects to donate to, and one with lots of people in the lottery you think are not thoughtful about what projects to donate to. You’re considering which to add your 10k to. In either one the returns are good in expectation purely based on you getting a 20% chance to 5x your donation (which is good if you think there’s increasing marginal returns to money at this level), but also in the other 80% of worlds you have a preference for your money being allocated by people who are more thoughtful.
This isn’t the main consideration—unless you think the other people will do something actively very harmful with the money. You’d have to think that the other people will (in expectation) do something worse with a marginal 10k than you giving away 10k does good.
> but also in the other 80% of worlds you have a preference for your money being allocated by people who are more thoughtful.
For the CEA donor lottery, the pot size is fixed independent of one’s entry as the guarantor (Paul Christiano last year, the regranting pool I am administering this year) puts in funds for any unclaimed tickets. So the distribution of funding amounts for each entrant is unaffected by other entrants. It’s set up this way specifically so that people don’t even have to think about the sort of effect you discuss (the backstop fund has ~linear value of funds over the relevant range, so that isn’t an impact either).
The only thing that participating in the same lottery block as someone else matters for is correlations between your donations and theirs. E.g. if you would wind up choosing a different charity to give to depending on whether another participant won the lottery. But normally the behavior of one other donor wouldn’t change what you think is the best opportunity.
That all makes a lot of sense! Thanks.
Carl’s comment renders this irrelevant for CEA lotteries, but I think this reasoning is wrong even for the type of lotteries you imagine.
What you’re forgetting is that in the 20 % of worlds where you get your donation, you’d rather have been in the pool without thoughtful people. If you were, you will get to regrant 50k smartly, and a thoughtful person will get to regrant 40k. However, if you were in the pool with thoughtful people, the thoughtful people won’t get to regrant any money, and the 40k in the thoughtless group will go to some thoughtless cause.
When joining a group (under your assumptions, that aren’t true for CEA), you increase the winnings of everyone while decreasing the probability that they win. In expectation, they all get to regrant the same amount of money. So the only situation where the decision between groups matter is if you have some very specific ideas about marginal utility, e.g. if you want to ensure that there exists at least one thoughtful lottery winner, and don’t care much about the second.
Yes, the main effect balances out like that.
But insofar as the lottery enhances the effectiveness of donors (by letting them invest more in research if they win, amortized against a larger donation), then you want donors doing good to be enhanced and donors doing bad not to be enhanced. So you might want to try to avoid boosting pot size available to bad donors, and ensure good donors have large pots available. The CEA lottery is structured so that question doesn’t arise.
There is also the minor issue of correlation with other donors in the same block mentioned in the above comment, although you could ask CEA for a separate block if some unusual situation meant your donation plans would change a lot if you found out another block participant had won.
I think there are busy people will have the connections to make a good grant but won’t have the time to write a full report. In fact, I think there are many competent people that are very busy.
Other reasons why someone competent at picking grants may not feel comfortable with the thought of having to write a report might be because writing specifically isn’t their strength or because exposing their thinking to public scrutiny might be anxiety-inducing.
Or because their best granting opportunity can’t be justified with publically-available knowledge, or has other weird optics / reputational concerns.
I guess it depends on the details of the returns to scale for donors. If there are returns to scale across the whole range of possible values of the donor lottery, as long as one person who would do lots of work/has good judgment joins the donor lottery, we should be excited about less conscientious people joining as well.
To be more concrete, imagine the amount of good you can do with a donation goes with the square of the donation. Let’s suppose one person who will be a good donor joins the lottery with $1. Everyone else in the lottery will make a neutral donation if they win. The expected value of the lottery is (good person’s chance of winning) * (total pool)² = (1/total pool) * (total pool)² = total pool.
Obviously that exact model is a bit contrived, but it points at why non-report-writing people still bring value in a lottery
Except that the pot size isn’t constrained by the participation of small donors: the CEA donor lottery has fixed pot sizes guaranteed by large donors, and the largest donors could be ~risk-neutral over lotteries with pots of many millions of donors. So there is no effect of this kind, and there is unlikely to ever be one except at ludicrously large scales (where one could use derivatives or the like to get similar effects).
Thanks for highlighting in this comment. It don’t think I made that prominent enough in the post itself
My guess is reports in Adam’s style are likely net negative, because they will nudge the lottery winners toward donations which are publicly justifiable, and away from supporting things which are really new, small, high-risk, or their funding depends on knowledge which isn’t public or easily shareable.
Institutional funding sources are biased toward conservatism, big grants, and either established, or at least started by established individuals.
Lottery winners are in a good position to counter that bias (ca in the style of a pop-up foundation advocated for by Tyler Cowen). My guess is, donor lottery winners can have ~2-10x more impact that way in comparison to funding projects which more traditional funding sources would also fund. Hence, any pressure toward “be more like institutional funders” and away from “pop-up” is likely net negative.
What about a report along the lines of ‘I am donating in support of X, for highly illegible reasons relating to my intuition from looking at their work, and private information I have about them personally’?
That would be fine—my worry was about the 2017 report pushing standards / expectations in a direction which I think will lead to less impact long-term.
This worry is not entirely hypothetical: note for example the comment by aarongertler, an EA forum moderator
(To be clear, I also want to add that Adam did a great job carefully reviewing the organizations and approaching the problem with IMO similar level of rigor as an established funding organization, and I admire the work. I just want to avoid this approach becoming something which is expected.)
We can imagine three categories of grants:
A. Publically justifiable
B. Privately justifiable
C. Unjustifiable :)
I agree reports like Adam’s will move people from B to A, but I think they will also move people from C to A, by forcing them to examine their choices more carefully and hold themselves to a higher standard.
This model prompts two possible sources of disagreement: you could disagree about the relative proportions of people moving from B vs. from C, or you could disagree about how bad it is to have a mix of B and C vs. more A.
To address the second question, if you think that B is 2-10x more valuable than A, then even if donations in category C are worthless (leaving aside the chance they are net negative), an equal mix of B and C is better than just A, and towards the 10x end of that spectrum, you can justify up to 90% C and 10% B.
But let’s return to that parenthetical – could more C donations be net negative, even aside from opportunity cost? I think this risk is underexamined. I suspect most projects won’t directly do harm, but well-funded blunders are more visible and reputationally damaging.
I think 2-10x is the wrong average multiplier across lottery winners (though, in fairness, you didn’t explicitly claim it was an average). In order to make good grants to new small high-risk things, you need to hear about them, and I suspect most lottery participants don’t have the necessary networks and don’t have special access to significant private information – after all, private information doesn’t spread well.
Concretely I’m suggesting that the median lottery participant doesn’t get any benefit at all from the ability to use private information.
I disagree. You should not have as a central example some sort of secret, but trust. Transitivity of trust is limited, and everybody has a unique position in the trust network. Many will have interesting opportunities in their network neighborhoods. (I don’t claim to be typical, but still: I can easily list maybe a dozen of such not easily justifiable opportunities where I could send money; even if I’m somewhere on the tail on the distribution, I’d guess typical lottery winner has at leas 1 or 2 such opportunitites)
That the use of the funds will be worse when writing a report is plausible. Do you also think that reports change others’ giving either negligibly or negatively?
It’s hard to estimate.
Winning the lottery likely amplifies the voice of the winner, but the effect may be conditional on how much credibility the winner had beforehand. So far, the lottery winners were highly trusted people working in central organizations.
Overall, I would estimate the indirect effect on giving by other individual donors is with 90% confidence within 3x the size of the direct effect, with an unclear sign. There is a significant competition for the attention (and money) of individual donors