Yes, but ex ante. The higher up the distribution the harder they will get to identify because ‘if it’s transformative it’s not measurable; if it’s measurable it’s not transformative’. The weakness of the measurements mean you’re going to be hit with a lot of regression to the mean.
Also that stuff is likely to be weird (must be extreme on neglectedness if it’s so important and still useful to put more money in to), and so just as it’s hard to get someone to give enormous amounts, it will probably also be hard to move donations there.
I’m not talking about in-practice difficulties like convincing people to donate. I’m just talking about statistics.
Can you point to actual parameters for a toy model where changing the distribution from normal to log-normal (holding median and variance constant) decreases the benefits you can get from convincing people to switch charities? The model can include things like regression to the mean and weirdness penalties. My intuition is that the parameter space of such models is very small, if it exists at all.
If we thought that the charity they were switching to were only at the 95th percentile, it could be worse in a log-normal case than a normal case (indeed it could be worse than not getting them to switch).
However that would be an unusual belief for us. More reasonably we might think it were uniformly drawn from the top ten percent (or something). And then log-normal is again much better than normal. I agree with the thrust of your intuition.
Yeah, in GiveWell classic, you’re generally going to estimate that a high-impact charity is on the 95th percentile but with uncertainty around that. Which is in-between the cases that you describe.
I can imagine that knowing that something is on the 95th percentile with high certainty might be worse than guessing that something is on the 95th percentile with high uncertainty, if you have a prior that is some mixture of log-normal, normal and power-law. That’s what we’d have to show to really question the classic GiveWell model.
I’m sure you’re right about the math, but I am concerned with the in-practice difficulties.
My point about the mean was simply that one shouldn’t compare the max-median on the log-normal—which would be natural if you visualise where ‘typical donations’ that you see go—but rather the max-mean, which is a less extreme ratio. I wasn’t drawing a contrast with a normal or any other distribution.
I’m also not sure about the answer to my question to Ryan—maybe the effectiveness is still the better focus, but I’m prompting him with possible counterarguments.
Ah, from your first comment it sounded like you were comparing the mean of the log-normal to the mean of a less-skewed distribution, rather than to the median of the log-normal. That sounds more reasonable.
In practice, isn’t it relatively easy to identify whether someone is already up the right tail in their giving? I don’t recall struggling with this in initial conversations. You can just ask whether they give abroad for instance, which will seemingly get you most of the way since most people don’t (i.e. it is true that those who do vastly skew the mean, but you can just exclude almost all of them ex ante...).
What you’re saying might be appropriate for mass marketing I suppose where you can’t cut off the right tail.
Sorry, pretty unclear post on my part. Owen basically got it right though; if we’re talking practically rather than theoretically, you don’t have to decide to always focus on effectiveness or always focus on quantity. You can choose, and your choice can be influenced by the information you have about your audience.
Since most individual people are around the median/mode and a long way below the mean, for most individual people talking about effectiveness is correct. There are a few exceptions to this (those ‘up the right tail’), and then you can talk about amount...or just accept that you aren’t going to achieve that much here and find more people where you can talk about effectiveness!
This is obviously dependent on how much ability you have to discriminate based on your audience, which in turn depends on context, hence my ‘mass marketing’ point.
I think it’s the reverse—if you exclude the people who already do give effectively then you’ve brought the mean of those who remain down closer to the vicinity of the median.
Yes, but ex ante. The higher up the distribution the harder they will get to identify because ‘if it’s transformative it’s not measurable; if it’s measurable it’s not transformative’. The weakness of the measurements mean you’re going to be hit with a lot of regression to the mean.
Also that stuff is likely to be weird (must be extreme on neglectedness if it’s so important and still useful to put more money in to), and so just as it’s hard to get someone to give enormous amounts, it will probably also be hard to move donations there.
I’m not talking about in-practice difficulties like convincing people to donate. I’m just talking about statistics.
Can you point to actual parameters for a toy model where changing the distribution from normal to log-normal (holding median and variance constant) decreases the benefits you can get from convincing people to switch charities? The model can include things like regression to the mean and weirdness penalties. My intuition is that the parameter space of such models is very small, if it exists at all.
If we thought that the charity they were switching to were only at the 95th percentile, it could be worse in a log-normal case than a normal case (indeed it could be worse than not getting them to switch).
However that would be an unusual belief for us. More reasonably we might think it were uniformly drawn from the top ten percent (or something). And then log-normal is again much better than normal. I agree with the thrust of your intuition.
Yeah, in GiveWell classic, you’re generally going to estimate that a high-impact charity is on the 95th percentile but with uncertainty around that. Which is in-between the cases that you describe.
I can imagine that knowing that something is on the 95th percentile with high certainty might be worse than guessing that something is on the 95th percentile with high uncertainty, if you have a prior that is some mixture of log-normal, normal and power-law. That’s what we’d have to show to really question the classic GiveWell model.
I’m sure you’re right about the math, but I am concerned with the in-practice difficulties.
My point about the mean was simply that one shouldn’t compare the max-median on the log-normal—which would be natural if you visualise where ‘typical donations’ that you see go—but rather the max-mean, which is a less extreme ratio. I wasn’t drawing a contrast with a normal or any other distribution.
I’m also not sure about the answer to my question to Ryan—maybe the effectiveness is still the better focus, but I’m prompting him with possible counterarguments.
Ah, from your first comment it sounded like you were comparing the mean of the log-normal to the mean of a less-skewed distribution, rather than to the median of the log-normal. That sounds more reasonable.
Yep my bad making the original comment ambiguous.
In practice, isn’t it relatively easy to identify whether someone is already up the right tail in their giving? I don’t recall struggling with this in initial conversations. You can just ask whether they give abroad for instance, which will seemingly get you most of the way since most people don’t (i.e. it is true that those who do vastly skew the mean, but you can just exclude almost all of them ex ante...).
What you’re saying might be appropriate for mass marketing I suppose where you can’t cut off the right tail.
Sorry Alex, I can’t quite follow what you’re arguing here. Are you saying you can just focus on people who already give fairly effectively?
Sorry, pretty unclear post on my part. Owen basically got it right though; if we’re talking practically rather than theoretically, you don’t have to decide to always focus on effectiveness or always focus on quantity. You can choose, and your choice can be influenced by the information you have about your audience.
Since most individual people are around the median/mode and a long way below the mean, for most individual people talking about effectiveness is correct. There are a few exceptions to this (those ‘up the right tail’), and then you can talk about amount...or just accept that you aren’t going to achieve that much here and find more people where you can talk about effectiveness!
This is obviously dependent on how much ability you have to discriminate based on your audience, which in turn depends on context, hence my ‘mass marketing’ point.
I think it’s the reverse—if you exclude the people who already do give effectively then you’ve brought the mean of those who remain down closer to the vicinity of the median.