In response to your first paragraph, I think it’s true that GiveWell will have more information about any changes in the returns function. For the reasons given the in the second post, I think it’s unlikely that GiveWell charities do have inflection points in their returns functions. I’m not sure from GiveWell’s writing whether they think that there are inflection points or not (In particular, I don’t think they take a clear stance on this in the linked post).
I think your second paragraph is answered by footnote 1 of the first post. I don’t fully understand how your third and fourth paragraphs relate to the posts. Are you simply arguing that a fuller analysis would incorporate the size of individual donations, not the total level of funding? This seems like a plausible extension.
I see that you said “claiming that expected returns are normally diminishing is compatible with expecting that true returns increase over some intervals. I think that true returns often do increase over some intervals, but that returns generally decrease in expectation.”
I wasn’t sure why this would be true in a model that describes the organization’s behavior, so I spent some time thinking it through. Here is a way to reconcile increasing returns and decreasing expected returns, with a graph. Note that when talking about “funding” here (and the x-axis of the graph) I mean “funding the organization will receive over the next planning period, i.e. calendar year”, and assume there’s no uncertainty over funding received, same as in Max’s model.
I think it’s reasonable to assume that “increasing returns” in organization’s impact often come from cases of “lumpy investments”, i.e. things with high impact and high fixed costs. In this case nothing would happen until a certain level of funding is reached, and at that point there is a discrete jump in impact. For the sake of the argument let’s assume that everything the organization does has this nature (we’ll relax this later). So you’d expect the true returns function to be a step function (see the black curve on graph).
How does the organization makes decision? First, let’s assume that these “lumpy investments” (call them “projects”) aren’t actually 0 or 1; rather, the closer the level of funding is to the “required” level, the more likely the project will happen (e.g. maybe AMF is trying to place an order for bed nets and the minimum requirement is 1000 nets, but it’s possible that they can convince the supplier to make an order of 900 nets with probability less than 1). For simplicity let’s assume the probability grows linearly (we’ll relax this later). Then the expected returns is actually the red piecewise linear function in the graph. Note that overall the marginal returns are still weakly diminishing (but they are constant within each project) because given the red expected returns function the organization would choose to first do the project with the highest marginal return (i.e. slope), then the second highest, etc.
Note: We assume the probability grows linearly. If we relax this assumption, things get more complicated. I illustrate the case where probabilities grow in a convex way within each project with the ugly green curves (note that this also covers the case with no uncertainty in the project happening or not, but rather the project has a “continuous” nature and increasing marginal returns). It’s true that you cannot call the whole thing concave (and I don’t know if mathematicians have a word to describe something like this). But from the perspective of a donor who, IN ADDITION to the model here that assumes certainty in funding levels, has uncertainty over how much funding the organization has, the “expected-expected” returns function they face (with expectation over funding level and impact) would probably be closer to the earlier piecewise linear thing, or concave. If the probabilities grow in some weird ways that are not completely convex (note that this also covers the case with no uncertainty in the project happening or not, but rather the project has a “continuous” nature and weirdly shaped, non-convex marginal returns), things may get more complicated (e.g. switching projects half way may happen if the organization always spends the next dollar on the next thing with highest marginal return) -- maybe we should abandon such possibilities since they are unintuitive.
Note: If the organization does some projects that look more like linear in the relationship between impact and funding, 1) we can still use the red piecewise linear graph, and organizations will still start with projects with the highest slopes; 2) at a fine level things are still discrete so we’ll be back to (mini) step functions.
Note: We also assumed the only uncertainty here is whether a project would happen at a funding level less than “required”. There could also be uncertainty over impact, conditional the project happening—this is not in our model, but my guess is it shouldn’t change the main results much (of course it might depend on the shape of the new layer of uncertainty, and I haven’t thought about it carefully).
All of the above is essentially based on the old idea that organizations do highest returns things first. The main addition is to look at a model where there are discrete projects (with elements of increasing returns) and still arrive at the same general conclusion.
I don’t know how many people find this useful, but I was very confused by this issue (and said some incoherent things in my earlier comments, which I’ve delete to avoid confusing people), and found that I had to think through what the organization actually does in the case of lumpy investments.
Other important issues that are related but out of the scope of this discussion include how organizations and donors act under uncertainty over donation to be received by the organization.
In response to your first paragraph, I think it’s true that GiveWell will have more information about any changes in the returns function. For the reasons given the in the second post, I think it’s unlikely that GiveWell charities do have inflection points in their returns functions. I’m not sure from GiveWell’s writing whether they think that there are inflection points or not (In particular, I don’t think they take a clear stance on this in the linked post).
I think your second paragraph is answered by footnote 1 of the first post. I don’t fully understand how your third and fourth paragraphs relate to the posts. Are you simply arguing that a fuller analysis would incorporate the size of individual donations, not the total level of funding? This seems like a plausible extension.
On increasing and decreasing (marginal) returns:
I see that you said “claiming that expected returns are normally diminishing is compatible with expecting that true returns increase over some intervals. I think that true returns often do increase over some intervals, but that returns generally decrease in expectation.”
I wasn’t sure why this would be true in a model that describes the organization’s behavior, so I spent some time thinking it through. Here is a way to reconcile increasing returns and decreasing expected returns, with a graph. Note that when talking about “funding” here (and the x-axis of the graph) I mean “funding the organization will receive over the next planning period, i.e. calendar year”, and assume there’s no uncertainty over funding received, same as in Max’s model.
I think it’s reasonable to assume that “increasing returns” in organization’s impact often come from cases of “lumpy investments”, i.e. things with high impact and high fixed costs. In this case nothing would happen until a certain level of funding is reached, and at that point there is a discrete jump in impact. For the sake of the argument let’s assume that everything the organization does has this nature (we’ll relax this later). So you’d expect the true returns function to be a step function (see the black curve on graph).
How does the organization makes decision? First, let’s assume that these “lumpy investments” (call them “projects”) aren’t actually 0 or 1; rather, the closer the level of funding is to the “required” level, the more likely the project will happen (e.g. maybe AMF is trying to place an order for bed nets and the minimum requirement is 1000 nets, but it’s possible that they can convince the supplier to make an order of 900 nets with probability less than 1). For simplicity let’s assume the probability grows linearly (we’ll relax this later). Then the expected returns is actually the red piecewise linear function in the graph. Note that overall the marginal returns are still weakly diminishing (but they are constant within each project) because given the red expected returns function the organization would choose to first do the project with the highest marginal return (i.e. slope), then the second highest, etc.
Note: We assume the probability grows linearly. If we relax this assumption, things get more complicated. I illustrate the case where probabilities grow in a convex way within each project with the ugly green curves (note that this also covers the case with no uncertainty in the project happening or not, but rather the project has a “continuous” nature and increasing marginal returns). It’s true that you cannot call the whole thing concave (and I don’t know if mathematicians have a word to describe something like this). But from the perspective of a donor who, IN ADDITION to the model here that assumes certainty in funding levels, has uncertainty over how much funding the organization has, the “expected-expected” returns function they face (with expectation over funding level and impact) would probably be closer to the earlier piecewise linear thing, or concave. If the probabilities grow in some weird ways that are not completely convex (note that this also covers the case with no uncertainty in the project happening or not, but rather the project has a “continuous” nature and weirdly shaped, non-convex marginal returns), things may get more complicated (e.g. switching projects half way may happen if the organization always spends the next dollar on the next thing with highest marginal return) -- maybe we should abandon such possibilities since they are unintuitive.
Note: If the organization does some projects that look more like linear in the relationship between impact and funding, 1) we can still use the red piecewise linear graph, and organizations will still start with projects with the highest slopes; 2) at a fine level things are still discrete so we’ll be back to (mini) step functions.
Note: We also assumed the only uncertainty here is whether a project would happen at a funding level less than “required”. There could also be uncertainty over impact, conditional the project happening—this is not in our model, but my guess is it shouldn’t change the main results much (of course it might depend on the shape of the new layer of uncertainty, and I haven’t thought about it carefully).
All of the above is essentially based on the old idea that organizations do highest returns things first. The main addition is to look at a model where there are discrete projects (with elements of increasing returns) and still arrive at the same general conclusion.
I don’t know how many people find this useful, but I was very confused by this issue (and said some incoherent things in my earlier comments, which I’ve delete to avoid confusing people), and found that I had to think through what the organization actually does in the case of lumpy investments.
Other important issues that are related but out of the scope of this discussion include how organizations and donors act under uncertainty over donation to be received by the organization.