I think your critique of the ITN framework might be flawed. (though I haven’t read section 2 yet). I assume some of my critique must be wrong as I still feel a bit confused about it, but I really need to get back to work...
One point that I think is a bit confusing is that you use the term marginal cost-effectiveness. To my knowledge this is not an acknowledged term in economics or elsewhere. What I think you mean instead is the average benefit given a certain amount of money.
Cost-effectiveness is (according to wikipedia at least) generally expressed at something like: 100USD/QALY. This is done by looking at how much a program costs and how many QALYs it created. So we get the average benefit of each $100 dollars for the program by doing this. However, we gain no insight as to what happened inside of the program. Maybe the first 100USD did all the work and the rest ended up being fluff, we don’t know. More likely would be that the money had diminishing marginal returns.
When talking about tractability you say:
with importance and tractability alone, you could calculate the marginal cost-effectiveness of work on a problem, which is ultimately what we care about
You would know cost-effectiveness if you knew the amount spent so far/amount of good done. You know the amount spent from neglectedness but don’t know the amount already done with the money spent. I guess marginal cost-effectiveness = average benefit from X more dollars. Let’s say that this is doubling the amount spent so far. I don’t think we can construe this as marginal though as doubling the money is not an ‘at the margin’ change. I think then that tractability gives you average benefit from X more dollars (so no need for scale).
We still need neglectedness and scale though to do a proper analysis.
Scale because if something wasn’t a big problem, why solve it? And to look at neglectedness let’s use some made-up numbers:
Say that we as humanity have already spent 1 trillion USD on climate change (we use this to measure neglectedness) and got a 1% reduction in risk of an extinction event (use this to calculate the amount of good = .01* present value of all future lives). That gives us cost-effectiveness (cost/good done). We DON’T know however what happens at the margin (if we put more money in). We just have an average. Assuming constant returns may seem (almost) reasonable on an intervention like bed net distribution but it seems less reasonable when we’ve already spent 1 trillion USD on a problem. Then what we really need to know is the benefit of, say, another 1 trillion USD. This I think is what 80k’s tractability measure is trying to get at. The average benefit (or cost-effectiveness) of another hunk of money/resources.
So defending neglectedness a bit. If we think that the marginal benefit to more money is not constant (which seems eminently reasonable) then it makes sense to try to find out where we are on the curve. Neglectedness helps to show us where we might be on the curve, even though we have little idea what the curve looks like (though I would generally find it safe to assume decreasing marginal returns). If we’re on the flat bit of the diminishing marginal returns curve then we sure as hell want to know, or at least find evidence which would indicate that to be likely.
So then neglectedness is trying to find where we are on the curve, which will help us understand the marginal return to one more person/dollar entering (the true margin). This might mean that even if a problem is unsolvable there might be easy gains to be had in terms of reducing risk on the margin. For something that is neglected but not tractable we might be able to have huge benefits by throwing a few people/dollars in (get x-risk reductions f.ex) but that might peter off really quickly thus making it untractable. It would be less attractive overall then because putting a lot of people in would not be worth it.
Tractability says, if we were to dump lot’s more money, what are the average returns going to look like. If we are now at the flat part of the curve average returns might be FAR lower than they were in a cost-effectiveness analysis (average returns of past spending) of what we already spent.
Maybe new intuitions for these:
Neglectedness: How much bang for the buck do we get for one more person/dollar?
Tractability: Is it worth dumping lot’s of resources into this problem?
I think you identified the same problem i saw. If you have a small problem, then there no reason to call it ‘neglected’ if you put enough resources into solving that small problem. You have to put all problems into context—no reason to spend alot of resources to 100% solve a small problem when you put no resources into trying to solve a big problem. This is like spending alot of money to give sandwiches to solve temporary hunger problem for a few people, while ‘neglecting ’ the entire issue of global hunger or food scarcity.
I think your critique of the ITN framework might be flawed. (though I haven’t read section 2 yet). I assume some of my critique must be wrong as I still feel a bit confused about it, but I really need to get back to work...
One point that I think is a bit confusing is that you use the term marginal cost-effectiveness. To my knowledge this is not an acknowledged term in economics or elsewhere. What I think you mean instead is the average benefit given a certain amount of money.
Cost-effectiveness is (according to wikipedia at least) generally expressed at something like: 100USD/QALY. This is done by looking at how much a program costs and how many QALYs it created. So we get the average benefit of each $100 dollars for the program by doing this. However, we gain no insight as to what happened inside of the program. Maybe the first 100USD did all the work and the rest ended up being fluff, we don’t know. More likely would be that the money had diminishing marginal returns.
When talking about tractability you say:
You would know cost-effectiveness if you knew the amount spent so far/amount of good done. You know the amount spent from neglectedness but don’t know the amount already done with the money spent. I guess marginal cost-effectiveness = average benefit from X more dollars. Let’s say that this is doubling the amount spent so far. I don’t think we can construe this as marginal though as doubling the money is not an ‘at the margin’ change. I think then that tractability gives you average benefit from X more dollars (so no need for scale).
We still need neglectedness and scale though to do a proper analysis.
Scale because if something wasn’t a big problem, why solve it? And to look at neglectedness let’s use some made-up numbers:
Say that we as humanity have already spent 1 trillion USD on climate change (we use this to measure neglectedness) and got a 1% reduction in risk of an extinction event (use this to calculate the amount of good = .01* present value of all future lives). That gives us cost-effectiveness (cost/good done). We DON’T know however what happens at the margin (if we put more money in). We just have an average. Assuming constant returns may seem (almost) reasonable on an intervention like bed net distribution but it seems less reasonable when we’ve already spent 1 trillion USD on a problem. Then what we really need to know is the benefit of, say, another 1 trillion USD. This I think is what 80k’s tractability measure is trying to get at. The average benefit (or cost-effectiveness) of another hunk of money/resources.
So defending neglectedness a bit. If we think that the marginal benefit to more money is not constant (which seems eminently reasonable) then it makes sense to try to find out where we are on the curve. Neglectedness helps to show us where we might be on the curve, even though we have little idea what the curve looks like (though I would generally find it safe to assume decreasing marginal returns). If we’re on the flat bit of the diminishing marginal returns curve then we sure as hell want to know, or at least find evidence which would indicate that to be likely.
So then neglectedness is trying to find where we are on the curve, which will help us understand the marginal return to one more person/dollar entering (the true margin). This might mean that even if a problem is unsolvable there might be easy gains to be had in terms of reducing risk on the margin. For something that is neglected but not tractable we might be able to have huge benefits by throwing a few people/dollars in (get x-risk reductions f.ex) but that might peter off really quickly thus making it untractable. It would be less attractive overall then because putting a lot of people in would not be worth it.
Tractability says, if we were to dump lot’s more money, what are the average returns going to look like. If we are now at the flat part of the curve average returns might be FAR lower than they were in a cost-effectiveness analysis (average returns of past spending) of what we already spent.
Maybe new intuitions for these:
Neglectedness: How much bang for the buck do we get for one more person/dollar?
Tractability: Is it worth dumping lot’s of resources into this problem?
I think you identified the same problem i saw. If you have a small problem, then there no reason to call it ‘neglected’ if you put enough resources into solving that small problem. You have to put all problems into context—no reason to spend alot of resources to 100% solve a small problem when you put no resources into trying to solve a big problem. This is like spending alot of money to give sandwiches to solve temporary hunger problem for a few people, while ‘neglecting ’ the entire issue of global hunger or food scarcity.