amc comments on Announcing the Future Fund’s AI Worldview Prize

• Sorry I don’t have a link. Here’s an example that’s a bit more spelled out (but still written too quickly to be careful):

Suppose there are two possible worlds, S and L (e.g. “short timelines” and “long timelines”). You currently assign 50% probability to each. You invest in actions which help with either until your expected marginal returns from investment in either are equal. If the two worlds have the same returns curves for actions on both, then you’ll want a portfolio which is split 5050 across the two (if you’re the only investor; otherwise you’ll want to push the global portfolio towards that).

Now you update either that S is 1% more likely (51%, with L at 49%).

This changes your estimate of the value of marginal returns on S and on L. You rebalance the portfolio until the marginal returns are equal again—which has 51% spending on S and 49% spending on L.

So you eliminated the marginal 1% spending on L and shifted it to a marginal 1% spending on S. How much better spent, on average, was the reallocated capital compared to before? Around 1%. So you got a 1% improvement on 1% of your spending.

If you’d made a 10% update you’d get roughly a 10% improvement on 10% of your spending. If you updated all the way to certainty on S you’d get to shift all of your money into S, and it would be a big improvement for each dollar shifted.

• I think this particular example requires an assumption of logarithmically diminishing returns, but is right with that.

(I think the point about roughly quadratic value of information applies more broadly than just for logarithmically diminishing returns. And I hadn’t realised it before. Seems important + underappreciated!)

One quirk to note: If a funder (who I want to be well-informed) is 5050 on S vs L, but my all-things-considered belief is 6040, then I would value the first 1% they shift towards my position much more than they do (maybe 10x more?) and will put comparatively little value on shifting them all the way (ie the last percent from 59% to 60% is much less important). You can get this from a pretty similar argument as in the above example.

(In fact, the funder’s own much greater valuation of shifting 10% than 1% can be seen as a two-step process where (i) they shift to 6040 beliefs, and then (ii) they first get a lot of value from shifting their allocation from 50 to 51, then slightly less from shifting from 51 to 52, etc...)

• I agree with all this. I meant to state that I was assuming logarithmic returns for the example, although I do think some smoothness argument should be enough to get it to work for small shifts.