TL;DR’s for the EA Forum/Welcome: ”Effective altruists are trying to figure out how to build a more effective AI, using paperclips, but we’re not really sure how it’s possible to do so.
Ouch.
matthewp
Karma: 219
Perhaps EA’s roots in philosophy lead it more readily to this failure mode?
Take the diminishing marginal returns framework above. Total benefit is not likely to be a function of a single variable ‘invested resources’. If we break ‘invested resources’ out into constituent parts we’ll hit the buffers OP identifies.
Breaking into constituent parts would mean envisaging the scenario in which the intervention was effective and adding up the concrete things one spent money on to get there: does it need new PhDs minted? There’s a related operational analysis about time lines: how many years for the message to sink in?
Also, for concrete functions, it is entirely possible that the sigmoid curve is almost flat up to an extraordinarily large total investment (and regardless of any subsequent heights it may reach). This is related to why ReLU functions are popular in neural networks: because zero gradients prevent learning.
I would spend every penny unblocking the pathway to a vaccine.
Multithreading stages of the clinical trials. E.g. have combination vaccine safety trials started ahead of proof of efficacy for individual trials?
Reviewing what the target for effectiveness really is. E.g. would a vaccine which tips the population reproduction rate below 1 without providing individual level guarantees? How long would we wait to see if anything better would become available? Would we be prepared to risk needing two phases of vaccination? Would we make an earlier safe vaccine with low efficacy available optionally?
Cohort recruitment should be easy, volunteerism is at max right now. If it is not, the problem must be logistical. What support is needed (coach services, apps etc)?
Streamlining regulatory hurdles. Assume tests go well, is there any blocking legislation which does not make sense? What bills could we predict are necessary now to pre-empt the blockage?
Preparing for mass production of different kinds of vaccines. Assume each system works and ask what would be needed to scale it out. Make bets on any cheap elements of any relevant systems. Use the time before production starts to duplicate existing production facilities.
Readying the logistics for administration. E.g. given PPE shenanigans, it wouldn’t surprise me if we had a lack of disposable syringes etc. Most of this could be prepared in advance of actually having a vaccine.
The basic ideas and test candidates are already known. The lag between now and mass roll out is therefore (mostly) dependent on our organizational skills.
<waving hands> UK GDP is ~£2.9 trillion. The recession will shave at least 10% off that. The government takes ~30% of GDP in tax. If bringing forward mass vaccination could shave a quarter off an 18 month recession, it would be revenue neutral to pay £100 billion to do it. So, if some of the above sounds too expensive, it’s because a bigger budget is necessary and likely justified. It would take an order 1000x correction to change this reasoning. </waving hands>
“Our actions have dominating long-term effects that we cannot ignore.”
To me, this is a strange intuition. Most actions by most people most of the time disappear like ripples in a stream.
If this were not the case, reality would tear under the weight of schemes past people had for the present. Perhaps it is actually hard to change the course of history?
This is a nice piece of accessible scholarship. It would perhaps benefit from an explicit note on why the question is interesting in this context and to this audience.
Ah, that’s interesting and the nub of a difference.
The way I see it, a ‘good’ impact function would upweight the impact of low probability downside events and, perhaps, downweight low probability upside events. Maximising the expectation of such a function would push one toward policies which more reliably produce good outcomes.
So, what do you think of the idea that aiming for high expected returns in long term investments might not be the best thing to do, given the skewed distribution? This is, we want to ensure that most futures are ‘good’; not just a few that are ‘excellent’ lost in a mass of ‘meh’ or worse.
BTW, I did like the podcast—it does take something to make me tap out forum posts :)
Thanks for the response. To clarify: in the second model both the drift and the diffusion term impact on the expected returns. If you substitute in a model return e^{q + sz}, with z a standard normal:
E[V(1)] = E[e^{q + s z}] = E[e^{sz}]e^q = e^{s^2/2} e^q > e^q
So, if we have fixed from some source that E[V(1)]=1.07=e^r then we cannot set q=r in the model with randomness while maintaining the equality. Where the equality cashes out as ‘the expected rate of return a year from now is 7%’.
Empirically estimated long run rates already take into account the effects of randomness since they are typically some sort of mean of observed returns. If this were not the case one would always have to, at least, quote the parameters in pairs (drift=such and such, vol=such and such) and perform a calculation in order to get out the expected returns.
“People don’t want to be associated with something low status and are likely to subject anything they perceive as low status to a lot of scrutiny.”
Ouch! Alas, it is true in general. However, I think it’s a dangerous heuristic when not backed by the kinds of substantive comments made in 1-6.
I do think toning down 5 might foster a better culture. Perhaps there is more information here I don’t know. But this kinda sounds like someone tried something it didn’t work out, and they don’t get a second chance. That’s not a great rubric to establish if you want people to take risks.
Ego depletion is quite a narrow psychological effect. If the idea that people’s moment to moment fatigue saps moment to moment willpower is debunked, that’s far from showing that akrasia isn’t a thing in general.
In a world where general-sense akrasia was not a thing there would be a far higher rate of people being ripped like movie stars, a far lower rate of smoking, a much high rate of personal savings etc than there is in the world we inhabit.
The willpower argument is actually quite good. There are ways to reduce the amount of willpower required, but the kernel of the argument applies.
My prediction for people who constantly feel bad for not living up to an exacting standard is that a majority will fall off the boat entirely.
Maximising paperclips is a misunderstood human value. Some lazy factory owners says, gee wouldn’t it be great if I could get an AI to make my paperclips for me? Then builds an AGI and asks it to make paperclips, and it then makes everything into paperclips its utility function being unreflective of its owners true desire to also have a world.
If there is a flaw here it’s probably somewhere in thinking that AGI will get built as some sort of intermediate tool and that it will be easy to rub the lamp and ask the genie to do something in easy to misunderstand natural language.
Nice point.
‘I also wish we didn’t accidentally make donating to AMF or GiveDirectly so uncool.’
This reminds me of the pattern where we want to do something original, so we don’t take the obvious solution.
“Making rationality more accessible.”
Sounds great, and I’ve thought about this too. But what does it look like?
Seminar series. Probably in the workplace—this would not be so scalable but for me would be highly targeted.
Video lectures. Costly, probably get wide reach though. Maybe better done in short form, slick and well marketed.
Podcast. IMHO hard to beat Rationally Speaking. However, this content should be more introductory so perhaps more of an audio series than a podcast.
How to assess what the main topics should be though? I feel the pedagogy for rationality is lacking, because for many people who are interested they picked up the basics by osmosis before getting into it in a more organised way. I.e. what is the first thing someone should learn, the second etc. For me, everything revolves around an understanding of probability—but that’s a long and somewhat indirect road to walk.
Perhaps there are also some good reasons that people with different life experience both a) don’t make it to ‘core’ and b) prioritize more near term issues.
There’s an assumption here that weirdness alone is off-putting. But, for example, technologists are used to seeing weird startup ideas and considering the contents.
This suggests a next thing to find out is: who disengages and why.