Nice point.
‘I also wish we didn’t accidentally make donating to AMF or GiveDirectly so uncool.’
This reminds me of the pattern where we want to do something original, so we don’t take the obvious solution.
Nice point.
‘I also wish we didn’t accidentally make donating to AMF or GiveDirectly so uncool.’
This reminds me of the pattern where we want to do something original, so we don’t take the obvious solution.
Maximising paperclips is a misunderstood human value. Some lazy factory owners says, gee wouldn’t it be great if I could get an AI to make my paperclips for me? Then builds an AGI and asks it to make paperclips, and it then makes everything into paperclips its utility function being unreflective of its owners true desire to also have a world.
If there is a flaw here it’s probably somewhere in thinking that AGI will get built as some sort of intermediate tool and that it will be easy to rub the lamp and ask the genie to do something in easy to misunderstand natural language.
The willpower argument is actually quite good. There are ways to reduce the amount of willpower required, but the kernel of the argument applies.
My prediction for people who constantly feel bad for not living up to an exacting standard is that a majority will fall off the boat entirely.
Ego depletion is quite a narrow psychological effect. If the idea that people’s moment to moment fatigue saps moment to moment willpower is debunked, that’s far from showing that akrasia isn’t a thing in general.
In a world where general-sense akrasia was not a thing there would be a far higher rate of people being ripped like movie stars, a far lower rate of smoking, a much high rate of personal savings etc than there is in the world we inhabit.
“People don’t want to be associated with something low status and are likely to subject anything they perceive as low status to a lot of scrutiny.”
Ouch! Alas, it is true in general. However, I think it’s a dangerous heuristic when not backed by the kinds of substantive comments made in 1-6.
I do think toning down 5 might foster a better culture. Perhaps there is more information here I don’t know. But this kinda sounds like someone tried something it didn’t work out, and they don’t get a second chance. That’s not a great rubric to establish if you want people to take risks.
Thanks for the response. To clarify: in the second model both the drift and the diffusion term impact on the expected returns. If you substitute in a model return e^{q + sz}, with z a standard normal:
E[V(1)] = E[e^{q + s z}] = E[e^{sz}]e^q = e^{s^2/2} e^q > e^q
So, if we have fixed from some source that E[V(1)]=1.07=e^r then we cannot set q=r in the model with randomness while maintaining the equality. Where the equality cashes out as ‘the expected rate of return a year from now is 7%’.
Empirically estimated long run rates already take into account the effects of randomness since they are typically some sort of mean of observed returns. If this were not the case one would always have to, at least, quote the parameters in pairs (drift=such and such, vol=such and such) and perform a calculation in order to get out the expected returns.
So, what do you think of the idea that aiming for high expected returns in long term investments might not be the best thing to do, given the skewed distribution? This is, we want to ensure that most futures are ‘good’; not just a few that are ‘excellent’ lost in a mass of ‘meh’ or worse.
BTW, I did like the podcast—it does take something to make me tap out forum posts :)
Ah, that’s interesting and the nub of a difference.
The way I see it, a ‘good’ impact function would upweight the impact of low probability downside events and, perhaps, downweight low probability upside events. Maximising the expectation of such a function would push one toward policies which more reliably produce good outcomes.
This is a nice piece of accessible scholarship. It would perhaps benefit from an explicit note on why the question is interesting in this context and to this audience.
“Our actions have dominating long-term effects that we cannot ignore.”
To me, this is a strange intuition. Most actions by most people most of the time disappear like ripples in a stream.
If this were not the case, reality would tear under the weight of schemes past people had for the present. Perhaps it is actually hard to change the course of history?
Perhaps EA’s roots in philosophy lead it more readily to this failure mode?
Take the diminishing marginal returns framework above. Total benefit is not likely to be a function of a single variable ‘invested resources’. If we break ‘invested resources’ out into constituent parts we’ll hit the buffers OP identifies.
Breaking into constituent parts would mean envisaging the scenario in which the intervention was effective and adding up the concrete things one spent money on to get there: does it need new PhDs minted? There’s a related operational analysis about time lines: how many years for the message to sink in?
Also, for concrete functions, it is entirely possible that the sigmoid curve is almost flat up to an extraordinarily large total investment (and regardless of any subsequent heights it may reach). This is related to why ReLU functions are popular in neural networks: because zero gradients prevent learning.
TL;DR’s for the EA Forum/Welcome: ”Effective altruists are trying to figure out how to build a more effective AI, using paperclips, but we’re not really sure how it’s possible to do so.
Ouch.
As long as the core focuses on unusual priorities – which using neglectedness as a heuristic for prioritization will mean is likely – there’s a risk that new members get surprised when they find out about these unusual priorities
Perhaps there are also some good reasons that people with different life experience both a) don’t make it to ‘core’ and b) prioritize more near term issues.
There’s an assumption here that weirdness alone is off-putting. But, for example, technologists are used to seeing weird startup ideas and considering the contents.
This suggests a next thing to find out is: who disengages and why.
Thanks—that last link was one I’d come across and liked when looking for previous coverage. My sole previous blog post was about Pascal’s Wager. I’d found though when speaking about it that I was assuming too much for some of the audience I wanted to bring along; notwithstanding my sloppy writing :D So, I’m going to attempt to stay focused and incremental.
I also think it would be a lot more helpful to walk through how this mistake could happen in some real scenarios in the context of EA
Hopefully, we’ll get there! It’ll be mostly Bayesian though :)
“Making rationality more accessible.”
Sounds great, and I’ve thought about this too. But what does it look like?
Seminar series. Probably in the workplace—this would not be so scalable but for me would be highly targeted.
Video lectures. Costly, probably get wide reach though. Maybe better done in short form, slick and well marketed.
Podcast. IMHO hard to beat Rationally Speaking. However, this content should be more introductory so perhaps more of an audio series than a podcast.
How to assess what the main topics should be though? I feel the pedagogy for rationality is lacking, because for many people who are interested they picked up the basics by osmosis before getting into it in a more organised way. I.e. what is the first thing someone should learn, the second etc. For me, everything revolves around an understanding of probability—but that’s a long and somewhat indirect road to walk.