Nice point.
‘I also wish we didn’t accidentally make donating to AMF or GiveDirectly so uncool.’
This reminds me of the pattern where we want to do something original, so we don’t take the obvious solution.
matthewp
Karma: 217
Maximising paperclips is a misunderstood human value. Some lazy factory owners says, gee wouldn’t it be great if I could get an AI to make my paperclips for me? Then builds an AGI and asks it to make paperclips, and it then makes everything into paperclips its utility function being unreflective of its owners true desire to also have a world.
If there is a flaw here it’s probably somewhere in thinking that AGI will get built as some sort of intermediate tool and that it will be easy to rub the lamp and ask the genie to do something in easy to misunderstand natural language.
The willpower argument is actually quite good. There are ways to reduce the amount of willpower required, but the kernel of the argument applies.
My prediction for people who constantly feel bad for not living up to an exacting standard is that a majority will fall off the boat entirely.
Ego depletion is quite a narrow psychological effect. If the idea that people’s moment to moment fatigue saps moment to moment willpower is debunked, that’s far from showing that akrasia isn’t a thing in general.
In a world where general-sense akrasia was not a thing there would be a far higher rate of people being ripped like movie stars, a far lower rate of smoking, a much high rate of personal savings etc than there is in the world we inhabit.
“People don’t want to be associated with something low status and are likely to subject anything they perceive as low status to a lot of scrutiny.”
Ouch! Alas, it is true in general. However, I think it’s a dangerous heuristic when not backed by the kinds of substantive comments made in 1-6.
I do think toning down 5 might foster a better culture. Perhaps there is more information here I don’t know. But this kinda sounds like someone tried something it didn’t work out, and they don’t get a second chance. That’s not a great rubric to establish if you want people to take risks.
Thanks for the response. To clarify: in the second model both the drift and the diffusion term impact on the expected returns. If you substitute in a model return e^{q + sz}, with z a standard normal:
E[V(1)] = E[e^{q + s z}] = E[e^{sz}]e^q = e^{s^2/2} e^q > e^q
So, if we have fixed from some source that E[V(1)]=1.07=e^r then we cannot set q=r in the model with randomness while maintaining the equality. Where the equality cashes out as ‘the expected rate of return a year from now is 7%’.
Empirically estimated long run rates already take into account the effects of randomness since they are typically some sort of mean of observed returns. If this were not the case one would always have to, at least, quote the parameters in pairs (drift=such and such, vol=such and such) and perform a calculation in order to get out the expected returns.
So, what do you think of the idea that aiming for high expected returns in long term investments might not be the best thing to do, given the skewed distribution? This is, we want to ensure that most futures are ‘good’; not just a few that are ‘excellent’ lost in a mass of ‘meh’ or worse.
BTW, I did like the podcast—it does take something to make me tap out forum posts :)
Ah, that’s interesting and the nub of a difference.
The way I see it, a ‘good’ impact function would upweight the impact of low probability downside events and, perhaps, downweight low probability upside events. Maximising the expectation of such a function would push one toward policies which more reliably produce good outcomes.
This is a nice piece of accessible scholarship. It would perhaps benefit from an explicit note on why the question is interesting in this context and to this audience.
“Our actions have dominating long-term effects that we cannot ignore.”
To me, this is a strange intuition. Most actions by most people most of the time disappear like ripples in a stream.
If this were not the case, reality would tear under the weight of schemes past people had for the present. Perhaps it is actually hard to change the course of history?
Perhaps EA’s roots in philosophy lead it more readily to this failure mode?
Take the diminishing marginal returns framework above. Total benefit is not likely to be a function of a single variable ‘invested resources’. If we break ‘invested resources’ out into constituent parts we’ll hit the buffers OP identifies.
Breaking into constituent parts would mean envisaging the scenario in which the intervention was effective and adding up the concrete things one spent money on to get there: does it need new PhDs minted? There’s a related operational analysis about time lines: how many years for the message to sink in?
Also, for concrete functions, it is entirely possible that the sigmoid curve is almost flat up to an extraordinarily large total investment (and regardless of any subsequent heights it may reach). This is related to why ReLU functions are popular in neural networks: because zero gradients prevent learning.
TL;DR’s for the EA Forum/Welcome: ”Effective altruists are trying to figure out how to build a more effective AI, using paperclips, but we’re not really sure how it’s possible to do so.
Ouch.
As long as the core focuses on unusual priorities – which using neglectedness as a heuristic for prioritization will mean is likely – there’s a risk that new members get surprised when they find out about these unusual priorities
Perhaps there are also some good reasons that people with different life experience both a) don’t make it to ‘core’ and b) prioritize more near term issues.
There’s an assumption here that weirdness alone is off-putting. But, for example, technologists are used to seeing weird startup ideas and considering the contents.
This suggests a next thing to find out is: who disengages and why.
Thanks—that last link was one I’d come across and liked when looking for previous coverage. My sole previous blog post was about Pascal’s Wager. I’d found though when speaking about it that I was assuming too much for some of the audience I wanted to bring along; notwithstanding my sloppy writing :D So, I’m going to attempt to stay focused and incremental.
I also think it would be a lot more helpful to walk through how this mistake could happen in some real scenarios in the context of EA
Hopefully, we’ll get there! It’ll be mostly Bayesian though :)
Nice example, I see where you’re going with that.
I share the intuition that the second case would be easier to get people motivated for, as it represents more of a confirmed loss.
However, as your example shows actually the first case could lead to an ‘in it together’ effect on co-ordination. Assuming the information is taken seriously. Which is hard as, in advance, this kind of situation could encourage a ‘roll the dice’ mentality.
One of the topics I hope to return to here is the importance of histograms. They’re not a universal solvent. However they are easily accessible without background knowledge. And as a summary of results, they require fewer parametric assumptions.
I very much agree about the reporting of means and standard deviations, and how much a paper can sweep under the rug by that method.
Now that is a big philosophical question.
One answer is that there is no difference between ‘orders’ of random variables in Bayesian statistics. You’ve either observed something or you haven’t. If you haven’t, then you figure out what distribution the variable has.
The relationship between that distribution and the real world is a matter of your assiduousness to scientific method in constructing the model.
Lack of a reported distribution on a probability, e.g. p=0.42, isn’t the same as a lack of one. It could be taken as the assertion that the distribution on the probability is a delta function at 0.42. Which is to say the reporter is claiming to be perfectly certain what the probability is.
There is no end to how meta we could go, but the utility of going one order up here is to see that it can actually flip our preferences.
But collectively we are all better off if everyone stops holding protests for now.
Who is the ‘we’ here and by whose yardstick the benefit measured?
Animal rights activists are not turning out in large numbers to get tear gassed and beaten for the cause. This is pretty good evidence that they are not in the set of ‘everyone else who thinks their reason is as good as I think this one is’.
As usual, there are better alternatives being neglected here. Those who want more lockdown have, in this situation, two options to get it: more violence or more concessions.
Negotiation is certainly possible. So, a consequentialist might lay additional covid deaths at the step of a government which failed to negotiate.
Add to this the obvious virtue of the demand to end police brutality and recognize that black lives matter. That being an option now, it seems particularly bizarre, and wrong, to delay granting the wish.
“Making rationality more accessible.”
Sounds great, and I’ve thought about this too. But what does it look like?
Seminar series. Probably in the workplace—this would not be so scalable but for me would be highly targeted.
Video lectures. Costly, probably get wide reach though. Maybe better done in short form, slick and well marketed.
Podcast. IMHO hard to beat Rationally Speaking. However, this content should be more introductory so perhaps more of an audio series than a podcast.
How to assess what the main topics should be though? I feel the pedagogy for rationality is lacking, because for many people who are interested they picked up the basics by osmosis before getting into it in a more organised way. I.e. what is the first thing someone should learn, the second etc. For me, everything revolves around an understanding of probability—but that’s a long and somewhat indirect road to walk.