This has the ring of truth to it.
matthewp
Karma: 219
Do you know if there are any orgs in the UK housing Ukrainian refugees?
> How difficult should we expect AI alignment to be?
With many of the AI questions, one needs to reason backwards rather than pose the general question.
Suppose we all die because unaligned AI. What form did the unaligned AI take? How did it work? Which things that exist now were progenitors of it, and what changed to make it dangerous? How could those problems have been avoided, technically? Organisationally?
I don’t see how useful alignment research can be done quite separately to capabilities research. Otherwise we’ll get will be people coming in at the wrong time with a bunch of ideas that lack technical purchase.
Similarly, the questions about what applications we’ll see first are already hinted at in capabilities research.That being the case, it will take way more energy than 1 year for someone to upskill because they actually need to understand something about capabilities work.
As someone with a mathematical background, I see a claim about a general implication (the RC) arising from Total Utilitarianism. I ask ‘what is Total Utilitarianism?’ I understand ‘add up all the utilities’. I ask ‘what would the utility functions have to look like for the claim to hold?’ The answer is, ‘quite special’.
I don’t think any of us should be comfortable with not checking the claim works at a gears level. The claim here being, approximately, that the RC is implied under Total Utilitarianism regardless of the choice of utility function. Which is false, as demonstrated above.
> This subdiscipline treats distributions of wellbeing across individuals in different hypothetical worlds as a given input, and seeks to find a function that outputs a plausible ranking of those worlds.If you’d be interested formalising what this means, I could try and show that either the formalisation is uninteresting or that some form of my counterexamples to the RC still holds.
Thanks for the considered reply :)
The crux I think lies in, “is not meant to be sensitive to how resources are allocated or how resources convert to wellbeing.” I guess the point established here is that it is, in fact, sensitive to these parameters.
In particular if one takes this ‘total utility’ approach of adding up everyone’s individual utility we have to ask what each individual’s utility is a function of.
It seems easy to argue that the utility of existing individuals will be affected by expanding or contacting the total pool of individuals. There will be opposing forces of division of scarce resources vs network effects etc.
are ruled out by stipulation.
A way the argument above could be taken down would be writing down some example of a utility function, plugging it into the total utility calculation and showing the RC does hold. Then pointing out that the function comes from a broad class which covers most situations of practical interest.
If the best defence is indeed just pointing out that it’s true for a narrow range of assumptions, my reaction will be like, “OK, but that means I don’t have to pay much attention whenever it crops up in arguments because it probably doesn’t apply.”
Well, on the basis of the description in the SEP article:
The idea behind this view is that the value of adding worthwhile lives to a population varies with the number of already existing lives in such a way that it has more value when the number of these lives is small than when it is large
It’s not the same thing, since above we’re saying that each individual’s utility is a function of the whole setup. So when you add new people you change the existing population’s utilities. The SEP description instead sounds like changing only what happens at the margin.
The main argument above is more or less technical, rather than ‘verbal’. And reliance on verbal argument is pretty much the root of the original issue.
In the event someone else said something similar some other time, there’s still value in a rederivation from a different starting position. I’m not so much concerned with credit for coming up with an idea than that I less frequently encounter instances of this issue.
I think it’s more of a comment that one would find the number of academics ‘excited’ about AIS would increase as the number of venues for publication grew.
This doesn’t seem to have been said, so I will: $1m is enough to live off as an endowment. You can use this to work your entire life on any cause you want to, and then donate as much of it in your will as you wish to.
Upvoted because I think that this should not be downvoted without comment. However I think OP will get more engagement and generate a fuller respose here if:
The arguments that there is a mass extinction going on are summarized rather than linked to.
Some facts about funding levels are given. E.g. how much £ overall goes to eco / wildlife / conservation charities overall versus EA causes. Otherwise some may respond that funding from EA sources is not required, or there are more neglected priorities.
The case for impact and why biodiversity loss is worse than other causes the community focuses on. E.g. there will be those happy to concede that it is bad, but meaningless in the event of x-risks coming to pass.
Note: I am sympathetic generally to the need for a diversity of causes, I’m just pointing out some elements I’d expect to see in an argument which proved persuasive.
Would suggest at least forming a ‘control group’, performing the same analysis and looking at differences in the sets of popular feeds. Following Obama doesn’t tell you much about a person.
One would also need to figure out what it is about those accounts that separates them from other , similar accounts ’IPW’s could have followed but didn’t.
Might get hold of it and confirm my biases :D.
I feel fairly confident though that this argument doesn’t hinge too much on a particular technology such as IoT (as I see in the blurb). To unnecessarily recapitulate: something like the above argument on GDP falls out as the consequence of marginal costs being driven to zero. By whatever means, and relying only on micro 101 theory. In the limit, GDP will provide very little information about utility. There’ll be a lot of good, cool stuff in the world which will be free.
GDP is a very leaky measure for growth in this context. To see this, consider a utopian scenario with dirt cheap fusion energy powered Star Trek replicators. The marginal cost of most traded goods drops to near zero and GDP tanks (setting aside services). You have for traditional industry writ large a similar dynamic to that napster triggered for the music industry.
Assuming we don’t all die sometime soon and things ‘carry on’ the solution is likely to lie at least in part, eventually, in giving up on trying to summarise all of technology in a scalar value.
On another note, as a piece of futurism we see here that technology / the economy is predicted to either:
Get much bigger.
Get much smaller.
Stay about the same.
Do something else.
This covers just about every possibility, so it can’t constitute much evidence on which to update.
[edit: tidied up]
You suggest that concessions will help reduce the scale of the protests, but my impression is that the literature suggests that actually repression is effective.
Presented with options to get largely non-violent protestors-for-justice to go home quickly:
a) Justice
b) Repression
Your response is that b) is a tried and tested intervention. Seriously?
That is not the path to human flourishing.
But collectively we are all better off if everyone stops holding protests for now.
Who is the ‘we’ here and by whose yardstick the benefit measured?
Animal rights activists are not turning out in large numbers to get tear gassed and beaten for the cause. This is pretty good evidence that they are not in the set of ‘everyone else who thinks their reason is as good as I think this one is’.
As usual, there are better alternatives being neglected here. Those who want more lockdown have, in this situation, two options to get it: more violence or more concessions.
Negotiation is certainly possible. So, a consequentialist might lay additional covid deaths at the step of a government which failed to negotiate.
Add to this the obvious virtue of the demand to end police brutality and recognize that black lives matter. That being an option now, it seems particularly bizarre, and wrong, to delay granting the wish.
Now that is a big philosophical question.
One answer is that there is no difference between ‘orders’ of random variables in Bayesian statistics. You’ve either observed something or you haven’t. If you haven’t, then you figure out what distribution the variable has.
The relationship between that distribution and the real world is a matter of your assiduousness to scientific method in constructing the model.
Lack of a reported distribution on a probability, e.g. p=0.42, isn’t the same as a lack of one. It could be taken as the assertion that the distribution on the probability is a delta function at 0.42. Which is to say the reporter is claiming to be perfectly certain what the probability is.
There is no end to how meta we could go, but the utility of going one order up here is to see that it can actually flip our preferences.
One of the topics I hope to return to here is the importance of histograms. They’re not a universal solvent. However they are easily accessible without background knowledge. And as a summary of results, they require fewer parametric assumptions.
I very much agree about the reporting of means and standard deviations, and how much a paper can sweep under the rug by that method.
Nice example, I see where you’re going with that.
I share the intuition that the second case would be easier to get people motivated for, as it represents more of a confirmed loss.
However, as your example shows actually the first case could lead to an ‘in it together’ effect on co-ordination. Assuming the information is taken seriously. Which is hard as, in advance, this kind of situation could encourage a ‘roll the dice’ mentality.
I also think it would be a lot more helpful to walk through how this mistake could happen in some real scenarios in the context of EA
Hopefully, we’ll get there! It’ll be mostly Bayesian though :)
Thanks—that last link was one I’d come across and liked when looking for previous coverage. My sole previous blog post was about Pascal’s Wager. I’d found though when speaking about it that I was assuming too much for some of the audience I wanted to bring along; notwithstanding my sloppy writing :D So, I’m going to attempt to stay focused and incremental.
My quick answer would be: since writing the comment I noticed plenty of people made first contact via hpmor :D
I still don’t know the answer though. I’d guess a startupy algorithm to answer this might lookw like:
identify audience (is it local folks, all peeps on web, ‘agenty’ people) and desired outcomes (more active community members, or just spread the concepts)
find channels to first reach that audience (go viral on tiktok or guest lecture at stanford)
funnel into a broader learning programme (is it a mooc, a YT playlist)
But obvs this is a pretty involved effort and perhaps something one would go for a grant for :o