JesseClifton

Karma: 324

JesseClifton 13 May 2021 16:08 UTC
12 points
0 ∶ 0
in reply to: Jan_Kulveit’s comment on: Launching the EAF Fund
We at CLR are now using a different definition of s-risks.

New definition:

S-risks are risks of events that bring about suffering in cosmically significant amounts. By “significant”, we mean significant relative to expected future suffering.

Note that it may turn out that the amount of suffering that we can influence is dwarfed by suffering that we can’t influence. By “expectation of suffering in the future” we mean “expectation of action-relevant suffering in the future”.

JesseClifton 15 Mar 2021 18:28 UTC
13 points
0 ∶ 0
in reply to: Anthony DiGiovanni’s comment on: Against neutrality about creating happy lives

I found it surprising that you wrote: …

Because to me this is exactly the heart of the asymmetry. It’s uncontroversial that creating a person with a bad life inflicts on them a serious moral wrong. Those of us who endorse the asymmetry don’t see such a moral wrong involved in not creating a happy life.

+1. I think many who have asymmetric sympathies might say that there is a strong aesthetic pull to bringing about a life like Michael’s, but that there is an overriding moral responsibility not to create intense suffering.

JesseClifton 22 Nov 2020 4:16 UTC
5 points
0 ∶ 0
in reply to: weeatquince’s comment on: How to think about an uncertain future: lessons from other sectors & mistakes of longtermist EAs
Very late here, but a brainstormy thought: maybe one way one could start to make a rigorous case for RDM is to suppose that there is a “true” model and prior that you would write down if you had as much time as you needed to integrate all of the relevant considerations you have access to. You would like to make decisions in a fully Bayesian way with respect to this model, but you’re computationally limited so you can’t. You can only write down a much simpler model and use that to make a decision.

We want to pick a policy which, in some sense, has low regret with respect to the Bayes-optimal policy under the true model. If we regard our simpler model as a random draw from a space of possible simplified models that we could’ve written down, then we can ask about the frequentist properties of the regret incurred by different decision rules applied to the simple models. And it may be that non-optimizing decision rules like RDM have a favorable bias-variance tradeoff, because they don’t overfit to the oversimplified model. Basically they help mitigate a certain kind of optimizer’s curse.

JesseClifton 17 Nov 2020 19:29 UTC
4 points
0 ∶ 0
in reply to: Anthony DiGiovanni’s comment on: some concerns with classical utilitarianism
nil already kind of addressed this in their reply, but it seems important to keep in mind the distinction between the intensity of a stimulus and the moral value of the experience caused by the stimulus. Statements like “experiencing pain just slightly stronger than that threshold” risk conflating the two. And, indeed, if by “pain” you mean “moral disvalue” then to discuss pain as a scalar quantity begs the question against lexical views.

Sorry if this is pedantic, but in my experience this conflation often muddles discussions about lexical views.

JesseClifton 7 Aug 2020 13:55 UTC
12 points
0 ∶ 0
on: What are some low-information priors that you find practically useful for thinking about the world?
Some Bayesian statisticians put together prior choice recommendations. I guess what they call a “weakly informative prior” is similar to your “low-information prior”.

JesseClifton 5 Aug 2018 2:20 UTC
12 points
0 ∶ 0
in reply to: RandomEA’s comment on: Problems with EA representativeness and how to solve it
Nice comment; I’d also like to see a top-level post.

One quibble: Several of your points risk conflating “far-future” with “existential risk reduction” and/or “AI”. But there is far-future work that is non-x-risk focused (e.g. Sentience Institute and Foundational Research Institute) and non-AI-focused (e.g. Sentience Institute) which might appeal to someone who shares some of the concerns you listed.

JesseClifton 9 Jan 2018 23:47 UTC
1 point
0 ∶ 0
in reply to: kbog’s comment on: “Just take the expected value” – a possible reply to concerns about cluelessness

Distribution P is your credence. So you are saying “I am worried that my credences don’t have to do with my credence.” That doesn’t make sense. And sure we’re uncertain of whether our beliefs are accurate, but I don’t see what the problem with that is.

I’m having difficulty parsing the statement you’ve attributed to me, or mapping it what I’ve said. In any case, I think many people share the intuition that “frequentist” properties of one’s credences matter. People care about calibration training and Brier scores, for instance. It’s not immediately clear to me why it’s nonsensical to say “P is my credence, but should I trust it?”

JesseClifton 8 Jan 2018 22:21 UTC
0 points
0 ∶ 0
in reply to: kbog’s comment on: “Just take the expected value” – a possible reply to concerns about cluelessness
It sounds to me like this scenario is about a difference in the variances of the respective subjective probability distributions over future stock values. The variance of a distribution of credences does not measure how “well or poorly supported by evidence” that distribution is.

My worry about statements of the form “My credences over the total future utility given intervention A are characterized by distribution P” does not have to do with the variance of the distribution P. It has to do with the fact that I do not know whether I should trust the procedures that generated P to track reality.

JesseClifton 7 Jan 2018 22:22 UTC
1 point
0 ∶ 0
in reply to: kbog’s comment on: “Just take the expected value” – a possible reply to concerns about cluelessness

whether you are Bayesian or not, it means that the estimate is robust to unknown information

I’m having difficulty understanding what it means for a subjective probability to be robust to unknown information. Could you clarify?

subjective expected utility theory is perfectly capable of encompassing whether your beliefs are grounded in good models.

Could you give an example where two Bayesians have the same subjective probabilities, but SEUT tells us that one subjective probability is better than the other due to better robustness / resulting from a better model / etc.?

JesseClifton 27 Dec 2017 20:38 UTC
5 points
0 ∶ 0
in reply to: kbog’s comment on: “Just take the expected value” – a possible reply to concerns about cluelessness
For a Bayesian, there is no sense in which subjective probabilities are well or poorly supported by the evidence, unless you just mean that they result from calculating the Bayesian update correctly or incorrectly.

Likewise there is no true expected utility to estimate. It is a measure of an epistemic state, not a feature of the external world.

I am saying that I would like this epistemic state to be grounded in empirical reality via good models of the world. This goes beyond subjective expected utility theory. As does what you have said about robustness and being well or poorly supported by evidence.

JesseClifton 27 Dec 2017 7:46 UTC
2 points
0 ∶ 0
in reply to: kbog’s comment on: “Just take the expected value” – a possible reply to concerns about cluelessness

But that just means that people are making estimates that are insufficiently robust to unknown information and are therefore vulnerable to the optimizer’s curse.

I’m not sure what you mean. There is nothing being estimated and no concept of robustness when it comes to the notion of subjective probability in question.

JesseClifton 22 Dec 2017 17:20 UTC
10 points
0 ∶ 0
in reply to: Owen Cotton-Barratt’s comment on: “Just take the expected value” – a possible reply to concerns about cluelessness
I can’t speak for the author, but I don’t think the problem is the difficulty of “approximating” expected value. Indeed, in the context of subjective expected utility theory there is no “true” expected value that we are trying to approximate. There is just whatever falls out of your subjective probabilities and utilities.

I think the worry comes more from wanting subjective probabilities to come from somewhere — for instance, models of the world that have a track-record of predictive success. If your subjective probabilities are not grounded in such a model, as is arguably often the case with EAs trying to optimize complex systems or the long-run future, then it is reasonable to ask why they should carry much epistemic / decision-theoretic weight.

(People who hold this view might not find the usual Dutch book or representation theorem arguments compelling.)

JesseClifton 24 Nov 2017 21:28 UTC
4 points
0 ∶ 0
on: What consequences?
Thanks for writing this. I think the problem of cluelessness has not received as much attention as it should.

I’d add that, in addition to the brute good and x-risks approaches, there are approaches which attempt to reduce the likelihood of dystopian long-run scenarios. These include suffering-focused AI safety and values-spreading. Cluelessness may still plague these approaches, but one might argue that they are more robust to both empirical and moral uncertainty.

JesseClifton 20 Jul 2017 22:02 UTC
16 points
0 ∶ 0
in reply to: Wei Dai’s comment on: An Argument for Why the Future May Be Good
Lazy solutions to problems of motivating, punishing, and experimenting on digital sentiences could also involve astronomical suffering.

JesseClifton 9 Jul 2017 17:17 UTC
0 points
0 ∶ 0
in reply to: WillPearson’s comment on: My current thoughts on MIRI’s “highly reliable agent design” work
Right, I’m asking how useful or dangerous your (1) could be if it didn’t have very good models of human psychology—and therefore didn’t understand things like “humans don’t want to be killed”.

JesseClifton 7 Jul 2017 22:13 UTC
1 point
0 ∶ 0
on: My current thoughts on MIRI’s “highly reliable agent design” work
Great piece, thank you.

Regarding “learning to reason from humans”, to what extent do you think having good models of human preferences is a prerequisite for powerful (and dangerous) general intelligence?

Of course, the motivation to act on human preferences is another matter—but I wonder if at least the capability comes by default?

JesseClifton 9 Jun 2017 17:53 UTC
0 points
0 ∶ 0
on: Introducing Sentience Institute
Have animal advocacy organizations expressed interest in using SI’s findings to inform strategic decisions? To what extent will your choices of research questions be guided by the questions animal advocacy orgs say they’re interested in?

JesseClifton 5 Jun 2017 16:32 UTC
7 points
0 ∶ 0
on: Cognitive Science/Psychology As a Neglected Approach to AI Safety
Strong agreement. Considerations from cognitive science might also help us to get a handle on how difficult the problem of general intelligence is, and the limits of certain techniques (e.g. reinforcement learning). This could help clarify our thinking on AI timelines as well as the constraints which any AGI must satisfy. Misc. topics that jump to mind are the mental modularity debate, the frame problem, and insight problem solving.

This is a good article on AI from a cog sci perspective: https://arxiv.org/pdf/1604.00289.pdf

JesseClifton 12 Apr 2017 11:57 UTC
0 points
0 ∶ 0
in reply to: Chriswaterguy’s comment on: Scenarios for cellular agriculture
Yes, I think you’re right, at least when prices are comparable.

JesseClifton 9 Feb 2017 20:11 UTC
3 points
0 ∶ 0
in reply to: Linch’s comment on: Outcome of GWWC Outreach Experiment
More quick Bayes: Suppose we have a Beta(0.01, 0.32) prior on the proportion of people who will pledge. I choose this prior because it gives a point-estimate of a ~3% chance of pledging, and a probability of ~95% that the chance of pledging is less than 10%, which seems prima facie reasonable.

Updating on your data using a binomial model yields a Beta(0.01, 0.32 + 14) distribution, which gives a point estimate of < 0.1% and a ~99.9% probability that the true chance of pledging is less than 10%.