I do independent research on EA topics. I write about whatever seems important, tractable, and interesting (to me).
I have a website: https://mdickens.me/ Much of the content on my website gets cross-posted to the EA Forum, but I also write about some non-EA stuff over there.
I used to work as a software developer at Affirm.
It is hilarious(ly sad) that the largest present-day problem in the entire world only receives on the order of $5 million/year. I really hope these orgs get more funding in the near future.
The argument isn’t that the variance on your EV is very high. The argument is that you can’t have an EV at all. The challenge of unawareness sequence makes this argument in detail.
That looks like a link to an unpublished draft that only you can view. I can’t see it
I agree that it doesn’t make sense to make extremely risky investments and then be very risk-averse with your donations.
I’m not sure about the donation recommendations. Why should donors have a small core of low-risk donations? Why not donate all their money to the single highest-EV opportunity? Or why not give to a few different opportunities that they believe are the best candidates for highest-EV, without concern for risk?
To clarify, I’m not saying that’s because LLMs don’t have welfare. It could also be that they do have welfare, but their statements are not correlated to their internal experiences in the obvious way.
We know that we can get LLMs to express any welfare statement by saying e.g. “pretend you are a sentient AI that’s having an awesome time / being tortured”. It could be that all LLM outputs are roleplaying, but there is some true underlying experience that we don’t (yet?) know how to observe.
Do we have much reason to believe that when a model outputs tokens describing good welfare, that’s because it is having good subjective experiences? It seems to me that we don’t.
Increased care by AIs for some kinds of entities will have substantial spillovers to other kinds (e.g. humans to digital minds and vice versa)
This is more likely to be true if the care comes from some general theory of concern for sentient welfare, and less likely if it comes from something more arbitrary like values-learned-via-RL.
Current AIs are capable of suffering
The question isn’t, “Are current AIs capable of suffering?” The question is, “How big a deal is their suffering in expectation?” Probability is low, but expected importance is high.
Benchmarks will become useless due to eval awareness¹
The question isn’t whether benchmarks will become useless. The question is, “is the probability high enough that we can’t count on benchmarks?” To which the answer is yes.
None of these are from “core AI safety funders”. Views in the EA community were all across the map, but I share James’s impression that big funders (especially CG) were too reluctant to fund anything that’s critical of AI companies.
We’ve only known about evolution for two centuries, and the modern prosperous environment (compared to the ancestral environment), post-demographic-transition, is about equally new. This is the first time in evolutionary history that someone who wants to have a dozen children can pretty much just do that.
In the ancestral environment, the fittest humans weren’t the ones who wanted to have the most children, or who thought it was moral to maximize their genetic influence over the future. They were the ones who were good at making tools, or good at making friends with people who make the tools, etc. That’s different now. Over millions of generations of simulated evolution, I expect the gene pool to become overwhelmingly dominated by people who want to have a lot of kids.
I, however, also think that the creator believing X is true is some evidence that X is true, since the creator would reasonably have more power and knowledge, which makes me guess that they might be more likely than us to be right (hence the update toward their view).
This is only evidence for moral realism inasmuch as the creator has access to evidence about the true morality. If the creator went to you and said “here is why I know Kantianism is true” and then gave some evidence, then that evidence would be evidence.
Basically, if the creator has good justification for their belief, they should be able to give you that justification. If they can’t, then why does their belief mean anything?
I think this assumes that moral intuitions are entirely downstream of the design of the mind.
It is not clear to me that there’s a way to set up this experiment that doesn’t bake in the answer from the beginning.
If you design minds as idealized versions of already-existing minds (a la CEV), then they’ll be starting from the same moral intuitions. At least among extant human minds, moral intuitions don’t vary by all that much.
If you run natural selection on highly intelligent minds, they will inevitably converge on the belief that the true morality is to maximize how many descendants they have—because minds that hold that belief will, in the long run, out-compete minds that don’t. It does not follow that the true morality is to maximize how many descendants you have.
If you have AI super-philosophers reasoning about ethics, then I’d think the relevant evidence wouldn’t be the fact that you can get different AIs to agree with each other, but the actual content of their reasoning. Like, the evidence for moral realism would be that a super-philosopher comes up with an argument about what morality is, and that argument itself contains evidence.
Imagine we point a telescope at some far-off galaxy and find “KANT IS RIGHT ABOUT EVERYTHING” written in stars, in clean sans-serif and at a resolution of 1,000 stars per degree. That would make Kantian ethics seem more likely to be true.
Disagree. It would be evidence that the creator of the universe believes Kantian ethics is true. It would only be evidence if we have reason to believe that the creator of the universe is especially likely to be right about ethics, which I don’t think we do.
AI super-philosophers, built a few different ways, agreeing on the same ethics.
Compared to the idea of different human philosophers taking different approaches and ultimately agreeing on ethics, it seems to me that this is only different in degree, in that the super-philosophers are better at philosophy.
Minds with different architectures converging on the same moral intuitions.
This seems highly contingent on what space of minds you look at. If you have the ability to design minds, then it should be trivial to cause those minds to believe any ethical framework you want them to believe.
They probably just came from LessWrong and got the ordering of the buttons mixed up
Hmmm I think the new version is harder to understand than the old version. The old version gave a colloquial discussion with illustrative examples. The new version uses a formal philosophical argument with a lot of technical terms, and it’s taken from an article that sort of assumes readers are already familiar with cluelessness. Premise P3 asserts that our understanding of the far future is “sufficiently coarse-grained” and doesn’t explain what that means or why to believe it, but IMO that explanation is the key thing that should be on a wiki page. The primary goal should be to explain to the reader what cluelessness is.