Two people mentioned the CEA not being very effective as an unpopular opinion they hold; has any good recent criticism of the CEA been published?
You mention the Jhanas and metta meditation as both being immensely pleasurable experiences. Since these come from meditation, they seem like they might be possible for people to do “at home” at very little risk (save for the opportunity costs from the time investment). Do you have any thoughts on encouraging meditation aimed towards achieving these highly pleasurable states specifically as a cause area and/or something we should be doing personally?
In a building somewhere, tucked away in a forgotten corner, there are four clocks. Each is marked with a symbol: the first with a paperclip, the second with a double helix, the third with a trefoil, and the fourth with a stormcloud.
As you might expect from genre convention, these are not ordinary clocks. In fact, they started ticking when the first human was born, and when they strike midnight, a catastrophe occurs. The type depends on the clock, but what is always true is the disaster kills at least one in ten.
The times currently remaining on the clocks are:
AI Clock: 3:00 to midnight
Biotech Clock: 3:50 to midnight
Nuclear Clock: 4:30 to midnight
Climate Clock: 3:10 to midnight
Since there are many clocks, ticking somewhat randomly, they can be combined to estimate how long until at least one strikes midnight. 40 seconds of humanity.
These numbers were calculated using the Metaculus community median predictions of the probability of 10% of people dying from each of the causes from the Ragnarök question series.
I took those values as a constant probability of extinction over a period of 81 years (sort of like what I brought up in my previous shortform post), and calculated the mean time until catastrophe given this.
I mapped 350,000 years (the duration for which anatomically modern humans have existed according to Wikipeda) to 24 hours.
It is of course possible for human activity to push on the hands of these clocks, just as the clocks can influence humanity. An additional person working full time on those activities that would wind back the clocks could expect to delay them by this amount:
AI Clock: 20,000 microseconds
Biotech Clock: 200 microseconds
Nuclear Clock: 30 microseconds
Climate Clock: 20 microseconds
And these were calculated even more tenuously, by taking 80,000 hours’ order-of-magnitude guesses at how much of the problem an additional full-time worker would solve completely literally and then finding the difference in the Doomsday clock time for that.
The division-by-zero type error is that EV(preventing holocaust|universe is infinite) would be calculated as ∞-∞, which in the extended reals is undefined rather than zero. If it was zero, then you could prove 0 = ∞-∞ = (∞+1)-∞ = (∞-∞)+1 = 1.
This reminds me of the most important AMA question of all:
McAskill, would you rather fight 1 horse-sized chicken, or 100 chicken-sized horses?
One way that x-risk outreach is done outside of EA is by evoking the image of some sort of countdown to doom. There are 12 years until climate catastrophe. There are two minutes on the Doomsday clock, etc.However, in reality, instead of doomsday being some fixed point of time on the horizon that we know about, all the best-calibrated experts have is probability distribution smeared over a wide range of times, mostly sitting on “never” simply for the purposes of just taking the median time not working.And yet! The doomsday clock, so evocative! And I would like to make a bot that counts down on Twitter, I would like to post vivid headlines to really get the blood flowing. (The Twitter bot question is in fact what prompted me to start thinking about this.)Some thoughts on ways to do this in an almost-honest way:
Find the instantaneous probability, today. Convert this to a timescale until disaster. If there is a 0.1% chance of a nuclear war this year, then this is sort of like there being 1,000 years until doom. Adjust the clock with the probability each year. Drawback is that this both understates and overstates the urgency: there’s a good chance disaster will never happen once the acute period is over, but if it does happen it will be much sooner than 100 years. This is what the Doomsday clock seems to want to do, though I think it’s just a political signalling tool for the most part.
Make a conditional clock. If an AI catastrophe happens in the next century (11% chance), it will on average happen in 2056 (50% CI: 2040 − 2069), so have the clock tick down until that date. Display both the probability and the timer prominently, of course, as to not mislead. Drawback is that this is far too complicated and real clocks don’t only exist with 1⁄10 probability. This is what I would do if I was in charge of the Bulletin of the Atomic Scientists.
Make a countdown instead to the predicted date of an evocative milestone strongly associated with acute risk, like the attainment of human level AI or the first time a superbug is engineered in a biotech lab. Drawback is that this will be interpreted as a countdown until doomsday approximately two reblogs in (one if I’m careless in phrasing), and everyone will laugh at me when the date passes and the end of the world has not yet happened. This is the thing everyone is ascribing to AOC on Twitter.
Will there be anything in the book new for people already on board with longtermism?
In 2017, 80k estimated that $10M of extra funding could solve 1% of AI xrisk (todo: see if I can find a better stock estimate for the back of my envelope than this). Taking these numbers literally, this means that anyone who wants to buy AI offsets should, today, pay $1G*(their share of the responsibility).
There are 20,000 AI researchers in the world, so if they’re taken as being solely responsible for the totality of AI xrisk the appropriate pigouvian AI offset tax fine is $45,000 per researcher hired per year. This is large but not overwhelmingly so.
Additional funding towards AI safety will probably go to hiring safety researchers for $100,000 per year each, so continuing to take these cost effectiveness estimates literally, to zeroth order another way of offsetting is to hire one safety researcher for every two capabilities researchers.
“How targeted should donation recommendations be” (sorta)
I’ve noticed that Givewell targets specific programs (e.g. their recommendation), ACE targets whole organisations, and among far future charities you just kinda get promising-sounding cause areas.
I’m interested in what kind of differences between cause areas lead to this, and also whether anything can be done to make more fine-grained evaluations more desirable in practice.
The total number of cows probably stays about the same, because if they had space to raise more cows they would have just done that—I don’t think that availability of semen is the main limiting factor. So the amount of suffering averted by this intervention can be found by comparing the suffering per cow per year in either cases.
Model a cow as having two kids of experiences: normal farm life where it experiences some amount of suffering x in a year, and slaughter where it experiences some amount of suffering y all at once.
In equilibrium, the population of cows is 5⁄6 female and 1⁄6 male. A female cow can, in the next year, expect to suffer an amount (x+y/10), and a male cow can expect to suffer an amount (x+y/2). So a randomly chosen cow suffers (x+y/6).
If male cows are no longer created, this changes to just the amount for female cows, (x+y/10).
So the first-order effect of the intervention is to reduce the suffering per cow per year by the difference between these two, y/15; i.e. averting an amount of pain equal to 1⁄15 of that of being slaughtered per cow per year.
If you want to make a decision, you will probably agree with me that it’s more likely that you’ll end up making that decision, or at least that it’s possible to alter the likelyhood that you’ll make a certain decision by thinking (otherwise your question would be better stated as “if physics is deterministic, does ethics matter”). And, under many worlds, if something is more likely to happen, then there will be more worlds where that happens, and more observers that see that happen (I think this is usually how it’s posed, anyway). So while there’ll always be some worlds where you’re not altruistic, no matter what you do, you can change how many worlds are like that.
When I have a question about the future, I like to ask it on Metaculus. Do you have any operationalisations of synthetic biology milestones that would be useful to ask there?
Tangential to the main point, but what is agmatine, and how would it help someone who suspects they’ve been brainwashed?
This 2019 article has some costs listed:
Fish: “it costs Finless slightly less than $4,000 to make a pound of tuna”
Beef: “Aleph said it had gotten the cost down to $100 per lb.”
Beef(?): “industry insiders say American companies are getting the cost to $50 per lb.”
GiveWell did an intervention report on maternal mortality 10 years ago, and at the time concluded that the evidence is less compelling than for their top charities (though they say that it is now probably out of date).
The amount of carbon that they say could be captured by restoring these trees is 205 GtC, which for $300bn to restore comes to ~70¢/ton of CO2 ~40¢/ton of CO2. Founders Pledge estimates that, on the margin, Coalition for Rainforest Nations averts a ton of CO2e for 12¢ (range: factor of 6) and the Clean Air Task Force averts a ton of CO2e for 100¢ (range: order of magnitude). So those numbers do check out.
You can’t just ask the AI to “be good”, because the whole problem is getting the AI to do what you mean instead of what you ask. But what if you asked the AI to “make itself smart”? On the one hand, instrumental convergence implies that the AI should make itself smart. On the other hand, the AI will misunderstand what you mean, hence not making itself smart. Can you point the way out of this seeming contradiction?
(Under the background assumptions already being made in the scenario where you can “ask things” to “the AI”:) If you try to tell the AI to be smart, but fail and instead give it some other goal (let’s call it being smart’), then in the process of becoming smart’ it will also try to become smart, because no matter what smart’ actually specifies, becoming smart will still be helpful for that. But if you want it to be good and mistakenly tell it to be good’, it’s unlikely that being good will be helpful for being good’.
The signup form for the Learning-by-doing AI Safety workshop currently links to the edit page for the form on google docs, rather than the page where one actually fills out the form; the link should be this one (and the form should probably not be publicly editable).