This has been the case for quite a while now. There was a small discussion back in December 2016 where some people expressed similar opinions. My guess is that 2015 is the last year the group regularly had interesting posts, but I might be remembering incorrectly.
riceissa(Issa Rice)
Karma: 783
How did you decide on “blog posts, cross-posted to EA Forum” as the main output format for your organization? How deliberate was this choice, and what were the reasons going into it? There are many other output formats that could have been chosen instead (e.g. papers, wiki pages, interactive/tool website, blog+standalone web pages, online book, timelines).
wikieahuborg_w-20180412-history.xmlcontains the dump, which can be imported to a MediaWiki instance.
Re: The old wiki on the EA Hub, I’m afraid the old wiki data got corrupted, it wasn’t backed up properly and it was deemed too difficult to restore at the time :(. So it looks like the information in that wiki is now lost to the winds.
I think a dump of the wiki is available at https://archive.org/details/wiki-wikieahuborg_w.
The full metrics report gives the breakdown of number of donors by donation size and year (for 2016-2018), both as an estimate and for known number of donors.
Do you have any thoughts on Qualia Research Institute?
Over the years, you have published several pieces on ways you’ve changed your mind (e.g. about EA, another about EA, weird ideas, hedonic utilitarianism, and a bunch of other ideas). While I’ve enjoyed reading the posts and the selection of ideas, I’ve also found most of the posts frustrating (the hedonic utilitarianism one is an exception) because they mostly only give the direction of the update, without also giving the reasoning and additional evidence that caused the update* (e.g. in the EA post you write “I am erring on the side of writing this faster and including more of my conclusions, at the cost of not very clearly explaining why I’ve shifted positions”). Is there a reason you keep writing in this style (e.g. you don’t have time, or you don’t want to “give away the answers” to the reader), and if so, what is the reason?
*Why do I find this frustrating? My basic reasoning is something like this: I think this style of writing forces the reader to do a weird kind of Aumann reasoning where they have to guess what evidence/arguments Buck might have had at the start, and what evidence/arguments he subsequently saw, in order to try to reconstruct the update. When I encounter this kind of writing, I mostly just take it as social information about who believes what, without bothering to go through the Aumann reasoning (because it seems impossible or would take way too much effort). See also this comment by Wei Dai.
Do you think non-altruistic interventions for AI alignment (i.e. AI safety “prepping”) make sense? If so, do you have suggestions for concrete actions to take, and if not, why do you think they don’t make sense?
(Note: I previously asked a similar question addressed at someone else, but I am curious for Buck’s thoughts on this.)
How do you see success/an “existential win” playing out in short timeline scenarios (e.g. less than 10 years until AGI) where alignment is non-trivial/turns out to not solve itself “by default”? For example, in these scenarios do you see MIRI building an AGI, or assisting/advising another group to do so, or something else?
[Meta] During the AMA, are you planning to distinguish (e.g. by giving short replies) between the case where you can’t answer a question due to MIRI’s non-disclosure policy vs the case where you won’t answer a question simply because there isn’t enough time/it’s too much effort to answer?
The 2017 MIRI fundraiser post says “We plan to say more in the future about the criteria for strategically adequate projects in 7a” and also “A number of the points above require further explanation and motivation, and we’ll be providing more details on our view of the strategic landscape in the near future”. As far as I can tell, MIRI hasn’t published any further explanation of this strategic plan. Is MIRI still planning to say more about its strategic plan in the near future, and if so, is there a concrete timeframe (e.g. “in a few months”, “in a year”, “in two years”) for publishing such an explanation?
(Note: I asked this question a while ago on LessWrong.)
I asked a question on LessWrong recently that I’m curious for your thoughts on. If you don’t want to read the full text on LessWrong, the short version is: Do you think it has become harder recently (say 2013 vs 2019) to become a mathematician at MIRI? Why or why not?
In November 2018 you said “we want to hire as many people as engineers as possible; this would be dozens if we could, but it’s hard to hire, so we’ll more likely end up hiring more like ten over the next year”. As far as I can tell, MIRI has hired 2 engineers (Edward Kmett and James Payor) since you wrote that comment. Can you comment on the discrepancy? Did hiring turn out to be much more difficult than expected? Are there not enough good engineers looking to be hired? Are there a bunch of engineers who aren’t on the team page/haven’t been announced yet?
On the SSC roadtrip post, you say “After our trip, I’ll write up a post-mortem for other people who might be interested in doing things like this in the future”. Are you still planning to write this, and if so, when do you expect to publish it?
Back in July, you held an in-person Q&A at REACH and said “There are a bunch of things about AI alignment which I think are pretty important but which aren’t written up online very well. One thing I hope to do at this Q&A is try saying these things to people and see whether people think they make sense.” Could you say more about what these important things are, and what was discussed at the Q&A?
I read the paper (skipping almost all the math) and Philip Trammell’s blog post. I’m not sure I understood the paper, and in any case I’m pretty confused about the topic of how growth influences x-risk, so I want to ask you a bunch of questions:
-
Why do the time axes in many of the graphs span hundreds of years? In discussions about AI x-risk, I mostly see something like 20-100 years as the relevant timescale in which to act (i.e. by the end of that period, we will either go extinct or else build an aligned AGI and reach a technological singularity). Looking at Figure 7, if we only look ahead 100 years, it seems like the risk of extinction actually goes up in the accelerated growth scenario.
-
What do you think of Wei Dai’s argument that safe AGI is harder to build than unsafe AGI and we are currently putting less effort into the former, so slower growth gives us more time to do something about AI x-risk (i.e. slower growth is better)?
-
What do you think of Eliezer Yudkowsky’s argument that work for building an unsafe AGI parallelizes better than work for building a safe AGI, and that unsafe AGI benefits more in expectation from having more computing power than safe AGI, both of which imply that slower growth is better from an AI x-risk viewpoint?
-
What do you think of Nick Bostrom’s urn analogy for technological developments? It seems like in the analogy, faster growth just means pulling out the balls at a faster rate without affecting the probability of pulling out a black ball. In other words, we hit the same amount of risk but everything just happens sooner (i.e. growth is neutral).
-
Looking at Figure 7, my “story” for why faster growth lowers the probability of extinction is this: The richer people are, the less they value marginal consumption, so the more they value safety (relative to consumption). Faster growth gets us sooner to the point where people are rich and value safety. So faster growth effectively gives society less time in which to mess things up (however, I’m confused about why this happens; see the next point). Does this sound right? If not, I’m wondering if you could give a similar intuitive story.
-
I am confused why the height of the hazard rate in Figure 7 does not increase in the accelerated growth case. I think equation (7) for might be the cause of this, but I’m not sure. My own intuition says accelerated growth not only condenses along the time axis, but also stretches along the vertical axis (so that the area under the curve is mostly unaffected).
As an extreme case, suppose growth halted for 1000 years. It seems like in your model, the graph for hazard rate would be constant at some fixed level, accumulating extinction probability during that time. But my intuition says the hazard rate would first drop near zero and then stay constant, because there are no new dangerous technologies being invented. At the opposite extreme, suppose we suddenly get a huge boost in growth and effectively reach “the end of growth” (near period 1800 in Figure 7) in an instant. Your model seems to say that the graph would compress so much that we almost certainly never go extinct, but my intuition says we do experience a lot of risk for extinction. Is my interpretation of your model correct, and if so, could you explain why the height of the hazard rate graph does not increase?
This reminds me of the question of whether it is better to walk or run in the rain (keeping distance traveled constant). We can imagine a modification where the raindrops are motionless in the air.
-
Can you give some examples of EA organizations that have done things the “right way” (in your view)?
Several background variables give rise to worldviews/outlooks about how to make the transition to a world with AGIs go well. Answering this question requires assigning values to the background variables or placing weights on the various worldviews, and then thinking about how likely “Disneyland with no children” scenarios are under each worldview, by e.g. looking at how they solve philosophical problems (particularly deliberation) and how likely obvious vs non-obvious failures are.
That is to say, I think answering questions like this is pretty difficult, and I don’t think there are any deep public analyses about it. I expect most EAs who don’t specialize in AI alignment to do something on the order of “under MIRI’s views the main difficulty is getting any sort of alignment, so this kind of failure mode isn’t the main concern, at least until we’ve solved alignment; under Paul’s views we will sort of have control over AI systems, at least in the beginning, so this kind of failure seems like one of the many things to be worried about; overall I’m not sure how much weight I place on each view, and don’t know what to think so I’ll just wait for the AI alignment field to produce more insights”.
The inconsistency is itself a little concerning.
I am one of the contributors to the Donations List Website (DLW), the site you link to. DLW is not affiliated with the EA Hotel in anyway (although Vipul, the maintainer of DLW, made a donation to the EA Hotel). Some reasons for the discrepancy in this case:
As stated in bold letters at the top of the page, “Current data is preliminary and has not been completely vetted and normalized”. I don’t think this is the main reason in this case.
Pulling data into DLW is not automatic, so there is a lag between when the donations are made and when they appear on DLW.
DLW only tracks public donations.
I think that’s one of the common ways for a post to be interesting, but there are other ways (e.g. asking a question that generates interesting discussion in the comments).