MichaelA

• MichaelA 14 Aug 2022 10:50 UTC

  4 points

  on: Pri­ori­tiz­ing x-risks may re­quire car­ing about fu­ture people

  I haven’t yet read this post*, but wanted to mention that readers of this might also find interesting the EA Wiki article Ethics of existential risk and/​or some of the readings it links to/​collects.

  So readers can have more of a sense of what that page is about, here are the first two paragraphs of that page:

  The ethics of existential risk is the study of the ethical issues related to existential risk, including questions of how bad an existential catastrophe would be, how good it is to reduce existential risk, why those things are as bad or good as they are, and how this differs between different specific existential risks. There is a range of different perspectives on these questions, and these questions have implications for how much to prioritise reducing existential risk in general and which specific risks to prioritise reducing.

  In The Precipice, Toby Ord discusses five different “moral foundations” for assessing the value of existential risk reduction, depending on whether emphasis is placed on the future, the present, the past, civilizational virtues or cosmic significance.^[1]

  *Though I did read the Introduction, and think these seem like important & correct points. ETA: I’ve now read the rest and still broadly agree and think these points are important.

• MichaelA 13 Aug 2022 10:03 UTC

  3 points

  in reply to: MarkusAnderljung’s comment on: An­nounc­ing the GovAI Policy Team

  Michael Aird, I think, sporadically updates this collection of lists of EA-relevant research questions)

  Yeah, I update that whenever I learn of a new relevant collection of research questions.

  That said, fwiw, I’d generally recommend that people interested in getting into research in some area:

  • Focus mostly on things like applying to jobs, expressing interest in working with some mentor, or applying to research training programs like the ERIs.

  • See independent research as only (a) a “cheap test of fit” that you spend a few days on on weekends and such, rather than a few months on, or (b) a backup option if applying to lots of roles isn’t yet working on, or a thing you do while waiting to hear back.

  Some people/​situations would be exceptions to that general advice, but generally I think having more structure, mentorship, feedback, etc. is better.

• MichaelA 13 Aug 2022 10:00 UTC

  9 points

  in reply to: MarkusAnderljung’s comment on: An­nounc­ing the GovAI Policy Team

  One other option: My AI Governance and Strategy team at Rethink Priorities offers 3-5 month fellowships and permanent research assistant roles, either of which can be done at anywhere from 20h/​w to 40h/​w depending on the candidates’ preference. And we hire almost entirely based on performance in our work tests & interviews rather than on credentials/​experience (though of course experience often helps people succeed in our work tests & interviews), and have sometimes hired people during or right after undergrad degrees.

  We aren’t currently actively hiring, but people can express interest here.

  (I just happened to read this post because I’m interested in GovAI, and then realised my team’s roles seem relevant to this thread—I don’t originally come here to do recruiting :)

  Also, I’m really excited about GovAI’s work and about them getting great hires, and I’d suggest people typically apply to many orgs/​roles and see what happens rather than trying to just choose one or a few to apply to.)

• MichaelA 13 Aug 2022 9:53 UTC

  4 points

  in reply to: Lexley Villasis’s comment on: An­nounc­ing the GovAI Policy Team

  Just popping in to say you might find this post (of mine) useful: Interested in EA/​longtermist research careers? Here are my top recommended resources Also this comment I left on it:

  “Resources that are only relevant to people interested in AI governance and (to some extent) technical AI safety

  • You could participate in the AGI Safety Fundamentals course’s Governance track, or—when the course isn’t running—work through all or part of the curriculum independently. This seems like an unusually good way for most people to learn about AI risk and AI governance (from a longtermist or existential-risk-focused perspective).

  • Description of some organizations relevant to long-term AI governance (non-exhaustive) (2021) collects and overviews some organizations you might be interested in applying to. (This link is from week 7 of the AGI Safety Fundamentals course’s Governance track.)

  • I think Some AI Governance Research Ideas would be my top recommendation for a public list of AI governance research ideas.

    • But I’d suggest being discerning with this list, as I also think that some of those ideas are relatively low-priority and that the arguments presented for prioritizing those particular ideas are relatively weak, at least from a longtermist/​existential-risk-focused perspective.”

Col­lec­tion of work on ‘Should you should fo­cus on the EU if you’re in­ter­ested in AI gov­er­nance for longter­mist/​x-risk rea­sons?’

• MichaelA 6 Aug 2022 9:13 UTC

  8 points

  on: A list of EA-re­lated podcasts

  Radio Bostrom—“Audio narrations of academic papers by Nick Bostrom.”

• MichaelA 31 Jul 2022 12:56 UTC

  6 points

  on: Re-An­nounc­ing High Im­pact Eng­ineers (FKA EA Eng­ineers)

  As we’ve discussed, I’m tentatively excited about HIE’s plans, I think changing your name was a good idea, and I think this new name is good.

  But I wanted to push back on /​ hopefully clarify one part of the reasoning (partly because I think I was a/​the source for that bit of the reasoning, so maybe I didn’t express it clearly when we chatted):

  To not monopolise funding for engineering-related EA organisations. This is significant consideration from a funder’s point of view as they may be hesitant to fund another organisation that may help to serve engineers in a different way to ours because they are under the impression that we are already fully serving this space.

  I would instead phrase this more like:

  To not monopolise the space of engineering-related EA organisations.

  • People in/​around the EA community often put a lot of emphasis on neglectedness and cooperation.

  • This basically makes sense, but has the unfortunate consequence that often people are too quick to dismiss an area/​idea as “covered” simply because one person/​group is working on it or appears to be working on it.

    • This can lead to missed opportunities, given that the existing project may later fold, may not be highly effective, may only cover part of the space, may have limited capacity, or may take one specific approach to it even though experimentation and multiple approaches would be valuable.

  • This problem is probably exacerbated when a project/​org is called something like “EA [x]”.

    • Because that may increase the chance that other people understandably but incorrect assume that that project/​org is sufficiently covering the intersection of EA and [x].

    • And because then people may worry that that org/​project is trying to stake a claim to the whole space, or would feel insulted /​ confused if another org stepped in to the space.

    • This seems like an argument in favor of names like 80,000 Hours and GiveWell rather than “EA Career Advice” or “Global Health & Development Charity Evaluators”.

  • (Another, unrelated way to mitigate this problem is to explicitly and repeatedly (a) mention that you think other projects in the same space could still be valuable, and (b) explain what your org/​project expects to do and what people might guess you’d do that you don’t expect to do.)

  ...ok, that phrasing definitely seems too long for the post, but I guess I wanted to unpack the reasoning more fully while I was at it.

  So a key thing I’m highlighting here is that I’m not really worried about funders overly assuming orgs called “EA [x]” will sufficiently cover everything at the intersection of EA and [x] and hence not fund other projects; rather, I’m worried that various potential entrepreneurs-or-similar will think that or will worry about what the existing org will think about them starting a new thing.

  I think where funders come in is that they may be less likely to fund an org that is itself going to be called “EA [x]”, because that name probably somewhat increases the extent to which this org may crowd out other projects.

  (Btw, I can’t “take credit” for spotting this consideration; I’ve seen/​heard it discussed by others, and am essentially just a messenger.)

• MichaelA 31 Jul 2022 6:53 UTC

  18 points

  on: AI timelines via bioan­chors: the de­bate in one place

  Thanks, this seems like a useful collection to have made!

  Here are some additional places where Ajeya provides a somewhat more conversational/​accessible overview or discussion of her report:

  • A ~40 minute segment of Ajeya’s 80,000 Hours interview where she talks about AI timelines − 2021, audio

  • Timelines for Transformative AI and Language Model Alignment | Ajeya Cotra − 2022, Q&A, video

  • Q&A with Ajeya Cotra − 2021, video

    • I can’t remember how focused this was on timelines, but I’d guess it was at least 25% focused on that

  • AXRP Episode 7.5 - Forecasting Transformative AI from Biological Anchors with Ajeya Cotra − 2021, transcript of an interview

• MichaelA 30 Jul 2022 11:13 UTC

  4 points

  on: EA Op­por­tu­ni­ties: A new and im­proved col­lec­tion of in­tern­ships, con­tests, events, and more.

  Thanks for your work on this! I imagine this will indeed be valuable.

  We would especially appreciate it if you submitted niche opportunities that would be easy to miss (e.g., Buck’s call for language model investigators, Rethink Priorities Nuclear research projects)

  All collections of EA-relevant research questions which I’m aware of and which are publicly shareable are in A central directory for open research questions. So you could harvest that directory for other collections that seem sufficiently well-scoped/​high-priority/​packaged with a chance of light mentorship that it’s worth including them in your site? And/​or you could include a link to that directory on the site so people who are interested can poke around themselves based on their own interests, skills, and plans?

  (Also, btw, the nuclear risk research projects post is based on work supported by RP but was posted in a personal capacity, and the offer of assistance for people who work on those things is from me rather than from RP. But that distinction is a bit murky and not very important, so it’s fine by me if you don’t make any edits based on that.)

• MichaelA 30 Jul 2022 11:07 UTC

  2 points

  in reply to: MichelJusten’s comment on: EA Op­por­tu­ni­ties: A new and im­proved col­lec­tion of in­tern­ships, con­tests, events, and more.

  I think actually you mean “permanent” vs “temporary”, rather than “full-time” vs “part-time”.

  I think a lot of the opportunities you do/​will/​should cover will either will be or can be full-time but just temporary, e.g. GovAI’s fellowships are on your board and are intended as full-time, and CERI and SERI’s fellowships can be done full-time and probably should be on your board when they next have openings. And I think your target audience is probably often constrained to not taking permanent things yet or not being (demonstrably) skilled enough to get into the relevant permanent things yet, but could do temporary things at a full-time load.

  So I suggested editing the phrasings in light of that in this post and (if the site says relevant things) on the site.

• MichaelA 27 Jul 2022 16:17 UTC

  10 points

  in reply to: Peter Wildeford’s comment on: Why EA needs Oper­a­tions Re­search: the sci­ence of de­ci­sion making

  I think what RP means by that term is “a researcher focused on figuring out & improving various operations-related things”, which is closer to industrial/​organizational psychology + business administration + some other stuff than to the field of “operations research” (“a discipline that deals with the development and application of advanced analytical methods to improve decision-making [...] considered to be a subfield of mathematical sciences” (Wikipedia)).

  So I think this is just an unfortunate overlap of terminology, rather than us actually doing something fairly similar to what this post is about.

  (I guess I blame whoever named the field of operations research, since the name really doesn’t feel intuitive, but that ship has probably long since sailed.)

  (Unnecessary disclaimer: I work at RP and Peter is my manager, but that’s not relevant to me reading this post or leaving this comment)

• MichaelA 25 Jul 2022 9:09 UTC

  14 points

  on: En­light­en­ment Values in a Vuln­er­a­ble World

  I think this post contains some major and some minor errors and is overall fairly “one-sided”, and that the post will therefore tend to overall worsen & confuse (rather than improve & clarify) debates and readers’ beliefs. Below I discuss what I see as some of the errors or markers of one-sidedness in this post. I then close with some other points, e.g. emphasising that I do think good critiques and red-teaming is valuable, noting positives of this post, and acknowledging that this comment probably feels kind-of rude :)

  ---

  Here are some of the things I see as issues in this post. Some are in themselves important, and others are in themselves minor but seem to me like indications of the post generally seeming quite inclined to support a given conclusion rather than more neutrally surveying a topic and seeing what it lands on.* I’ve bolded key points to help people skim this.

  • As Zach mentioned, I think you at least somewhat overstate the extent to which Bostrom is recommending as opposed to analyzing these interventions.

    • Though I do think Bostrom probably could and should have been clearer about this, given that many people have gotten this impression from the paper.

  • You seem to argue (or at least give the vibe that) that there’s there’s so little value in trying to steer technological development for the better than we should mostly not bother and instead just charge ahead as fast as possible. It seems to me that this conclusion is probably incorrect (though I do feel unsure), that the arguments you’ve presented for it are somewhat weak, and that you haven’t adequately discussed arguments against it.

    • Your arguments for this conclusion include that it’s hard to predict the potential benefits and harms of various technologies, that some dangerous and powerful techs like AI can also protect us from other things, and that actors who would steer technological development have motives other than just making the world better.

      • I think these are in fact all true and important points, and it’s good for people to consider them.

      • But I think there are still many cases where we can be pretty confident that our best bet is that some tech will reduce risk or will increase it and that some way of steering tech will have net positive effects.

        • I don’t mean that we can be confident that this will indeed happen this way, but that we can be confident that even after another 10,000 hours of thinking and research we’d still conclude these actions are net positive in expectation (or net negative, in the cases where that’s our guess). And we should take action on that basis.

        • (I won’t try to justify this here due to time constraints, and it would be fair to not be convinced. But hopefully readers can try to think of examples and realise for themselves that my stance seems right.)

        • And if we can’t be confident of that right now, then it seems to me that we should try to (a) gain greater clarity on what tech steering would be good and greater ability to learn that or implement our learnings effectively, and (b) avoid actively accelerating tech dev in the meantime. (As opposed to treating our inability to usefully steer things as so unchangeable that we should just charge ahead and hope for the best.)

      • It seems odd to me to act as though we should be so close to agnostic about the net benefits or harms of all techs and so close to untrusting of any actors who could steer tech development that we should instead just race ahead as fast as we can in all directions.

      • In some cases, I think making simple models, Fermi estimates, or forecasts could help make “each side”’s claims more clear and help us figure out which should get more weight. An example of what this could look like is here: https://​​blog.givewell.org/​​2015/​​09/​​30/​​differential-technological-development-some-early-thinking/​​ (This actually overall highlights the plausibility of the “maybe accelerating AI is good” stance. And I agree that that’s plausible. I’m not saying this post supports my conclusion, just that it seems like an example of a productive way to advance this discussion.)

    • I haven’t re-read your post closely to check what your precise claims and are if you somewhere provide appropriate caveats. But I at least think that the impression people would walk away with is something like “we should just race ahead as fast as possible”.

  • A core premise/​argument in your post appears to be that pulling a black ball and an antidote (i.e., discovering a very dangerous technology and a technology that can protect us from it) at the same time means we’re safe. This seems false, and I think that substantially undermines the case for trying to rush forward and grab balls from the urn as fast as possible.

    • I think the key reason this is false is that “discovering” a technology or “pulling a ball from the urn” does not mean it has reached maturity and been deployed globally. So even if we’ve discovered both the dangerous and protective technology, it’s still possible for the dangerous technology to be deployed in a sufficiently bad way before the protective technology has been deployed in a sufficiently good way.

      • I think there are also reasons why that might be likely, e.g. in some ways it seems easier to destroy than to create, and some dangerous technologies would just need to be deployed once somewhere whereas some protective technologies would need to be deployed continuously and everywhere. (That might be the same point stated in two separate ways—not sure.)

      • OTOH, there are also reasons why that might be unlikely, e.g. far more people want to avoid existential catastrophe than to enact it.

      • Overall I’m not sure which is more likely, but it definitely seems at least plausible that we could end up with disaster if we discover both a very dangerous tech and a paired protective tech at the same time.

    • I’ll illustrate with one of your own examples: “[Increasing our technological ability] slowly, one ball at a time, just means less chance at pulling antidote technologies in time to disable black ball risks.For example, terraforming technology which allows small groups of humans to make changes to a planet’s atmosphere and geography may increase existential risk until space-settling technology puts people on many planets. If terraforming technology typically precedes space-settling then accelerating the pace of progress reduces risk.” But I think if we develop such terraforming technology and such space-settling technology at the same time, or even develop space-settling technology somewhat earlier, that does guarantee we will in fact have built self-sustaining settlements on many places before an individual uses the terraforming technology in a bad way.

      • It’s still totally possible for us to all die due to the terraforming technology before those self-sustaining settlements are set up.

    • Another way to illustrate this: You write “If we discovered all possible technologies at once (which in Bostrom’s wide definition of technology in the VWH paper includes ideas about coordination and insight), we would be in the safe region.” I encourage readers to genuinely try to imagine that literally tomorrow literally the ~8 billion people who exist collectively discover literally all possible technologies at once, and then consider whether they’re confident humanity will exist and be on track to thrive in 2023. Do you (the reader) feel confident that everything will go well in that world where all possible techs and insights on dumped on us at once?

  • I also don’t agree, and don’t think Bostrom would claim, that technological maturity means having discovered all possible technologies, or that we would necessarily be safe if we’d discovered & deployed all possible technologies (even if we survive the initial transition to that world).

    • Bostrom writes “By ‘technological maturity’ we mean the attainment of capabilities affording a level of economic productivity and control over nature
      close to the maximum that could feasibly be achieved (in the fullness of time) (Bostrom, 2013).” That phrasing a bit vague, but I think that attaining that level of capabilities doesn’t mean that we’ve actually got all possible technologies or that every given individual has the maximum possible capabilities.

    • It seems plausible/​likely that some technologies are sufficiently dangerous that we’ll only be safe if we’re in a world where them ever being discovered or ever being deployed is prevented—i.e., that no protective measure would be adequate except prevention.

      • iirc, Bostrom’s discussion of “Type-0 vulnerabilities” is relevant here.

  • I think the following bolded claim is false, and I think it’s very weird to make this empirical claim without providing any actual evidence for it: “AI safety researchers argue over the feasibility of ‘boxing’ AIs in virtual environments, or restricting them to act as oracles only, but they all agree that training an AI with access to 80+% of all human sense-data and connecting it with the infrastructure to call out armed soldiers to kill or imprison anyone perceived as dangerous would be a disaster.”

    • I am 100% confident that not all AI safety researchers have even considered that question, let alone formed the stance you suggest they all agree on.

    • Perhaps you meant they “would all agree”? Still though, it would seem odd to be confident of that without providing any justification.

    • And I think in fact many would disagree if asked. In fact, I expect that many of them would believe that what the future should look like would technically or basically involve this happening; we have a properly aligned superintelligent AI that either already has access to those things or could gain access to those things if it simply chose to do so.

  • I think “If it is to fulfill its mission of preventing anthropogenic risk long into the future, the global surveillance state cannot afford to risk usurpation” and related claims are basically false or misleading.

    • It appears to me that we’re fairly likely to be in or soon be in a “time of perils”, where existential risk is unusually high. There are various reasons to expect this to subside in future besides a global surveillance state. So it seems pretty plausible that it would be best to temporarily have unusually strong/​pervasive surveillance, enforcement, etc. for particular types of activities.

    • And if we’ve set this actor up properly, then it should be focused on what’s net positive overall and should not conflate “ensuring this actor has an extremely high chance of maintaining power helps reduce some risks” with “ensuring this actor has an extremely high chance of maintaining power is overall net beneficial”.

    • To be clear, I’m not saying that we should do things like this or that it’d work if we tried; I’m just saying that thinking that increased surveillance, enforcement, moves towards global governance, etc. would be good doesn’t require thinking that permanent extreme levels (centralised in a single state-like entity) would be good.

  • The following seems like a misrepresentation of Bostrom, and one which is in line with what I perceive as a general one-sidedness or uncharitability or in this post: “Bostrom continues to assume that the power to take a socially beneficial action is sufficient to guarantee that the state will actually do it. “States have frequently failed to solve easier collective action problems … With effective global governance, however, the solution becomes trivial: simply prohibit all states from wielding the black-ball technology destructively.””

    • That quote does not state that the power to take a socially beneficial action is sufficient to guarantee that a state will actually take it. A solution can be trivial but not taken.

      • Also, the “effective” in “effective global governance” might be adding something beyond “power” along the lines of “this governance is pointed in the right direction”?

    • I haven’t read the VWH paper in a while, so maybe he does make this claim elsewhere, or maybe he repeatedly implies it without stating it. But that quote does not seem to demonstrate this.

  ---

  Some other things I want to make sure I say (not issues with the post):

  • To be clear, I do think it’s valuable to critically discuss & red-team the VWH paper in particular and also other ideas and writings that are prominent within longtermism. And I personally wish Bostrom had written the VWH paper somewhat differently, and I don’t feel confident that the interventions it discusses are net positive. So this comment is not meant to discourage other critical discussions or to strongly defend the interventions discussed in VWH.

  • But I do think it’s important to counter mistaken and misleading posts in general, even if the posts are good-faith and are attempting to play a valuable role of criticizing prominent ideas.

  • I wrote this comment pretty quickly, so I don’t fully justify things and my tone is sometimes a bit sharp or uncharitable—apologies in advance for that.

    • (I expect that if the original poster and I instead had a call we would get on the same page faster and feel more positively toward each other, and that I would come across as a bit less rude than this comment might.)

  • I do think there are some good elements of this post (e.g., the writing is generally clear, you include a decent summary at the start, you keep things organized nicely with headings, and some of your points seem true and important). I focus on the negatives since they seem more important and due to time constraints.

  • As a heads up, I’m unlikely to reply to replies to this, since I’m trying to focus on my main work atm.

  *To be clear, I’m a fan of red-teaming, which is not neutral surveying but rather deliberately critical. But that should then be framed explicitly as red-teaming.

• MichaelA 23 Jul 2022 7:25 UTC

  3 points

  in reply to: Will Aldred’s comment on: Will Al­dred’s Shortform

  I’m also pretty sure this wasn’t a totally original brainstorm, and that I was remembering these examples having read something on a similar topic somewhere, probably here on the Forum, though I can’t recall which post it was.

  Maybe it was some combination of the posts with the megaprojects tag?

• MichaelA 23 Jul 2022 7:20 UTC

  11 points

  in reply to: Will Aldred’s comment on: Will Al­dred’s Shortform

  Hmm, it feels unclear to me what you’re claiming here. In particular, I’m not sure which of the following is your claim:

  1. “Right now all money committed to EA could be spent on things that we currently (should) think are at least slightly net positive in expectation. (Even if we maybe shouldn’t spend on those things yet, since maybe we should wait for even better opportunities.)”

  2. “Right now all money committed to EA could be spent on things that might be net positive in expectation. (But there aren’t enough identified opportunities that we currently think are net positive to absorb all current EA money. Some of the things currently look net negative but with high uncertainty, and we need to do further research or wait till things naturally become closer and clearer to find out which are net positive. We also need to find more opportunities.)”

  1 is a stronger and more interesting claim than 2. But you don’t seem to make it clear which one you’re saying.

  If 2 is true, then we still are “severely bottlenecked by good funding opportunities” + by strategic clarity. So it might be that the people you’re talking to are already thinking 2, rather than that EA funding is effectively infinite?

  To be clear, I do think 2 is importantly different from “we have effectively infinite money”, in particular in that it pushes in favor of not spending on extremely slightly net positive funding opportunities now since we want to save money for when we’ve learned more about which of the known maybe-good huge funding opportunties are good.* So if there are people acting and thinking as though we have effectively infinite money, I do think they should get ~this message. But I think your shortform could maybe benefit from distinguishing 1 and 2.

  (Also, a nit-picky point: I’d suggest avoiding phrasing like “could plausibly absorb all EA funding” without a word like “productively”, since of course there are things that can literally just absorb our funding—literally just spending is easy.)

  *E.g., personally I think trying to spend >$1b in 2023 on each of the AI things you mentioned would probably require spending on some net negative in expectation things, but I also think that we should keep those ideas in mind for future and spend a bit slower on other things for that reason.

• MichaelA 23 Jul 2022 7:04 UTC

  7 points

  in reply to: JP Addison’s comment on: Us­ing the “ex­ec­u­tive sum­mary” style: writ­ing that re­spects your reader’s time

  Cool, glad to hear that.

  fwiw, I think a good summary can & should also summarise the reasoning, not only the conclusions. (Though of course it’ll still miss many details of both the reasoning and the conclusions.)

  I’d also flag that I think this should be some sort of nudge, suggestion, optional box to fill in, or whatever, rather than a required box. So people could still decide to not have a summary if they wanted, e.g. if that would mess with their engaging discursive style in a given piece.

• MichaelA 22 Jul 2022 18:18 UTC

  3 points

  in reply to: Adam Binks’s comment on: Us­ing the “ex­ec­u­tive sum­mary” style: writ­ing that re­spects your reader’s time

  Yeah, I’ve actually discussed that idea briefly with the EA Forum team and I think it’d probably be good. I’ll send a link to this thread to them to give them one more data point in favor of doing this. (Though it’s plausible to me that there’s some reason they shouldn’t do this which I’m overlooking—I’d trust their bottom-line views here more than mine.)

  But yeah, I’m also thinking of GDocs, blog posts posted elsewhere, and any other format, so I think we also need nudges like this post.

• MichaelA 22 Jul 2022 7:21 UTC

  2 points

  on: Database of orgs rele­vant to longter­mist/​x-risk work

  I suggested as one possible next step “People could duplicate and then adapt this database in order to make [a] version that’s relevant to all EA cause areas”

  I think such a database has now been made! (Though I’m not sure if that was done by duplicating & adapting my one.) Specifically, Michel Justen has made A Database of EA Organizations & Initiatives. I imagine this’d be useful to some people who find their way to this post.*

  Here’s the summary section of their post, for convenience:

  “I’ve created a new database of EA organizations and initiatives that I host on the recently revamped EA Opportunities page. Here’s the raw Airtable.

  • I think this is the most comprehensive collection of organizations in or closely involved with EA to date. It features orgs explicitly within or adjacent to EA, as well as a non-comprehensive list of other orgs working on global catastrophic risks, even if they have little involvement with EA. As of writing this, there are 276 organizations in this database. Of these, 130 are labeled as “Part of EA community” and the rest are labeled as either “aware of and friendly to EA” or uninvolved.

  • I still recommend this database as the most valuable database of organizations doing longtermist/​x-risk work given its more comprehensive indicators for how orgs are aiming to reduce x-risk.

  • If you see any mistakes in this database, please let us know. You can also submit new organizations.”

  *I guess I should flag that I haven’t looked closely at Michel’s post or database, so can’t personally vouch for its accuracy, comprehensiveness, etc.

• MichaelA 22 Jul 2022 6:56 UTC

  11 points

  in reply to: MichaelA’s comment on: Us­ing the “ex­ec­u­tive sum­mary” style: writ­ing that re­spects your reader’s time

  Here are a couple other resources that I like and that people might also find useful on approximately this topic (besides just the “related work” you mention):

  • Honesty about reading

  • The value of content density

  And here are my own quick tips on summaries, from Tips & readings on good writing/​communication, which are less useful than but somewhat complementary to this post:

  • “You should usually include an actual tl;dr/​summary/​key takeaways section right near the start—even in most cases where you feel it’s unimportant or inappropriate.

    • (At least when writing for e.g. EAs. Sometimes when writing for mass audiences, you’ll better engage people by deliberately not making it clear what you’re writing about or what you’ll ultimately claim.)

    • See Reasoning Transparency and Using the “executive summary” style: writing that respects your reader’s time

    • Usually don’t skip this section

    • Usually don’t just have a section with that sort of name but where you actually just say “This post will cover x, y, and z”

      • I don’t just want to know you say something about x, y, z; I want to know the core of what you actually say!

    • There’s a good chance you—whoever you are—think “The key takeaways are too complex to be explained briefly before someone has actually read my introduction, how I explain the terms, etc.” You’re probably wrong.

      • I kept thinking this for ~8 months, till finally the many many times I was advised to add summaries got to me and I started really trying to do that, at which point I realised it really was typically possible & valuable.

      • Have you actually spent 5 minutes, by the clock, really trying to summarise the key takeaways in a way that will make sense to a reader who hasn’t read the whole thing?

    • (There are some exceptions, e.g. for extremely short posts)”

• MichaelA 22 Jul 2022 6:53 UTC

  21 points

  on: Us­ing the “ex­ec­u­tive sum­mary” style: writ­ing that re­spects your reader’s time

  Thanks for this post! As you know*:

  • I agree that it’d be really valuable for more EA-aligned people to goddamn write summaries at all, to write them well in the ways this post explains, and to use this writing style in general (not just for the summary sections).

  • I think my own writing is much better as a result of adopting these practices.

  • I think this post is a really valuable resource for advocating for and explaining these tips, and I intend to often share it with people and to add it to my (somewhat messy) doc of Tips & readings on good writing/​communication.

  *Disclaimer: The author and I interact a lot and our views aren’t super independent.

• MichaelA 21 Jul 2022 9:53 UTC

  12 points

  in reply to: MichaelA’s comment on: Will Al­dred’s Shortform

  My comment on Ajeya Cotra’s AMA, from Feb 2021 (so probably I’d write it differently today):

  “[I’m not sure if you’ve thought about the following sort of question much. Also, I haven’t properly read your report—let me know if this is covered in there.]

  I’m interested in a question along the lines of “Do you think some work done before TAI is developed matters in a predictable way—i.e., better than 0 value in expectation—for its effects on the post-TAI world, in ways that don’t just flow through how the work affects the pre-TAI world or how the TAI transition itself plays out? If so, to what extent? And what sort of work?”

  An example to illustrate: “Let’s say TAI is developed in 2050, and the ‘TAI transition’ is basically ‘done’ by 2060. Could some work to improve institutional decision-making be useful in terms of how it affects what happens from 2060 onwards, and not just via reducing x-risk (or reducing suffering etc.) before 2060 and improving how the TAI transition goes?”

  But I’m not sure it’s obvious what I mean by the above, so here’s my attempt to explain:

  The question of when TAI will be developed[1] is clearly very important to a whole bunch of prioritisation questions. One reason is that TAI—and probably the systems leading up to it—will very substantially change how many aspects of how society works. Specifically, Open Phil has defined TAI as “AI that precipitates a transition comparable to (or more significant than) the agricultural or industrial revolution” (and Muehlhauser has provided some more detail on what is meant by that).

  But I think some EAs implicitly assume something stronger, along the lines of:

  The expected moral value of actions we take now is entirely based on those actions’ effects on what happens before TAI is developed and those actions’ effects on the development, deployment, etc. of TAI. That is, the expected value of the actions we take now is not partly based on how the actions affect aspects of the post-TAI world in ways unrelated to how TAI is developed, deployed, etc. This is either because we just can’t at all predict those effects or because those effects wouldn’t be important; the world will just be very shaken up and perhaps unrecognisable, and any effects of pre-TAI actions will be washed out unless they affect how the TAI transition occurs.

  E.g., things we do now to improve institutional decision-making or reduce risks of war can matter inasmuch as they reduce risks before TAI and reduce risks from TAI (and maybe also reduce actual harms, increase benefits, etc.). But they’ll have no even-slightly-predictable or substantial effect on decision-making or risks of war in the post-TAI world.

  But I don’t think that necessarily follows from how TAI is defined. E.g., various countries, religious, ideologies, political systems, technologies, etc., existed both before the Industrial Revolution and for decades/​centuries afterwards. And it seems like some pre-Industrial-Revolution actions—e.g. people who pushed for democracy or the abolition of slavery—had effects on the post-Industrial-Revolution world that were probably predictably positive in advance and that weren’t just about affecting how the Industrial Revolution itself occurred.

  (Though it may have still been extremely useful for people taking those actions to know that, when, where, and how the IR would occur, e.g. because then they could push for democracy and abolition in the countries that were about to become much more influential and powerful.)

  So I’m tentatively inclined to think that some EAs are assuming that short timelines pushes against certain types of work more than it really does, and that certain (often “broad”) interventions could be in expectation useful for influencing the post-TAI world in a relatively “continuous” way. In other words, I’m inclined to thinks there might be less of an extremely abrupt “break” than some people seem to think, even if TAI occurs. (Though it’d still be quite extreme by many standards, just as the Industrial Revolution was.)

  [1] Here I’m assuming TAI will be developed, which is questionable, though it seems to me pretty much guaranteed unless some existential catastrophe occurs beforehand.”