lukeprog

• lukeprog 17 Sep 2021 20:01 UTC

  15 points

  on: Great Power Conflict

  If you know of work on how AI might cause great power conflict, please let me know

  Phrases to look for include “accidental escalation” or “inadvertent escalation” or “strategic stability,” along with “AI” or “machine learning.” Michael Horowitz and Paul Scharre have both written a fair bit on this, e.g. here.

• lukeprog 15 Sep 2021 22:22 UTC

  34 points

  on: The mo­ti­vated rea­son­ing cri­tique of effec­tive altruism

  [EA has] largely moved away from explicit expected value calculations and cost-effectiveness analyses.

  How so? I hadn’t gotten this sense. Certainly we still do lots of them internally at Open Phil.

  Re: cost-effectiveness analyses always turning up positive, perhaps especially in longtermism. FWIW that hasn’t been my experience. Instead, my experience is that every time I investigate the case for some AI-related intervention being worth funding under longtermism, I conclude that it’s nearly as likely to be net-negative as net-positive given our great uncertainty and therefore I end up stuck doing almost entirely “meta” things like creating knowledge and talent pipelines.

• lukeprog 25 Aug 2021 21:04 UTC

  15 points

  on: What are the EA move­ment’s most no­table ac­com­plish­ments?

  Much of the concrete life saving and life improvement that GiveWell top charities have done with GiveWell-influenced donations.

• lukeprog 17 Aug 2021 21:34 UTC

  7 points

  in reply to: RyanCarey’s comment on: In fa­vor of more an­throp­ics research

  Is the claimed dissolution by MIRI folks published somewhere?

• lukeprog 17 Aug 2021 21:33 UTC

  4 points

  on: What is the clos­est thing you know to EA that isn’t EA?

  Maybe the John A Hartford Foundation.

  Various utilitarianism- and Peter Singer-motivated efforts in global poverty and animal welfare, decades before the modern effective altruism community emerged.

  Mohism.

  Empirical development economics and GBD-prioritized global health interventions.

  Of course, the “rationalist” and “transhumanist” communities have strong similarities, and large chunks of them have essentially merged with EA.

  There are various efforts aimed at more widespread use of cost-benefit analysis, e.g. see Sunstein’s book.

• lukeprog 8 Aug 2021 12:47 UTC

  2 points

  in reply to: tamgent’s comment on: AMA: The new Open Philan­thropy Tech­nol­ogy Policy Fellowship

  It’s mostly about skillsets, context/​experience with both the DC policy world, and familiarity with Open Philanthropy’s programmatic priorities.

• lukeprog 6 Aug 2021 19:06 UTC

  12 points

  in reply to: Flodorner’s comment on: AMA: The new Open Philan­thropy Tech­nol­ogy Policy Fellowship

  A large portion of the value from programs like this comes from boosting fellows into career paths where they spend at least some time working in the US government, and many of the most impactful government roles require US citizenship. We are therefore mainly focused on people who have (a plausible pathway to) citizenship and are interested in US government work. Legal and organizational constraints means it is unlikely that we will be able to sponsor visas even if we run future rounds.

  This program is US-based because the US government is especially important to our programmatic priorities. That said, it’s possible we’ll run (or fund someone else to run) a similar program in one or more non-US countries in the future, perhaps most likely in the UK.

• lukeprog 6 Aug 2021 19:04 UTC

  3 points

  in reply to: tamgent’s comment on: AMA: The new Open Philan­thropy Tech­nol­ogy Policy Fellowship

  I expect Open Philanthropy will want to fund more fellowships like this in the future, but we have some uncertainty about (1) the supply of applicants who are a good fit for the program, and especially (2) the availability of staff and contractors who can run time-intensive programs like this. If we don’t run a similar program in the future, I think the most likely reason will be a lack of (2).

  What links here?

  • Take­aways on US Policy Ca­reers (Part 1): Paths to Im­pact and Per­sonal Fit by Mauricio (26 Aug 2021 2:45 UTC; 74 points)

AMA: The new Open Philan­thropy Tech­nol­ogy Policy Fellowship

• lukeprog 22 Jul 2021 21:47 UTC

  2 points

  in reply to: quinn’s comment on: Ap­ply to the new Open Philan­thropy Tech­nol­ogy Policy Fel­low­ship!

  Oops! Should be fixed now.

Ap­ply to the new Open Philan­thropy Tech­nol­ogy Policy Fel­low­ship!

• lukeprog 19 Jul 2021 21:36 UTC

  13 points

  in reply to: weeatquince’s comment on: A per­sonal take on longter­mist AI governance

  As far as I know it’s true that there isn’t much of this sort of work happening at any given time, though over the years there has been a fair amount of non-public work of this sort, and it has usually failed to convince people who weren’t already sympathetic to the work’s conclusions (about which intermediate goals are vs. aren’t worth aiming for, or about the worldview cruxes underlying those disagreements). There isn’t even consensus about intermediate goals such as the “make government generically smarter about AI policy” goals you suggested, though in some (not all) cases the objection to that category is less “it’s net harmful” and more “it won’t be that important /​ decisive.”

  What links here?

  • A prac­ti­cal guide to long-term plan­ning – and sug­ges­tions for longtermism by weeatquince (10 Oct 2021 15:37 UTC; 99 points)

A per­sonal take on longter­mist AI governance

• lukeprog 4 Jul 2021 12:26 UTC

  14 points

  in reply to: MichaelA’s comment on: EA needs consultancies

  A couple quick replies:

  • Yes, there are several reasons why Open Phil is reluctant to hire in-house talent in many cases, hence the “e.g.” before “because our needs change over time, so we can’t make a commitment that there’s much future work of a particular sort to be done within our organizations.”

  • I actually think there is more widespread EA client demand (outside OP) for EA consulting of the types listed in this post than the post itself represents, because there were several people who gave me feedback on the post and said something like “This is great, I think my org has lots of demand for several of these services if they can be provided to a sufficient quality level, but please don’t quote me on that because I haven’t thought hard enough about this and don’t want people to become over-enthusiastic about this on the basis of my OTOH reaction.” Perhaps I should’ve mentioned this in the original post.

• lukeprog 30 Jun 2021 22:38 UTC

  12 points

  in reply to: Linch’s comment on: EA needs consultancies

  I don’t feel strongly. You all have more context than I do on what seems feasible here. My hunch is in favor of RP maintaining current quality (or raising it only a tiny bit) and scaling quickly for a while — I mostly wanted to give some counterpoints to your suggestion that maybe RP should lower its quality to get more quantity.

• lukeprog 30 Jun 2021 15:33 UTC

  19 points

  in reply to: Linch’s comment on: EA needs consultancies

  I don’t think EAs have a comparative advantage in policy/​research in general, but I do think some EAs have a comparative advantage in doing some specific kinds of policy/​research for other EAs, since EAs care more than many (not all) clients about certain analytic features, e.g. scope-sensitivity, focus on counterfactual impact, probability calibration, reasoning transparency of a particular sort, a tolerance for certain kinds of weirdness, etc.

• lukeprog 30 Jun 2021 15:30 UTC

  18 points

  in reply to: Linch’s comment on: EA needs consultancies

  Other Rethink Priorities clients (including at Open Phil) might disagree, but my hunch is that if anything, higher quality and lower quantity is the way to go, because a client like me has less context on consultants doing some project than I do on someone I’ve directly managed (internally) on research projects for 2 years. So e.g. Holden vetted my Open Phil work pretty closely for 2 years and now feels less need to do so because he has a sense of what my strengths and weaknesses are, where he can just defer to me and where he should make sure to develop his own opinion, etc. That’s part of the massive cost of hiring, training, and managing internal talent, but it eventually gets you to a place where you don’t need to be so nervous about major crippling flaws (of some kinds) in someone’s work. But a major purpose of outsourcing analysis work is to get some information you need without needing to first have built up months or years of deep context with them. But how can I trust the work of someone I have so little context with? I think “go kinda overboard on legibility /​ reasoning transparency” and “go kinda overboard on quality /​ thoroughness /​ vetting” are two major strategies, especially when the client is far more time-constrained than funding constrained (as Open Phil is).

• lukeprog 30 Jun 2021 15:30 UTC

  7 points

  in reply to: Ozzie Gooen’s comment on: EA needs consultancies

  Thanks for your thoughtful comment!

  Re: reluctance. Can you say more about the concern about donor perceptions? E.g. maybe grantmakers like me should be more often nudging grantees with questions like “How could you get more done /​ move faster by outsourcing some work to consultants/​contractors?” I’ve done that in a few cases but haven’t made a consistent effort to signal willingness to fund subcontracts.

  What do you mean about approval from a few parties? Is it different than other expenditures?

  Re: university rules. Yes, very annoying. BERI is trying to help with that, and there could be more BERIs.

  Re: “isolated to Open Phil.” Agree that the consultancy model doesn’t help much if in practice there’s only one client, or just a few — hence my attempt (mostly in the footnotes) to get some sense of how much demand there is for these services outside Open Phil. Of course, with Open Phil being the largest funder in the EA space, many potential clients of EA consultancies are themselves in part funded by Open Phil, but that doesn’t seem too problematic so long as Open Phil isn’t institutionally opposed to subgranting/​subcontracting.

  (Even within Open Phil, a bit of robustness could come from multiple teams demanding a particular genre of services, e.g. at least 3 pretty independent teams at Open Phil have contracted Rethink Priorities for analysis work. But still much safer for contractors if there are several truly independent clients.)

  Re: prices. Seems like an education issue. If you find you need additional validation for the fact that contractors have good reasons for costing ~1.3x to 2x as much as an employee per hour worked, feel free to point people to this comment. :)

  Re: subsidizing. Yes, this would be interesting to think more about. There’s even a model like Founders Pledge and Longview where donors fund the service entirely and then the consultant provides the services for free to clients (in this case, donor services to founders and high-net-worth individuals).

  I’m struggling to parse “Many contractors that organizations themselves come from those organizations.” Could you rephrase?

  Definitely agree that understanding the internal needs of clients is difficult. Speaking from the side of someone trying to communicate my needs/​desires to various grantees and consultants, it also feels difficult on this end of things. This difficulty is often a major reason to do something in-house even if it would in theory be simpler and more efficient to outsource. E.g. it’s a major part of why Open Phil as built a “worldview investigations” team: it’s sort-of weird to have a think tank within a grantmaker instead of just funding external think tanks, but it was too hard to communicate to external parties exactly what we needed to make our funding decisions, so the only way forward was to hire that talent internally so we could build up more shared context etc. with the people doing that work. That was very expensive in staff time, but ultimately the only way to get what we needed. But in other cases it should be possible (and has been possible) for clients to communicate what they need to consultants. One person I spoke to recently suggested that programs like RSP could be a good complement to consultancy work because it allows more people to hang out and gain context on how potential future clients (in that case FHI, but also sort-of “veteran hardcore longtermists in general”) think about things and what they need.

• lukeprog 30 Jun 2021 15:29 UTC

  6 points

  in reply to: Linch’s comment on: EA needs consultancies

  The problem I’m trying to solve (at the top of the post) is that (non-consultancy) EA organizations like Open Phil, for a variety of reasons, can’t hire the talent we need to accomplish everything we’d like to accomplish. So when we do manage to hire someone into a specific role, I think their work in that role can be highly valuable, and if they’re performing well in that role after the first ~year then my hunch is they should stay in that role for at least a few years. That said, we’ve had staff leave and become a grantee/​similar instead, and I could imagine some staff leaving to become an EA consultant at some point if they think they can accomplish more good that way and/​or if they think that’s a better fit for them personally.

• lukeprog 30 Jun 2021 15:29 UTC

  7 points

  in reply to: Larks’s comment on: EA needs consultancies

  I don’t think that would play to Open Phil’s comparative advantages especially well. I think Open Phil should focus on figuring out how to move large amounts of money toward high-ROI work.

  What links here?

  • Arepo's comment on Part 3: Com­par­ing agency or­gani­sa­tional models by Arepo (26 Jul 2021 9:43 UTC; 1 point)