Daniel_Eth

Some po­ten­tial les­sons from Car­rick’s Con­gres­sional bid

Some quotes from Tues­day’s Se­nate hear­ing on AI

[Linkpost] “Gover­nance of su­per­in­tel­li­gence” by OpenAI

• Daniel_Eth 30 Mar 2023 9:56 UTC

  44 points

  33

  in reply to: Dr. David Mathers’s comment on: Paus­ing AI Devel­op­ments Isn’t Enough. We Need to Shut it All Down by Eliezer Yudkowsky

  Clarification for anyone who’s reading this comment outside of having read the article – the article calls for governments to adopt clear policies involving potential preemptive military strikes in certain circumstances (specifically, against a hypothetical “rogue datacenter”, as these datacenters could be used to build AGI), but it is not calling for any specific military strike right now.

The Im­por­tance of AI Align­ment, ex­plained in 5 points

• Daniel_Eth 17 Jul 2020 13:55 UTC

  38 points

  on: Ob­jec­tions to Value-Align­ment be­tween Effec­tive Altruists

  I think there’s some interesting points here! A few reactions:

  • I don’t think advocates of traditional diversity are primarily concerned with cognitive diversity. I think the reasoning is more (if altruistic) to combat discrimination/​bigotry or (if self-interested) good PR/​a larger pool of applicants to choose from.

  • I think in some of the areas that EAs have homogeneity it’s bad (eg it’s bad that we lack traditional diversity, it’s bad that we lack so much geographic diversity, it’s bad that we have so much homogeneity of mannerisms, it’s bad that certain intellectual traditions like neoliberalism or the Pinkerian progress narrative are overwhelmingly fashionable in EA, etc), but I’d actually push back against the claim that it’s bad that we have such a strong consequentialist bent (this just seems to go so hand-in-hand with EA—one doesn’t have to be a consequentialist to want to improve the external world as much as possible, but I’d imagine there’s a strong tendency for that) or that we lack representation of certain political leanings (eg I wouldn’t want people in the alt-right in EA).

  • If people don’t feel comfortable going against the grain and voicing opposition, I’d agree that’s bad because we’d lack ability to self-correct (though fwiw my personal impression is that EA is far better on this metric than almost all other subcultures or movements).

  • It’s not clear to me that hierarchy/​centralization is bad—there are certain times when I think we err too much on this side, but then I think others where we err too much the other way. If we had significantly less centralization, I’d have legitimate concerns about coordination, info-hazards, branding, and evaluating quality of approaches/​organizations.

  • I agree that some of the discussion about intelligence is somewhat cringe, but it seems to me that we’ve gotten better on that metric over time, not worse.

  • Agree that the fandom culture is… not a good feature of EA

  • There probably are some feedback loops here as you mention, but there are other mechanisms going the other direction. It’s not clear to me that the situation is getting worse and we’re headed for “locking in” unfortunate dynamics, and if anything I think we’ve actually mostly improved on these factors over time (and, crucially, my inside view is that we’ve improved our course-correction ability over time).

• Daniel_Eth 13 Mar 2022 2:18 UTC

  35 points

  in reply to: Jackson Wagner’s comment on: A new me­dia out­let fo­cused on philanthropy

  FWIW, I found the interview with SBF to be quite fair, and imho it presented Sam in a neutral-to-positive light (though perhaps a bit quirky). Teddy’s more recent reporting/​tweets about Sam also strike me as both fair and neutral to positive.

• Daniel_Eth 6 Aug 2022 5:59 UTC

  32 points

  in reply to: Larks’s comment on: [link post] The Case for Longter­mism in The New York Times

  Hmm, I don’t read it that way. My read of this passage is: the risk of WWIII by 2070 might be as high as somewhat over 20% (but that estimate is probably picked from the higher end of serious estimates), WWIII may or may not lead to all-out nuclear war, all-out nuclear war has some unknown chance of leading to the collapse of civilization, and if that happened then there would also be some further unknown chance of never recovering. So all-in-all, I’d read this as Will thinking that X-risk from nuclear war in the next 50 years was well below 20%.

  I also don’t think NYT readers have particularly clear prejudices about nuclear war (they probably have larger prejudices about things like overpopulation), so this would be a weird place to make a concession, in my mind.

[Linkpost] AI Align­ment, Ex­plained in 5 Points (up­dated)

• Daniel_Eth 13 Jul 2022 23:47 UTC

  30 points

  on: Pass­ing Up Pay

  My personal view is that targeted small-dollar political donations (which large donors cannot simply fill, due to campaign finance laws) are likely to be vastly higher value on the margin than corresponding-sized (equivalent size plus tax savings) non-political donations to organizations that large donors can fill, insofar as such targeted political opportunities arise. So if I was in the situation you’re describing, I’d accept the higher salary with the intention of donating to such political opportunities when they arose. Of course, this logic is specific to a particular kind of donation opportunity, and won’t generalize to most areas that EAs currently donate to.

• Daniel_Eth 31 Mar 2023 0:07 UTC

  29 points

  22

  in reply to: Dr. David Mathers’s comment on: Paus­ing AI Devel­op­ments Isn’t Enough. We Need to Shut it All Down by Eliezer Yudkowsky

  This goes considerably beyond ‘international treaties with teeth are plausibly necessary here’… Eliezer is proposing attacks on any countries that are building AI-above-a-certain-level, whether or not they sign up to the treaty.

  Is this actually inconsistent? If a country doesn’t sign up for the Biological Weapons Convention, and then acts in flagrant disregard of it, would they not be expected to be faced with retaliatory action from signatories, including, depending on specifics, plausibly up to military force? My sense was that people who pushed for the introduction and enforcement of the BWC would have imaged such a plausible response as within bounds.

  I also think “with teeth” kind of obscures by abstraction here (since it doesn’t necessarily sound like it means war/​violence, but that’s what’s being proposed.)

  I don’t think what Eliezer is proposing would necessarily mean war/​violence either – conditional on a world actually getting to the point were major countries are agreeing to such a treaty, I find it plausible that smaller countries would simply acquiesce in shutting down rogue datacenters. If they didn’t, before military force was used, diplomacy would be used. Then probably economic sanctions would be used. Eliezer is saying that governments should be willing to escalate to using military force if necessary, but I don’t think it’s obvious that in such a world military force would be necessary.

• Daniel_Eth 9 Nov 2021 10:34 UTC

  29 points

  on: Daniel_Eth’s Shortform

  Should there be an “EA Donation Index Fund” that allows people to simply “donate the market” (similar to how index funds like the S&P500 allow for simply buying the market)? This fund could allocate donations to EA orgs in proportion to the total donations that those funds receive (from EA sources?) over the year (it would perhaps make sense for there to be a few such funds – such as one for EA as a whole, one for longtermism, one for global health and development, etc).
  
  I see a few potential benefits:
  • People who want to donate effectively (and especially if wanting to diversify donations) but don’t have the knowledge/​expertise/​time/​etc, and for whatever reason don’t necessarily trust EA funds to donate appropriately on their behalf, could do so. I expect there may be many people holding back from donating now for lack of a sense of how to donate best (including from people on the periphery of EA), so this might increase donations. I further expect the quality of donations would increase from those not as knowledgable, if they simply donated the market.
  • Could be lower overhead and more scalable compared to other funds.
  • Aesthetically, I’d imagine this sort of setup might appeal to finance people, and finance people have a lot of money, so it may widen to pool of donors to EA.
  • Index fund donations would effectively be matching donations – if, for instance, half of all EA donations were through an EA index fund, then that would mean direct donations to specific charities would be matched by moving money from the index fund towards the specific charity as well (of course, at the expense of other charities in the fund) – this would arguably provide greater incentive for direct donors to donate more (at least insofar as they thought they knew more than/​had better values than the market, but that would be their revealed preference from choosing to be direct donors instead of just donating to the index fund).

• Daniel_Eth 21 Feb 2018 0:23 UTC

  28 points

  2

  on: Why I pri­ori­tize moral cir­cle ex­pan­sion over ar­tifi­cial in­tel­li­gence alignment

  I thought this piece was good. I agree that MCE work is likely quite high impact—perhaps around the same level as X-risk work—and that it has been generally ignored by EAs. I also agree that it would be good for there to be more MCE work going forward. Here’s my 2 cents:

  You seem to be saying that AIA is a technical problem and MCE is a social problem. While I think there is something to this, I think there are very important technical and social sides to both of these. Much of the work related to AIA so far has been about raising awareness about the problem (eg the book Superintelligence), and this is more a social solution than a technical one. Also, avoiding a technological race for AGI seems important for AIA, and this also is more a social problem than a technical one.

  For MCE, the 2 best things I can imagine (that I think are plausible) are both technical in nature. First, I expect clean meat will lead to the moral circle expanding more to animals. I really don’t see any vegan social movement succeeding in ending factory farming anywhere near as much as I expect clean meat to. Second, I’d imagine that a mature science of consciousness would increase MCE significantly. Many people don’t think animals are conscious, and almost no one thinks anything besides animals can be conscious. How would we even know if an AI was conscious, and if so, if it was experiencing joy or suffering? The only way would be if we develop theories of consciousness that we have high confidence in. But right now we’re very limited in studying consciousness, because our tools at interfacing with the brain are crude. Advanced neurotechnologies could change that—they could allow us to potentially test hypotheses about consciousness. Again, developing these technologies would be a technical problem.

  Of course, these are just the first ideas that come into my mind, and there very well may be social solutions that could do more than the technical solutions I mentioned, but I don’t think we should rule out the potential role of technical solutions, either.

• Daniel_Eth 6 Nov 2021 2:11 UTC

  26 points

  0

  in reply to: BrownHairedEevee’s comment on: There’s a role for small EA donors in cam­paign finance

  DSCC’s goals are just to elect democrats – they don’t consider, for instance, how different democrats differ on EA criteria such as biosecurity. Donating to particularly aligned candidates (especially in primaries) is probably higher value than donating to existing (non-EA) funds.

[Linkpost] Given Ex­tinc­tion Wor­ries, Why Don’t AI Re­searchers Quit? Well, Sev­eral Reasons

[Link post] Paths To High-Level Ma­chine Intelligence

• Daniel_Eth 21 Jun 2021 4:12 UTC

  23 points

  on: 2018-2019 Long Term Fu­ture Fund Gran­tees: How did they do?

  “There is a part of me which finds the outcome (a 30 to 40% success rate) intuitively disappointing”
  
  Not only do I somewhat disagree with this conclusion, but I don’t think this is the right way to frame it. If we discard the “Very little information” group, then there’s basically a three-way tie between “surprisingly successful”, “unsurprisingly successful”, and “surprisingly unsuccessful”. If a similar amount of grants are surprisingly successful and surprisingly unsuccessful, the main takeaway to me is good calibration about how successful funded grants are likely to be.

• Daniel_Eth 8 Mar 2023 23:22 UTC

  20 points

  19

  in reply to: Kirsten’s comment on: Sugges­tion: A work­able ro­man­tic non-es­ca­la­tion policy for EA com­mu­nity builders

  I feel like the power differential between community builders and new members decreases over time as the new member “graduates” from being a new member and becomes a longer-term members, so perhaps the policy could apply for the first few months of the member’s involvement?

• Daniel_Eth 8 Oct 2020 19:49 UTC

  20 points

  on: Open Com­mu­ni­ca­tion in the Days of Mal­i­cious On­line Actors

  Thanks, I think this is interesting and these sorts of considerations may become increasingly important as EA grows. One other strategy that I think is worth pursuing is preventative measures. IMHO, ideally EA would be the kind of community that selectively repels people likely to be malicious (eg I think it’s good if we repel people who are generally fueled by anger, people who are particularly loud and annoying, people who are racist, etc). I think we already do a pretty good job of “smacking down” people who are very brash or insulting to other members, and I think the epistemic norms in the community probably also select somewhat against people who are particularly angry or who have a tendency to engage in ad hominem. Might also be worth considering what other traits we want to select for/​against, and what sort of norms we could adopt towards those ends.

• Daniel_Eth 31 Mar 2023 0:22 UTC

  18 points

  8

  in reply to: Wil Perkins’s comment on: Paus­ing AI Devel­op­ments Isn’t Enough. We Need to Shut it All Down by Eliezer Yudkowsky

  Worse, Elizier is calling for literal nuclear strikes

  He’s calling for a policy that would be backed by whatever level of response was necessary to enforce it, including, if it escalated to that level, military response (plausibly including nuclear). This is different from, right now, literally calling for nuclear strikes. The distinction may be somewhat subtle, but I think it’s important to keep this distinction in mind during this discussion.

  I want to publicly state that pushing this maximized narrative about AI x risk will lead to terrorist actions against GPU clusters or individuals involved in AI

  This statement strikes me as overconfident. While the narrative presumably does at least somewhat increase the personal security concerns of individuals involved in AI, I think we need to be able to have serious discussions on the topic, and public policy shouldn’t be held hostage to worries that discussions about problems will somewhat increase the security concerns of those involved in those problems (e.g., certain leftist discourse presumably somewhat increases the personal security concerns of rich people, but I don’t think that fact is a good argument against leftism or in favor of silencing leftists).