Will Aldred

Karma: 4,106

Will Aldred 22 Apr 2024 2:22 UTC
6 points
1 ∶ 0
in reply to: Habryka’s comment on: Future of Humanity Institute 2005-2024: Final Report
Perhaps this old comment from Rohin Shah could serve as the standard link?
(Note that it’s on the particular case of recommending people do/don’t work at a given org, rather than the general case of praise/criticism, but I don’t think this changes the structure of the argument other than maybe making point 1 less salient.)
Excerpting the relevant part:
On recommendations: Fwiw I also make unconditional recommendations in private. I don’t think this is unusual, e.g. I think many people make unconditional recommendations not to go into academia (though I don’t).
I don’t really buy that the burden of proof should be much higher in public. Reversing the position, do you think the burden of proof should be very high for anyone to publicly recommend working at lab X? If not, what’s the difference between a recommendation to work at org X vs an anti-recommendation (i.e. recommendation not to work at org X)? I think the three main considerations I’d point to are:
1. (Pro-recommendations) It’s rare for people to do things (relative to not doing things), so we differentially want recommendations vs anti-recommendations, so that it is easier for orgs to start up and do things.
2. (Anti-recommendations) There are strong incentives to recommend working at org X (obviously org X itself will do this), but no incentives to make the opposite recommendation (and in fact usually anti-incentives). Similarly I expect that inaccuracies in the case for the not-working recommendation will be pointed out (by org X), whereas inaccuracies in the case for working will not be pointed out. So we differentially want to encourage the opposite recommendations in order to get both sides of the story by lowering our “burden of proof”.
3. (Pro-recommendations) Recommendations have a nice effect of getting people excited and positive about the work done by the community, which can make people more motivated, whereas the same is not true of anti-recommendations.
Overall I think point 2 feels most important, and so I end up thinking that the burden of proof on critiques / anti-recommendations should be lower than the burden of proof on recommendations—and the burden of proof on recommendations is approximately zero. (E.g. if someone wrote a public post recommending Conjecture without any concrete details of why—just something along the lines of “it’s a great place doing great work”—I don’t think anyone would say that they were using their power irresponsibly.)
I would actually prefer a higher burden of proof on recommendations, but given the status quo if I’m only allowed to affect the burden of proof on anti-recommendations I’d probably want it to go down to ~zero. Certainly I’d want it to be well below the level that this post meets.

Will Aldred 16 Apr 2024 23:10 UTC
2 points
0 ∶ 0
on: Cooperative AI: Three things that confused me as a beginner (and my current understanding)
Thanks, I found this post helpful, especially the diagram.
What (if any) is the overlap of cooperative AI […] and AI safety?
One thing I’ve thought about a little is the possiblility of there being a tension wherein making AIs more cooperative in certain ways might raise the chance that advanced collusion between AIs breaks an alignment scheme that would otherwise work.^[1]
1. ^
  I’ve not written anything up on this and likely never will; I figure here is as good a place as any to leave a quick comment pointing to the potential problem, appreciating that it’s but a small piece in the overall landscape and probably not the problem of highest priority.

Will Aldred 8 Mar 2024 20:26 UTC
8 points
3 ∶ 0
in reply to: metachirality’s comment on: NIST staffers revolt against expected appointment of ‘effective altruist’ AI researcher to US AI Safety Institute
Hard to tell from the information given. Two sources saying an unknown number of people are threatening to resign could just mean that two people are disgruntled and might themselves resign.

Will Aldred 2 Mar 2024 20:16 UTC
6 points
2 ∶ 0
in reply to: Vasco Grilo’s comment on: Cooperating with aliens and AGIs: An ECL explainer
Hmm, okay, so it sounds like you’re arguing that even if we measure the curvature of our observable universe to be negative, it could still be the case that the overall universe is positively curved and therefore finite? But surely your argument should be symmetric, such that you should also believe that if we measure the curvature of our observable universe to be positive, it could still be the case that the overall universe is negatively curved and thus infinite?

Will Aldred 2 Mar 2024 18:47 UTC
6 points
0 ∶ 0
in reply to: Vasco Grilo’s comment on: Cooperating with aliens and AGIs: An ECL explainer
Thanks for replying, I think I now understand your position a bit better. Okay, so if your concern is around measurements only being finitely precise, then my exactly-zero example is not a great one, because I agree that it’s impossible to measure the universe as being exactly flat.
Maybe a better example: if the universe’s large-scale curvature is either zero or negative, then it necessarily follows that it’s infinite.
—(I didn’t give this example originally because of the somewhat annoying caveats one needs to add. Firstly, in the flat case, that the universe has to be simply connected. And then in the negatively curved case, that our universe isn’t one of the unusual finite types of hyperbolic 3-manifold given by Mostow’s rigidity theorem in pure math. (As far as I’m aware, all cosmologists believe that if the universe is negatively curved, then it’s infinite.))—
I think this new example might address your concern? Because even though measurements are only finitely precise, and contain uncertainty, you can still be ~100% confident that the universe is negatively curved based on measurement. (To be clear, the actual measurements we have at present don’t point to this conclusion. But in theory one could obtain measurements to justify this kind of confidence.)
(For what it’s worth, I personally have high credence in eternal inflation, which posits that there are infinitely many bubble/pocket universes, and that each pocket universe is negatively curved—very slightly—and infinitely large. (The latter on account of details in the equations.))

Will Aldred 2 Mar 2024 17:15 UTC
4 points
0 ∶ 0
in reply to: Vasco Grilo’s comment on: Cooperating with aliens and AGIs: An ECL explainer
Hi Vasco, I’m having trouble parsing your comment. For example, if the universe’s large-scale curvature is exactly zero (and the universe is simply connected), then by definition it’s infinite, and I’m confused as to why you think it could still be finite (if this is what you’re saying; apologies if I’m misinterpreting you).
I’m not sure what kind of a background you already have in this domain, but if you’re interested in reading more, I’d recommend first going to the “Shape of the universe” Wikipedia page, and then, depending on your mileage, lectures 10–13 of Alan Guth’s introductory cosmology lecture series.

Will Aldred 21 Feb 2024 18:02 UTC
12 points
2 ∶ 2
in reply to: SuperDuperForecasting’s comment on: New Open Philanthropy Grantmaking Program: Forecasting
I’m confused about why you think forecasting orgs should be trying to acquire commercial clients.^[1] How do you see this as being on the necessary path for forecasting initiatives to reduce x-risk, contribute to positive trajectory change, etc.? Perhaps you could elaborate on what you mean by “real-world impact”?
COI note: I work for Metaculus.
1. ^
  The main exception that comes to mind, for me, is AI labs. But I don’t think you’re talking about AI labs in particular as the commercial clients forecasting orgs should be aiming for?

Will Aldred 18 Feb 2024 16:38 UTC
4 points
0 ∶ 0
in reply to: VictorW’s comment on: VictorW’s Quick takes
What you describe in your first paragraph sounds to me like a good updating strategy, except I would say that you’re not updating your “natural independent opinion,” you’re updating your all-things-considered belief.
Related short posts I recommend—the first explains the distinction I’m pointing at, and the second shows how things can go wrong if people don’t track it:
- ‘Independent impressions’
- ‘When reporting AI timelines, be clear who you’re deferring to’

Will Aldred 3 Feb 2024 21:22 UTC
2 points
0 ∶ 0
in reply to: Ryan Greenblatt’s comment on: Matthew_Barnett’s Shortform
Fair point, I’ve added a footnote to make this clearer.

Will Aldred 3 Feb 2024 19:56 UTC
13 points
0 ∶ 0
in reply to: Matthew_Barnett’s comment on: Matthew_Barnett’s Shortform
AI x-risk is unique because humans would be replaced by other beings, rather than completely dying out. This means you can’t simply apply a naive argument that AI threatens total extinction of value
Paul Christiano wrote a piece a few years ago about ensuring that misaligned ASI is a “good successor” (in the moral value sense),^[1] as a plan B to alignment (Medium version; LW version). I agree it’s odd that there hasn’t been more discussion since.^[2]
Here’s a non-exhaustive list of guesses for why I think EAs haven’t historically been sympathetic [...]: A belief that AIs won’t be conscious, and therefore won’t have much moral value compared to humans.
I’ve wonderedabout this myself. My take is that this area was overlooked a year ago, but there’s now some good work being done. See Jeff Sebo’s Nov ’23 80k podcast episode, as well as Rob Long’s episode, and the paper that the two of them co-authored at the end of last year: “Moral consideration for AI systems by 2030”. Overall, I’m optimistic about this area becoming a new forefront of EA.
accelerationism would have, at best, temporary effects
I’m confused by this point, and for me this is the overriding crux between my view and yours. Do you really not think accelerationism could have permanent effects, through making AI takeover, or some other irredeemable outcome, more likely?
Are you sure there will ever actually be a “value lock-in event”?
I’m not sure there’ll be a lock-in event, in the way I can’t technically be sure about anything, but such an event seems clearly probable enough that I very much want to avoid taking actions that bring it closer. (Insofar as bringing the event closer raises the chance it goes badly, which I believe to be a likely dynamic. See, for example, the Metaculus question, “How does the level of existential risk posed by AGI depend on its arrival time?”, or discussion of the long reflection.)
1. ^
  Although, Paul’s argument routes through acausal cooperation—see the piece for details—rather than through the ASI being morally valuable in itself. (And perhaps OP means to focus on the latter issue.) In Paul’s words:
  Clarification: Being good vs. wanting good
  We should distinguish two properties an AI might have:
  - Having preferences whose satisfaction we regard as morally desirable.
  - Being a moral patient, e.g. being able to suffer in a morally relevant way.
  These are not the same. They may be related, but they are related in an extremely complex and subtle way. From the perspective of the long-run future, we mostly care about the first property.
2. ^
  There was a little discussion a few months ago, here, but none of what was said built on Paul’s article.

Will Aldred 2 Feb 2024 0:44 UTC
13 points
0 ∶ 0
on: Managing risks while trying to do good
Individually, altruists [...] can make a habit of asking themselves and others what risks they may be overlooking, dismissing, or downplaying.
Institutionally, we can rearrange organizational structures to take these individual tendencies into account, for example by creating positions dedicated to or focused on managing risk.
I’ve been surprised by how this seems to be a bit of a blind spot in our community.^[1] I’ve previously written a couple of comments—excerpted below—on this theme, about the state of community building. These garnered a decent number of upvotes, but I don’t think they led to any concrete actions or changes. (For instance, the second comment never received a reply from Open Phil.)
My attempts to raise this concern [about optimizing for numbers/hype at the expense of i) cause prio, ii) addressing particular talent bottlenecks, and iii) mitigating downside risks] with other community builders, including those above me, were mostly dismissed. This worried me. It seemed like the community building machine was not open to the hypothesis that (some of) what it was doing might be ineffective, or, worse, net negative. (More on the latter below.) On top of this, there seemed to be a tricky second-order effect at play: evaporative cooling whereby the community builders who developed concerns like mine exited, only to be replaced by more bullish community builders. The result: a disproportionately bullish community building machine. And there didn’t appear to be any countermeasures in place. For example, there was plenty of funding available if one wanted a paid role doing community building. But, in addition to the social disincentive, there was no funding available for evaluating/critiquing the impact of community building—at least, not that I was aware of.
(link)
There was near-consensus that Open Phil should generously fund promising AI safety community/movement-building projects they come across
Would you be able to say a bit about to what extent members of this working group have engaged with the arguments around AI safety movement-building potentially doing more harm than good? For instance, points 6 through 11 of Oli Habryka’s second message in the “Shutting Down the Lightcone Offices” post (link). If they have strong counterpoints to such arguments, then I imagine it would be valuable for these to be written up.
(link)
1. ^
  I mean, if one has a high prior on one’s actions being robustly positive, then it makes sense to continue full steam ahead without worrying about risks. (Because there is a tradeoff: spending time considering risks means spending less time acting.) However, I don’t think this level of confidence is warranted for the vast majority of longtermist interventions. For more, see this comment by Linch.

Will Aldred 30 Jan 2024 17:19 UTC
17 points
1 ∶ 0
in reply to: weeatquince’s comment on: EV investigation into Owen and Community Health
(Fwiw, the Forum moderation team does this for many of our cases.)

Will Aldred 28 Jan 2024 23:48 UTC
2 points
0 ∶ 0
in reply to: Matthew_Barnett’s comment on: Matthew_Barnett’s Shortform
Objection 1: It is unlikely that there will be a point at which a unified agent will be able to take over the world, given the existence of competing AIs with comparable power
For what it’s worth, the Metaculus crowd forecast for the question “Will transformative AI result in a singleton (as opposed to a multipolar world)?” is currently “60%”. That is, forecasters believe it’s more likely than not that there won’t be competing AIs with comparable power, which runs counter to your claim.
(I bring this up seeing as you make a forecasting-based argument for your claim.)

Will Aldred 15 Jan 2024 15:58 UTC
10 points
0 ∶ 0
on: AI doing philosophy = AI generating hands?
Following on from your saner world illustration, I’d be curious to hear what kind of a call to action you might endorse in our current world.
I personally find your writings on metaphilosophy, and the closely related problem of ensuring AI philosophical competence, persuasive. In other words, I think this area has been overlooked, and that more people should be working in it given the current margin in AI safety work. But I also have a hard time imagining anyone pivoting into this area, at present, given that:^[1]
1. There’s no research agenda with scoped out subproblems (as far as I’m aware), only the overall, wicked problem of trying to get advanced AIs to do philosophy well.
2. There are no streams within junior research programs, like MATS, to try one’s hand^[2] in this area while gaining mentorship.
1. ^
  A third reason, which I add here as a footnote since it seems far less solvable: Monetary and social incentives are pushing promising people into empirical/ML-based intent alignment work. (To be clear, I believe intent alignment is important. I just don’t think it’s the only problem that deserves attention.) It takes agency—and financial stability, and a disregard for status—to strike out on one’s own and work on something weirder, such as metaphilosophy or other neglected, non-alignment AI topics.
  [ETA: Ten days after I posted this comment, Will MacAskill gave an update on his work: he has started looking into neglected, non-alignment AI topics, with a view to perhaps founding a new research institution. I find this encouraging!]
2. ^
  Pun intended.

Will Aldred 10 Jan 2024 19:39 UTC
2 points
0 ∶ 0
in reply to: Greg_Colbourn’s comment on: MIRI 2024 Mission and Strategy Update
Yes, this is a fair point; Holden has discussed these dangers a little in “Digital People Would Be An Even Bigger Deal”. My bottom-line belief, though, is that mind uploads are still significantly more likely to be safe than ML-derived ASI, since uploaded minds would presumably work, and act, much more similarly to (biological) human minds. My impression is that others also hold this view? I’d be interested if you disagree.
To be clear, I rank moratorium > mind uploads > ML-derived ASI, but I think it’s plausible that our strategy portfolio should include mind uploading R&D alongside pushing for a moratorium.

Will Aldred 9 Jan 2024 23:57 UTC
8 points
3 ∶ 0
in reply to: Guy Raveh’s comment on: Will Aldred’s Shortform
Hmm, based on what you’ve said here—and I acknowledge that what you’ve said is a highly compressed version of your experience, thus I may well be failing to understand you (and I apologize in advance if I mischaracterize your experience)—I think I’m not quite seeing how this refutes my framing? I accept that my type-A/B framing rounds off a bunch of nuance, but to me, within that framing, it sounds like you’re type-A?
Like, I’m not sure how long the transition period was for you, and I expect different people’s transition periods will vary considerably, but my model, viewed through this lens, is that a type-A person will make it out of their transition period and be able to maintain a vegan diet thereafter at little to no cost. Whereas a type-B can spend weeks, months—even a year or more, like myself^[1]—planning out and iterating on their vegan diet; making sure, through doing research, taking blood tests, and so on, that they’re avoiding the known pitfalls, and still never make it out of the transition period.^[2]^[3]
1. ^
  I’ve written about my experience here.
2. ^
  I like this comment from Jason: “Nutritional research is hard, and we’d need a significantly stronger body of research (e.g., random assignment, very large samples) to say that a vegan diet is maximally healthful for everyone at an individual level (as opposed to healthier on the a [sic] population average).” (link)
3. ^
  Moreover, for me, the vegan experience actually got increasingly unpleasant with time, if anything, so I don’t think it’s the case that type-B’s will eventually asymptote onto becoming costlessly vegan if only they stick with it for long enough.
  (Additionally, if asymptoting really does occur, but “long enough” means months or years, then I have sympathy for those who give up in the meantime.)

Will Aldred 8 Jan 2024 21:35 UTC
4 points
0 ∶ 0
in reply to: Elizabeth’s comment on: Will Aldred’s Shortform
Thanks for your comments, both. I agree that the personal versus universal statements distinction is noteworthy (and missing from my take above).

Will Aldred 7 Jan 2024 22:17 UTC
37 points
14 ∶ 1
on: Will Aldred’s Shortform
Shower thought: A lot of the talking past each other that happens between vegan and non-vegan^[1] EAs^[2] might come from selection effects plus typical mind fallacy.^[3]
Let’s say there are two types of people: type-A, for whom a vegan diet imposes little or no costs, and type-B, for whom a vegan diet imposes substantial costs (in things like health, productivity,^[4] social life). My hunch is that most long-time vegans are type-A, while most type-B people who try going vegan bounce.
Now, to a type-A vegan who doesn’t realize type-B is a thing, those who claim that a vegan diet has costs are obviously lying. And to a type-B person who’s tried and failed to go vegan, vegans are obviously lying about how easy it is. (To put things crudely.)
I think an awareness from both sides of the vegan/non-vegan divide that the other type—B/A—exists, and that the other side is mostly made up of the other type, could go quite a long way toward circumventing frustrating or otherwise unproductive debates.
(I doubt this take is original to me, but I didn’t find it on this forum already with a quick search, so I figured I’d share.)
1. ^
  I expect the same argument to apply to vegetarians versus non-vegetarians, but for brevity I’ll just write “vegan” throughout. (I’m avoiding the veg*n shorthand just because I don’t find *’s aesthetically appealing in prose.)
2. ^
  I’ve specified “EAs” rather than just people in general because my argument works better if most non-vegans have actually tried to go vegan. My sense is that this is true for the EA population, but (very) not true for the general population.
3. ^
  My preferred definition of typical mind fallacy: The mistake of jumping to conclusions—often subconsciously—about other people’s experiences based on your own experiences. In other words, the mistake of assuming that other people are more like you than they really are. (Here’s the LessWrong Wiki’s definition.)
4. ^
  I’ve previously written about my experience here.

Will Aldred 6 Jan 2024 19:20 UTC
14 points
0 ∶ 1
in reply to: lilly’s comment on: Survey of 2,778 AI authors: six parts in pictures
Just to give your final point some context: the average in-depth research project by Rethink Priorities reportedly costs $70K-$100K. So, if this AI Impacts survey cost $138K in participant compensation, plus some additional amount for things like researcher time, then it looks like this survey was two or three times more expensive than the average research project in its approximate reference class.
I haven’t thought hard about whether the costs of EA-funded research make sense in general, but I thought I’d leave this comment so that readers don’t go away thinking that this survey cost like an order of magnitude more than what’s standard.

Will Aldred 5 Jan 2024 15:49 UTC
11 points
1 ∶ 1
on: MIRI 2024 Mission and Strategy Update
I’m curious, since it sounds like MIRI folks may have thought about this, if you have takes on how best to allocate marginal effort between pushing for cooperation-to-halt-AI-progress on the one hand, and accelerating cognitive enhancement (e.g., mind uploading) on the other?^[1]
Like, I see that you list promoting cooperation as a priority, but to me, based on your footnote 3, it doesn’t seem obvious that promoting cooperation to buy ourselves time is a better strategy at the margin than simply working on mind uploading.^[2] (At least, I don’t see this being obviously true for people-trying-to-reduce-AI-risk at large, and I’d be interested in your—or others’—thoughts here, in case there’s something I’m missing. It may well be clearly true for MIRI given your comparative advantages; I’m asking this question from the perspective of overall AI risk reduction strategy.) Here’s that footnote 3:
Nate and Eliezer both believe that humanity should not be attempting technical alignment at its current level of cognitive ability, and should instead pursue human cognitive enhancement (e.g., via uploading), and then having smarter (trans)humans figure out alignment.
Related recent discussion:
- “Does davidad’s uploading moonshot work?”
  - Context: David Dalrymple (aka davidad) recently outlined a concrete plan for mind uploading by 2040.
Related prediction markets:
- Eliezer’s Manifold market, “If Artificial General Intelligence has an okay outcome, what will be the reason?”
  - At present, the leading answer is: “Humanity successfully coordinates worldwide to prevent the creation of powerful AGIs for long enough to develop human intelligence augmentation, uploading, or some other pathway into transcending humanity’s window of fragility.”
- My Metaculus question, “Will mind uploading happen before AGI?”
  - The current community prediction is 1%.^[3]
1. ^
  ETA: I’ve just noticed that earlier today, another Forum user posted a quick take on a similar theme, asking why there’s been no EA funding for cognitive enhancement projects. See here.
2. ^
  The immediate lines of reasoning I can think of for why “put all marginal effort towards pausing AI” is the best strategy right now are: i) uploading is intractable given AGI timelines, and ii) future, just-before-the-pause models—GPT-7, say—could help significantly with mind uploading R&D. But then, assuming that uploading is our best bet for getting alignment right, I think ii just shifts the discussion to things like “where is the best place to pause (with respect to the tradeoff between powerful automation of uploading R&D versus not pausing too late)?” and “are there ways to push for differential progress in models’ capabilities? (e.g., narrow superhuman ability in neuroscience research).”
  What’s more, as counters to i: Firstly, most problems fall within a 100x tractability range. Secondly, even if cooperation+pause efforts are clearly higher impact right now than object-level uploading work, I think there’s still the argument that field-building for mind uploading should start now, rather than once the pause is in place. Because if field-building starts now, then with luck there’ll be a body of uploading researchers ready to make the most of a future pause. (This argument doesn’t go through if the pause lasts indefinitely, because in that case there’s time to build up the mind uploading field from scratch in the pause. But it does go through if the pause is limited or fragile, which I tentatively believe are more likely possibilities. See also Scott Alexander’s taxonomy of AI pauses.)
3. ^
  Taken together, these two prediction markets arguably paint a grim picture. Namely, the trades on Eliezer’s question imply that mind uploading is the most likely way that AGI goes well for humanity, but the forecasts on my question imply that we’re very unlikely to get mind uploading before AGI.