You can give me anonymous feedback here.
elifland
Karma: 2,695
All views are my own rather than those of any organizations/groups that I’m affiliated with. Trying to share my current views relatively bluntly. Note that I am often cynical about things I’m involved in. Thanks to Adam Binks for feedback.
Edit: See also child comment for clarifications/updates.
Edit 2: I think the grantmaking program has different scope than I was expecting; see this comment by Benjamin for more.
Following some of the skeptical comments here, I figured it might be useful to quickly write up some personal takes on forecasting’s promise and what subareas I’m most excited about (where “forecasting” is defined as things I would expect to be in the scope of OpenPhil’s program to fund).
Overall, most forecasting grants that OP has made seem much lower EV than the AI safety grants (I’m not counting grants that seem more AI-y than forecasting-y, e.g. Epoch, and I believe these wouldn’t be covered by the new grantmaking program). Due to my ASI timelines (10th percentile ~2027, median ~late 2030s), I’m most excited about forecasting grants that are closely related to AI, though I’m not super confident that no non-AI related ones are above the bar.
I generally agree with the view that I’ve heard repeated a few times that EAs significantly overrate forecasting as a cause area, while the rest of the world significantly underrates it.
I think EAs often overrate superforecasters’ opinions, they’re not magic. A lot of superforecasters aren’t great (at general reasoning, but even at geopolitical forecasting), there’s plenty of variation in quality.
General quality: Becoming a superforecaster selects for some level of intelligence, open-mindedness, and intuitive forecasting sense among the small group of people who actually make 100 forecasts on GJOpen. There are tons of people (e.g. I’d guess very roughly 30-60% of AI safety full-time employees?) who would become superforecasters if they bothered to put in the time.
Some background: as I’ve written previously I’m intuitively skeptical of the benefits of large amounts of forecasting practice (i.e. would guess strong diminishing returns).
Specialties / domain expertise: Contra a caricturized “superforecasters are the best at any forecasting questions” view, consider a grantmaker deciding whether to fund an organization. They are, whether explicitly or implicitly, forecasting a distribution of outcomes for the grant. But I’d guess most would agree that superforecasters would do significantly worse than grantmakers at this “forecasting question”. A similar argument could be made for many intellectual jobs, which could be framed as forecasting. The question on whether superforecasters are relatively better isn’t “Is this task answering a forecasting question“ but rather “What are the specific attributes of this forecasting question”.
Some people seem to think that the key difference between questions superforecasters are good at vs. smart domain experts are in questions that are *resolvable* or *short-term*. I tend to think that the main differences are along the axes of *domain-specificity* and *complexity*, though these are of course correlated with the other axes. Superforecasters are selected for being relatively good at short-term, often geopolitical questions.
As I’ve written previously: It varies based on the question/domain how much domain expertise matters, but ultimately I expect reasonable domain experts to make better forecasts than reasonable generalists in many domains.
There’s an extreme here where e.g. forecasting what the best chess move is obviously better done by chess experts rather than superforecasters.
So if we think of a spectrum from geopolitics to chess, it’s very unclear to me where things like long-term AI forecasts land.
This intuition seems to be consistent with the lack of quality existing evidence described in Arb’s report (which debunked the “superforecasters beat intelligence experts without classified information” claim!).
Similarly, I’m skeptical of the straw rationalist view that highly liquid well-run prediction markets would be an insane societal boon, rather than a more moderate-large one (hard to operationalize, hope you get the vibe). See here for related takes. This might change with superhuman AI forecasters though, whose “time” might be more plentiful.
Historically, OP-funded forecasting platforms (Metaculus, INFER) seem to be underwhelming on publicly observable impact per dollar (in terms of usefulness for important decision-makers, user activity, rationale quality, etc.). Maybe some private influence over decision-makers makes up for it, but I’m pretty skeptical.
Tbh, it’s not clear that these and other platforms currently provide more value to the world than the opportunity cost of the people who spend time on them. e.g. I was somewhat addicted to Metaculus then later Manifold for a bit and spent more time on these than I would reflectively endorse (though it’s plausible that they were mostly replacing something worse like social media). I resonate with some of the comments on the EA Forum post mentioning that it’s a very nerd-sniping activity; forecasting to move up a leaderboard (esp. w/quick-resolving questions) is quite addicting to me compared to normal work activities.
I’ve heard arguments that getting superforecasted probabilities on things is good because they’re more legible/credible because they’re “backed by science”. I don’t have an airtight argument against this, but it feels slimy to me due to my beliefs above about superforecaster quality.
Regarding whether forecasting orgs should try to make money, I’m in favor of pushing in that direction as a signal of actually providing value, though it’s of course a balance re: the incentives there and will depend on the org strategy.
The types of forecasting grants I’d feel most excited about atm are, roughly ordered, and without a claim that any are above OpenPhil’s GCR bar (and definitely not exhaustive, and biased toward things I’ve thought about recently):
Making AI products for forecasting/epistemics in the vein of FutureSearch and Elicit. I’m also interested in more lightweight forecasting/epistemic assistants.
FutureSearch and systems in recent papers are already pretty good at forecasting, and I expect substantial improvements soon with next-gen models.
I’m excited about making AIs push toward what’s true rather than what sounds right at first glance or is pushed by powerful actors.
However, even if we have good forecasting/epistemics AIs, I’m worried that it won’t convince people of the truth since people are irrational and often variance in their beliefs is explained by gaining status/power, vibes, social circles, etc. It seems especially hard to change people’s minds on very tribal things, which seem correlated with the most important beliefs to change.
AI friends might actually be more important than AI forecasters for epistemics, but that doesn’t mean AI forecasters are useless.
I might think/write more about this soon. See also Lukas’s Epistemics Project Ideas and ACX on AI for forecasting
Judgmental forecasting of AI threat models, risks, etc. involving a mix of people who have AI / dangerous domain expertise and/or very strong forecasting track record (>90th percentile superforecaster), ideally as many people as possible who have both. Not sure how helpful it will be but it seems maybe worth more people trying.
In particular, forecasting that can help inform risk assessment / RSPs seems like a great thing to try. See also discussion of the Delphi technique in the context of AGI risk assessment here. Malcolm Murray at GovAI is running a Delphi study to get estimates of likelihood and impact of various AI risks from experts.
This is related to a broader class of interventions that might look somewhat like a “structured review process” in which one would take an in-depth threat modeling report and have various people review and contribute their own forecasts in addition to qualitative feedback. My sense is that when superforecasters reviewed Joe Carlsmith’s p(doom) forecast in a similar vein that the result wasn’t that useful, but the exercise could plausibly be more useful with better quality reviews/forecasts. It’s unclear whether this would be a good use of resources above the usual ad-hoc/non-forecasting review process, but might be worth trying more.
Forecasting tournaments on AI questions with large prize pools: I think these historically have been meh (e.g. the Metaculus one attracted few forecasters, wasn’t fun to forecast on (for me at least), and I’d guess significantly improved ~no important decisions), but I think it’s plausible things could go better now as AIs are much more capable, there are many more interesting and maybe important things to predict, etc.
Crafting forecasting questions that are cruxy on threat models / intervention prioritization between folks working on AI safety
It’s kind of wild that there has been so little success on this front. See frustrations from Alex Turner “I think it’s not a coincidence that many of the “canonical alignment ideas” somehow don’t make any testable predictions until AI takeoff has begun.” I worry that this will take a bunch of effort and not get very far (see Paul/Eliezer finding only a somewhat related bet re: their takeoff speeds disagreement), but it seems worth giving a more thorough shot with different participants.
I’m relatively excited about doing things within the AI safety group rather than between this group and others (e.g. superforecasters) because I expect the results might be more actionable for AI safety people.
I incorporated some snippets of a reflections section from a previous forecasting retrospective above, but there’s a little that I didn’t include if you’re inclined to check it out.
I’m grateful to you guys for making this post! :)
I think a lot of the criticisms shared recently have been very valid but overall the EA community is amazing and has also accomplished some great things and I’m super thankful for it. Great idea to create this thread to help keep that in mind!
Background on my views on the EA community and epistemics
Epistemic status: Passionate rant
I think protecting and improving the EA community’s epistemics is extremely important and we should be very very careful about taking actions that could hurt it to improve on other dimensions.
First, I think that the EA community’s epistemic advantage over the rest of the world in terms of both getting to true beliefs via a scout mindset, and taking the implications seriously is extremely important for the EA community’s impact. I think it might be even more important than the moral difference between EA and the rest of the world. See Ngo and Kwa for more here. In particular, in seems like we’re very bottlenecked on epistemics in AI safety, perhaps the most important cause area. See Muelhauser and MIRI conversations.
Second, I think the EA community’s epistemic culture is an extremely important thing to maintain as an attractor for people with a scout mindset and taking-ideas-seriously mentality. This is a huge reason that I and I’m guessing many others love spending time with others in the community, and I’m very very wary about sacrificing it at all. This includes people being transparent and upfront about their beliefs and the implications.
Third, the EA community’s epistemic advantage and culture are extremely rare and fragile. By default, they will erode over time as ~all cultures and institutions do. We need to try really hard to maintain them.
Fourth, I think we really need to be pushing the epistemic culture to improve rather than erode! There is so much room for improvement in quantification of cost-effectiveness, making progress on long-standing debates, making it more socially acceptable and common to critique influential organizations and people, etc. There’s a long way to go and we need to move forward not backwards.
On (2b): I’m a bit sceptical that politicians or policymakers are sufficiently nitpicky for this to be a big issue, but I’m not confident here. WWOTF might just have the effect of bringing certain issues closer to the edges of the Overton window. I find it plausible that the most effective way to make AI risk one of these issues is in the way WWOTF does it: get mainstream public figures and magazines talking about it in a very positive way. I could see how this might’ve been far harder with a book that allows people to brush it off as tech-bro BS more easily.
I think this is a fair point, but even if it’s right I’m worried about trading off some community epistemic health to appear more palatable to this crowd. I think it’s very hard to consistently present your views in a fairly different way publicly than they are presented in internal conversations, and it hinders intellectual progress of the movement. I think we need to be going in the other direction; Rob Bensinger has a twitter thread on how we need to be much more open and less scared of saying weird things in public, to make faster progress.
On there being intellectually dishonesty: I worry a bit about this, but maybe Will is just providing his perspective and that’s fine. We can still have others in the longtermist community disagree on various estimates. Will for one has explicitly tried not to be seen as a leader of a movement of people who just follow his ideas. I’d be surprised if differences within the community become widely seen as intellectual dishonesty from the outside (though of course isolated claims like these have been made already).
Sorry if I wasn’t clear here: I’m most worried about Will not being fully upfront about the implications of his own views.
On alternative uses of time: Those three project seem great and might be better EV per effort spent, but that’s consistent with great writers and speakers like Will having a comparative advantage in writing WWOTF.
Seems plausible, though I’m concerned about community epistemic health from the book and the corresponding big media push. If a lot of EAs get interested via WWOTF they may come in with a very different mindset about prioritization, quantification, etc.
The mechanism I have in mind is a bit nebulous. It’s in the vein of my response to (2a), i.e., creating intellectual precedent, making odd ideas seem more normal, etc. to create an environment (e.g., in politics) more receptive to proposals and collaboration. This doesn’t have to be through widespread understanding of the topics. One (unresearched) analogue might be antibiotic resistance. People in general, including myself, know next to nothing about it, but this weird concept has become respectable enough that when a policymaker Googles it, they know it’s not just some kooky fear than nobody outside strangely named research centres worry about or respectfully engage with.
Seems plausible to me, though I’d strongly prefer if we could do it in a way where we’re also very transparent about our priorities.
(also, sorry for just bringing up the community epistemic health thing now. Ideally I would have brought it up earlier in this thread and discussed it more in the post but have been just fleshing out my thoughts on it yesterday and today.)
- My take on What We Owe the Future by (1 Sep 2022 18:07 UTC; 353 points)
- Quantified Intuitions: An epistemics training website including a new EA-themed calibration app by (20 Sep 2022 22:25 UTC; 86 points)
- Quantified Intuitions: An epistemics training website including a new EA-themed calibration app by (LessWrong; 20 Sep 2022 22:25 UTC; 28 points)
- 's comment on My take on What We Owe the Future by (5 Sep 2022 5:08 UTC; 4 points)
EDIT: Scott has admitted a mistake, which addresses some of my criticism:
(this comment has overlapping points with titotal’s)
I’ve seen a lot of people strongly praising this article on Twitter and in the comments here but I find some of the arguments weak. Insofar as the goal of the post is to say that EA has done some really good things, I think the post is right. But I don’t think it convincingly argues that EA has been net positive for the world.[1]First: based on surveys, it seems likely that most (not all!) highly-engaged/leader EAs believe GCRs/longtermist causes are the most important, with a plurality thinking AI x-risk / x-risk more general are the more important.[2] I will analyze the post from a ~GCR/longtermist-oriented worldview that thinks AI is the most important cause area during the rest of this comment; again I don’t mean to suggest that everyone has it, but if something like it is held by the plurality of highly-engaged/leader EAs it seems highly relevant for the post to be convincing from that perspective.
My overall gripe is exemplified by this paragraph (emphasis mine):
And I think the screwups are comparatively minor. Allying with a crypto billionaire who turned out to be a scammer. Being part of a board who fired a CEO, then backpedaled after he threatened to destroy the company. These are bad, but I’m not sure they cancel out the effect of saving one life, let alone 200,000.
(Somebody’s going to accuse me of downplaying the FTX disaster here. I agree FTX was genuinely bad, and I feel awful for the people who lost money. But I think this proves my point: in a year of nonstop commentary about how effective altruism sucked and never accomplished anything and should be judged entirely on the FTX scandal, nobody ever accused those people of downplaying the 200,000 lives saved. The discourse sure does have its priorities.)
I’m concerned about the bolded part; I’m including the caveat for context. I don’t want to imply that saving 200,000 lives isn’t a really big deal, but I will discuss from the perspective of “cold hard math” .
200,000 lives equals roughly a ~.0025% reduction in extinction risk, or a ~.25% reduction in risk of a GCR killing 80M people, if we care literally zero about future people. To the extent we weight future people, the numbers obviously get much lower.
The magnitude of the effect size of the board firing Sam, of which the sign is currently unclear IMO, seems arguably higher than .0025% extinction risk and likely higher than 200,000 lives if you weight the expected value of all future people >~100x of that of current people.
The FTX disaster is a bit more ambiguous because some of the effects are more indirect; quickly searching for economic costs didn’t find good numbers, but I think a potentially more important thing is that it is likely to some extent an indicator of systemic issues in EA that might be quite hard to fix.
The claim that “I’m not sure they cancel out the effect of saving one life” seems silly to me, even if we just look at generally large “value of a life” estimates compared to the economic costs of the FTX scandal.
Now I’ll discuss the AI section in particular. There is little attempt to compare the effect sizes of “accomplishments” (with each other and also with potential negatives, with just a brief allusion to EAs accelerating AGI) or argue that they are net positive. The effect sizes seem quite hard to rank to me, but I’ll focus on some ones that seem important but potentially net negative (not claiming that they definitely are!), in order of their listing:
“Developed RLHF, a technique for controlling AI output widely considered the key breakthrough behind ChatGPT.”
This is needless to say controversial in the AI safety community
Got two seats on the board of OpenAI, held majority control of OpenAI for one wild weekend, and still apparently might have some seats on the board of OpenAI, somehow?
As I said above the sign of this still seems unclear, and I’m confused why it’s included when later Scott seems to consider it a negative
Helped found, and continue to have majority control of, competing AI startup Anthropic, a $30 billion company widely considered the only group with technology comparable to OpenAI’s.
Again, controversial in the AI safety community
- ^
My take is that EA has more likely than not been positive, but I don’t think it’s that clear and either way, I don’t think this post makes a solid argument for it.
- ^
As of 2019, EA Leaders thought that over 2x (54% vs. 24%) more resources should go to long-term causes than short-term with AI getting the most (31% of resources), and the most highly-engaged EAs felt somewhat similarly. I’d guess that the AI figure has increased substantially given rapid progress since 2019/2020 (2020 was the year GPT-3 was released!). We have a 2023 survey of only CEA staff in which 23⁄30 people believe AI x-risk should be a top priority (though only 13⁄30 say “biggest issue facing humanity right now”, vs. 6 for animal welfare and 7 for GHW). CEA staff could be selected for thinking AI is less important than those directly working on it, but would think it’s more important than those at explicitly non-longtermist orgs.
I’m guessing I have a lower opinion of Leverage than you based on your tone, but +1 on Kerry being at CEA for 4 years making it more important to pay serious attention to what he has to say even if it ultimately doesn’t check out. We need to be very careful to minimize tribalism hurting our epistemics.
Sharing an update on my last 6 months that’s uncomfortably personal for me to want to share as more than a shortform for now, but I think is worth sharing somewhere on the Forum: Personal update: EA entrepreneurship, mental health, and what’s next
tl;dr In the last 6 months I started a forecasting org, got fairly depressed and decided it was best to step down indefinitely, and am now figuring out what to do next. I note some lessons I’m taking away and my future plans.
Adversarial collaborations on important topics
Epistemic Institutions
There are many important topics, such as the level of risk from advanced artificial intelligence and how to reduce it, among which there are reasonable people with very different views. We are interested in experimenting with various types of adversarial collaborations, which we define as people with opposing views working to clarify their disagreement and either resolve the disagreement or identify an experiment/observation that would resolve it. We are especially excited about combining adversarial collaborations with forecasting on any double cruxes identified from them. Some ideas for experimentation might be varying the number of participants, varying the level of moderation and strictness of enforced structure, and introducing AI-based aids.
Existing and past work relevant to this space include the Adversarial Collaboration Project, SlateStarCodex’s adversarial collaboration contests, and the Late 2021 MIRI Conversations.
what makes you think our expected value calculations will ever become good enough, and how will you know if they do?
Agree with this: it seems unclear to me that they’ll become good enough in many cases since our reasoning capabilities are fairly limited and the world is really complicated. I think this point is what Eliezer is trying to communicate with the tweet pictured in the post:
Hey Joshua, appreciate you sharing your thoughts (strong upvoted)! I think we actually agree about the effects of sharing numerical credences more than you might think, but disagree about the solution.
But it also causes people to anchor on what may ultimately be an extremely shaky and speculative guess, hindering further independent analysis and leading to long citation trails. For example, I think the “1-in-6” estimate from The Precipice may have led to premature anchoring on that figure, and likely is relied upon too much relative to how speculative it necessarily is.
I agree that this is a substantial downside of sharing numerical credences. I saw it first-hand with people taking the numbers in my previous post more seriously than I had intended (as you also mentioned!)
However, I think there are large benefits to sharing numerical credences, such that the solution isn’t to share credences less but instead to improve the culture around them.
I think we should shift EA’s culture be more favorable of sharing numerical credences even (especially!) when everyone involved knows they’re tentative, brittle, etc. And we should be able to have discussions involving credences and worry less that others will take them too seriously.
I’ve been hopefully contributing to this some, e.g. by describing my motivation for including confidence numbers as: “I decided it was worth it to propose a definition and go ahead and use as many made-up numbers as possible for transparency.” And I’ve attempted to push back when I’ve perceived others as having taken credences/BOTECs I’ve given too seriously in the past.
Some more ideas for shifting the culture around numerical credences:
Use resilience to demonstrate how brittle your beliefs are.
Highlight how much other reasonable people disagree with your credences.
Openly change your mind and publicly shift your credences when new evidence comes in, or someone presents a good counter-argument.
Explicitly encourage others not to cite your numbers, if you believe they are too brittle (you mention this in your other comment).
I’d love to get others’ ideas for shifting the culture here!
(Comments re-worded from those on a draft)
Overall I like the direction this post pushes in.
I shared a briefing with the participants summarizing the nine Open Philanthropy grants above, with the idea that it might speed the process along.
In hindsight, this was suboptimal, and might have led to some anchoring bias. Some participants complained that the summaries had some subjective component. These participants said they used the source links but did not pay that much attention to these opinions.
On the other hand, other participants said they found the subjective estimates useful. And because the briefing was written in good faith, I am personally not particularly worried about it. Even if there are anchoring issues, we may not necessarily care about it if we think that the output is accurate, in the same way that we may not care about forecasters anchoring on the base rate. [emphasis mine]
If I were redoing this experiment, I would probably limit myself even more to expressing only factual claims and finding sources. A better scheme may have been share a writeup with a minimal subjective component, then strongly encouraging participants to make their own judgments before looking at a separate writeup with more subjective summaries, which they can optionally use to adjust their estimates
I disagree with the opinions expressed in the bolded paragraph. I wouldn’t want forecasters to anchor on a specific base rate I gave them! I’d want them to find their own. Of course you think that the forecasters are anchoring on something accurate since the opinions they’re anchoring on are your own! This isn’t reassuring to me at all.
Thoughts on scaling up this type of estimation up [section header]
I’m more excited about in-depth evaluation of agendas/organizations as a whole than trying to scale up shallow estimations to all grants.
Giving some very quick numbers to this, say:
a 12% chance of AGI being built before 2030,
a 30% of it being built in Britain by then if so,
a 90% of it being built by DeepMind if so,
an initial 50% chance of it going well if so
GovAI efforts shift the probability of it going well from 50% to 55%.
Punching those numbers into a calculator, a rough estimate is that GovAI reduces existential risk by around 0.081%, or 8.1 basis points.
This BOTEC feels too optimistic about GovAI’s impact to me, and I trust it even less than most BOTECs because it’s not directly modeling the channel though which I (and I believe GovAI) think GovAI will have the most impact, which is field-building.
I think a lot of the value of university is providing a peer group for social and intellectual development. To the extent you hasn’t found a great group of friends at university, I think you should actively try very hard to find a group that you enjoy spending time with then lean into this. To the extent you can fill this need outside of university, it seems very reasonable to drop out.
This is the advice I’d give to my younger self: I didn’t make many great friends in college and should have tried much harder to find my niche. But I don’t think I was ready to just drop out and start working, since I hadn’t found my niche outside of university yet either. That being said, it might be a better idea for people who are more emotionally mature than me on some dimensions to “drop out and start working” even if they haven’t quite found their niche.
I’d also link to The Case Against Education. It’s probably a bit hyperbolic but I think it’s pointing at a real dynamic: a large portion of the purpose of college is signaling rather than building human capital.
(I’ve only skimmed the post)
This seems right theoretically, but I’m worried that people will read this and think this consideration ~conclusively implies fewer people should go into AI alignment, when my current best guess is the opposite is true. I agree sometimes people make the argmax vs. softmax mistake and there are status issues, but I still think not enough people proportionally go into AI for various reasons (underestimating risk level, it being hard/intimidating, not liking rationalist/Bay vibes, etc.).
I agree that these can technically all be true at the same time, but I think the tone/vibe of comments is very important in addition to what they literally say, and the vibe of Arepo’s comment was too tribalistic.
I’d also guess re: (3) that I have less trust in CEA’s epistemics to necessarily be that much better than Leverage’s , though I’m uncertain here (edited to add: tbc my best guess is it’s better, but I’m not sure what my prior should be if there’s a “he said / she said” situation, on who’s telling the truth. My guess is closer to 50⁄50 than 95⁄5 in log odds at least).
It also strikes against recent work on patient philanthropy, which is supported by Will MacAskill’s argument that we are not living in the most influential time in human history.
Note that patient philanthropy includes investing in resources besides money that will allow us to do more good later; e.g. the linked article lists “global priorities research” and “Building a long-lasting and steadily growing movement” as promising opportunities from a patient longtermist view.
Looking at the Future Fund’s Areas of Interest, at least 5 of the 10 strike me as promising under patient philanthropy: “Epistemic Institutions”, “Values and Reflective Processes”, “Empowering Exceptional People”, “Effective Altruism”, and “Research That Can Help Us Improve”
I also think using the association between EA and Oxford Uni seems counterintuitive to me; people seem to often associate Oxford with “elitism”.
Agree with this, and this could be further hurt by focusing too much on the areas with more elite universities on the East Coast (of course I am in favor of recruiting from them to a large extent, but shifting the community is a different question). Right now I think the Silicon Valley and Oxford hubs balance each other out well on this dimension.
One aspect of Silicon Valley culture I really like relative to East Coast is that people care very little whether you went to Harvard or dropped out of high school, and they don’t care at all that I prefer to wear t-shirts and shorts every day because it’s much more comfortable for me. To a larger extent than other places, I feel like Silicon Valley culture judges me on what I actually get done.
I think we have good reason to believe veg*ns will underestimate the cost of not-eating-meat for others due to selection effects. People who it’s easier for are more likely to both go veg*n and stick with it. Veg*ns generally underestimating the cost and non-veg*ns generally overestimating the cost can both be true.
The cost has been low for me, but the cost varies significantly based on factors such as culture, age, and food preferences. I think that in the vast majority of cases the benefits will still outweigh the costs and most would agree with a non-speciesist lens, but I fear down-playing the costs too much will discourage people who try to go veg*n and do find it costly. Luckily, this is becoming less of an issue as plant-based substitutes are becoming more widely available.
These were the 3 snippets I was most interested in
Under pure risk-neutrality, whether an existential risk intervention can reduce more than 1.5 basis points per billion dollars spent determines whether the existential risk intervention is an order of magnitude better than the Against Malaria Foundation (AMF).
If you use welfare ranges that are close to Rethink Priorities’ estimates, then only the most implausible existential risk intervention is estimated to be an order of magnitude more cost-effective than cage-free campaigns and the hypothetical shrimp welfare intervention that treats ammonia concentrations. All other existential risk interventions are competitive with or an order of magnitude less cost-effective than these high-impact animal interventions.
Even if you think that Rethink Priorities’ welfare ranges are far too high, many of the plausible existential risk interventions are not an order of magnitude more cost-effective than the hypothetical ammonia-treating shrimp welfare intervention or cage-free campaigns.
The estimate being too low by 1-2 orders of magnitude seems plausible to me independently (e.g. see the wide distribution in my Squiggle model [1]), but my confidence in the estimate is increased by it being the aggregated of several excellent forecasters, who were reasoning independently to some extent. Given that, my all-things-considered view is that 1 order of magnitude off[2] feels plausible but not likely (~25%?), and 2 orders of magnitude seems very unlikely (~5%?).
- ^
EDIT: actually looking closer at my Squiggle model I think it should be more uncertain on the first variable, something like russiaNatoNuclearexchangeInNextMonth=.0001 to .01 rather than .0001 to .003
- ^
Compared to a reference of what’s possible given the information we have, with e.g. a group of 100 excellent forecasters would get if spending 1000 hours each.
- ^
I’ve read it. I’d guess we have similar views on Leverage, but different views on CEA. I think it’s very easy for well-intentioned, generally reasonable people’s epistemics to be corrupted via tribalism, motivated reasoning, etc.
But as I said above I’m unsure.
Edited to add: Either way, might be a distraction to debate this sort of thing further. I’d guess that we both agree in practice that the allegations should be taken seriously and investigated carefully, ideally by independent parties.
I thought I would like this post based on the title (I also recently decided to hold off for more information before seriously proposing solutions), but I disagree with much of the content.
A few examples:
I think we can safely say with at this point >95% confidence that SBF basically committed fraud even if not technically in the legal sense (edit: but also seems likely to be fraud in the legal sense), and it’s natural to start thinking about the implications of this and in particular be very clear about our attitude toward the situation if fraud indeed occurred as looks very likely. Waiting too long has serious costs.
If we were to wait until we close to fully knew “whether or why fraud occurred” this might take years as the court case plays out. I think we should get on with it reasonably quickly given that we are pretty confident some really bad stuff went down. Delaying the investigation seems generally more costly to me than the costs of conducting it, e.g. people’s memories decay over time and people have more time to get alternative stories straight.
This seems wrong, e.g. EA leadership had more personal context on Sam than investors. See e.g. Oli here with a personal account and my more abstract argument here.