You can give me anonymous feedback here. I often change my mind and don’t necessarily endorse past writings.
elifland
Karma: 2,914
74 (= 0.074/0.01)
.074/.01 is 7.4, not 74
Eli Lifland on Navigating the AI Alignment Landscape
In an update on Sage introducing quantifiedintuitions.org, we described a pivot we made after a few months:
As stated in the grant summary, our initial plan was to “create a pilot version of a forecasting platform, and a paid forecasting team, to make predictions about questions relevant to high-impact research”. While we build a decent beta forecasting platform (that we plan to open source at some point), the pilot for forecasting on questions relevant to high-impact research didn’t go that well due to (a) difficulties in creating resolvable questions relevant to cruxes in AI governance and (b) time constraints of talented forecasters. Nonetheless, we are still growing Samotsvety’s capacity and taking occasional high-impact forecasting gigs.
[...]
Meanwhile, we pivoted to building the apps contained in Quantified Intuitions to improve and maintain epistemics in EA.
Ought has pivoted ~twice: from pure research on factored cognition to forecasting tools to an AI research assistant.
Discussing how to align Transformative AI if it’s developed very soon
Personally the FTX regrantor system felt like a nice middle ground between EA Funds and donor lotteries in terms of (de)centralization. I’d be excited to donate to something less centralized than EA Funds but more centralized than a donor lottery.
Which part of my comment did you find as underestimating how grievous SBF/Alameda/FTX’s actions were? (I’m genuinely unsure)
Nitpick, but I found the sentence:
Based on things I’ve heard from various people around Nonlinear, Kat and Emerson have a recent track record of conducting Nonlinear in a way inconsistent with EA values [emphasis mine].
A bit strange in the context of the rest of the comment. If your characterization of Nonlinear is accurate, it would seem to be inconsistent with ~every plausible set of values and not just “EA values”.
Appreciate the quick, cooperative response.
I want you to write a better post arguing for the same overall point if you agreed with the title, hopefully with more context than mine.
Not feeling up to it right now and not sure it needs a whole top-level post. My current take is something like (very roughly/quickly written):
New information is currently coming in very rapidly.
We should at least wait until the information comes in a bit slower before thinking seriously in-depth about proposed mitigations so we have a better picture of what went wrong. But “babbling” about possible mitigations seems mostly fine.
An investigation similar to the one proposed here should be started fairly quickly, with the goal of producing an initial version of a report within ~2 months so we can start thinking pretty seriously about what mitigations/changes are needed, even if a finalized report would take longer.
My main thought is that I don’t know why he committed fraud. Was it actually to utility maximize, or because he was just seeking status, or got too prideful or what?
I think either way most of the articles you point to do more good than harm. Being more silent on the matter would be worse.
I’d agree with this if I thought EA right now had a cool head. Maybe I should have said we should wait until EA has a cooler head before launching investigations.
I’d hope that the investigation would be conducted mostly by an independent, reputable entity even if commissioned by EA organizations. Also, “EA” isn’t a fully homogeneous entity and I’d hope that the people commissioning the investigation might be more cool-headed than the average Forum poster.
I thought I would like this post based on the title (I also recently decided to hold off for more information before seriously proposing solutions), but I disagree with much of the content.
A few examples:
It is uncertain whether SBF intentionally committed fraud, or just made a mistake, but people seem to be reacting as if the takeaway from this is that fraud is bad.
I think we can safely say with at this point >95% confidence that SBF basically committed fraud even if not technically in the legal sense (edit: but also seems likely to be fraud in the legal sense), and it’s natural to start thinking about the implications of this and in particular be very clear about our attitude toward the situation if fraud indeed occurred as looks very likely. Waiting too long has serious costs.
We could immediately launch a costly investigation to see who had knowledge of fraud that occurred before we actually know if fraud occured or why. In worlds where we’re wrong about whether or why fraud occurred this would be very costly. My suggestion: wait for information to costlessly come out, discuss what happened when not in the midst of the fog and emotions of current events, and then decide whether we should launch this costly investigation.
If we were to wait until we close to fully knew “whether or why fraud occurred” this might take years as the court case plays out. I think we should get on with it reasonably quickly given that we are pretty confident some really bad stuff went down. Delaying the investigation seems generally more costly to me than the costs of conducting it, e.g. people’s memories decay over time and people have more time to get alternative stories straight.
Adjacently, some are arguing EA could have vetted FTX and Sam better, and averted this situation. This reeks of hindsight bias! Probably EA could not have done better than all the investors who originally vetted FTX before giving them a buttload of money!
Maybe EA should investigate funders more, but arguments for this are orthogonal to recent events, unless CEA believes their comparative advantage in the wider market is high-quality vetting of corporations. If so, they could stand to make quite a bit of money selling this service, and should possibly form a spinoff org.
This seems wrong, e.g. EA leadership had more personal context on Sam than investors. See e.g. Oli here with a personal account and my more abstract argument here.
I’m not as sure about advisors, as I wrote here. Agree on recipients
It’s a relevant point but I think we can reasonably expect EA leadership to do better at vetting megadonors than Sequoia due to (a) more context on the situation, e.g. EAs should have known more about SBF’s past than Sequoia and/or could have found it out more easily via social and professional connections (b) more incentive to avoid downside risks, e.g. the SBF blowup matters a lot more for EA’s reputation than Sequoia’s.
To be clear, this does not apply to charities receiving money from FTXFF, that is a separate question from EA leadership.
Thanks for clarifying. To be clear, I didn’t say I thought they were as bad as Leverage. I said “I have less trust in CEA’s epistemics to necessarily be that much better than Leverage’s , though I’m uncertain here”
I’ve read it. I’d guess we have similar views on Leverage, but different views on CEA. I think it’s very easy for well-intentioned, generally reasonable people’s epistemics to be corrupted via tribalism, motivated reasoning, etc.
But as I said above I’m unsure.
Edited to add: Either way, might be a distraction to debate this sort of thing further. I’d guess that we both agree in practice that the allegations should be taken seriously and investigated carefully, ideally by independent parties.
I agree that these can technically all be true at the same time, but I think the tone/vibe of comments is very important in addition to what they literally say, and the vibe of Arepo’s comment was too tribalistic.
I’d also guess re: (3) that I have less trust in CEA’s epistemics to necessarily be that much better than Leverage’s , though I’m uncertain here (edited to add: tbc my best guess is it’s better, but I’m not sure what my prior should be if there’s a “he said / she said” situation, on who’s telling the truth. My guess is closer to 50⁄50 than 95⁄5 in log odds at least).
I’m guessing I have a lower opinion of Leverage than you based on your tone, but +1 on Kerry being at CEA for 4 years making it more important to pay serious attention to what he has to say even if it ultimately doesn’t check out. We need to be very careful to minimize tribalism hurting our epistemics.
Does the “deliberate about current evidence” part includes thinking a lot about AI alignment to identify new arguments or considerations that other people on Earth may not have thought of, or would that count as new evidence?
It seems like if that would not count as new evidence, that the team you described might be able to come up with much better forecasts than we have today, and I’d think their final forecast would be more likely to end up much lower or much higher than e.g. your forecast. One consequence of this might then be be that your 90% confidence about MacAskill’s misaligned AI takeover credence is too high, even if your 35% point estimate is reasonable.I added this line in response to Lukas pointing out that the researchers could just work on the agendas to get information. As I mentioned in a comment, the line between deliberating and identifying new evidence is fuzzy but I think it’s better to add a rough clarification than nothing.
I intend identifying new arguments or considerations based on current evidence to be allowed, but I’m more skeptical than you that this would converge that much closer to 0% or 100%. I think there’s a ton of effectively irreducible uncertainty in forecasting something as complex as whether misaligned AI will takeover this century.
Thanks Max! Was great seeing you as well. I did take some time off and was a bit more chill for a little while blogging however much I felt like. I’ve been doing a lot better for the past 2 months.
These were the 3 snippets I was most interested in
Under pure risk-neutrality, whether an existential risk intervention can reduce more than 1.5 basis points per billion dollars spent determines whether the existential risk intervention is an order of magnitude better than the Against Malaria Foundation (AMF).
If you use welfare ranges that are close to Rethink Priorities’ estimates, then only the most implausible existential risk intervention is estimated to be an order of magnitude more cost-effective than cage-free campaigns and the hypothetical shrimp welfare intervention that treats ammonia concentrations. All other existential risk interventions are competitive with or an order of magnitude less cost-effective than these high-impact animal interventions.
Even if you think that Rethink Priorities’ welfare ranges are far too high, many of the plausible existential risk interventions are not an order of magnitude more cost-effective than the hypothetical ammonia-treating shrimp welfare intervention or cage-free campaigns.