I lead the DeepMind mechanistic interpretability team
Neel Nanda
Karma: 4,362
What does reclaim give you? I’ve never heard of it, and the website is fairly uninformative
It looks like the total number of lives saved by all Singer- and EA-inspired donors over the past 50 years may be small, or even zero
This conclusion from the first half of the letter seems unjustified by the prior text?
You seem to be arguing that there’s a credit allocation problem, where there’s actually many actors who contribute to a bednet saving a life, but Givewell style calculations ignore this, and give all the credit to the donor, which leads to over counting. I would describe this as GiveWell computing the marginal impact, which I think is somewhat reasonable (how is the world different if I donate vs don’t donate), but agree this has issues and there are arguments for better credit allocation methods. I think this is a fair critique.
But, I feel like at best this dilutes the impact by a factor of 10, maybe 100 at an absolute stretch. If we take rough estimates like 200,000 lives saved via GiveWell (rough estimate justified in footnote 1 of this post), that’s still 20,000 or 2,000 lives saved. I don’t see how you could get to “small or even zero” from this argument
I like Caleb’s answer. Some more thoughts
Clearly writing out the evidence that you’re a good candidate for the grant. This can look like conventional credentials, past projects, references, etc, just anything which increases my probability that it’s a good idea
Scoping out the purpose of the grant. In particular, I generally expect that when someone without a track record of AI Safety/similar research does an independent research project, it’s fairly unlikely to actually result in impactful research (doing good research is really hard!), and that most of the impact comes from helping the person skill up, test fit, and either get a job or do better projects in future. You tend to skill up best when making a sincere effort to do good research, so this doesn’t mean don’t think about it all, but I would also discuss the skilling up benefits to you and why that matters
apply to several funders where possible
I’m surprised by this one! I see how it’s in the applicant’s interests, but why does it matter to you?
How is it silly? It seems perfectly acceptable, and even preferable, for people to be involved in shaping EA only if they agree for their leadership to be scrutinized.
My argument is that barring them doesn’t stop them from shaping EA, just mildly inconveniences them, because much of the influence happens outside such conferences
With all the scandals we’ve seen in the last few years, I think it should be very evident how important transparency is
Which scandals do you believe would have been avoided with greater transparency, especially transparency of the form here (listing the names of those involved, with no further info)? I can see an argument that eg people who have complaints about bad behaviour (eg Owen’s, or SBF/Alameda’s) should make them more transparently (though that has many downsides), but that’s a very different kind of transparency.
I’m not expressing an opinion on that. The post makes a clear claim that their legal status re tax deductibility will change if more EU citizens sign up. This surprises me and I want to understand it better. I agree there are other benefits to having more members, I’m not disputing that
I’m surprised that having more members let’s you offer better tax deductions (and that they don’t even need to be Danish taxpayers!), what’s up with that?
Seems like she’ll have a useful perspective that adds value to the event, especially on brand. Why do you think it should be arms length?
This seems fine to me—I expect that attending this is not a large fraction of most attendee’s impact on EA, and that some who didn’t want to be named would have not come if they needed to be on a public list, so barring such people seems silly (I expect there’s some people who would tolerate being named as the cost of coming too, of course). I would be happy to find some way to incentivise people being named.
And really, I don’t think it’s that important that a list of attendees be published. What do you see as the value here?
Seems reasonable (tbh with that context I’m somewhat OK with the original ban), thanks for clarifying!
when they’re considering buying mansions in the Oxford countryside/other controversial multimillion dollar calculations, publishing the cost-benefit calculation rather than merely asserting its existence
Huh? That wasn’t CEAs decision, they just fiscally sponsored Wytham
1 is very true, 2 I agree with apart from the word main, it seems hard to label any factor as “the main” thing, and there’s a bunch of complex reasoning about counterfactuals—eg if GDM stopped work that wouldn’t stop Meta, so is GDM working on capabilities actually the main thing?
I’m pretty unconvinced that not sharing results with frontier labs is tenable—leaving aside that these labs are often the best places to do certain kinds of safety work, if our work is to matter, we need the labs to use it! And you often get valuable feedback on the work by seeing it actually used in production. Having a bunch of safety people who work in secret and then unveil their safety plan at the last minute seems very unlikely to work to me
I personally think that “does this advance capabilities” is the wrong question to ask, and instead you should ask “how much does this advance capabilities relative to safety”. Safer models are just more useful, and more profitable a lot of the time! Eg I care a lot about avoiding deception. But honest models are just generally more useful to users (beyond white lies I guess). And I think it would be silly for no one to work on detecting or reducing deception. I think most good safety work will inherently advance capabilities in some sense, and this is a sign that it’s actually doing anything real. I struggle to think of any work I think is both useful and doesn’t advance capabilities at all
Ah, thanks, that’s important context—I semi-retract my strongly worded comment above, depending on exactly how bad the removed post was, but can imagine posts in this genre that I think are genuinely bad
Strong +1 to Richard, this seems a clear incorrect moderation call and I encourage you to reverse it.
I’m personally very strongly opposed to killing people because they eat meat, and the general ethos behind that. I don’t feel in the slightest offended or bothered by that post, it’s just one in a string of hypothetical questions, and it clearly is not intended as a call to action or to encourage action.
If the EA Forum isn’t somewhere where you can ask a perfectly legitimate hypothetical question like that, what are we even doing here?
I think this is a valid long term concern but takes at least a few months to properly propagate—if someone qualified tells you that when hiring they look at a github profile, that’s probably pretty good for the duration of your job search
I made a reasonably large donation to LTFF at the time of the match, and it felt very clear to me exactly what the situation was, that the matching funds were questionably counterfactual, and felt like just a small bonus to me. I thought the comms there were good.
I imagine you can get a lot of the value here more cheaply by reaching out to people in the field and asking them a bunch of questions about what signals do and do not impress them?
Doing internships etc is valuable to get the supervision to DO the impressive projects, of course.
EDIT: Speaking as someone who does hiring of interpretability researchers, I think there’s a bunch of signals I look for and ones I don’t care about, and sometimes people new to the field have very inaccurate guesses here
Ah, fair. Yes, I agree that’s a plausible factor, especially for nicher areas
Sure, I agree that under the (in my opinion ridiculous and unserious) accounting method of looking at the last actor, zero is a valid conclusion.
I disagree that small is accurate—I feel like even if I’m being incredibly charitable and say that the donor is only 1% of the overall ecosystem saving the life, we still get to 2000 lives saved, which seems highly unreasonable to call small—to me small is at best <100