ML safety researcher, working on NN interpretability.
Adrià Garriga Alonso
Great thanks, I’ve set up a recurring donation!
EDIT: apparently they’re very time-constrained, so I’ll give $13.3k as a lump sum instead.
No, I think your table is substantially better than chatgpt’s because it factors out the two alignment dimensions into two spatial dimensions.
Literally everyone knows he was the Masculine Mongoose. Superheros don’t even try to hide their identity any more.
You can get research taste by doing research at all, it doesn’t have to be a PhD. You may argue that PIs have very good research taste that you can learn from. But their taste is geared towards satisfying academic incentives! It might not be good taste for what you care about. As Chris Olah points out, “Your taste is likely very influenced by your research cluster”.
Thank you for the write-up. I wish I had this advice, and (more crucially) kept reminding myself of it, during my PhD. As you say, academic incentives did poison my brain, and I forgot about my original reasons for entering the programme. I only realised one month ago that it had been happening slowly; my brain is likely still poisoned, but I’m working on it.
I’m curious about your theory of change, if you have time to briefly write about it. You wrote that
addressing these risks goes substantially through EAs taking on a lot more object level work— founding organizations, engineering systems, making scientific progress— than I expect is the median view
and that you don’t think gunning for a faculty position is a good thing. What kind of job is the right one to “make scientific progress”, then? I thought that the best way to do that is to run a lab, managing a bunch of smart PhD students and postdocs, and steering them towards useful research directions.
My impression is that PIs manage the same or more people than the equivalent seniority position in industry, at least in machine learning; but that they have freedom to set research priorities, instead of having to follow a boss. (On the flipside, they have to pander to grant givers, but that seems to give more freedom in research direction).
In summary, what do you think is the kind of job where you can make the most scientific progress?
I was scared when seeing the title. Then I read a little bit more:
That is, the entire good of humanity may be outweighed by the cumulative suffering of farmed animals, with total animal suffering growing faster than human wellbeing is increasing, especially in recent decades
I already thought that was the case. It’s really sad, but at least it’s not another massive source of suffering to add to my list. Thank you for substantiating this with a calculation!
It is unclear to me if this is a good idea. Sci-hub is great, but whoever does this would face a good amount of legal risk. If EA organisations (eg American ones) are known to be funding this, they face the risk of lawsuits and reputational damage.
I think at least this post should not be publicized too widely. Maybe nobody else commented on this post for precisely this reason?
I think even among such selected crowd, Anita would stand out like a bright star. The average top-university PhD student doesn’t end up holding a top faculty job. (This may seem elitist, but it is important: becoming a trainer of mediocre PhD students is likely not more effective than non-profit work). A first-author Nature paper in undergrad (!) is quite rare too.
Very cool, I didn’t actually believe that other regulatory regimes emulated the EU, but I believe it a little bit now. The large number of GDPR emulations surprised me.
One thing I don’t quite get
This complicated architecture has also had a 5.2% growth rate in all its bodies combined, with most of the staff being highly educated (usually possessing a master’s degree).
This is a growth in the number of staff?
Both of these factors resulted in a signal of competence to other countries in the world, which results in a higher degree of trust in the EU’s decisions and policies.
Why does having more staff signal competence? Is it because more budget implies the agencies are being taken seriously?
Yes.
In my read, this post is not about whether having children (literal ‘pro’ ‘natalism’) is correct or not. I think having a debate about that is great, and I’m inclined towards the ‘yes’ side.
It’s about pointing to signs suggesting the existence of power-seeking faction within EA, that (by their own admission) is attempting to coopt the movement for their own aims.
(Why the hedging in the previous paragraph: stating that your faction is “now 100X more likely to become a real, dominant faction” is not quite stating your intention to make it dominate, it just suggests it.)
I think coopting a broad altruism movement for particular people’s pet causes (or selfish aims) is bad. There is a very real possibility that EA will be coopted by power-seekers and become another club for career-conscious people who want to get ahead. I support good attempts to prevent that.
This pointing is asymmetrical with respect to the question of whether the purported ‘faction’ in question is in fact a faction, and is in fact trying to coopt the movement.
My impression is that it might be easy to miss some amino acid types if you’re not careful (e.g. tryptophan is almost exclusively found in meat/dairy and is the only way your body can make serotonin
I am pretty confident that this particular impression is incorrect. The essential amino-acid profiles of the protein of most plant sources is very close to human requirements. See in particular Figure 14 of the WHO report on amino-acid requirements. (https://apps.who.int/iris/bitstream/handle/10665/43411/WHO_TRS_935_eng.pdf?sequence=1&isAllowed=y, page 165 of the PDF). It compares the human percentage amino-acid requirements with the content of various animal and vegetal sources. They are incredibly similar, and also the percentage of tryptophan required is larger than the human pattern in all plant sources (except perhaps maize if we scale down the bars).
That said, thank you for the post! I am now 70% confident that I am in fact stressed; but I don’t see a way to stop it, the work just keeps on piling up.
I don’t see how this is a counterargument. Do you mean to say that, once you are on track to tenure, you can already start doing the high-impact research?
It seems to me that, if this research is too diverged from the academic incentives, then our hypothetical subject may become one of these rare cases of CS tenure-track faculty that does not get tenure.
Good insight, thank you for writing this post! I agree with it. Now that you point it out, I find striking how knowlege has compounded, even more impressively than money.
I would like to add another contestant: influence, within or out of mainstream institutions. As a movement, social capital and influence on other people (especially politicians) could prove very useful to be able to have a large impact when the time is right. I’m thinking especially of the Mont Pelerin society: how they spread in economics academia by convincing people and placing people in positions of (mostly academic) influence; and how they eventually became orthodox economic policy.
The EA community also seems to be very aware of the MPS. What I’m pointing out is that, under your framework, community building is also an intervention for patient longtermism.
No, getting rid of factory farming (“fiat iustitia”) won’t increase X-risk (“pereat mundus”).
Or are you implying that resources are in competition for the two? (Perhaps weakly true)
thank machine doggo
In 1951, Alan Turing argued that at some point computers would probably exceed the intellectual capacity of their inventors, and that “therefore we should have to expect the machines to take control.” Whether that is a good or a bad thing depends on whether the machines are benevolent towards us or not. (Partial source: https://www.newyorker.com/magazine/2015/11/23/doomsday-invention-artificial-intelligence-nick-bostrom )
[for policy makers]
It is a mistake to assume that AI researchers are driven by the positive consequences of their work. Geoffrey Hinton, winner of a Turing Award for his enormous contribution to deep neural networks, is not optimistic about the effects of advanced AI, or whether humans can decide what it does. In a 2015 meeting of the Royal Society, he stated that “there is not a good track record of less intelligent things controlling things of greater intelligence”, and that “political systems will use [AI] to terrorize people”. Nevertheless, he presses on with his research, because “the prospect of discovery is too sweet”.
(source for the quotes: https://www.newyorker.com/magazine/2015/11/23/doomsday-invention-artificial-intelligence-nick-bostrom )
[for any audience]
The chief asset of the human species is our intelligence: with it, we have settled all over and transformed the world. Most machine learning researchers expect AI to surpass human intelligence in all areas within a lifetime (source ). When that happens, humanity will find ourselves in the same place as chimpanzees: with our fate at the mercy of the most intelligent species. As deep learning Geoffrey Hinton noted, “there is not a good track record of less intelligent things controlling things of greater intelligence”.
What does the WANBAM acronym (assuming it is one) stand for? Presumably Women And Non Binary… Altruism Movement?
(apologies if the question is irrelevant but I’m very curious, and I couldn’t find this in the post or the website)
Work trials (paid, obviously) are awesome for better hiring, especially if you’re seeking to get good candidates that don’t fulfill the traditional criteria (e.g. coming from an elite US/UK university). Many job seekers don’t have a current employment.
Living with other EAs or your coworkers is mostly fine too, especially if you’re in a normal living situation, like most EA group houses are.
These suggestions aren’t great. I agree with the “Don’t date” ones, but these were already argued for before.