Tell me more about these “luxurious AI safety retreats”? I haven’t been to an AI safety workshop in several years, and wonder if something has changed. From searching the web, I found this:
I was there for an AI workshop earlier this year in Spring and stayed for 2 or 3 days, so let me tell you about the ‘luxury’ of the ‘EA castle’: it’s a big, empty, cold, stone box, with an awkward layout. (People kept getting lost trying to find the bathroom or a specific room.) Most of the furnishings were gone. Much of the layout you can see in Google Maps was nonfunctional, and several wings were off-limits or defunct, so in practice it was maybe a quarter of the size you’d expect from the Google Maps overview. There were clearly extensive needs for repair and remodeling of a lot of ancient construction, and most of the gardens are abandoned as too expensive to maintain. It is, as a real estate agent might say, very ‘historical’ and a ‘good fixer-upper’.
Can I suggest you make this a new top-level post and link to it here? It sounds like you’ve been thinking about it a lot, and I think continued discussion would probably be better in its own post rather than here (although your original comment makes sense here for sure!)
Thanks for saying this. Sadly there is a lot of deference when it comes to AI safety and its questionable researchers, and while EA claims it loves criticism, I didn’t meet much love when raising my concerns.
In a group that is composed at 80% by rich white males who have a STEM background where AI safety allows them to get the recognition of their technical skills AND a huge pay, raising such concerns never goes well.
I’m actually preparing a series of post on the lack of diversity within AI and cultural biases will be part of it—how your critical thinking shuts down when it comes to doing work you love, and how evidence that existential risks should be prioritized falls apart under hard criticism (see David Thorsad’s criticism of Bostrom’s famous number 10^16). I expect much pushback and blind denial, as I can see with the comments under my own post that are pretty much just saying ‘AI researchers deserve to be paid well because ML is hard’. I have news: it’s far from unique to AI safety, sadly.
the comments under my own post that are pretty much just saying ‘AI researchers deserve to be paid well because ML is hard’
Which of the comments under your post do you read that way?
I understand the standard argument to be more like “AI researchers have commercial options that will pay them very highly, so it’s hard to get good AI researchers to work on altruistic projects if you offer too far below what they could be making elsewhere”.
Fellowships where you seek people with excellent machine learning skills should be well-paid to attract talent, especially given how much such people can make doing capacity research.
3000 per month for beginners in an AI fellowship is way, way too much.
We need to stop considering machine learning engineers as la crème de la crème and justify these exorbitant salaries based on that assumption. The tractability and impact measuring of the work of these people is highly questionable (the causality series written by RP rates existential risk research at 2 on a scale from 1 to 5, tractability-wise).
$3,000 a month in San Francisco is literally under minimum wage. Entry level salaries in data science in the US, for people fresh out of college or a boot camp program, is like $8,000 a month.
I should have specified: the fellowships I’m talking about are in London/Switzerland. Still expensive but nothing that justifies paying people with barely a bachelor degree and no work experience
sounds like it’s also below Zurich minimum wage (not totally sure if that minimum wage is currently in effect or not) and similar to the London “living wage” (which isn’t a required thing)
How much should a technical researcher be paid during an AI safety fellowship in your opinion? 3000 Euro per month does not sound like a lot to me.
(Actually, I think that many AI safety researchers are being paid a lot more than just 3000 Euro. My guess is that some at Anthropic might earn 6 times as much).
We’re talking fellows, so people with very little experience and no certainty of impact at all. You’re comparing this with fully-fledged Anthropic researchers, which doesn’t make sense at all.
And I could talk how these researchers at Anthropic are probably paid way too much for the tractability of their work, but I guess this asks for another post.
Tell me more about these “luxurious AI safety retreats”? I haven’t been to an AI safety workshop in several years, and wonder if something has changed. From searching the web, I found this:
and this:
And not much visible evidence of luxury.
That’s one example, it is only one though; many other fellowships are very well-paid, up to 3000 euros per month, I’m thinking SERI/CHERI/CERI
Can I suggest you make this a new top-level post and link to it here? It sounds like you’ve been thinking about it a lot, and I think continued discussion would probably be better in its own post rather than here (although your original comment makes sense here for sure!)
Thanks for saying this. Sadly there is a lot of deference when it comes to AI safety and its questionable researchers, and while EA claims it loves criticism, I didn’t meet much love when raising my concerns.
In a group that is composed at 80% by rich white males who have a STEM background where AI safety allows them to get the recognition of their technical skills AND a huge pay, raising such concerns never goes well.
I’m actually preparing a series of post on the lack of diversity within AI and cultural biases will be part of it—how your critical thinking shuts down when it comes to doing work you love, and how evidence that existential risks should be prioritized falls apart under hard criticism (see David Thorsad’s criticism of Bostrom’s famous number 10^16). I expect much pushback and blind denial, as I can see with the comments under my own post that are pretty much just saying ‘AI researchers deserve to be paid well because ML is hard’. I have news: it’s far from unique to AI safety, sadly.
Which of the comments under your post do you read that way?
I understand the standard argument to be more like “AI researchers have commercial options that will pay them very highly, so it’s hard to get good AI researchers to work on altruistic projects if you offer too far below what they could be making elsewhere”.
Fellowships where you seek people with excellent machine learning skills should be well-paid to attract talent, especially given how much such people can make doing capacity research.
3000 per month for beginners in an AI fellowship is way, way too much.
We need to stop considering machine learning engineers as la crème de la crème and justify these exorbitant salaries based on that assumption. The tractability and impact measuring of the work of these people is highly questionable (the causality series written by RP rates existential risk research at 2 on a scale from 1 to 5, tractability-wise).
$3,000 a month in San Francisco is literally under minimum wage. Entry level salaries in data science in the US, for people fresh out of college or a boot camp program, is like $8,000 a month.
I should have specified: the fellowships I’m talking about are in London/Switzerland. Still expensive but nothing that justifies paying people with barely a bachelor degree and no work experience
I don’t think that helps much? If it was in, say, Geneva then $3k/month is under minimum wage.
(I agree $3k/month is above minimum wage in London)
Is your actual objection to hiring new grads to at all?
sounds like it’s also below Zurich minimum wage (not totally sure if that minimum wage is currently in effect or not) and similar to the London “living wage” (which isn’t a required thing)
How much should a technical researcher be paid during an AI safety fellowship in your opinion? 3000 Euro per month does not sound like a lot to me.
(Actually, I think that many AI safety researchers are being paid a lot more than just 3000 Euro. My guess is that some at Anthropic might earn 6 times as much).
We’re talking fellows, so people with very little experience and no certainty of impact at all. You’re comparing this with fully-fledged Anthropic researchers, which doesn’t make sense at all.
And I could talk how these researchers at Anthropic are probably paid way too much for the tractability of their work, but I guess this asks for another post.