There’s an old Facebook group, New EA hub search and planning, that might be worth reviving in light of the incresed amount of funding coming into the space.
Even with more availability of funding, many talented folks won’t get funded or will only recieve small amounts. Establishing new hubs that provide an easier entry-way to being surrounded by community seems like potentially quite high value. After all, it takes a village to raise an EA and being surrounded by others is an accelerator to impact.
Chris Leong
Karma: 7,790
Sydney has an AI Safety Hub, but it’s far from it’s potential. Unfortunately, this isn’t anyone’s main focus right now.
I’d talk to Yanni who runs TARA. He’s running ML bootcamps in a bunch of cities and he’s planning to scale this further. The graduates of his program could form the seeds of a new hub.
I think this is a cool idea, esp. re: sharing of resources and I’d be excited to see if your free agent plan works. However, I’m honestly pretty skeptical of your city prioritisation and that dramatically reduces my estimation of the value of this project as far in my books. I expect this would also reduce the chance of the project being funded.
One worry, which only applies to certain classes of MCP, is that folks may try to take what they’re doing already and reframe it as AIS, without checking whether the ToC makes sense, BC of the increasing status and funding associated with AIS.
They just need to be upskilled on the AI safety component, which I believe could be done to a reasonable degree with maybe a week’s worth of reading and a weekend-long workshop.
I’m pretty skeptical of this. Then again, it depends on the actual role. Maybe for an ops person you just want them to understand the basics and then they can pick up the rest over time.It’s a bit unreasonable to ask someone established to quit their job, take a massive pay cut, and join a weird three-month thing with no guarantee of a stable job on the other side.
It’s not clear to me that this isn’t good actually in that it filters for people who really are committed.
Actually, I don’t want to go that far because some people have family commitments that would be a blocker regardless of how committed they are, but IMHO commitment selection filters are underrated.That’s a big ask. Every ounce of friction and doubt you can remove matters.
I don’t actually know if we want to remove all of the frictions. See Chesteron’s Fence. I would be surprised if you knew who was or wasn’t value-aligned or committed before the workshop.
Step one: more Praxis-like events.
I would be excited to see this too. Though honestly, I would be less excited by the ability to recruit some of the participants (immediate effect) than the potential to perculate these ideas into a particular scene (medium and long-term effect). I am also somewhat wary of recruiting too heavily from the debating scene—there are many brilliant debaters, but are they just going to very persuasively convince us to pursue the wrong strategy?
Cover the field wide, not deep
Agreed. People can only learn so much in a short program. It’s best to accept this and to see if you can spark a passion or interest in them, because if you succeed here, they’ll learn about the area themselves.
Resist it. A participant who leaves motivated will go learn the camps on their own
I’m more pessimistic than you. I really like the work of Wittgensten, a philosopher of language (“a langauge is a way of life”, “if a lion could speak, we couldn’t understand him”). People can pick up a shallow version of the other perspectives themselves, but learning how to step into another way of seeing the world is actually something that most people need scaffolding for.
I think that the strongest argument for a lot of your positions is that if you think RSI is imminent we should make a lot of decisions that would be long-term damaging for the community in order to gain short-term value. That could be the right decision, but if we decide to do that, we should pursue that consciously.
Reading List on Wise AI
People talk about “democratising” AI. I prefer the term “despotising” AI. That is, uploading the weights on the internet so that despots around the world can download AI to use to oppress their people in a permissionless fashion. Even better (for the despots, at least), it doesn’t just provide them with the tools to oppress their population, but it also provides them with an credible excuse to do so. After all, with the right AI tools individuals are more than capable of causing significant harm. The despots don’t just get to oppress their populations, they also get to present themselves as heroes for doing so.
Could staple some money to it but prestige is probably the main prize
Money can help bootstrap prestige.
Hey Jen,
Just thought I’d share my essay, which won a runner-up prize in the Manifund Essay Competition on Building the EA/AI Safety Scene in San Francisco with you in case it is of any use (I tried to PM you, but I ran into a bug).
Obviously, take it with a massive grain of salt: 1) I don’t live in SF 2) you’ve actually been running things in SF which is much more informative than just theorising about what should be done like I did with my post 3) The best strategy for you to pursue is highly contingent on your own individual skills and fit.
Best,
Chris
Astral Codex: Breakdown in Pakistan
[Question] What should the EA community be doing now that the (AIS) Overton window has opened up again?
At AI StopWatch, we typically post between two and five times a day, seven days a week. Every evening, we compile the day’s posts — which are low on jargon, high on water cooler discussability (we hope) — into an email-friendly “Daily Digest,” which we also release as a ~10-20 minute audio podcast a few hours later.
I would guess that would be very resource-intensive. How much FTE staff do you have working on this?
Is this US only?
80% chance I can add a few comments if someone produces a draft curriculum. (Alternatively, I might be able to join a call).
Someone really needs to create a course based on Will’s EA in the Age of AGI: https://forum.effectivealtruism.org/posts/R8AAG4QBZi5puvogR/effective-altruism-in-the-age-of-agi
Using the term “cult” in in a purported terminology post isn’t helpful.
I know it’s annoying, but I would have suggested splitting this into two posts, one introducing the terminology in a neutral fashion and another where you use the terminology to make your criticism.
There’s been a massive wave of AIbiosec programs recently. I uncertain how fast this can be scaled up, then again, this isn’t my area.
I think I might have previously posted this as a comment on another post, but I’ll share here as well:
”Recently I’ve been thinking about how the EA community is not what it used to be and how it’s unlikely to ever be like that again (EA resonated strongly in a social context that no longer exists). But, of course, there’s a saying ‘when one door closes, another opens’. This leads to the question: ’What could EA be instead?’.
I’ll quote from Communities are Not Fungible:
“But a community that took twenty years to develop its particular structure of norms and mutual knowledge cannot be regrown in twenty years, because the conditions that shaped it no longer exist. The people are older, the context has changed, and the specific convergance of circumstances that brought those particular individuals together in that particular configuration at that particular time is gone. Communities are path-dependent in the strongest possible sense: their current state is a function of their entire history, and you can’t rerun the history.”
Contexts change and what resonates during one time period is different from what resonates in another.”
I can see two paths forward:
• Defining a new vision for where we find ourselves today
• Managed decline[1]- ^
Which sounds bad, but sometimes may actually be the best strategy.
- ^
How was the 40% figure calculated?
The funds we’re donating to are aware of AI’s impact on the world and seem likely to take steps to use AI to improve their outcomes
There is definitely some correlation between current competence and ability to adapt to a changing world, though I suspect that there’s also a huge amount of stickiness that undermines this.
All that said, if you think there should be a higher discount, you can apply it, but that doesn’t substantially change the results regarding what goes to AW. Indeed, if you increase the discount to 90%, you don’t get much more to AI (and less overall to GCRs as a whole, likely because the discount also applies to biorisk and nuclear weapons)
Fascinating. Is biorisk depreciating because some of those projects have a long payoff time?
My intuition is that biorisk talent will be more easily redeployable in a world undergoing an AI transition than a lot of the animal talent (more sticky).
I haven’t dived into the figures/calculation, but I’m pretty skeptical about donating 46% to regular animal welfare projects given AI progress. Is the AI discount just based on x-risk or does it account for the possibility of AI radically transforming the world in such a way that the impact of current charities becomes moot?
A few thoughts:
a) “Rejected or never heard back at the first stage and left confused”—if someone clearly isn’t good cultural fit, then identifying this at the first stage is the ideal outcome as it saves the time of both parties.
b) “Rejected at the final stage having wasted lots of time without useful feedback”—unfortunately, if someone is a bad cultural fit, then it’s hard to provide feedback without coming off as rude. Additionally, it’s hard to provide any feedback on this at all without making it easier for candidates to hide their lack of cultural fit.
c) “Many of the “unicorns” fall into one of two buckets with very few of the people I’ve spoken to falling in the middle. Many also swap between buckets without consistency”—Honestly, I’d take wrapping between the buckets without consistency as a red flag rather than a sign that the selectors were wrong. After all, in this case they seem to have failed both the immediate gut check and the more considered reflection of people who wanted to be open minded and let them advanced.
d) “It also seems like many orgs want ops people (even those on the hard ops side) to have excellent AIS knowledge and fluency, even though this is not required for the role”—I think there’s a lot of benefits of having a unified organisational culture so you’re all rowing in the same direction. If you hire someone who is only suited for a particular role, then you have less flexibilty in the future to promote them into other roles. You’re also providing them with an implicit endorsement which may lead to them being hired into roles they’d be unsuited for in the future by other folks.
e) “A few hiring managers I’ve spoken to are very wary of hiring senior folks who haven’t come from EA/ rat circles citing individual bad experiences with such people. This feels pretty uncharitable given how many bad students there are compared to the few good students who get hired”—Is it charitable to dismiss the accounts of folks who wanted to be open minded and tried hiring such folks and then learned lessons from it? I’m not saying these hires can’t work, just that the reasons for why they may not work are very subtle and I’d be much more enthuastic about your project if you were better able to steelman the opposing perspective (see also Chesterton’s Fence).
f) “While it is true a bad senior hire can be more expensive than a bad junior hire, we can get far more information about the track record of senior people from their experience.”—that doesn’t really help if the issue is that the candidate is highly competent, but a poor cultural fit.
g) “Frustratingly, lots of these candidates do well/ are silver medalists in hiring rounds and are never passed on anywhere else”—If a candidate was just not quite good enough to land the role, you might want to pass them on. On the other hand, if a candidate is a poor culture fit, you definitely wouldn’t want to pass them on.
h) “I would push these people to consider whether increased legibility of AI Safety due to recent warning shots and political salience could unlock much faster transitions that we aren’t capitalising on”—warning shots likely helps because it reduces the epistemic distance between the AIS community and the rest of the world, but I’m unsure why you think political salience would help rather than hurt. I tend to agree with Eliezer that Politics is the Mindkiller.
i) “Most of these people have done several BlueDot courses”—I’ve also facilitated for BlueDot. These courses do a lot of things well, but we shouldn’t be expecting these courses to be fixing the problem of cultural fit. Cultural fit is the kind of thing that is hard to address with an online format. I expect BlueDot’s Context Week is much more likely to be the right format for this. I also agree that local hubs are a good way of supporting this (I’m trying to do what I can at the Sydney AI Safety Space).
i) Despite everything I’ve said, seeing how many rounds closed without a hire suggests that we desperately need more talent and so I really do hope your project goes well, even if I think the challenge is much harder than you anticipate. Feel free to reach out if I can assist in any way.