I work primarily on AI Alignment. My main direction at the moment is to accelerate alignment work via language models and interpretability.
jacquesthibs
I’d be interested in this if I moved to NYC. I’m currently at the very early beginnings of preparing for interviews and I’m not sure where I’ll land yet so I won’t answer the survey. Definitely a great idea, though. The decently-sized EA community in NYC is one of the reasons it’s my top choice for a place to move to.
Who am I? Until recently, I worked as a data scientist in the NLP space. I’m currently preparing for a new role, but unsure if I want to:
Work as a machine learning engineer for a few years then transition to alignment, founding a startup/org or continue working as ML engineer.
Or, try to get a role as close to alignment as possible.
When I first approached Yonatan, I told him that my goal was to become “world-class in ml within 3 years” in order to make option 1 work. My plan involved improving my software engineering skills since it was something I felt I was lacking. I told him my plan on how to improve my skills and he basically told me I was going about it all wrong. In the end, he said I should seek mentorship with someone who has the incentive to help me improve my programming skills (via weekly code reviews) ASAP. I had subconsciously avoided this approach because my experiences with mentorship were less than stellar. I took a role with the promise that I would be mentored and, in the end, I was the one doing all the mentoring...
Anyway, after a few conversations with Yonatan, it became clear that seeking mentorship would be at least 10X more effective than my initial plan.
Besides helping me change my approach to becoming a better programmer (and everything else in general), our chats have allowed me to change my career approach in a better direction. Yonatan is good at helping you avoid spouting vague, bad arguments for why you want to do x.
I’m still in the middle of the job search process so I will update this comment in a few months once the dust has settled. For now, I need to go, things have changed recently and I need to get in touch with Yonatan for feedback. :)
I highly recommend this service. It is lightyears ahead of a lot of other “advice” I’ve found online.
Avast is telling me that the following link is malicious:
Ding’s China’s Growing Influence over the Rules of the Digital Road describes China’s approach to influencing technology standards, and suggests some policies the US might adopt. #Policy
If anything, this is a claim that people have been bringing up on Twitter recently, the parallels between EA and religion. It’s certainly something we should be aware of since, having ”blind faith” in religion is something that be good, we don’t seem to actually want to do this within EA. I could explain why I think AI risk is different from messiah thing, but Rob Miles explains it well here:
Given limited information (but information nonetheless), I think AI risk could potentially lead to serious harm or not at all, and it’s worth hedging our bets on this cause area (among others). This feels different then choosing to have blind faith in a religion, but I can see why outsiders think this. Though we can be victims of post-rationalization, I think religious folks have reasons to believe in a religion. I think some people might gravitate towards AI risk as a way to feel more meaning in their lives (or something like that), but my impression is that this is not the norm.
At least in my case, it’s like, “damn we have so many serious problems in the world and I want to help with them all, but I can’t. So, I’ll focus on areas of personal fit and hedge my bets even though I’m not so sure about this AI thing and donate what I can to these other serious issues.”
Fantastic work. And thank you for transcribing!
Saving for potential future use. Thanks!
One thing that may backfire with the slow rollout of talking to journalists is that people who mean to write about EA in bad faith will be the ones at the top of the search results. If you search something like “ea longtermism”, you might find bad faith articles many of us are familiar with. I’m concerned we are setting ourselves up to give people unaware of EA a very bad faith introduction.
Note: when I say “bad faith“ here, it may just be a matter of semantics with how some people are seeing it as. I think I might not have the vocabulary to articulate what I mean by “bad faith.” I actually agree with pretty much everything David has said in response to this comment.
Great points, here’s my impression:
Meta-point: I am not suggesting we do anything about this or that we start insulting people and losing our temper (my comment is not intended to be prescriptive). That would be bad and it is not the culture I want within EA. I do think it is, in general, the right call to avoid fanning the flames. However, my first comment is meant to point at something that is already happening: many people uninformed about EA are not being introduced in a fair and balanced way, and first impressions matter. And lastly, I did not mean to imply that Torres’ stuff was the worse we can expect. I am still reading Torres’ stuff with an open-mind to take away the good criticism (while keeping the entire context in consideration).
Regarding the articles: Their way of writing is by telling the general story in a way that it’s obvious they know a lot about EA and had been involved in the past, but then they bends the truth as much as possible so that the reader leaves with a misrepresentation of EA and what EAs really believe and take action on. Since this is a pattern in their writings, it’s hard not to believe they might be doing this because it gives them plausible deniability since what they’re saying is often not “wrong”, but it is bent to the point that the reader ends up inferring things that are false.
To me, in the case of their latest article, you could leave with the impression that Bostrom and MacAskill (as well as the entirety of EA) both think that the whole world should stop spending any money towards philanthropy that helps anyone in the present (and if you do, only to those who are privileged). The uninformed reader can leave with the impression that EA doesn’t even actually care about human lives. The way they write gives them credibility to the uninformed because it’s not just an all-out attack where it is obvious to the reader what they’re intentions are.
Whatever you want to call it, this does not seem good faith to me. I welcome criticism of EA and longtermism, but this is not criticism.
*This is a response to both of your comments.
It’s a good project because, you know, doing good is important and we should want to do good better rather than worse. It’s utterly absurd because everyone who has ever wanted to do good has wanted to do good well, and acting as though you and your friends alone are the first to hit upon the idea of trying to do it is the kind of galactic hubris that only subcultures that have metastasized on the internet can really achieve.
This seems wrong to me. Just this week, I went on a date with someone who told me the only reason she volunteers is that it makes her feel good about herself, and she doesn’t particularly care much about the impact. And you know what, props to her for admitting something that I expect a lot of other people do as well. I don’t think there’s something wrong with it, I’m just saying that “everyone who has ever wanted to do good has wanted to do good well” seems wrong to me.
I had 4 coaching calls with her for free after 80,000 Hours directed me to her.
The following tweet is being shared now: https://twitter.com/autismcapital/status/1590551673721991168?s=46&t=q60fxwumlq0Mq8CpGV3bxQ
This is obviously just a random unverified source, but I think it will be worth reflecting on this deeply once this is all said and done. It feeds directly into how EA’s maximizing behaviour can lead to these outcomes. Whether the above is true or not, it will certainly be painted as such by those who have been critical of EA.
One worry one might have is the following reaction: “I don’t need mental health help, I need my money back! You con artists have ruined my life and now want to give me a pat on the back and tell me it’s going to be ok?”
Then again, I do want us to do something if it makes sense. :(
Personally, I’ve mostly seen people confused and trying to demonstrate willingness to re-evaluate what might have led to these bad outcomes. They may overly sway in one direction, but this only just happened and they are re-assessing their worldview in real-time. Some are just asking questions about how decisions were made in the past so we just have more information and can improve things going forward (which might mean doing nothing differently in some instances). My impression is that a lot of the criticism about EA leadership are overblown and most (if not all) were blindsided.
That said, I haven’t really had the impression it’s as bad and widespread as this post makes it seem though. Maybe I just haven’t read the same posts/comments and tweets.
I do think that working together so we can land on our feet and continue to help those in need sounds nice and hope you’ll still be there since critical posts like this are obviously needed.
Honestly, I’m happy with this compromise. I want to hear more about what ‘leadership’ is thinking, but I also understand the constraints you all have.
This obviously doesn’t answer the questions people have, but at least communicating this instead of radio silence is very much appreciated. For me at least, it feels like it helps reduce feelings of disconnectedness and makes the situation a little less frustrating.
I feel like anyone reaching out to Elon could say “making it better for the world” because that’s exactly what would resonate with Elon. It’s probably what I’d say to get someone on my side and communicate I want to help them change the direction of Twitter and “make it better.”
I think the information you are sharing is useful (some parts less so, I agree with pseudonym), just don’t deadname/misgender them. EA is better than that.
Here’s a comment I wrote on LessWrong in order to provide some clarification:
———
So, my difficulty is that my experience in government and my experience in EA-adjacent spaces has totally confused my understanding of the jargon. I’ll try to clarify:
In the context of my government experience, forecasting is explicitly trying to predict what will happen based on past data. It does not fully account for fundamental assumptions that might break due to advances in a field, changes in geopolitics, etc. Forecasts are typically used to inform one decision. It does not focus on being robust across potential futures or try to identify opportunities we can take to change the future.
In EA / AGI Risk, it seems that people are using “forecasting” to mean something somewhat like foresight, but not really? Like, if you go on Metaculus, they are making long-term forecasts in a superforecaster-mindset, but are perhaps expecting their long-term forecasts are as good as the short-term forecasts. I don’t mean to sound harsh, it’s useful what they are doing and can still feed into a robust plan for different scenarios. However, I’d say what is mentioned in reports typically does lean a bit more into (what I’d consider) foresight territory sometimes.
My hope: instead of only using “forecasts/foresight” to figure out when AGI will happen, we use it to identify risks for the community, potential yellow/red light signals, and golden opportunities where we can effectively implement policies/regulations. In my opinion, using a “strategic foresight” approach enables us to be a lot more prepared for different scenarios (and might even have identified a risk like SBF much sooner).
My understanding of forecasting is that you would optimally want to predict a distribution of outcomes, i.e. the cone but weighted with probabilities. This seems strictly better than predicting the cone without probabilities since probabilities allow you to prioritize between scenarios.
Yes, in the end, we still need to prioritize based on the plausibility of a scenario.
I understand some of the problems you describe, e.g. that people might be missing parts of the distribution when they make predictions and they should spread them wider but I think you can describe these problems entirely within the forecasting language and there is no need to introduce a new term.
Yeah, I care much less about the term/jargon than the approach. In other words, what I’m hoping to see more of is to come up with a set of scenarios and forecasting across the cone of plausibility (weighted by probability, impact, etc) so that we can create a robust plan and identify opportunities that improve our odds of success.
“DM me if you’re interested in dating me”
Before EAGSF this year, (on Twitter) I mentioned putting this on your SwapCard profile as a way to prevent the scenarios above where people ask others for meetings because they are romantically interested in them. So, instead, they could contact them off-site if interested and EAGs would hopefully have more people just focused on going to it for career reasons. My thought was that if you don’t do something like this, people are just going to continue hiding their intentions (though I’m sure some would still do this regardless).
I was criticized for saying this. Some people said they have an uncomfortable feeling after hearing that suggestion because they now have it in their minds that you might be doing a 1-on-1 with them because you find them attractive. Fair enough! Even if you, let’s say, link to a dating doc off-site or contact info that they can reach after the conference. I hoped that we could make it more explicit the fact that people in the community are obviously looking to date others in the community and are finding that very difficult. Instead, my guess is that we are placed in a situation where people will set-up 1-on-1s because they find someone attractive even if they don’t admit it. I do not condone this, and it’s not something I’ve done (for all the reasons listed in this thread).
Personally, I do not plan to ask anyone out from the community at any point. Initially, I had hoped to find someone with similar values, but I just don’t think there is any place it seems appropriate. Not even parties. It’s just not worth the effort to figure out how to ask out an EA lady in a way that’s considered acceptable. This might sound extreme to some, but I just don’t find it worth the mental energy to navigate my way through this and just want to be in career-mode (and, at most, friendship-mode) when engaging with other EAs. And, more importantly, there’s too much work and fun mixed, and it just leads to uncomfortable situations and posts like this.
I’m not making a judgement on what others should do, but hopefully whichever way the community goes, it becomes more welcoming for people who want to do good.
- 6 Dec 2022 15:39 UTC; 10 points) 's comment on I’m a 22-year-old woman involved in Effective Altruism. I’m sad, disappointed, and scared. by (
- 24 Dec 2022 21:30 UTC; 5 points) 's comment on I’m a 22-year-old woman involved in Effective Altruism. I’m sad, disappointed, and scared. by (
Thank you for writing this post. I’m currently a technical alignment researcher who spent 4 years in government doing various roles, and my impression has been the same as yours regarding the current “strategy” for tackling x-risks. I talk about similar things (foresight) in my recent post. I’m hoping technical people and governance/strategy people can work together on this to identify risks and find golden opportunities for reducing risks.
I just want to say that this course curriculum is amazing and I really appreciate that you’ve made it public. I’ve already gone through about a dozen articles. I’m an ML engineer who wants to learn more about AGI safety, but it’s unfortunately not a priority for me at the moment. That said, I will still likely go through the curriculum on my own time, but since I’m focusing on more technical aspects of building ML models at the moment, I won’t be applying since I can’t strongly commit to the course. Anyways, again, I appreciate making the curriculum public. As I slowly go through it, I might send some questions for clarification along the way. I hope that’s ok. Thanks!