Trying to figure out how EA can use AI better, both to uplift current orgs and to create new tools to positively reshape the world.
Iām really keen to meet people building things with AI, and people interested in making more of that happen within EA. If youāre any of those, please have a really low bar for emailing or booking a call with me.
What are some groups or communities outside mainstream EA that would be worth engaging in our efforts to reduce existential risks? What are some specific areas or projects we could work on together?
Hereās a few other questions that approach that from different angles:
* Whoās the audience for your new book? Within them, who would be most keen to help reduce x-risk, even if itās not through EA-affiliated orgs or projects?
* What points in common does EA have with the UK Progress community? What can we do on the margin to work more closely together on our shared priorities?
Well, AI sentience probably has <100 people and Iād be joyful if that increased to 1k. Iām still interested even if the question is āHow can we fail less misserably at creating a talent pipeline?ā
Yep, agree that linear averages might not be the best tool for that. The product of odds sounds interesting. I might give it a read and see if itās easy to keep track of in my mind
Oh, amazing! I didnāt knew Alex had a Substack. And indeed, itās full of advice for using LLMs for work.
Hereās a relevant link for those interested: https://āālawsen.substack.com/āāp/āālean-into-laziness
On the same genre thereās also these posts from Shakeel Hashim and Peter Hartree on how they use LLMs:
For anyone considering working on the ORCID-TAXID mapping tool, I suspect it might be an unusually approachable project for those with some familiarity with biological publications and programming. Even without what ORCID or TAXID stood for before reading this post, I managed to construct a barebones demo in 30 minutes using ChatGPT and the Europe PMC API (which has an option to search by ORCID ID, though some quick manual searches suggest it isnāt comprehensive). I think in under 30 hours you could build a decently useful product by adding features like:
Querying PubMed too alongside Europe PMC to get better publication coverage
Searching publications by author name (since PubMed doesnāt offer filtering by ORCID ID) and filtering out clear false positives using simple heuristics like author affiliation and publication area
Creating more fine-grained criteria for identifying organisms in publication abstracts and texts
I will message the post authors and offer to do this myself, but if you already have background in biological sciences and are looking for a cool upskilling project, you would probably be a better fit than me for this.
For those wanting a quick encapsulation of Nietzscheās morality, I recommend Arjunās other post on the topic. Itās both unusually succinct and well-written.
Oh, sorry. Iāll expand the abbreviation in the original comment. Itās āCommunity Building resourcesā.
Thanks for putting numbers to my argument! I was expecting a greater proportion of left-leaning individuals among the college educated, so this was a useful update.
A reason why the political orientation gap might be less worrying than it appears at first sight is that it probably stems partly from the overwhelmingly young bent of EA. Young people from many countries (and perhaps especially in the countries where EA has greater presence) tend to be more left leaning than the general population.
This might be another reason to try onboarding older people to EA more relative to the pool of new members, but if you thought that would involve significant costs (e.g. having less young talented EAs because less community building resources were directed towards that demographic), then perhaps in equilibrium we should have a somewhat skewed distribution of political orientations.
Thanks for taking the time to respond! I find your point of view more plausible now that I understand it a little bit better (though Iām still not sure of how convincing I find it overall).
Iām not sure if I understand where youāre coming from, but Iād be curious to know: do you think similarly of EAs who are Superforecasters or have a robust forecasting record?
In my mind, updating may as well be a ritual, but if itās a ritual that allows us to better track reality then thereās little to dislike about it. As an example of how precise numerical reasoning could help, the book Superforcasting describes how rounding Superforecasters predictions (interpreting .67 probability of X happens as .7 probability of X happening) increases the error of the prediction. The book also includes many other examples where I think numerical reasoning confers a sizable advantage to its user.
What stops AI Safety orgs from just hiring ML talent outside EA for their junior/āmore generic roles?
Oh, I totally agree that giving people the epistemics is mostly preferable to hanging them the bottom line. My doubts come more from my impression that forming good epistemics in a relatively unexplored environment (e.g. cause prioritization within Colombia) is probably harder than in other contexts.
I know that at least our explicit aim with the group was to exhibit the kind of patience and rigour you describe and that I ended up somewhat underwhelmed with the results. I initially wanted to try to parse out where our differing positions came from, but this comment eventually got a little long and rambling.
For now Iāll limit myself to thanking you for making what I think itās a good point.
Hi, thanks for the detailed reply. I mostly agree with the main thrust of your comment, but I think I feel less optimistic about what happens when you actually try to implement it.
Like, weāve had discussions in my group about how to prioritize cause areas and in principle everyone agrees with how we should work on causes that are bigger, more neglected and tractable, but when it comes to specific causes it turns out that the unmeasured effects are the most important thing and the flow through effects of the intervention Iāve always liked turn out to compensate for its lack of demonstrated impact.
Iām not saying itās impossible to have those discussions, just that for a group constrained on people whoāve engaged with EA cause priorization arguments, being able to rely on the arguments that others have put forward (like we can often do for international causes) makes the job much easier. However, Iām open to the possibility that that the best compromise might be to simply to let people focus on local causes and double down on cultivating better epistemics.
(P.s.: I now realize that answering to every comment on your post might be quite demanding, so feel free to not answer. Iāll make sure to still update on the fact that you werenāt convinced by my initial comment. If anything at least your comment is making me consider alternatives that donāt restrict the growth and avoid the epistemic consequences I delineated. Iām not sure if Iāll find something that pleases me, but Iāll muse on it a little further)
Hmm, itās funny, this post comes at a moment when Iām heavily considering moving in the opposite direction with my EA university group (towards being more selective and focused on EA-core cause areas). Iād like to know what you think of the reason I thought for doing so.
My main worry is that as the interests of EA members broaden (e.g. to include helping locally), the EA establishment will have less concrete recommendations to offer and people will not have a chance to truly internalize some core EA principles (e.g. amounts matter, doubt in the absence of measurement).
That has been an especially salient problem for my group, given that we live in a middle-income country (Colombia) and many people feel most excited about helping within our country. However, when Iāve heard them make plans for how they would help, I struggle to see what difference we made by presenting them EA ideas. They tend to choose causes more by their previous emotional connection than by attributes that suggest a better opportunity to help (e.g. by using the SNT framework). My expectations are that if we emphasize more the distinctive aspects of EA (and the concrete recommendations they imply), people will have a better chance to update on the ways that mainstream EA differs from what they already believed and we will have a better shot at producing some counterfactual impact.
(Though, as a caution, itās possible that the tendency for members of my group to not realize when EA ideas differ from their own may come from my particular aversion to openly questioning or contradicting people, rather than from the memberās interest in less-explored areas for helping.)
If I wanted to be charitable to their answer of the cost of saving a life Iād point out that $5000 is roughly the cost of saving a life reliably and at scale. If you relax any of those conditions, saving a life might be cheaper (e.g. Givewell sometimes finances opportunities more cost-effective than AMF, or perhaps youāre optimistic about some highly leveraged interventions like political advocacy). However, I wouldnāt bet that this phenomenon would be behind a significant fraction of the divergence of their answers.
Thanks for the post, Jan! I follow AI Alignment debates only superficially and I had heard of the continuity assumption as a big source of disagreement, but I didnāt have a clear concept of where it stemmed from and what were itās practical implications. I think your post does a very good job at grounding the concept and filling those gaps.
These are just the first questions that came to mind, but may not necessarily overlap with Adreasā interests or knowledge:
Given his deontological leanings, is there something he would like to see people in the EA community doing less/āmore of?
Whatās the paper/āline of investigation from GPI that has changed his view on practical priorities for EA the most?
How involved in philosophical discussions should the median EA be? (e.g. should we all read Parfit or just muddle through with what we hear from informal discussions of ethics within the community?)
Whatās the thrust of his argument in āAgainst Large Number Sceptismā? How would he characterize how people who feel uncomfortable with arguments resting on large numbers think about the subject?
Where does the interest in Decision Theroy among EAs come from? Is it because it could have practical implications or something else entirely? What would change if we had an answer to the top open questions in Decision Theory?
Thank you Shen, this is wonderful! With my local group in Colombia weāre getting ready to stage a fellowship for the second time and hearing about your experience gave me many ideas for things we may try to improve on.
I loved how both the title of the post and the first section foreshadow the wild animal welfare turn in the middle of the post. It was truly enjoyable read!