đ¶
Esben Kran
Karma: 848
ïAI safety needs to scale, and hereâs how you can do it
I work as co-director of Apart Research, specifically with research management, AI safety research consulting, and field-building. Iâm entrepreneurially focused.
Thank you for hosting this! Iâll repost a question on Asyaâs retrospective post regarding response times for the fund.
our median response time from January 2022 to April 2023 was 29 days, but our current mean (across all time) is 54 days (although the mean is very unstable)
I would love to hear more about the numbers and information here. For instance, how did the median and mean change over time? What does the global distribution look like? The disparity between the mean and median suggests there might be significant outliers; how are these outliers addressed? I assume many applications become desk rejects; do you have the median and mean for the acceptance response times?
Agency FounÂdaÂtions Challenge: SeptemÂber 8th-24th, $10k Prizes
I was incredibly impressed by the tables of numbers in their impact evaluation. After conversing with the team, Iâve witnessed their high ability to produce results, and their evaluation research methods certainly attest to this. This appears to be one of those rare opportunities where donations could have a significant counterfactual impact.
Edit: I am not in any way affiliated with FEM and randomly met one of the co-founders on a flight where we had a conversation about their work.
Thank you for sharing your reflections and for the work youâve done on the EA Funds, Asya! I appreciate the role the Funds have played over the past years.
our median response time from January 2022 to April 2023 was 29 days, but our current mean (across all time) is 54 days (although the mean is very unstable)
A few questions arise from your mention of the Fundsâ response times. I would love to hear more about the numbers and information here. For instance, how did the median and mean change over time? What does the global distribution look like? The disparity between the mean and median suggests there might be significant outliers; how are these outliers addressed? I assume many applications become desk rejects; do you have the median and mean for the acceptance response times?
The focus of FLI on lethal autonomous weapons systems (LAWS) generally seems like a good and obvious framing for a concrete extinction scenario. Currently, a world war will without a doubt use semi-autonomous drones with the possibility of a near-extinction risk from nuclear weapons.
A similar war in 2050 seems very likely to use fully autonomous weapons under a development race, leading to bad deployment practices and developmental secrecy (without international treaties). With these types of âslaughterbotsâ, there is the chance of dysfunction (e.g. misalignment) leading to full eradication. Besides this, cyberwarfare between agentic AIs might lead to broad-scale structural damage and for that matter, the risk of nuclear war brought about through simple orders given to artificial superintelligences.
The main risks to come from the other scenarios mentioned in the replies here are related to the fact that we create something extremely powerful. The main problems arise from the same reasons that one mishap with a nuke or a car can be extremely damaging while one mishap (e.g. goal misalignment) with an even more powerful technology can lead to even more unbounded (to humanity) damage.
And then there are the differences between nuclear and AI technologies that make the probability of this happening significantly higher. See Yudkowskyâs list.
This a unique, interesting and simple proposal I have not seen presented in academic form yet. With the development of the article, youâll of course need to change the framing of a few sections to introduce the idea, the viability, along with the multi-purpose potential of the proposal.
Despite unlikely effective enforcement of the policy, it seems like a valuable idea to publish. Combining it with newer work in GPU monitoring firmware (Shavit, 2023) and your own proposals for required GPU server tracking.
To comment on kpurens comment, carbon taxation was a non-political issue before it became contentious and if the lobbying hadnât hit as hard, it seems like there would be a larger chance for a global carbon tax. At the same time, compute governance seems more enforceable because of the centralization of data centers.
ïJoin the AI govÂerÂnance and inÂterÂpretabilÂity hackathons!
AnÂnouncÂing the EuroÂpean NetÂwork for AI Safety (ENAIS)
The CE incubatees are an absolutely amazing bunch and exactly the types of people I would want on these world-bettering projects. Charity Entrepreneurship is no doubt one of the EA projects I am most excited about due to pure impact, research prowess and future potential.
When I compare the CE program to YC (see also OWID@YC), it feels even better due to the great co-founder matching process and the success rate along with the excellence within the focus areas (people donât come with their own esoteric tech startups).
For other commenters who talk about the use of reach numbers instead of e.g. QALY or WELLBYs, having seen some of their spreadsheets and programs, I am in deep awe and respect of some of the superheroes that have saved hundreds of lives (with many of these being counterfactual given the effectiveness and new vectors of impact) though Iâll let them summarize their numbers.
I am very excited about where the projects will be in one and even five years and commend everyone involved.
InÂtro to AlignÂment Jams
ïReÂsults from the AI testÂing hackathon
Very confused why people have downvoted here? Please explain why, if possible. And context: Iâve also started a private chat about the data with Peter and Ash.
Thereâs this project that a few people have been working on towards making an overview in Obsidian that seems to be in this vein as well (see the comment, too): aisi.ai/â?idea=75
ïJoin the AI TestÂing Hackathon this Friday
Wonderful to see an analysis of all this even more wonderful data coming out of the programs! Weâre also happy to share our survey results (privately and anonymized) with people interested:
Demographic, feedback, AIS attitudes before and after an Alignment Jam (for the first and second versions)
A 70 respondent in-depth survey on attitudes towards AI risk arguments (basically, âwhat converted youâ)
An AI safety pain points survey that was also done in-person with several top AIS researchers
An AI safety progress survey about P(doom)s etc.
We also have a few other surveys but the ones above are probably most interesting in the context of this post.
This is wonderful to hear! Every time Iâve talked with the Norwegian EA community, there has been a strong sense of integrity, ethics, and a will to do good along with tangible results. Iâm consistently impressed and itâs wonderful to see that effort get rewarded and for that impact to get a public voice.
Congratulations!
Regarding legalities, seeing that the grant is from April, I will also add that Molly mentions the general clawbacks are only relevant 3 months back into time from the dissolution of FTX organizations.
Thank you so much for the talk, Paul! It was exciting to see the vignettes besides the very practical first case. It will be interesting to see the entry of Straumli on the evaluations scene since I think you have a solid case for success.
CoI statement: Straumli donated the prize money for the Governance Sprint, though nothing goes to me or Apart, just the AI safety community.