Hi, I’m Max :)
looking for work in AI governance (general strategy, expert surveys, research infrastructure, EU tech policy fellow)
background in cognitive science & biology (did research on metacognition)
most worried about AI going badly for technical & coordination reasons
vegan for the animals
doing my own forecasts: https://www.metaculus.com/accounts/profile/110500/
I just watched Cover-Up, a Netflix documentary about Seymour Hersh, the journalist who later received the Pulitzer for his reporting on the My Lai massacre. Came to the forum with the hunch that you might’ve written about him as well, as I’m sure he’d deserve it.
We only briefly crossed paths twice (I think), you helped me find accomodation when I kinda stranded at the EAGx Oxford 2016 (nostalgia group pic). And then we ran into each other on a certain island located within the Lucayan Archipelago in the Atlantic Ocean. Still getting warm fuzzies when seeing your name, glad you left that hole.
This also seems to be true for Germany. From the respective German Doctors Without Borders page:
Beim Verkauf von Wertpapieren fällt in der Regel Kapitalertragsteuer (25% zzgl. Solidaritätszuschlag) an. Als steuerbegünstigt anerkannte gemeinnützige Organisation sind wir von der Körperschaftsteuer und dem Solidaritätszuschlag befreit. Den von uns erzielten Veräußerungserlös Ihrer Wertpapiere können wir ohne Abzug von Körperschaftsteuer und Solidaritätszuschlag in unseren Projekten einsetzen. Somit sparen Sie sich nicht nur den Aufwand eines Verkaufs, es fließt auch mehr Geld in unsere Hilfsprojekte.
@Sebastian Schwiecker Does Effektiv Spenden support donating stocks? Haven’t seen it on the website.
It’s great that you already have a rationale prompt for each question. I would probably recommend having one prompt like this at the end, with “(Optional)” in front so experts can share all further thoughts they think might be useful.
Fwiw, I also think the name is a bit complicated, and less memorable than Open Philanthropy. Here the reasoning from the Vox interview:
> Why “coefficient”? As CEO, Alexander Berger, puts it in my conversation with him, “coefficient is a multiplier”: the “co-” nods to collaboration with other givers; the “efficient” is a reminder of the north star of effectiveness.
Huge fan of your work, one of the few newsletters I read every week.
Random question, I wonder whether prediction markets are a potentially promising income stream for the team? E.g. Polymarket seems to have a bunch of overlap with the topics you’re covering.
Also, thanks for making your news-parsing code open source, was often curious how it looks like under the hood.
Hi Connacher! Thanks for the responses, makes sense.
On your question, one example I often miss from expert surveys is something like this open-ended question: “Do you have any other considerations that would help with understanding this topic?”
I generally agree that quantitative questions are intimately connected with identifying cruxes. Being quantitative about concrete events is a neat way of forcing the experts get more concrete and incentivize them to not get lost in a vague story, etc. But I suspect that often the individual insights from the experts might seem like cruxes to them, as they’re not used to think like that. So I think giving experts some prompts to just pour out their thoughts is often neglected. Furthermore, sometimes quantitative questions don’t fully capture all important angles of an issue and so it’s useful to give responders many chances to add additional comments.
Thanks for the work, this is great!
I especially appreciate the rationale summaries, and generally I’d encourage you to lean more into identifying underlying cruxes as opposed to quantitative estimates. (E.g. I’m skeptical on experts being sufficiently well calibrated to give particularly informative timeline forecasts).
I’m looking forward to the risk-related surveys. Would be interesting to hear their thoughts on the likelihood of concrete risks. One idea that comes to mind would be conditional forecasts on specific interventions to reduce risks.
Also, I wonder whether the presentation on the website could also feature some more “snack-sized” insights, next to the more long-form report-focussed presentation. E.g. the Chicago Booth expert surveys on economic experts focus on ~2-3 questions per article, with a handful of rationales of experts quoted in full. It keeps me coming back because it’s informative and takes up less than 5 minutes of my time.
https://kentclarkcenter.org/us-economic-experts-panel/
PS: Just in case something went wrong:
The wave 1 report questions imply there would be a section “Cognitive Limitations I” that is not included in the wave 1 report
Same for “Adoption Barriers I” that is not in the wave 2 report.
Thanks for the interesting interview!
Fwiw, this section made me feel like it should be thought through more deeply:
Luisa Rodriguez: [...] How confident are you that leaders in the countries that are set up to race and are already racing a little bit are going to see this as close to existential?
Daniel Kokotajlo: I think it will be existential if one side is racing and the other side isn’t. And even if they don’t see that yet, by the time they have superintelligences, then they will see it — because the superintelligences, being superintelligent, will be able to correctly identify this strategic consideration, and probably communicate that to the humans around them.
Luisa Rodriguez: Right. Once you have AGI, the AGI is like, “This is existential. We should do this big wartime effort to create a robot economy that’s going to give us this big advantage.”
Daniel Kokotajlo: That’s right. [...]
Some random thoughts:
Leaders already abstractly believe that AI is existentially important (e.g. Putin since 2017 apparently) and don’t really behave accordingly.
I think it’s very likely that there are already advisors in China and the U.S. that frequently argue for the existential importance of AI / ASI to the leader or people close to them. It’s unclear to me that e.g. Trump would trust an ASI much more than personal advisors.
I think it’s unclear that an ASI, especially from the current paradigm, will advise to race, and probably will strongly urge for cooperation and at most limited racing, e.g. defensive acceleration.
(Just quick random thoughts.)
The more that Trump is perceived as a liability for the party, the more likely they would go along with an impeachment after a scandal.
Reach out to Republicans in your state about your unhappiness about the recent behavior of the Trump administration.
Financially support investigative reporting on the Trump administration.
Go to protests?
Comment on Twitter? On Truth Social?
It’s possibly underrated to write concise and common sense pushback in the Republican Twitter sphere?
I relate hard with the career struggles, thanks for sharing! :’) Also very sweet (and again relatable) to drop everything for true love. :3
Thanks for writing this up, I think it’s a really useful benchmark for tracking AI capabilities.
One minor feedback point, I feel like instead of reporting on statistical significance in the summary, I’d report on effect sizes, or maybe even better just put the discrimination plots in the summary as they give a very concrete and striking sense of the difference in performance. Statistical significance is affected by how many datapoints you have, which makes lack of a difference especially hard to interpret in terms of how real-world significant the difference is.
Most of my donations were forgone payments for hiring rounds at organizations I consider among the most promising at reducing risks from AI (e.g. Horizon and MIRI).
Thanks for sharing, I really appreciate your committment, and that you announce it.
Fwiw, my immediate reaction is that this type of protest might be a little too soon and it will cause more ridicule and backlash because the general public’s and newsmedia’s impression is that there is currently no immediate danger. Would be interested in learning more about the timing considerations. Like, I’d imagine that doing this barricading in the aftermath of some concrete harm happening would make favorable reporting for newsmedia much more likely, and then you could steer the discourse towards future and greater harms.
Cool, thanks for sharing!
We can sponsor US visas for technical roles
Does this apply to any of the roles you list here?
I love the new profiles, and also all the new formatting options for comments, plus the filtering for private notes. Thanks so much! :)
Thanks so much for all your contributions Lizka! :) I really appreciated your presence on the forum, like a friendly, alive, and thoughtful soul that was attending to and helping grow this part of our ecosystem.
I can relate to the part about how unthankful it can be to be a mediator… it’s a pretty interesting dynamic where something really useful is being disincentivized, would be interested in hearing more of your, or others’ thoughts about it.
And I feel sorry that your work on moderation had negative effects on your interpersonal relationships. :| I don’t know how that exactly looked like for you, but could also imagine an exploration into dynamics here might be pretty interested and potentially helpful to understand more widely.
The part about people being confused about CEA’s role and activities, and the autonomy of the invidual teams, made me think that it might make sense to make the teams more prominent in comparison to CEA overall?
Like, giving teams more prominent names and brands (and cool logos!), emphasizing the teams more as the relevant entity that did something as opposed to CEA?
Thanks for doing this work, this seems like a particularly useful benchmark to track the world model of AI systems.
I found it pretty interesting to read the prompts you use, which are quite extensive and give a lot of useful structure to the reasoning. I was surprised to see in table 16 that the zero-shot prompts had almost the same performance level. The prompting kinda introduces a bunch of variance I imagine, and I wonder whether I should expect scaffolding (like https://futuresearch.ai/ are presumable focussing on) to cause significant improvements.
Thanks, that all makes sense and moderates my optimism a bit, and it feels like we roughly exhausted the depth of my thinking. Sigh… anyways, I’m really thankful and maybe also optimistic for the work that dedicated and strategically thinking people like you have been and will be doing for animals.
I’m reading “goes well for humans” as including “goes well for human values broadly, accounting for further refinement of human values”.