Hi, I’m Max :)
looking for work in AI governance (general strategy, expert surveys, research infrastructure, EU tech policy fellow)
background in cognitive science & biology (did research on metacognition)
most worried about AI going badly for technical & coordination reasons
vegan for the animals
doing my own forecasts: https://www.metaculus.com/accounts/profile/110500/
I love the new profiles, and also all the new formatting options for comments, plus the filtering for private notes. Thanks so much! :)
Thanks so much for all your contributions Lizka! :) I really appreciated your presence on the forum, like a friendly, alive, and thoughtful soul that was attending to and helping grow this part of our ecosystem.
I can relate to the part about how unthankful it can be to be a mediator… it’s a pretty interesting dynamic where something really useful is being disincentivized, would be interested in hearing more of your, or others’ thoughts about it.
And I feel sorry that your work on moderation had negative effects on your interpersonal relationships. :| I don’t know how that exactly looked like for you, but could also imagine an exploration into dynamics here might be pretty interested and potentially helpful to understand more widely.
The part about people being confused about CEA’s role and activities, and the autonomy of the invidual teams, made me think that it might make sense to make the teams more prominent in comparison to CEA overall?
Like, giving teams more prominent names and brands (and cool logos!), emphasizing the teams more as the relevant entity that did something as opposed to CEA?
Thanks for doing this work, this seems like a particularly useful benchmark to track the world model of AI systems.
I found it pretty interesting to read the prompts you use, which are quite extensive and give a lot of useful structure to the reasoning. I was surprised to see in table 16 that the zero-shot prompts had almost the same performance level. The prompting kinda introduces a bunch of variance I imagine, and I wonder whether I should expect scaffolding (like https://futuresearch.ai/ are presumable focussing on) to cause significant improvements.
Thanks, that all makes sense and moderates my optimism a bit, and it feels like we roughly exhausted the depth of my thinking. Sigh… anyways, I’m really thankful and maybe also optimistic for the work that dedicated and strategically thinking people like you have been and will be doing for animals.
That’s interesting, based on thinking that animal protein in the end comes from plant protein, and that animals use up a lot of space, food, and extra infrastructure that is not directly involved in turning plant protein into meat, I’d’ve guessed that plant protein would be much cheaper than animal protein.
I quickly asked chatGPT for the cheapest animal vs. plant proteins in the US:
Chicken: Approximately 6.6 cents per gram of protein
Lentils: Approximately 3.7 cents per gram of protein
Less difference than I’d’ve guessed.
Interesting, hard for me to judge. Reading the Innovations needed section, it seems like most hurdles are in the 3 OOM range, only the growth factor price is 6 OOM off.
My naive reaction is: AI + increased wealth + generally improving science & engineering + increased caring + those are “just” engineering problems → I’m much more bullish than the authors of the report, who are 9% for:
>50M metric tons of cellular meat will be sold at any price within a continuous 12-month span before the end of 2051. 9%
(For context, the annual production of conventional meat (excluding seafood) in 2018 was 346M metric tons)
I’m maybe at 40%, given that plant-based meat might just do it by itself, or that something disruptive happens that affects R&D broadly, or that it’s just unintuitively difficult.
I’m more hopeful this will be more comparable to civil rights / racism / sexism, as there are concrete victims who are suffering (and there is already a broad agreement that animals in human guardianship shouldn’t suffer), compared to climate change, which is much more abstract and indirect.
Yeah, I somewhat agree that the steady increase will probably bottom out at maybe 20%. But my hopeful vision is that at 20%, there will be critical mass effects for political action and for the demand of alternative products to lead to a much more mature industry.
Plus I expect health and climate change angles on meat consumption will also more likely than not steadily increase.
Finally, I’m also probably more optimistic about your last point, thinking that price/taste competitive meat alternatives will be huge. I think the Beyond and Impossible “moments” were huge milestones, and a few more “moments” like that will reduce resistance against higher welfare standards & higher prices for conventional meat.
Really interesting, thanks for sharing. I was particularly surprised about your changes of mind here:
We can make meaningful progress on abolishing factory farming or improving farmed animal welfare by 2050 75% 10% -65% We can make meaningful progress on abolishing factory farming or improving farmed animal welfare by 2100 85% 15% -70%
E.g. some spontaneous potential cruxes that might be interesting to hear your disagreement with, in case they capture your reasons for pessimism:
Plant protein sources will become price- and taste-competitive with more than half of all consumed meat products by 2040.
Cultured meat will become price- and taste-competitive with more than half of all consumed meat by 2050.
There is a critical mass of people caring about animal welfare at which animal rights will become major political issues in most democracies.
There will be a steady increase in people caring about animal welfare in the coming decade.
E.g. I expect Germany to have a >20% vegetarian population by 2040 (currently apparently 12%).
Thanks for this! :) I unfortunately only had time for skimming but I found the summary of pathways super useful and I had a positive impression of the level of rigor that went into this. Also appreciate the section on downside risks and how to address them.
Thanks so much for sharing your writing, it resonated deeply with me and made me cry more than once.
I feel incapable of being heard correctly at this point, so I guess it was a mistake to speak up at all and I’m going to stop now.
Noooo, sorry you feel that way. T_T I think you sharing your thinking here is really helpful for the broader EA and good-doer field, and I think it’s an unfortunate pattern that online communications quickly feels (or even is) somewhat exhausting and combative.
Just an idea, maybe you would have a much better time doing an interview with e.g. Spencer Greenberg on his Clearer Thinking podcast, or Robert Wiblin on the 80,000 Hours podcast? I feel like they are pretty good interviewers who can ask good questions that make for accurate and informative interviews.
(Just want to say, I really appreciate you sharing your thoughts and being so candid, Dustin. I find it very interesting and insightful to learn more about your perspective.)
Would it be possible to set up a fund that pays people for the damages they incurred for a lawsuit where they end up being innocent? That way the EA community could make it less risky for those who haven’t spoken up, and also signal how valuable their information is to them.
Meal replacement companies were there for us, through thick and slightly less thick.
Some commentary from Zvi that I found interesting, including pointers to some other people’s reactions:
https://thezvi.substack.com/p/ai-55-keep-clauding-along#§a-failed-attempt-at-adversarial-collaboration
Just in case someone interested in this has not done so yet, I think Zvi‘s post about it was worth reading.
https://thezvi.substack.com/p/openai-the-board-expands
Thanks for your work on this, super interesting!
Based on just quickly skimming, this part seems most interesting to me and I feel like discounting the bottom-line of the sceptics due to their points seeming relatively unconvincing to me (either unconvincing on the object level, or because I suspect that the sceptics haven’t thought deeply enough about the argument to evaluate how strong it is):
We asked participants when AI will displace humans as the primary force that determines what happens in the future. The concerned group’s median date is 2045 and the skeptic group’s median date is 2450—405 years later.
[Reasons of the ~400 year discrepancy:]
● There may still be a “long tail” of highly important tasks that require humans, similarto what has happened with self-driving cars. So, even if AI can do >95% of humancognitive tasks, many important tasks will remain.
● Consistent with Moravec’s paradox, even if AI has advanced cognitive abilities it willlikely take longer for it to develop advanced physical capabilities. And the latter wouldbe important for accumulating power over resources in the physical world.
● AI may run out of relevant training data to be fully competitive with humans in alldomains. In follow-up interviews, two skeptics mentioned that they would updatetheir views on AI progress if AI were able to train on sensory data in ways similar tohumans. They expected that gains from reading text would be limited.
● Even if powerful AI is developed, it is possible that it will not be deployed widely,because it is not cost-effective, because of societal decision-making, or for other reasons.
And, when it comes to outcomes from AI, skeptics tended to put more weight on possibilities such as
● AI remains more “tool”-like than “agent”-like, and therefore is more similar totechnology like the internet in terms of its effects on the world.
● AI is agent-like but it leads to largely positive outcomes for humanity because it isadequately controlled by human systems or other AIs, or it is aligned with humanvalues.
● AI and humans co-evolve and gradually merge in a way that does not cleanly fit theresolution criteria of our forecasting questions.
● AI leads to a major collapse of human civilization (through large-scale death events,wars, or economic disasters) but humanity recovers and then either controls or doesnot develop AI.
● Powerful AI is developed but is not widely deployed, because of coordinated humandecisions, prohibitive costs to deployment, or some other reason
I agree that things like confirmation bias and myside bias are huge drivers impeding “societal sanity”. And I also agree that it won’t help a lot here to develop tools to refine probabilities slightly more.
That said, I think there is a huge crowd of reasonably sane people who have never interacted with the idea of quantified forecasting as a useful epistemic practice and a potential ideal to thrive towards when talking about important future developments. Like other commentators say, it’s currently mostly attracting a niche of people who thrive for higher epistemic ideals, who try to contribute to better forecasts on important topics, etc. I currently feel like it’s not intractable for quantitative forecasts to become more common in epistemic spaces filled with reasonable enough people (e.g. journalism, politics, academia). Kinda similar to how tracking KPIs where probably once a niche new practice and are now standard practice.
Thanks, I think that’s a good question. Some (overlapping) reasons that come to mind that I give some credence to:
a) relevant markets are simply making an error in neglecting quantified forecasts
e.g. COVID was an example where I remember some EA adjacent people making money because investors were underrating the pandemic potential signifiantly
I personally find it plausible when looking e.g. at the quality of think tank reports which seems significantly curtailed due to the amount of vague propositions that would be much more useful if more concrete and quantified
b) relevant players train the relevant skills sufficiently well into their employees themselves (e.g. that’s my fairly uninformed impression from what Jane Street is doing, and maybe also Bridgewater?)
c) quantified forecasts are so uncommon that it still feels unnatural to most people to communicate them, and it feels cumbersome to be nailed down on giving a number if you are not practiced in it
d) forecasting is a nerdy practice, and those practices need bigger wins to be adopted (e.g. maybe similar to learning programming/math/statistics, working with the internet, etc.)
e) maybe more systematically I’m thinking that it’s often not in the interest of entrenched powers to have forecasters call bs on whatever they’re doing.
in corporate hierarchies people in power prefer the existing credentialism, and oppose new dimensions of competition
in other arenas there seems to be a constant risk of forecasters raining on your parade
f) maybe previous forecast-like practices (“futures studies”, “scenario planning”) maybe didn’t yield many benefits and made companies unexited about similar practices (I personally have a vague sense of not being impressed by things I’ve seen associated with these words)
I don’t think there’s actually a risk of CAISID damaging their EA networks here, fwiw, and I don’t think CAISID wanted to include their friendships in this statement.
My sense is that most humans are generally worried about disagreeing with what they perceive to be a social group’s opinion, so I spontaneously don’t think there’s much specific to EA to explain here.
I‘m really excited about more thinking and grant-making going into forecasting!
Regarding the comments critical of forecasting as a good investment of resources from a world-improving perspective, here some of my quick thoughts:
Systematic meritocratic forecasting has a track record of outperforming domain experts on important questions - Examples: Geopolitics (see Superforecasting), public health (see COVID), IIRC also outcomes of research studies
In all important domains where humans try to affect things, they are implicitly forecasting all the time and act on those forecasts. Random examples: - “If lab-grown meat becomes cheaper than normal meat, XY% of consumers will switch” - “A marginal supply of 10,000 bednets will decrease malaria infections by XY%” - Models of climate change projections conditional on emmissions
In many domains humans are already explicitly forecasting and acting on those forecasts - Insurance (e.g. forecasts on loan payments) - Finance (e.g. on interest rate changes) - Recidivism - Weather - Climate
Increases in use of forecasting has the potential to increase societal sanity - Make people more able to appreciate and process uncertainty in important domains - Clearer communication (e.g. less talking past one another by anchoring discussion on real world outcomes) - Establish feedback loops with resolvable forecasts ➔ stronger incentives for being correct & ability to select people who have better world models
That said, I also think that it’s often surprisingly difficult to ask actionable questions when forecasting, and often it might be more important to just have a small team of empowered people with expert knowledge combined with closely coupled OODA loops instead. I remember finding this comment from Jan Kulveit pretty informative:
In practice I’m a bit skeptical that a forecasting mindset is that good for generating ideas about “what actions to take”. “Successful planning and strategy” is often something like “making a chain of low-probability events happen”, which seems distinct, or even at tension with typical forecasting reasoning. Also, empirically, my impression is that forecasting skills can be broadly decomposed into two parts—building good models / aggregates of other peoples models, and converting those models into numbers. For most people, the “improving at converting non-numerical information into numbers” part has initially much better marginal returns (e.g. just do calibration trainings...), but I suspect doesn’t do that much for the “model feedback”.
Cool, thanks for sharing!
Does this apply to any of the roles you list here?