No
Peter Wildeford
Karma: 20,270
Policy career planning in the age of imminent superintelligence
There are two ways to interpret this claim.
One is to interpret this claim as causal—“the things that cause AGI to go well for humans also cause AGI to go well for animals”.
In general, my concern here is something like “AGI gets aligned primarily based on 2025-era human values by imitation learning and doesn’t magically converge on my ideal philosophy”. I think what happens to animals after that would be fairly contingent on human moral evolution.
Another is to interpret the claim as evidentiary—“what happens to animals conditional only on things going well for humans by any means?” In this sense, we likely do have ensuing transformative economic growth and rapid technological advancement, which likely shifts society away from factory farming. Especially as society becomes primarily digital and/or spacefaring, factory farming likely is not economically suited for that. Though I think this is far from guaranteed.
I’m hedging between both these interpretations, which is why I end up somewhere in the middle.
Wow, this situation is fucked up. You should never have had to go through all of this. I’m deeply disturbed by how this was handled.
Hi—thanks for this comment. As someone working on export control policy, let me give you my perspective.
Firstly, an important precondition for a cooperative pause is leverage. You don’t get China to agree to a mutual pause by first giving away your main strategic advantage. You get them to agree by making the alternative to be “a race they’re losing”, which is worse than cooperation. Export controls are thus part of what creates the conditions for being able to pause. If you equalize compute access first, China has no reason to agree to a pause because they’d be in a great position to race.
This is basic negotiation theory. You don’t just disarm and hand over your weapons before the arms control treaty; you disarm as part of the treaty.
More critically, export controls are already priced in. The US has maintained semiconductor restrictions on China since October 2022, tightening them in 2023 and again in 2024, before loosening them late last year. The core diplomatic costs of these controls have already been paid. Easing controls now doesn’t recoup that trust. China won’t say “oh great, all is forgiven”. But easing controls does give away the strategic advantage those controls purchased.
I’d also dispute that placing chip export controls on a state makes them a “pariah state”. Restricting dual-use technology exports to strategic competitors is completely normal behavior among non-pariah states. Similarly to how we might restrict F-35 technology, nuclear technology, satellite components, rocket launch components, etc., to many countries.
Hi! I’m a long time effective altruist (14+ years) and utilitarian/utilitarian-adjacent. This is a sweet and earnest post—you’re clearly bright and I admire your dedication at such a young age. You’re right to recognize that burnout isn’t utilitarian, but I worry that your “donate everything / camper van” framing is premature and probably wrong.
At age 14⁄15, the highest-EV move is almost always building optionality, not making binding commitments to extreme frugality. You’re right to maximize income, but I think you’re thinking too much about this in terms of “save money” and not enough in terms of “earn more money”. Your comparative advantage at 14 is completely unknown. You might become a researcher, policy person, entrepreneur, or earn-to-give—these all have very different optimal life strategies. And your ability to find and execute on the best path, and do that path well, will certainly lead to higher impact in the long-run than self-sacrificing immensely right now. I think moving into an apartment or a house is compatible with being a utilitarian. For example, Peter Singer lives in a house, as did a lot of other historical utilitarians.
Also… “Living alone would cause a huge debt of social interaction, causing me to die earlier” is… technically true but also a somewhat alienating way to think about relationships? Friends aren’t just longevity inputs. Be careful not to instrumentalize everything—that path leads to feeling disconnected and often to abandoning the whole framework.
Advice I’d give:
Read 80,000 Hours carefully—they’ve thought hard about sustainable high-impact careers.
Don’t plan the camper van. Plan to learn, explore, and keep options open. You can donate 20% of your income without giving up everything.
AP Calc will be fine if you’re doing well now.
My template for a quarterly review + plan
Hi. Thanks for writing this. I find electoral reform to be a genuinely interesting cause area, and I appreciate the effort to apply EA frameworks to it. I have a few concerns with the framing and some factual details:
On neglectedness: The claim that this is “the most neglected intervention in EA” doesn’t match the track record. The Center for Election Science has received over $2.4M from Open Philanthropy, $100K from EA Funds, and $40K+ from SFF. 80,000 Hours has a problem profile on voting reform calling it a “potential highest priority area” and did a full podcast episode with Aaron Hamlin. There’s also a dedicated EA Forum topic page, and another post appeared today advocating for CES funding. This doesn’t mean additional funding isn’t warranted, but framing this as EA’s “most neglected” area overstates the case.
On the theory of change: The post seems to conflate two distinct interventions: (1) running congressional candidates on electoral reform platforms to “prevent authoritarian consolidation” in 2026, and (2) actually implementing approval voting + top-four primaries long-term. The “$125M protects $400B” framing treats these as equivalent, but they’re quite different propositions. The 20% success probability is asserted without justification, and it’s unclear what “success” means… winning midterms? Passing state initiatives? Both?
I’d also find it helpful to understand the mechanism better: how does “50 congressional candidates running on electoral reform” prevent authoritarianism? Most swing voters care about bread-and-butter issues like affordability and healthcare—the claim that approval voting advocacy serves as an effective “organizing principle” would benefit from more evidence or argument.
Some factual corrections: The post advocates for approval voting but cites Alaska as evidence. Alaska uses ranked-choice voting, not approval voting. It’s also worth noting that the repeal measure in Alaska barely failed (50.1% to 49.9%) despite being outspent 100:1 - which cuts both ways on tractability. And while the post mentions Fargo’s approval voting was banned by the North Dakota legislature, this seems like an important cautionary tale about scaling that deserves more attention.
I appreciate you sharing but I’d just encourage tightening up the claims and being more precise about what evidence supports which interventions and what your suggested interventions are doing.
Correct!
Ronnie had a bit of a script but also improvised a lot. I had no script, no knowledge of Ronnie’s script, and was very blind to the whole thing. But we did many takes, so I was able to adapt accordingly.
See also “A Model Estimating the Value of Research Influencing Funders” that comes to a similar conclusion
This may be less fun but, for completeness, I want to present an alternative perspective—I think I know exactly how I’d spend it and don’t have any particular questions. Feel free to send the $20M over whenever works best.
You might like “A Model Estimating the Value of Research Influencing Funders” which makes a similar point, but quantitatively
Hi David—I work a lot on semiconductor/chip export policy, so very important to think about the strategy here.
My biggest issue is that “short vs. long” timelines is not a binary. I agree that under longer timelines, say post-2035, China likely can significantly catch up on chip manufacturing. (Seems much less likely pre-2035.) But I think the controls logic matters really strongly for timelines 2025-2035 and still might create a larger strategic advantage post-2035.
Who has the chips still matters, since it determines whether the country has enough compute to train their own models, run any models, and provision cloud providers. You treat “differential adoption” and “who owns chips” as separate when they’re deeply interconnected. If you control chip supply, you inherently influence adoption patterns. There would be diffusion of AI of course, but it would be much more likely to come from the US given chip controls, and potentially the AI would remain on US cloud under US control.
Furthermore, if you grant that AI can accelerate AI development itself, a 2-3 year compute advantage could be decisive… and not just in “fast take-off recursive self-improvement” but even in mundane ways where better AI leads to better chip design tools, better compiler optimization better datacenter cooling systems, and better materials science for next-gen chips.
You’re right that it is impossible to control 100% of the chips, but that’s not the goal. The goal is to control enough of the chips enough of the time to create a structural advantage. Maintaining a 10-to-1 compute advantage of the US over China will mean that even if we had AI parity, we’d still have 10x more AI agents than China. And we’d likely have better AI per agent as well.
For example, consider the same Russian oil example you discuss—yes, there’s significant leakage to India and China and these controls aren’t perfect, but Russia’s realized prices have stayed ~$15-20/barrel below Brent throughout 2024-2025 - forcing Russia to accept steep discounts while burning cash on shadow fleet operations and longer shipping routes.
And chips are much easier to control than oil right now. Currently, OpenAI can buy one million NVIDIA GB300s to power Stargate, but China and Russia can’t even come close. Chinese chips are currently much weaker in both quantity and quality, and this will persist for awhile as China lacks the relevant chipmaking equipment and likely will for some time—the EUV tech that prints chips at nanometer scale took decades to develop and is arguably the most advanced technology ever made. You seem to have some all-or-nothing thinking here or think that we can’t possibly block enough chips to matter, but we already have significantly reduced China’s compute stock and you even have people like DeepSeek’s CEO mentioning that chip controls are their biggest barrier. Chinese AI development would certainly be different if China could freely buy one million GB300s as well.
The key thing is that semiconductor manufacturing isn’t a commodity market with fungible goods flowing to equilibrium. You’re treating this as a standard economic problem where market forces inevitably equalize and assume a lot of frictionless markets—but neither of these seem true. The chip supply chain has unique characteristics with extreme manufacturing concentration, decades-long development cycles, and tacit knowledge that make it different. Additionally, network effects in AI development could create lock-in before economic pressure equalizes access. Moreover, American/Western AI and chip development isn’t going to freely flow to China because the US government would continue to stop that from happening as a matter of national security. Capital does flow, but this technology cannot flow quickly, freely, or easily.
It’s also not easy to just arbitrarily make up for chip disadvantage with energy advantage. It’s very difficult to train frontier AI models on ancient hardware. DeepSeek has been trying hard all year to train their model on Huawei chips and still haven’t succeeded. It doesn’t matter how cheap you make energy if chips remain a limiting factor. Arguably, TSMC’s lead over SMIC has grown, not shrunk, over the past decade despite massive Chinese investment.
All told, I think that China is at a significant AI disadvantage over the next decade or more and this is due to reasonably effective (albeit imperfect) chip controls. Ideally we would make the chip controls even better and stronger to press that advantage further (I have ideas on how), but that’s a different conversation from the strategic wisdom of the controls in the first place.
Congrats! I also thought it was great.
Sorry for the slightly off-topic question but I noticed EAG London 2025 talks are uploaded to YouTube but I didn’t see any EAG Bay Area 2025 talks. Do you know when those will go up?
I still stand by the book and I attribute a lot of my historical failures in management to not implementing this book well enough (especially the part about creating clarity around goals).
If you’re considering a career in AI policy, now is an especially good time to start applying widely as there’s a lot of hiring going on right now. I documented in my Substack over a dozen different opportunities that I think are very promising.
Thank you for sharing your perspective and I’m sorry this has been frustrating for you and people you know. I deeply appreciate your commitment and perseverance.
I hope to share you a bit of perspective from me as a hiring manager on the other side of things:
Why aren’t orgs leaning harder on shared talent pools (e.g. HIP’s database) to bypass public rounds? HIP is currently running an open search.
It’s very difficult to run an open search for all conceivable jobs and have the best fit for all of them. And even if you do have a list of the top candidates for everything, it’s still hard to sort and filter through that list without more screening. This makes HIP a valuable supplement but not a replacement.
~
I also think it would be worth considering how to provide some sort of job security/benefit for proven commitment within the movement
‘The movement’ is just the mix of all the people and orgs doing their own thing. Individual orgs themselves should be responsible for job security and rewarding commitment—the movement itself unfortunately isn’t an entity that is capable of doing that.
~
I know one lady who worked at a top EA org for eight years; she’s now struggling to find her next position within the movement, competing with new applicants! That seems like a waste of career capital.
Hopefully her eight years gives her a benefit against other applicants! That is, the career capital hasn’t been ‘wasted’ at all. But it still makes sense to view her against other applicants who may have other skills needed for the role—being good at one role doesn’t make you a perfect automatic fit for another role.
~
Moreover, I would avoid the expensive undertaking of a full hiring round until my professional networks had been exhausted. After all, if you’re in my network to begin with, you probably did something meritorious to get there.
While personal networks are a great place to source talent they’re far from perfect—in particular while personal networks are created by merit they are also formed by bias and preferencing ‘people like us’. A ‘full hiring round’ is thus more meritocratic—anyone can apply, you don’t need to figure out how to get into the right person’s network first.
~
You might like this article: Don’t be bycatch.
I think you should get the LLM to give you the citation and then cite that (ideally after checking it yourself).
I think Alex Bores winning his primary in NY-12 would be very valuable for AI safety and animal welfare—He championed and fought very hard for the NY RAISE Act and is willing to make AI safety his #1 issue in Congress. He’s also vegetarian and cares a lot about animal welfare. I’ve met Bores a few times and personally vouch for him understanding these issues very well.
But the election is very close. If you also support Alex Bores, one very valuable thing you could consider doing is reaching out to your friends who may live in NY-12 and tell them about your support.
In fact, you might know more people in NY-12 than you think! A LinkedIn filter (bit.ly/ny-li-filter) and a Facebook filter (bit.ly/ny-fb-filter) can help you find more people. I include more details and sample reach out messages in this document --> https://bit.ly/help-bores-win