Lizka

Karma: 16,914

I’m a researcher at Forethought; before that, I ran the non-engineering side of the EA Forum (this platform), ran the EA Newsletter, and worked on some other content-related tasks at CEA. [More about the Forum/CEA Online job.]

Selected posts

Background

I finished my undergraduate studies with a double major in mathematics and comparative literature in 2021. I was a research fellow at Rethink Priorities in the summer of 2021 and was then hired by the Events Team at CEA. I later switched to the Online Team. In the past, I’ve also done some (math) research and worked at Canada/USA Mathcamp.

Lizka Mar 18, 2025, 1:18 AM
15 points
1 ∶ 0
on: Lizka’s Shortform
Quick (visual) note on something that seems like a confusion in the current conversation:

Others have noted similar things (eg, and Will’s earlier take on total vs human extinction). You might disagree with the model (curious if so!), but I’m a bit worried that one way or another people are talking past each other (least from skimming the discussion).

(Commenting via phone, sorry for typos or similar!)

Lizka Mar 17, 2025, 4:59 PM
8 points
0 ∶ 0
on: Existential Choices Symposium with Will MacAskill and other special guests (3-5pm GMT Monday)
What actually changes about what you’d work on if you concluded that improving the future is more important on the current margin than trying to reduce the chance of (total) extinction (or vice versa)?
Curious for takes from anyone!

Lizka Mar 14, 2025, 7:32 PM
15 points
0 ∶ 0
on: AI Tools for Existential Security
I wrote a Twitter thread that summarizes this piece and has a lot of extra images (I probably went overboard, tbh.)
I kinda wish I’d included the following image in the piece itself, so I figured I’d share it here:
$As AI capabilities rise, AI systems will be responsible for a growing fraction of relevant work$

AI Tools for Existential Security

LizkaMar 14, 2025, 6:37 PM

44 points

8 comments11 min readEA link

(www.forethought.org)

Lizka Feb 17, 2025, 1:29 PM
9 points
1 ∶ 0
on: Fake thinking and real thinking
I just want to say that I really appreciated this post — it came at exactly the right time for me and I’ve referenced it several times since you shared it.

Some moderation team principles

JP Addison🔸Feb 4, 2025, 6:33 PM

61 points

2 comments5 min readEA link

Lizka Feb 4, 2025, 1:42 PM
4 points
0 ∶ 0
in reply to: Lizka’s comment on: Lizka’s Shortform
Follow-up:
Quick list of some ways benchmarks might be (accidentally) misleading^[1]
1. Poor “construct validity”^[2]( & systems that are optimized for the metric)
  1. The connection between what the benchmark is measuring and what it’s trying to measure (or what people think it’s measuring) is broken. In particular:
  2. Missing critical steps
    When benchmarks are trying to evaluate progress on some broad capability (like “engineering” or “math ability” or “planning”), they’re often testing specific performance on meaningfully simpler tasks. Performance on those tasks might be missing key aspects of true/real-world/relevant progress on that capability.
  3. Besides the inherent difficulty of measuring the right thing, it’s important to keep in mind that systems might have been trained specifically to perform well on a given benchmark.
    This is probably a bigger problem for benchmarks that have gotten more coverage.
  4. And some benchmarks have been designed specifically to be challenging for existing leading models, which can make new/other AI systems appear to have made more progress on these capabilities (relative to the older models) than they actually are.
    We’re seeing this with the “Humanity’s Last Exam” benchmark.
  5. ...and sometimes some of the apparent limitations are random or kinda fake, such that a minor improvement appears to lead to radical progress
2. Discontinuous metrics: (partial) progress on a given benchmark might be misleading.
  1. The difficulty of tasks/tests in a benchmark often varies significantly (often for good reason), and reporting of results might explain the benchmark by focusing on its most difficult tests instead of the ones that the model actually completed.
    I think this was an issue for Frontier Math, although I’m not sure how much strongly to discount some of the results as a result.
  2. -> This (along with issues like 1b above, which can lead to saturation) is part of what makes it harder to extrapolate from e.g. plots of progress on certain benchmarks.
3. Noise & poisoning of the metric: Even on the metric in question, data might have leaked into the training process, the measurement process itself can be easily affected by things like who’s running it, comparing performance of different models that were tested slightly differently might be messy, etc. Some specific issues here (I might try to add links later):
  1. Brittleness to question phrasing
  2. Data contamination (discussed e.g. here)
  3. Differences in how the measurement/testing process was actually set up (including how much inference compute was used, how many tries a model was given, access to tools, etc.)
4. Misc
  1. Selective reporting/measurement (AI companies want to report their successes)
  2. Tasks that appear difficult (because they’re hard for humans) might be might be especially easier for AI systems (and vice versa) — and this might cause us to think that more (or less) progress is being made than is true
    E.g. protein folding is really hard for humans
    This looks relevant, but I haven’t read it
  3. Some benchmarks seem mostly irrelevant to what I care about
  4. Systems are tested pre post-training enhancements or other changes
Additions are welcome! (Also, I couldn’t quickly find a list like this earlier, but I’d be surprised if a better version than what I have above wasn’t available somewhere; I’d love recommendations.)
Open Phil’s announcement of their now-closed benchmarking RFP has some useful notes on this, particularly the section on “what makes for a strong benchmark.”I also appreciated METR’s list of desiderata here.
1. ^
  To be clear: I’m not trying to say anything on ways benchmarks might be useful/harmful here. And I’m really not an expert.
2. ^
  This paper looks relevant but I haven’t read it.

Lizka Feb 3, 2025, 10:26 PM

28 points

3 ∶ 0

on: Lizka’s Shortform

TLDR: Notes on confusions about what we should do about digital minds, even if our assessments of their moral relevance are correct^[1]

I often feel quite lost when I try to think about how we can “get digital minds right.” It feels like there’s a variety of major pitfalls involved, whether or not we’re right about the moral relevance of some digital minds.

Digital-minds-related pitfalls in different situations
Reality ➡️ Our perception ⬇️	These digital minds are (non-trivially) morally relevant^[2]	These digital minds are not morally relevant
We see these digital minds as morally relevant	(1) We’re right But we might still fail to understand how to respond, or collectively fail to implement that response.	(2) We’re wrong We confuse ourselves, waste an enormous amount of resources on this^[3] (potentially sacrificing the welfare of other beings that do matter morally), and potentially make it harder to respond to the needs of other digital minds in the future (see also).
We don’t see these digital minds as morally relevant	(3) We’re wrong The default result here seems to be moral catastrophe through ignorance/sleepwalking (although maybe we’ll get lucky).	(4) We’re right All is fine (at least on this front).

Categories (2) and (3) above are ultimately about being confused about which digital minds matter morally — having an inappropriate level of concern, one way or another. A lot of the current research on digital minds seems to be aimed at avoiding this issue. (See e.g. and e.g..) I’m really glad to see this work; the pitfalls in these categories worry me a lot.

But even if we end up in category (1) — we realize correctly that certain digital minds are morally relevant — how can we actually figure out what we should do? Understanding how to respond probably involves answering questions like:

What matters for these digital minds? What improves their experience? (How can we tell?)
Is their experience overall negative/positive? Should we be developing such minds? (How can we tell??) (How can we ethically design digital minds?)
Should we respect certain rights for these digital minds? (..?)
??

Answering these questions seem really difficult.

In many cases, extrapolating from what we can tell about humans seems inappropriate.^[4] (Do digital minds systems find joy or meaning in some activities? Do they care about survival? What does it mean for a digital mind/AI system to be in pain? Is it ok to lie to AI systems? Is it ok to design AI systems that have no ~goals besides fulfilling requests from humans?)

And even concepts we have for thinking about what matters to humans (or animals) often seem ill-suited for helping us with digital minds.^[5] (When does it make sense to talk about freedoms and rights,^[6] or the sets of (relevant) capabilities a digital mind has, or even their preferences? What even is the right “individual” to consider? Self-reports seem somewhat promising, but when can we actually rely on them as signals about what’s important for digital minds?)

I’m also a bit worried that too much of the work is going towards the question of “which systems/digital minds are morally relevant” vs to the question of “what do we do if we think that a system is morally relevant (or if we’re unsure)?” (Note however that I’ve read a tiny fraction of this work, and haven’t worked in the space myself!) E.g. this paper focuses on the former and closes by recommending that companies prepare to make thoughtful decisions about how to treat the AI systems identified as potentially morally significant by (a) hiring or appointing a DRI (directly responsible individual) for AI welfare and (b) developing certain kinds of frameworks for AI welfare oversight. These steps do seem quite good to me (at least naively), but — as the paper explicitly acknowledges — they’re definitely not sufficient.

Some of the work that’s apparently focused at the question of “which digital minds matter morally” involves working towards theories of ~consciousness, and maybe that will also help us with the latter question. But I’m not sure.

(So my quite uninformed independent impression is that it might be worth investing a bit more in trying to figure out what we should do if we do decide that some digital minds might be morally relevant, or maybe what we should do if we find that we’re making extremely little progress on figuring out whether they are.)

These seem like hard problems/questions, but I want to avoid defeatism.

I appreciated @rgb‘s closing remark in this post (bold mine):

To be sure, our neuroscience tools are way less powerful than we would like, and we know far less about the brain than we would like. To be sure, our conceptual frameworks for thinking about sentience seem shaky and open to revision. Even so, trying to actually solve the problem by constructing computational theories which try to explain the full range of phenomena could pay significant dividends. My attitude towards the science of consciousness is similar to Derek Parfit’s attitude towards ethics: since we have only just begun the attempt, we can be optimistic.

Some of the other content I’ve read/skimmed feels like it’s pointing in useful directions on these fronts (and more recommendations are welcome!):

Improving the Welfare of AIs: A Nearcasted Proposal—Ryan Greenblatt on LW
To understand AI sentience, first understand it in animals—Jonathan Birch in Aeon (I’ve only skimmed this, though)
Jeff Sebo on digital minds, and how to avoid sleepwalking into a major moral catastrophe − 80,000 Hours (I’ve only skimmed this, again)

^
I’m continuing my quick take spree, with a big caveat that I’m not a philosopher and haven’t read nearly as much research/writing on digital minds as I want to.
And I’m not representing Forethought here! I don’t know what Forethought folks think about what I’m writing here.
^
The more appropriate term, I think, is “moral patienthood.” And we probably care about the degree of moral consideration that is merited in different cases.
^
This paper on “Sharing the world with digital minds” mentions this as one of the many failure modes — note that I’ve only skimmed it.
^
I’m often struck by how appropriate the War with the Newts is as an analogy/illustration/prediction for a bunch of what I’m seeing today, including on issues related to digital minds. But there’s one major way in which the titular newts differ significantly from potential morally relevant digital minds; the newts are quite similar to humans and can, to some extent, communicate their preferences in ways the book’s humans could understand if they were listening.
(One potential exception: there’s a tragic-but-brilliant moment in which “The League for the Protection of Salamanders” calls women to action sewing modest skirts and aprons for the Newts in order to appease the Newts’ supposed sense of propriety.)
^
This list of propositions touches on related questions/ideas.
^
See e.g. here:
Whether instrumentally-justified or independently binding, the rights that some AI systems could be entitled to might be different from the rights that humans are entitled to. This could be because, instrumentally, a different set of rights for AI systems promotes welfare. For example, as noted by Shulman and Bostrom (2021), naively granting both “reproductive” rights and voting rights to AI systems would have foreseeably untenable results for existing democratic systems: if AI systems can copy themselves at will, and every copy gets a vote, then elections could be won via tactical copying. This set of rights would not promote welfare and uphold institutions in the same way that they do for humans. Or AI rights could differ because, independently of instrumental considerations, their different properties entitle them to different rights—analogously to how children and animals are plausibly entitled to different rights than adults.

Lizka Feb 2, 2025, 11:11 PM
5 points
0 ∶ 0
in reply to: NickLaing’s comment on: Lizka’s Shortform
Thanks for saying this!
I’m trying to get back into the habit of posting more content, and aiming for a quick take makes it easier for me to get over perfectionism or other hurdles (or avoid spending more time on this kind of thing than I endorse). But I’ll take this as a nudge to consider sharing things as posts more often. :)

Lizka Feb 2, 2025, 10:59 PM
82 points
10 ∶ 0
on: Lizka’s Shortform
When I try to think about how much better the world could be, it helps me to sometimes pay attention to the less obvious ways that my life is (much) better than it would have been, had I been born in basically any other time (even if I was born among the elite!).
So I wanted to make a quick list of some “inconspicuous miracles” of my world. This isn’t meant to be remotely exhaustive, and is just what I thought of as I was writing this up. The order is arbitrary.
1. Washing machines
It’s amazing that I can just put dirty clothing (or dishes, etc.) into a machine that handles most of the work for me. (I’ve never had to regularly hand-wash clothing, but from what I can tell, it is physically very hard, and took a lot of time. I have sometimes not had a dishwasher, and really notice the lack whenever I do; my back tends to start hurting pretty quickly when I’m handwashing dishes, and it’s easier to start getting annoyed at people who share the space.)
Source: Illustrations from a Teenage Holiday Journal (1821). See also.
2. Music (and other media!)
Just by putting on headphones, I get better music performances than most royalty could ever hope to see in their courts. (The first recording was made in the 19th century. Before that — and also widespread access to radios — few people would get access to performances by “professional” musicians, maybe besides church or other gatherings. (Although I kind of expect many people were somewhat better at singing than most are today.)) (A somewhat silly intuition pump: apparently Franz Liszt inspired a mania. But most 19th century concert attendees wouldn’t be hearing him more than once or maybe a handful of times! So imagine listening to a piece that rocks your world, and then only being able to remember it by others’ recreations or memorabilia you’ve managed to get your hands on.)
In general, entertainment seems so much better today. Most people in history were illiterate — their entertainment might come from their own experiences or maybe telling each other stories, games, and occasional plays or other events. Of the people who could, at least, read books, most had access to very few. Today, we have a huge number of books and other texts, as well as TV/movies, and more. I have access to video demonstrations of a functionally endless number of things, (color!) photos of an enormous amount of art (I remember being struck by my mom mentioning that she’d always wished to see some building or other when she was younger, as she’d seen it in a book that had a catalogue of black-and-white architecture pictures at some point), etc.
People listening to a touring phonograph, from the 1900 inaugural issue of Boletín fonográfico. Source: Inventing the recording.
3. Glasses
I have pretty mild shortsightedness (with a mild astigmatism) but even then, I’m really glad that I have glasses; wearing them prevents headaches/ fatigue, makes it easier for me to see people’s faces (or street signs), and makes a huge difference if I’m trying to see a whiteboard (or watch a show or movie). And a number of my friends and family are near-blind without glasses, in ways that make me feel a lot of sympathy for people in the past; eye problems aren’t new. (In some ways, the modern world requires us to engage more with things like written signage, but I think many people in history had duller or more painful lives because they lacked glasses.)
Source: Highlights from Folger Shakespeare Library’s Release of almost 80,000 Images
4. Not having to marry someone against my will
By the standards of most of history, I’d be something like an old maid. I’m so grateful that I’ve never felt pressure of any kind to marry someone I didn’t want to.
Source.
5. Waterproof shoes / umbrellas/ coats
I live in Oxford, and have shoes in which I can walk through ankle-high puddles without wetting my feet. But whenever I get my feet wet and have to stay out in the cold — if I forget to bring an umbrella or coat and it rains heavily — I get quite miserable! Rain isn’t a modern development; a lot of people had to deal with these problems a lot. Moreover, most people’s houses were leakier and significantly less insulated than mine.
Source: Wikimedia
6. Photos and videos of loved ones, ability to (video) call them
We have very few pictures of my brother when he was very young (before my parents moved to the US), and very few childhood photos of my parents or older relatives. There are significantly more pictures of me, and I took tons of photos of my little sister — and love looking at them. And I can call my family and friends when we’re not in the same place! I don’t need to wait weeks to send and receive letters, hoping that they will arrive, or rely on sketches or miniatures.
7. Sewage systems, garbage disposal, menstrual products, diapers, etc.
I don’t need to use an outhouse, and walk along streets that are basically free of sewage and trash. I’m not chronically ill; I can be pretty confident that the water I drink isn’t contaminated by excrement. My life is basically normal when I’m on my period. When I’ve changed babies, I could just throw away the diaper.
Source.
I don’t want to pretend that life in earlier periods was all bad! (People had solutions to some of these problems that meant they could handle them, in some cases, better than I can today when they arose. And people had wonderful and meaningful moments — here’s a really touching collection of photos of friendship I saw as I was finding the pictures above.) But I’m so grateful for some things that have changed at least in some places.
What do I want to see in the future?
The world could be so much better. There are daily tragedies or horrors that need to stop, risks we face that I want to reduce, and privileges enjoyed by some that I want to see extended to everyone else. But even the ~daily standards of today’s privileged groups could go up by a lot. For instance, I think our treatment and practices around mental health / psychological pain are still far from what they could be — I often feel like we’re in the 18th century equivalent of medicine on this front. Chronic pain management and aging science (e.g handling dementia) is also significantly farther behind what it could be. Pain management (and healthcare more generally) has improved massively over the last decades/centuries, but people still have horrible migraines, long diseases, etc. I think we still have less quality time to spend with people we love and on things we find meaningful than we could. Technology could augment (or reduce the need for) our willpower in some areas, etc.
Source.

Lizka Feb 1, 2025, 7:56 PM
64 points
4 ∶ 0
on: Lizka’s Shortform
Notes on some of my AI-related confusions^[1]
It’s hard for me to get a sense for stuff like “how quickly are we moving towards the kind of AI that I’m really worried about?” I think this stems partly from (1) a conflation of different types of “crazy powerful AI”, and (2) the way that benchmarks and other measures of “AI progress” de-couple from actual progress towards the relevant things. Trying to represent these things graphically helps me orient/think.
First, it seems useful to distinguish the breadth or generality of state-of-the-art AI models and how able they are on some relevant capabilities. Once I separate these out, I can plot roughly where some definitions of “crazy powerful AI” apparently lie on these axes:
(I think there are too many definitions of “AGI” at this point. Many people would make that area much narrower, but possibly in different ways.)
Visualizing things this way also makes it easier for me^[2] to ask: Where do various threat models kick in? Where do we get “transformative” effects? (Where does “TAI” lie?)
Another question that I keep thinking about is something like: “what are key narrow (sets of) capabilities such that the risks from models grow ~linearly as they improve on those capabilities?” Or maybe “What is the narrowest set of capabilities for which we capture basically all the relevant info by turning the axes above into something like ‘average ability on that set’ and ‘coverage of those abilities’, and then plotting how risk changes as we move the frontier?”
The most plausible sets of abilities like this might be something like:
- Everything necessary for AI R&D^[3]
- Long-horizon planning and technical skills?
If I try the former, how does risk from different AI systems change?
And we could try drawing some curves that represent our guesses about how the risk changes as we make progress on a narrow set of AI capabilities on the x-axis. This is very hard; I worry that companies focus on benchmarks in ways that make them less meaningful, so I don’t want to put performance on a specific benchmark on the x-axis. But we could try placing some fuzzier “true” milestones along the way, asking what the shape of the curve would be in reference to those, and then trying to approximate how far along we are with respect to them by using a combination of metrics and other measures. (Of course, it’s also really difficult to develop a reasonable/useful sense for how far apart those milestones are on the most appropriate measure of progress — or how close partial completion of these milestones is to full completion.)
Here’s a sketch:
Overall I’m really unsure of which milestones I should pay attention to here, and how risk changes as we might move through them.
It could make sense to pay attention to real-world impacts of (future) AI systems instead of their ~intrinsic qualities, but real-world impacts seem harder to find robust precursors to, rely on many non-AI factors, and interpreting them involves trying to untangle many different cruxes or worldviews. (Paying attention to both intrinsic qualities and real-world impacts seems useful and important, though.)
All of this also complicates how I relate to questions like “Is AI progress speeding up or slowing down?” (If I ignore all of these confusions and just try to evaluate progress intuitively / holistically, it doesn’t seem to be slowing down in relevant ways.^[4])
Thoughts/suggestions/comments on any of this are very welcome (although I may not respond, at least not quickly).
Some content related to at least some of the above (non-exhaustive):
- METR’s RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts
- AI Impacts page on HLAI, especially “Human-level” is superhuman
- List of some definitions of advanced AI systems
- John Wentworth distinguishing between “early transformative AI” and “superintelligence”
- Holden Karnofsky’s recent piece for the Carnegie Endowment for International Peace: AI Has Been Surprising for Years
- Writing on (issues with) benchmarks/evals: Kelsey Piper in Vox, Anthropic’s “challenges in evaluating AI systems”, Epoch in 2023 on how well compute predicts benchmark performance (pretty well on average, harder individually)
- Recent post that I appreciated, which outlined timelines via some milestones and then outlined a picture of how the world might change in the background
1. ^
  Not representing “Forethought views” here! (I don’t know what Forethought folks think of all of this.)
  Written/drawn very quickly.
2. ^
  This diagram also makes me wonder how much pushing the bottom right corner further to the right (on some relevant capabilities) would help (especially as an alternative to pushing up or diagonally), given that sub-human general models don’t seem that safety-favoring but some narrow-but-relevant superhuman capabilities could help us deal with the risks of more general, human-level+ systems.
3. ^
  Could a narrower set work? E.g. to what extent do we just care about ML engineering?
4. ^
  although I’m somewhat surprised at the lack of apparent real-world effects

Lizka Feb 1, 2025, 3:40 PM
11 points
4 ∶ 0
on: The Upcoming PEPFAR Cut Will Kill Millions, Many of Them Children
I don’t think the situation is actually properly resolved by the recent waivers (as @Garrison points out below). See e.g. this thread and the linked ProPublica article. From the thread:
There is mass confusion and fear, both in and outside government. The aid organizations say they either don’t know how to get a waiver exempting them from Trump’s order or have no idea if/when theirs might get approved.
And from the article:
Despite an announcement earlier this week ostensibly allowing lifesaving operations to continue, those earlier orders have not been rescinded.
Many groups doing such lifesaving work either don’t know the right way to request an exemption to the order, known as a waiver, or have no sense of where their request stands. They’ve received little information from the U.S. government, where, in recent days, humanitarian officials have been summarily ousted or prohibited from communicating with the aid organizations.
...
By Tuesday evening, Trump and Rubio appeared to heed the international pressure and scale back the order by announcing that any “lifesaving” humanitarian efforts would be allowed to continue.
Aid groups that specialize in saving lives were relieved and thought their stop-work orders would be reversed just as swiftly as they had arrived.
But that hasn’t happened. Instead, more stop-work orders have been issued. As of Thursday, contractors worldwide were still grounded under the original orders and unable to secure waivers. Top Trump appointees arrested further funding and banned new projects for at least three months.
“We need to correct the impression that the waiver was self-executing by virtue of the announcement,” said Marcia Wong, the former deputy assistant administrator of USAID’s humanitarian assistance bureau.
Aid groups that had already received U.S. money were told they could not spend it or do any previously approved work. The contractors quoted in this article spoke on the condition of anonymity because they feared the administration might prolong their suspension or cancel their contracts completely.
(Thanks for writing the post! I referenced it in a mini thread on Twitter/X.)

Lizka Dec 12, 2024, 2:43 PM
11 points
4 ∶ 0
in reply to: titotal’s comment on: Probabilities might be off by one percentage point
As a datapoint: despite (already) agreeing to a large extent with this post,^[1] IIRC I answered the question assuming that I do trust the premise.
Despite my agreement, I do think there are certain kinds of situations in which we can reasonably use small probabilities. (Related post: Most* small probabilities aren’t pascalian, and maybe also related.)
More generally: I remember appreciating some discussion on the kinds of thought experiments that are useful, when, etc. I can’t find it quickly, but possible starting points could be this LW post, Least Convenient Possible World, maybe this post from Richard, and stuff about fictional evidence.
Writing quickly based on a skim, sorry for lack of clarity/misinterpretations!
1. ^
  My view is roughly something like:
  at least in the most obviously analogous situations, it’s very rare that we can properly tell the difference between 1.5% and 0.15% (and so the premise is somewhat absurd)

Lizka Nov 12, 2024, 9:56 PM
4 points
0 ∶ 0
in reply to: Lauren Gilbert’s comment on: The Structural Transformation Case For Peacekeeping
That makes sense and is roughly how I was interpreting what you wrote (sorry for potentially implying otherwise in my comment) — this is still a lot more positive on peacekeeping than I was expecting it to be :)

Lizka Nov 12, 2024, 9:48 PM
7 points
0 ∶ 0
on: The Structural Transformation Case For Peacekeeping
Before looking at what you wrote, I was most skeptical of the existence of (plausibly) cost-effective interventions on this front. In particular, I had a vague background view that some interventions work but are extremely costly (financially, politically, etc.), and that other interventions either haven’t been tried or don’t seem promising. I was probably expecting your post to be an argument that we/most people undervalue the importance of peace (and therefore costly interventions actually look better than they might) or an argument that there are some new ideas to explore.
So I was pretty surprised by what you write about UN peacekeeping:
...[UN] peacekeeping—no matter how useless individual peacekeepers seem—has been shown to work. The academic literature is very clear on this:
- Walter 2002 finds that if a third party is willing to enforce a signed agreement, warring parties are much more likely to make a bargain (5% vs. 50%) and the settlement is much more likely to hold (0.4% vs. 20%). 20% is not an amazing probability for sustained peace, but it sure beats 0.5%.
- ...
I haven’t actually looked at linked papers to check how how convincing I think they are, but thought it was interesting! And I wanted to highlight this in a comment in case any Forum users aren’t sure if they want to click through to the post but would be interested to read more with this context.
Another point that was new to me:
The UN Security Council seems to believe the endless articles about how useless peacekeepers are, and doesn’t seem all that enthusiastic about sending peacekeepers to new conflicts. Since 2010, the number of deployed peacekeepers has been largely flat, even as conflict deaths have increased
(Thanks for writing & sharing your post!)

Lizka Nov 5, 2024, 12:56 AM
5 points
0 ∶ 0
on: Remembering Joseph Rotblat (born on this day in 1908)
Addendum to the post: an exercise in giving myself advice
The ideas below aren’t new or very surprising, but I found it useful to sit down and write out some lessons for myself. Consider doing something similar; if you’re up for sharing what you write as a comment on this post, I’d be interested and grateful.
(1) Figure out my reasons for working on (major) projects and outline situations in which I would want myself to leave, ideally before getting started on the projects
I plan on trying to do this for any project that gives me any (ethical) doubts, and/or will take up at least 3 months of my full-time work. When possible, I also want to try sharing my notes with someone I trust. (I just did this. If you want to use my template / some notes, you can find it in this footnote.^[1] Related: Staring into the abyss.)
(2) Notice potential (epistemic and moral) “risk factors” in my environment
In many ways, the environment at Los Alamos seemed to elevate the risk that participants would ignore their ethical concerns (probably partly by design). Besides the fact that they were working on a deadly weapon,
- There was a lot of secrecy, and connections to people outside of the project were suspended
- There was a decent amount of ~blanket admiration for the leaders of the project (and for some of the more senior scientists)
- Relatedly, there was a sense of urgency and a collective mission (and there was a relatively clear set of “enemies” — this was during a war)
- Based on how people wrote about Los Alamos later, there seemed to be something playful or adventurous about how many treated their work; the bomb was being called a “gadget,” their material needs were taken care of, etc.
- And many of the participants were relatively young
(Related themes are also discussed in “Are you really in a race?”)
All else equal, I would like to avoid environments that exhibit these kinds of factors. But shunning them entirely seems impractical, so it seems worth finding ways to create some guardrails. Simply noticing that an environment poses these risks seems like it might already be quite useful. I think it would give me the chance to put myself on “high alert,” using that as a prompt to check back in with myself, talk to mentors, etc.
(3) Train some habits and mental motions
Reading about all of this made me want to do the following things more (and more deliberately):
1. Talking to people who aren’t embedded in my local/professional environment
  1. And talking to people who think very independently or in unusual (relative to my immediate circles) ways, seriously considering their arguments and conclusions
  2. (Also: remembering that I almost always have fallback options, and nurturing those options)
2. Explicitly prompting myself to take potential outcomes of my work seriously
  1. Imagine my work’s impact is more significant than I expected it to be. How do I feel? Is there anything I’m embarrassed about, or that I wish I had done differently?
3. Training myself to say — and be willing to believe/entertain — things that might diminish my social status in my community
  1. (This includes articulating confusion or uncertainty.)
I don’t have time right now to set up systems that could help me with these things, but I also just an event to my calendar, to try to figure out how I can do more of this. (E.g. I might want to add something like this to one of my weekly templates or just set up 1-2 recurring events.) Consider doing the same, if you’re in a similar situation.
And these ideas were generated very quickly — I’m sure there are more and likely better recommendations, so suggestions are welcome!
1. ^
  Here’s the rough format I just used:
  (1) Why I’m doing what I’m doing
  [Core worldview + 1-3 key goals, ideally including something that’s specific enough that people you know and like might disagree with it]
  (2) Situations in which I would want myself to leave [these are not necessarily things that I (or you, if you’re filling this out) think are plausible!]
  (2a) Specific red lines — I’d definitely leave if...
  (2b) Red flags: very seriously consider leaving if...
  (2c) [Optional] Other general notes on this (e.g. how changes in circumstances might affect why I’d leave)
  My notes included hypothetical situations like learning something that would cause me to significantly update on the integrity of the people in charge of my organization, situations in which important sources of feedback (sources of correction) seemed to be getting getting closed off, etc.
What links here?
- Remembering Joseph Rotblat (born on this day in 1908) by Lizka (Nov 5, 2024, 12:51 AM; 90 points)

Remembering Joseph Rotblat (born on this day in 1908)

LizkaNov 5, 2024, 12:51 AM

90 points

7 comments9 min readEA link

Lizka Oct 3, 2024, 6:40 PM
62 points
15 ∶ 0
on: Lizka’s Shortform
A note on how I think about criticism
(This was initially meant as part of this post,^[1] but while editing I thought it didn’t make a lot of sense there, so I pulled it out.)
I came to CEA with a very pro-criticism attitude. My experience there reinforced those views in some ways,^[2] but it also left me more attuned to the costs of criticism (or of some pro-criticism attitudes). (For instance, I used to see engaging with all criticism as virtuous, and have changed my mind on that.) My overall takes now aren’t very crisp or easily summarizable, but I figured I’d try to share some notes.
...
It’s generally good for a community’s culture to encourage criticism, but this is more complicated than I used to think.
Here’s a list of things that I believe about criticism:
1. Criticism or critical information can be extremely valuable. It can be hard for people to surface criticism (e.g. because they fear repercussions), which means criticism tends to be undersupplied.^[3] Requiring critics to present their criticisms in specific ways will likely stifle at least some valuable criticism. It can be hard to get yourself to engage with criticism of your work or things you care about. It’s easy to dismiss true and important criticism without noticing that you’re doing it.
  1. → Making sure that your community’s culture appreciates criticism (and earnest engagement with it), tries to avoid dismissing critical content based on stylistic or other non-fundamental qualities, encourages people to engage with it, and disincentivizes attempts to suppress it can be a good way to counteract these issues.
2. At the same time, trying to actually do anything is really hard.^[4] Appreciation for doers is often undersupplied. Being in leadership positions or engaging in public discussions is a valuable service, but opens you up to a lot of (often stressful) criticism, which acts as a disincentive for being public. Psychological safety is important in teams (and communities), so it’s unfortunate that critical environments lead more people to feel like they would be judged harshly for potential mistakes. Not all criticism is useful enough to be worth engaging with (or sharing). Responding to criticism can be time-consuming or otherwise costly and isn’t always worth it.^[5] Sometimes people who are sharing “criticism” hate the project for reasons that aren’t what’s explicitly stated, or just want to vent or build themselves up.^[6]
  1. … and cultures like the one described above can exacerbate these issues.
I don’t have strong overall recommendations. Here’s a post on how I want to handle criticism, which I think is still accurate. I also (tentatively) think that on the margin, the average person in EA who is sharing criticism of someone’s work should probably spend a bit more time trying to make that criticism productive. And I’d be excited to see more celebration or appreciation for people’s work. (I also discussed related topics in this short EAG talk last year.)
1. ^
  This was in that post because I ended up engaging with a lot of discussion about the effects of criticism in EA (and of the EA Forum’s critical culture) as part of running a Criticism Contest (and generally working on CEA’s Online Team).
2. ^
  I’ve experienced first-hand how hard it is to identify flaws in projects you’re invested in, I’ve seen how hard it is for some people to surface critical information, and noticed some ways in which criticism can be shut down or disregarded by well-meaning people.
3. ^
  See also the rationale in our Criticism Contest announcement post.
4. ^
  Kinda related: EA should taboo “EA should”
5. ^
  See the “transparency” example in my post on “missing moods”.
  Also: You don’t have to respond to every comment.
6. ^
  A lot of what Richard says in Moral Misdirection (and in Anti-Philanthropic Misdirection) also seems true and relevant here.
What links here?
- Criticism is sanctified in EA, but, like any intervention, criticism needs to pay rent by Holly Elmore ⏸️ 🔸 (Oct 11, 2024, 4:22 AM; 293 points)
- Announcing my departure from CEA (& sharing assorted notes) by Lizka (Oct 3, 2024, 6:30 PM; 192 points)

Lizka Oct 3, 2024, 6:31 PM
44 points
7 ∶ 0
on: Lizka’s Shortform
A note on mistakes and how we relate to them
(This was initially meant as part of this post^[1], but I thought it didn’t make a lot of sense there, so I pulled it out.)
“Slow-rolling mistakes” are usually much more important to identify than “point-in-time blunders,”^[2] but the latter tend to be more obvious.
When we think about “mistakes”, we usually imagine replying-all when we meant to reply only to the sender, using the wrong input in an analysis, including broken hyperlinks in a piece of media, missing a deadline, etc. I tend to feel pretty horrible when I notice that I’ve made a mistake like this.
I now think that basically none of my mistakes of this kind — I’ll call them “Point-in-time blunders” — mattered nearly as much as other “mistakes” I’ve made by doing things like planning my time poorly, delaying for too long on something, setting up poor systems, or focusing on the wrong things.
This second kind of mistake — let’s use the phrase “slow-rolling mistakes” — is harder to catch; I think sometimes I’d identify them by noticing a nagging worry, or by having multiple conversations with someone who disagreed with me (and slowly changing my mind), or by seriously reflecting on my work or on feedback I’d received.
...
This is not a novel insight, but I think it was an important thing for me to realize. Working at CEA helped move me in this direction. A big factor in this, I think, was the support and reassurance I got from people I worked with.
This was over two years ago, but I still remember my stomach dropping when I realized that instead of using “EA Forum Digest #84” as the subject line for the 84th Digest, I had used “...#85.” Then I did it AGAIN a few weeks later (instead of #89). I’ve screenshotted Ben’s (my manager’s) reaction.
...
I discussed some related topics in a short EAG talk I gave last year, and also touched on these topics in my post about “invisible impact loss”.
An image from that talk.
1. ^
  It was there because my role gave me the opportunity to actually notice a lot of the mistakes I was making (something that I think is harder if you’re working on something like research, or in a less public role), which also meant I could reflect on them.
2. ^
  If you have better terms for these, I’d love suggestions!
What links here?
- Announcing my departure from CEA (& sharing assorted notes) by Lizka (Oct 3, 2024, 6:30 PM; 192 points)

Announcing my departure from CEA (& sharing assorted notes)

LizkaOct 3, 2024, 6:30 PM

192 points

8 comments12 min readEA link

Lizka

AI Tools for Ex­is­ten­tial Security

Some mod­er­a­tion team principles

Re­mem­ber­ing Joseph Rot­blat (born on this day in 1908)

An­nounc­ing my de­par­ture from CEA (& shar­ing as­sorted notes)

AI Tools for Existential Security

Some moderation team principles

Remembering Joseph Rotblat (born on this day in 1908)

Announcing my departure from CEA (& sharing assorted notes)