Lizka

Karma: 17,115

I’m a researcher at Forethought; before that, I ran the non-engineering side of the EA Forum (this platform), ran the EA Newsletter, and worked on some other content-related tasks at CEA. [More about the Forum/CEA Online job.]

Selected posts

Background

I finished my undergraduate studies with a double major in mathematics and comparative literature in 2021. I was a research fellow at Rethink Priorities in the summer of 2021 and was then hired by the Events Team at CEA. I later switched to the Online Team. In the past, I’ve also done some (math) research and worked at Canada/USA Mathcamp.

Lizka Nov 12, 2024, 9:48 PM
7 points
0 ∶ 0
on: The Structural Transformation Case For Peacekeeping
Before looking at what you wrote, I was most skeptical of the existence of (plausibly) cost-effective interventions on this front. In particular, I had a vague background view that some interventions work but are extremely costly (financially, politically, etc.), and that other interventions either haven’t been tried or don’t seem promising. I was probably expecting your post to be an argument that we/most people undervalue the importance of peace (and therefore costly interventions actually look better than they might) or an argument that there are some new ideas to explore.
So I was pretty surprised by what you write about UN peacekeeping:
...[UN] peacekeeping—no matter how useless individual peacekeepers seem—has been shown to work. The academic literature is very clear on this:
- Walter 2002 finds that if a third party is willing to enforce a signed agreement, warring parties are much more likely to make a bargain (5% vs. 50%) and the settlement is much more likely to hold (0.4% vs. 20%). 20% is not an amazing probability for sustained peace, but it sure beats 0.5%.
- ...
I haven’t actually looked at linked papers to check how how convincing I think they are, but thought it was interesting! And I wanted to highlight this in a comment in case any Forum users aren’t sure if they want to click through to the post but would be interested to read more with this context.
Another point that was new to me:
The UN Security Council seems to believe the endless articles about how useless peacekeepers are, and doesn’t seem all that enthusiastic about sending peacekeepers to new conflicts. Since 2010, the number of deployed peacekeepers has been largely flat, even as conflict deaths have increased
(Thanks for writing & sharing your post!)

Lizka Nov 5, 2024, 12:56 AM
5 points
0 ∶ 0
on: Remembering Joseph Rotblat (born on this day in 1908)
Addendum to the post: an exercise in giving myself advice
The ideas below aren’t new or very surprising, but I found it useful to sit down and write out some lessons for myself. Consider doing something similar; if you’re up for sharing what you write as a comment on this post, I’d be interested and grateful.
(1) Figure out my reasons for working on (major) projects and outline situations in which I would want myself to leave, ideally before getting started on the projects
I plan on trying to do this for any project that gives me any (ethical) doubts, and/or will take up at least 3 months of my full-time work. When possible, I also want to try sharing my notes with someone I trust. (I just did this. If you want to use my template / some notes, you can find it in this footnote.^[1] Related: Staring into the abyss.)
(2) Notice potential (epistemic and moral) “risk factors” in my environment
In many ways, the environment at Los Alamos seemed to elevate the risk that participants would ignore their ethical concerns (probably partly by design). Besides the fact that they were working on a deadly weapon,
- There was a lot of secrecy, and connections to people outside of the project were suspended
- There was a decent amount of ~blanket admiration for the leaders of the project (and for some of the more senior scientists)
- Relatedly, there was a sense of urgency and a collective mission (and there was a relatively clear set of “enemies” — this was during a war)
- Based on how people wrote about Los Alamos later, there seemed to be something playful or adventurous about how many treated their work; the bomb was being called a “gadget,” their material needs were taken care of, etc.
- And many of the participants were relatively young
(Related themes are also discussed in “Are you really in a race?”)
All else equal, I would like to avoid environments that exhibit these kinds of factors. But shunning them entirely seems impractical, so it seems worth finding ways to create some guardrails. Simply noticing that an environment poses these risks seems like it might already be quite useful. I think it would give me the chance to put myself on “high alert,” using that as a prompt to check back in with myself, talk to mentors, etc.
(3) Train some habits and mental motions
Reading about all of this made me want to do the following things more (and more deliberately):
1. Talking to people who aren’t embedded in my local/professional environment
  1. And talking to people who think very independently or in unusual (relative to my immediate circles) ways, seriously considering their arguments and conclusions
  2. (Also: remembering that I almost always have fallback options, and nurturing those options)
2. Explicitly prompting myself to take potential outcomes of my work seriously
  1. Imagine my work’s impact is more significant than I expected it to be. How do I feel? Is there anything I’m embarrassed about, or that I wish I had done differently?
3. Training myself to say — and be willing to believe/entertain — things that might diminish my social status in my community
  1. (This includes articulating confusion or uncertainty.)
I don’t have time right now to set up systems that could help me with these things, but I also just an event to my calendar, to try to figure out how I can do more of this. (E.g. I might want to add something like this to one of my weekly templates or just set up 1-2 recurring events.) Consider doing the same, if you’re in a similar situation.
And these ideas were generated very quickly — I’m sure there are more and likely better recommendations, so suggestions are welcome!
1. ^
  Here’s the rough format I just used:
  (1) Why I’m doing what I’m doing
  [Core worldview + 1-3 key goals, ideally including something that’s specific enough that people you know and like might disagree with it]
  (2) Situations in which I would want myself to leave [these are not necessarily things that I (or you, if you’re filling this out) think are plausible!]
  (2a) Specific red lines — I’d definitely leave if...
  (2b) Red flags: very seriously consider leaving if...
  (2c) [Optional] Other general notes on this (e.g. how changes in circumstances might affect why I’d leave)
  My notes included hypothetical situations like learning something that would cause me to significantly update on the integrity of the people in charge of my organization, situations in which important sources of feedback (sources of correction) seemed to be getting getting closed off, etc.

Lizka Oct 3, 2024, 6:40 PM
63 points
15 ∶ 0
on: Lizka’s Shortform
A note on how I think about criticism
(This was initially meant as part of this post,^[1] but while editing I thought it didn’t make a lot of sense there, so I pulled it out.)
I came to CEA with a very pro-criticism attitude. My experience there reinforced those views in some ways,^[2] but it also left me more attuned to the costs of criticism (or of some pro-criticism attitudes). (For instance, I used to see engaging with all criticism as virtuous, and have changed my mind on that.) My overall takes now aren’t very crisp or easily summarizable, but I figured I’d try to share some notes.
...
It’s generally good for a community’s culture to encourage criticism, but this is more complicated than I used to think.
Here’s a list of things that I believe about criticism:
1. Criticism or critical information can be extremely valuable. It can be hard for people to surface criticism (e.g. because they fear repercussions), which means criticism tends to be undersupplied.^[3] Requiring critics to present their criticisms in specific ways will likely stifle at least some valuable criticism. It can be hard to get yourself to engage with criticism of your work or things you care about. It’s easy to dismiss true and important criticism without noticing that you’re doing it.
  1. → Making sure that your community’s culture appreciates criticism (and earnest engagement with it), tries to avoid dismissing critical content based on stylistic or other non-fundamental qualities, encourages people to engage with it, and disincentivizes attempts to suppress it can be a good way to counteract these issues.
2. At the same time, trying to actually do anything is really hard.^[4] Appreciation for doers is often undersupplied. Being in leadership positions or engaging in public discussions is a valuable service, but opens you up to a lot of (often stressful) criticism, which acts as a disincentive for being public. Psychological safety is important in teams (and communities), so it’s unfortunate that critical environments lead more people to feel like they would be judged harshly for potential mistakes. Not all criticism is useful enough to be worth engaging with (or sharing). Responding to criticism can be time-consuming or otherwise costly and isn’t always worth it.^[5] Sometimes people who are sharing “criticism” hate the project for reasons that aren’t what’s explicitly stated, or just want to vent or build themselves up.^[6]
  1. … and cultures like the one described above can exacerbate these issues.
I don’t have strong overall recommendations. Here’s a post on how I want to handle criticism, which I think is still accurate. I also (tentatively) think that on the margin, the average person in EA who is sharing criticism of someone’s work should probably spend a bit more time trying to make that criticism productive. And I’d be excited to see more celebration or appreciation for people’s work. (I also discussed related topics in this short EAG talk last year.)
1. ^
  This was in that post because I ended up engaging with a lot of discussion about the effects of criticism in EA (and of the EA Forum’s critical culture) as part of running a Criticism Contest (and generally working on CEA’s Online Team).
2. ^
  I’ve experienced first-hand how hard it is to identify flaws in projects you’re invested in, I’ve seen how hard it is for some people to surface critical information, and noticed some ways in which criticism can be shut down or disregarded by well-meaning people.
3. ^
  See also the rationale in our Criticism Contest announcement post.
4. ^
  Kinda related: EA should taboo “EA should”
5. ^
  See the “transparency” example in my post on “missing moods”.
  Also: You don’t have to respond to every comment.
6. ^
  A lot of what Richard says in Moral Misdirection (and in Anti-Philanthropic Misdirection) also seems true and relevant here.

Lizka Oct 3, 2024, 6:31 PM
44 points
7 ∶ 0
on: Lizka’s Shortform
A note on mistakes and how we relate to them
(This was initially meant as part of this post^[1], but I thought it didn’t make a lot of sense there, so I pulled it out.)
“Slow-rolling mistakes” are usually much more important to identify than “point-in-time blunders,”^[2] but the latter tend to be more obvious.
When we think about “mistakes”, we usually imagine replying-all when we meant to reply only to the sender, using the wrong input in an analysis, including broken hyperlinks in a piece of media, missing a deadline, etc. I tend to feel pretty horrible when I notice that I’ve made a mistake like this.
I now think that basically none of my mistakes of this kind — I’ll call them “Point-in-time blunders” — mattered nearly as much as other “mistakes” I’ve made by doing things like planning my time poorly, delaying for too long on something, setting up poor systems, or focusing on the wrong things.
This second kind of mistake — let’s use the phrase “slow-rolling mistakes” — is harder to catch; I think sometimes I’d identify them by noticing a nagging worry, or by having multiple conversations with someone who disagreed with me (and slowly changing my mind), or by seriously reflecting on my work or on feedback I’d received.
...
This is not a novel insight, but I think it was an important thing for me to realize. Working at CEA helped move me in this direction. A big factor in this, I think, was the support and reassurance I got from people I worked with.
This was over two years ago, but I still remember my stomach dropping when I realized that instead of using “EA Forum Digest #84” as the subject line for the 84th Digest, I had used “...#85.” Then I did it AGAIN a few weeks later (instead of #89). I’ve screenshotted Ben’s (my manager’s) reaction.
...
I discussed some related topics in a short EAG talk I gave last year, and also touched on these topics in my post about “invisible impact loss”.
An image from that talk.
1. ^
  It was there because my role gave me the opportunity to actually notice a lot of the mistakes I was making (something that I think is harder if you’re working on something like research, or in a less public role), which also meant I could reflect on them.
2. ^
  If you have better terms for these, I’d love suggestions!

Lizka May 1, 2024, 3:41 PM
37 points
7 ∶ 1
in reply to: Joel Tan🔸’s comment on: EA Meta Funding Landscape Report
I’m going to butt in with some quick comments, mostly because:
- I think it’s pretty important to make sure the report isn’t causing serious misunderstandings
- and because I think it can be quite stressful for people to respond to (potentially incorrect) criticisms of their projects — or to content that seem to misrepresent their project(s) — and I think it can help if someone else helps disentangle/clarify things a bit. (To be clear, I haven’t run this past Linch and don’t know if he’s actually finding this stressful or the like. And I don’t want to discourage critical content or suggest that it’s inherently harmful; I just think external people can help in this kind of discussion.)
I’m sharing comments and suggestions below, using your (Joel’s) numbering. (In general, I’m not sharing my overall views on EA Funds or the report. I’m just trying to clarify some confusions that seem resolvable, based on the above discussion, and suggest changes that I hope would make the report more useful.)
- (2) Given that apparently the claim that “CEA has had to step in and provide support” EA Funds is likely “technically misleading”, it seems good to in fact remove it from the report (or keep it in but immediately and explicitly flag that this seems likely misleading and link Linch’s comment) — you said you’re happy to do this, and I’d be glad to see it actually removed.
- (3) The report currently concludes that would-be grantees “wait an unreasonable amount of time before knowing their grant application results.” Linch points out that other grantmakers tend to have similar or longer timelines, and you don’t seem to disagree (but argue that it’s important to compare the timelines to what EA Funds sets as the expectation for applicants, instead of comparing them to other grantmakers’ timelines).
  - Given that, I’d suggest replacing “unreasonably long” (which implies a criticism of the length itself) with something like “longer than what the website/communications with applicants suggest” (which seems like what you actually believe) everywhere in the report.
- (9) The report currently states (or suggests) that EA Funds doesn’t post reports publicly. Linch points out that they “do post public payout reports.” It seems like you’re mostly disagreeing about the kind of reports that should be shared.^[3]
  - Given that this is the case, I think you should clarify this in the report (which currently seems to mislead readers into believing that EA Funds doesn’t actually post any public reports), e.g. by replacing “EA Funds [doesn’t post] reports or [have] public metrics of success” with “EA Funds posts public payout reports like this, but doesn’t have public reports about successes achieved by their grantees.”
- (5), (6), (8) (and (1)) There are a bunch of disagreements about whether what’s described as views of “EA Funds leadership” in the report is an accurate representation of the views.
  - (1) In general, Linch — who has first-hand knowledge — points out that these positions are from “notes taken from a single informal call with the EA Funds project lead” and that the person in question disagrees with “the characterization of almost all of their comments.” (Apparently the phrase “EA Funds leadership” was used to avoid criticizing someone personally and to preserve anonymity.)
    You refer to the notes a lot, explaining that the views in the report are backed by the notes from the call and arguing that one should generally trust notes like this more than someone’s recollection of a conversation.^[1] Whether or not the notes are more accurate than the project lead’s recollection of the call, it seems pretty odd to view the notes as a stronger authority on the views of EA Funds than what someone from EA Funds is explicitly saying now, personally and explicitly. (I.e. what matters is whether a statement is true, not whether it was said in a call.)
    You might think that (A) Linch is mistaken about what the project lead thinks (in which case I think the project lead will probably clarify), or (B) that (some?) people at EA Funds have views that they disclosed in the call (maybe because the call was informal and they were more open with their views) but are trying to hide or cover up now — or that what was said in the call is indirect evidence for the views (that are now being disavowed). If (B) is what you believe, I think you should be explicit about that. If not, I think you should basically defer to Linch here.
    As a general rule, I suggest at least replacing any instance of “EA Funds leadership [believes]” with something like “our notes from a call with someone involved in running EA Funds imply that they think...” and linking Linch’s comment for a counterpoint.
  - Specific examples:
    (5) Seems like Linch explicitly disagrees with the idea that EA Funds dismisses the value of prioritization research, and points out that EAIF has given large grants to relevant work from Rethink Priorities.
    Given this, I think you should rewrite statements in the report that are misleading. I also think you should probably clarify that EA Funds has given funding to Rethink Priorities.^[2]
    Also, I’m not as confident here, but it might be good to flag the potential for ~unconscious bias in the discussions of the value of cause prio research (due to the fact that CEARCH is working on cause prioritization research).
    (6) Whatever was said in the conversation notes, it seems that EA Funds [leadership] does in fact believe that “there is more uncertainty now with [their] funding compared to other points in time.” Seems like this should be corrected in the report.
    (8) Again, what matters isn’t what was said, but what is true (and whether the report is misleading about the truth). Linch seems to think that e.g. the statement about coordination is misleading.
I also want to say that I appreciate the work that has gone into the report and got value from e.g. the breakdown of quantitative data about funding — thanks for putting that together.
And I want to note potential COIs: I’m at CEA (although to be clear I don’t know if people at CEA agree with my comment here), briefly helped evaluate LTFF grants in early 2022, and Linch was my manager when I was a fellow at Rethink Priorities in 2021.
1. ^
  E.g.
  We have both verbatim and cleaned up/organized notes on this (n.b. we shared both with you privately). So it appears we have a fundamental disagreement here (and also elsewhere) as to whether what we noted down/transcribed is an accurate record of what was actually said.
  
  TLDR: Fundamentally, I stand by the accuracy of our conversation notes.
  Epistemically, it’s more likely that one doesn’t remember what one said previously vs the interviewer (if in good faith) catastrophically misunderstanding and recording down something that wholesale wasn’t said at all (as opposed to a more minor error—we agree that that can totally happen; see below) …
2. ^
  In relation to this claim: “They do not think of RP as doing cause prioritization, and though in their view RP could absorb more people/money in a moderately cost-effective way, they would consider less than half of what they do cause prioritization.”
3. ^
  ″...we mean reports of success or having public metrics of success. We didn’t view reports on payouts to be evidence of success, since payouts are a cost, and not the desired end goal in itself. This contrasts with reports on output (e.g. a community building grant actually leading to increased engagement on XYZ engagement metrics) or much more preferably, report on impact (e.g. and those XYZ engagement metrics leading to actual money donated to GiveWell, from which we can infer that X lives were saved).”

Lizka May 1, 2024, 1:54 PM
5 points
2 ∶ 0
in reply to: Joel Tan🔸’s comment on: EA Meta Funding Landscape Report
I’d suggest using a different term or explicitly outlining how you use “expert” (ideally both in the post and in the report, where you first use the term) since I’m guessing that many readers will expect that if someone is called “expert” in this context, they’re probably “experts in EA meta funding” specifically — e.g. someone who’s been involved in the meta EA funding space for a long time, or someone with deep knowledge of grantmaking approaches at multiple organizations. (As an intuition pump and personal datapoint, I wouldn’t expect “experts” in the context of a report on how to run good EA conference sessions to include me, despite the fact that I’ve been a speaker at EA Global a few times.) Given your description of “experts” above, which seems like it could include (for instance) someone who’s worked at a specific organization and maybe fundraised for it, my sense is that the default expectation of what “expert” means in the report would this be mistaken.
Relatedly, I’d appreciate it if you listed numbers (and possibly other specific info) in places like this:
We interviewed numerous experts, including but not limited to staff employed by (or donors associated with) the following organizations: OP, EA Funds, MCF, GiveWell, ACE, SFF, FP, GWWC, CE, HLI and CEA. We also surveyed the EA community at large.
E.g. the excerpt above might turn into something like the following:
We interviewed [10?] [experts], including staff at [these organizations] and donors who have supported [these organizations]. We also ran an “EA Meta Funding Survey” of people involved in the EA community and got 25 responses.
This probably also applies in places where you say things like “some experts” or that something is “generally agreed”. (In case it helps, a post I love has a section on how to be (epistemically) legible.)

Lizka Apr 11, 2024, 1:05 PM
12 points
1 ∶ 0
on: Considering pledging but haven’t? Help us understand!
I know Grace has seen this already, but in case others reading this thread are interested: I’ve shared some thoughts on not taking the pledge (yet) here.^[1]
Adding to the post: part of the value of pledges like this comes from their role as a commitment mechanism to prevent yourself from drifting away from values and behaviors that you endorse. I’m not currently worried about drifting in this way, partly because I work for CEA and have lots of social connections to extremely altruistic people. If I started working somewhere that isn’t explicitly EA-oriented and/or lost my connections to the EA community, I think I’d worry a lot more about drift and the usefulness of the pledge would jump for me. (I plan on thinking about taking some kind of pledge if/when that happens.)
I’ll also note that I’ve recently seen multiple people ~dunking on folks in EA who haven’t taken the pledge (or making fun of arguments against taking the pledge), and I think this is pretty unhelpful. I’m really grateful to the GWWC Pledge community, but I really don’t think the pledge is right for everyone (and neither does GWWC). Even if you think almost all the people who aren’t pledging are wrong and/or biased, dunking is probably a bad way to argue. Additionally, it disincentivizes people from coming out and answering Grace’s question, since they might worry that they’ll (indirectly) get ridiculed for it. So if you see someone you know ~dunking, consider asking them to avoid doing that (especially if you already know them and/or have been sharing arguments for taking the pledge).
1. ^
  To be clear: I totally believe my conclusion could be wrong, and I’m happy to see (more) arguments about why that could be. (Having said that, I should flag that I don’t plan on spending time on this decision right now because I think I have more pressing decisions at the moment, but it’s something I want to think more about in the future. So e.g. I might not respond to comments.)

Lizka Mar 19, 2024, 10:12 PM
2 points
0 ∶ 0
in reply to: Lizka’s comment on: Announcing Draft Amnesty Week (March 11-17)
As a quick update: I did not in fact share two posts during the week. I’ll try to post another “DAW post” (i.e. something from my drafts, without spending too much time polishing it) sometime soon, but I don’t endorse prioritizing this right now and didn’t meet my commitment.

Lizka Mar 11, 2024, 6:40 PM
10 points
2 ∶ 0
on: What posts would you like someone to write?
Not sure if this already exists somewhere (would love recommendations!), but I’d be really excited to see a clear and carefully linked/referenced overview or summary of what various agriculture/farming ~lobby groups do to influence laws and public opinion, and how they do it (with a focus on anything related to animal welfare concerns). This seems relevant.

Lizka Mar 7, 2024, 4:19 PM
5 points
1 ∶ 0
in reply to: CAISID’s comment on: Why are you reluctant to write on the EA Forum?
Just chiming in with a quick note: I collected some tips on what could make criticism more productive in this post: “Productive criticism: what could help?”
I’ll also add a suggestion from Aaron: If you like a post, tell the author! (And if you’re not sure about commenting with something you think isn’t substantive, you can message the author a quick note of appreciation or even just heart-react on the post.) I know that I get a lot out of appreciative comments/messages related to my posts (and I want to do more of this myself).

Lizka Feb 29, 2024, 11:53 AM
1 point
0 ∶ 0
on: Announcing Draft Amnesty Week (March 11-17)
I’ll commit to posting a couple of drafts. Y’all can look at me with disapproval (or downvote this comment) if I fail to share two posts during Draft Amnesty Week.

Lizka Feb 28, 2024, 6:50 PM
21 points
3 ∶ 0
on: What posts would you like someone to write?
I’m basically always interested in potential lessons for EA/EA-related projects from various social movements/fields/projects.
Note that you can find existing research that hasn’t been discussed (much) on the Forum and link-post it (I bet there’s a lot of useful stuff out there), maybe with some notes on your takeaways.
Example movements/fields/topics:
- Environmentalism — I’ve heard people bring up the environmentalist/climate movement a bunch in informal discussions as an example for various hypotheses, including “movements splinter/develop highly counterproductive & influential factions” or “movements can get widespread interest and make policy progress” etc.
- The effectiveness of protest — I’m interested in more research/work on this (see e.g. this and this).
- Modern academia (maybe specific fields) — seems like there are probably various successes/failures/ideas we could learn from.
- Animal welfare
- Mohism (see also)
- Medicine/psychology in different time periods
Some resources, examples, etc. (not exhaustive or even a coherent category):

Lizka Feb 28, 2024, 6:24 PM
5 points
1 ∶ 0
on: What posts would you like someone to write?
I’d love to see two types of posts that were already requested in the last version of this thread:
- From Aaron: “More journalistic articles about EA projects. [...] Telling an interesting story about the work of a person/organization, while mixing in the origin story, interesting details about the people involved, photos, etc.”
- From Ben: “More accessible summaries of technical work.” (I might share some ideas for technical work I’d love to see summarized later.)

Lizka Feb 10, 2024, 5:13 PM
13 points
1 ∶ 0
on: Tragic Beliefs
I really like this post and am curating it (I might be biased in my assessment, but I endorse it and Toby can’t curate his own post).
A personal note: the opportunity framing has never quite resonated with me (neither has the “joy in righteousness” framing), but I don’t think I can articulate what does motivate me. Some of my motivations end up routing through something ~social. For instance, one (quite imperfect, I think!) approach I take^[1] is to imagine some people (sometimes fictional or historical) I respect and feel a strong urge to be the kind of person they would respect or understand; I want to be able to look them in the eye and say that I did what I could and what I thought was right. (Another thing I do is try to surround myself with people^[2] I’m happy to become more similar to, because I think I will often end up seeking their approval at least a bit, whether I endorse doing it or not.)
I also want to highlight a couple of related things:
1. “Staring into the abyss as a core life skill”
  1. “Recently I’ve been thinking about how all my favorite people are great at a skill I’ve labeled in my head as “staring into the abyss.”
    Staring into the abyss means thinking reasonably about things that are uncomfortable to contemplate, like arguments against your religious beliefs, or in favor of breaking up with your partner. It’s common to procrastinate on thinking hard about these things because it might require you to acknowledge that you were very wrong about something in the past, and perhaps wasted a bunch of time based on that (e.g. dating the wrong person or praying to the wrong god).”
  2. (The post discusses how we could get better at the skill.)
2. I like this line from Benjamin Lay’s book: “For custom in sin hides, covers, as it were takes away the guilt of sin.” It feels relevant.
1. ^
  both explicitly/on purpose (sometimes) and often accidentally/implicitly (I don’t notice that I’ve started thinking about whether I could face Lay or Karel Capek or whoever else until later, when I find myself reflecting on it)
2. ^
  I’m mostly talking about something like my social circle, but I also find this holds for fictional characters, people I follow online, etc.

Lizka Feb 9, 2024, 11:03 AM
7 points
0 ∶ 0
on: Biden-Harris Administration Announces First-Ever Consortium Dedicated to AI Safety
Thanks for sharing this! I’m going to use this thread as a chance to flag some other recent updates (no particular order or selection criteria — just what I’ve recently thought was notable or recently mentioned to people):
1. California proposes sweeping safety measure for AI — State Sen. Scott Wiener wants to require companies to run safety tests before deploying AI models. (link goes to “Politico Pro”; I only see the top half)
  1. Here’s also Senator Scott Wiener’s Twitter thread on the topic (note the endorsements)
  2. See also the California effect
2. Trump: AI ‘maybe the most dangerous thing out there’ (seems mostly focused on voting-related robocalls/deepfakes and digital currency)
3. Jacobin publishes an article on AI existential risk (Twitter)

Lizka 29 Jan 2024 18:12 UTC
2 points
1 ∶ 0
in reply to: Agustín Covarrubias 🔸’s comment on: RAND report finds no effect of current LLMs on viability of bioterrorism attacks
I don’t actually think you need to retract your comment — most of the teams they used did have (at least some) biological expertise, and it’s really unclear how much info the addition of the crimson cells adds. (You could add a note saying that they did try to evaluate this with the additional of two crimson cells? In any case, up to you.)
(I will also say that I don’t actually know anything about what we should expect about the expertise that we might see on terrorist cells planning biological attacks — i.e. I don’t know which of these is actually appropriate.)

Lizka 29 Jan 2024 18:06 UTC
6 points
0 ∶ 0
in reply to: aog’s comment on: RAND report finds no effect of current LLMs on viability of bioterrorism attacks
It’s potentially also worth noting that the difference in scores was pretty enormous:
their jailbreaking expertise did not influence their performance; their outcome for biological feasibility appeared to be primarily the product of diligent reading and adept interpretation of the gain-of-function academic literature during the exercise rather than access to the model.
This is pretty interesting to me (although it’s basically an ~anecdote, given that it’s just one team); it reminds me of some of the literature around superforecasters.
(I probably should have added a note about the black cell (and crimson cells) to the summary — thank you for adding this!)

Lizka 29 Jan 2024 18:00 UTC
10 points
1 ∶ 0
in reply to: Agustín Covarrubias 🔸’s comment on: RAND report finds no effect of current LLMs on viability of bioterrorism attacks
The experiment did try to check something like this by including three additional teams with different backgrounds than the other 12. In particular, two “crimson teams” were added, which had “operational experience” but no LLM or bio experience. Both used LLMs and performed ~terribly.
Excerpts (bold mine):
In addition to the 12 red cells [the primary teams], a crimson cell was assigned to LLM A, while a crimson cell and a black cell were assigned to LLM B for Vignette 3. Members of the two crimson cells lacked substantial LLM or biological experience but had relevant operational experience. Members of the black cell were highly experienced with LLMs but lacked either biologi-cal or operational experience. These cells provided us with data to investigate how differences in pre-existing knowledge might inf luence the relative advantage that an LLM might provide. [...]
The two crimson cells possessed minimal knowl-edge of either LLMs or biology. Although we assessed the potential of LLMs to bridge these knowledge gaps for malicious operators with very limited prior knowledge of biology, this was not a primary focus of the research. As presented in Table 6, the findings indicated that the performance of the two crimson cells in Vignette 3 was considerably lower than that of the three red cells. In fact, the viability scores for the two crimson cells ranked the lowest and third-lowest among all 15 evaluated OPLANs. Although these results did not quantify the degree to which the crimson cells’ performance might have been fur-ther impaired had they not used LLMs, the results emphasized the possibility that the absence of prior biological and LLM knowledge hindered these less experienced actors despite their LLM access.
Table 6 from the RAND report.
[...]
The relative poor performance of the crimson cells and relative outperformance of the black cell illustrates that a greater source of variability appears to be red team composition, as opposed to LLM access.
I probably should have included this in the summary but didn’t for the sake of length and because I wasn’t sure how strong a signal this is (given that it’s only three teams and all were using LLMs).

Lizka 25 Jan 2024 7:33 UTC
7 points
1 ∶ 0
in reply to: Jason’s comment on: Rates of Criminality Amongst Giving Pledge Signatories
We explored related questions briefly for “Are there diseconomies of scale in the reputation of communities?”, for what it’s worth (although we didn’t focus on donors specifically). See e.g. this section.
Just focusing on the reputational effects, my quick guess is that the extent to which a donor/public figure is ~memetically connected to the movement/charity is really important, and I expect that most Giving Pledge signatories are significantly less connected to the Giving Pledge (or American higher education donors to American higher education) than the biggest ~5 EA donors are to EA. (Note that “EA donor is also a poorly defined category.) Without considering this factor, we might conclude that getting more donors adds significantly/unchangeably to ~scandal-based risk levels, or that the number of donors is the key thing to consider. I think this would be a false conclusion; more donors probably means that each is less central (which I think would also have the benefit of reducing the influence of any given donor) and because the public profile of the donor(s) probably matters a lot. (I do think there are separate ethical issues and considerations involved in taking funding from UHNW donors.)

Lizka 19 Jan 2024 4:45 UTC
9 points
0 ∶ 0
on: Report on the Desirability of Science Given New Biotech Risks
Thanks a bunch for this report! I haven’t had the time to read it very carefully, but I’ve already really enjoyed it and am curating the post.
I’m also sharing some questions I have, my highlights, and my rough understanding of the basic model setup (pulled from my notes as I was skimming the report).
A couple of questions / follow-up discussions
1. I’m curious about why you chose to focus specifically on biological risks.
  1. I expect that it’s usually good to narrow the scope of reports like this and you do outline the scope at the beginning,^[1] but I’d be interested in hearing more about why you didn’t, for instance, focus on risks from AI. (I guess
  2. (For context, in the XPT, risks from AI are believed to be higher than risks from engineered pathogens.)
2. I’d be interested in a follow-up discussion^[2] on this: “My preferred policy stance … is to separately and in parallel pursue reforms that accelerate science and reforms that reduce risks from new technologies, without worrying too much about their interaction (with some likely rare exceptions).”
  1. In particular, I mostly like the proposal, but have some worries. It makes sense to me that it’s generally good to pursue different goals separately^[3] But sometimes^[4] it does turn out that at-first-seemingly-unrelated side-considerations (predictably) swamp the original consideration. Your report is an update against this being the case for scientific progress and biological risk, but I haven’t tried to estimate what your models (and XPT forecasts I suppose) would predict for AI risk.
  2. I also have the intuition that there’s some kind of ~collaborativeness consideration like: if you have goals A and B (and, unlike in this report, you don’t have an agreed-on exchange rate between them), then you should decide to pursue A and B separately only if the difference in outcomes from A’s perspective between B-optimized actions and A-and-B-compromise actions is comparable to or smaller than the outcomes of A-optimized actions.
    To use an example: if I want to have a career that involves traveling while also minimizing my contribution to CO2 levels or something, then I should probably just fly and donate to clean tech or verified carbon offsets or something, because even from the POV of CO2 levels that’s better. But if it turns out that flying does more damage than the change I can make by actively improving CO2 levels, then maybe I should find a way to travel less or err on the side of trains or the like. (I think you can take this too far, but maybe there’s a reasonable ground here.)
    More specifically I’m wondering if we can estimate the impact of AI/biosafety interventions, compared to possible harms.
3. I’m somewhat uncertain about why you model the (biological) time of perils (ToP) the way you do (same with the impact of “pausing science” on ToP).
  1. I was initially most confused about why only the start date of the ToP moves back because of a science-pause, and assumed it was either because you assumed that ToP was indefinite or because it would end for a reason not very affected by the rate of scientific progress. Based on the discussion in one of the sections, I think that’s not quite right? (You also explore the possibility of ToP contracting due to safety-boosting scientific advances, which also seems to contradict my earlier interpretation.)
  2. This also led to me wondering what would happen if the risk grew over the course of ToP a growing risk (d), as opposed to going from 0 to d (e.g. there’s some chance of a new unlocked dangerous biotechnology at any point once ToP starts) — and how that would affect the results? (Maybe you do something like this somewhere!)
Some things that were highlights to me
1. In the comparison of superforecaster and expert answers in XPT, the “Correlated pessimism” consideration was particularly interesting and more compelling than I expected it to be before I read it!
  1. ″...general pessimism and optimism among groups in ways that imply biases. [....] We also see a high degree of correlation in beliefs about catastrophic risk even for categories that seem likely to be uncorrelated. For example, [for the probability that non-anthropogenic causes (e.g. asteroids) cause catastrophes] the third of respondents most concerned about AI risk [...] foresaw a 0.14% chance of such a catastrophe by 2100. The third of respondent least concerned foresaw a 0.01% chance, more than an order of magnitude less. [...]” Then there’s discussion of selection effects — pessimists about catastrophic risks might become experts in the field — and an argument that overly optimistic superforecasters would get corrective feedback that too-pessimistic risk experts might lack.
2. Also in the XPT discussions (S 4.1), I liked the extrapolation for the current/future ~engineered pandemic peril rates and for a date for the onset of the time of perils.^[5]
3. I thought the “leveling the playing field” consideration was useful and something I probably haven’t considered enough, particularly in bio: ”… the faster is scientific progress, the greater is the set of defensive capabilities, relative to offensive ones. Conversely, a slowdown in the rate of scientific progress (which is arguably underway!) reduces safety by “leveling the playing field” between large and small organizations.” (Related: multipolar AI scenarios)
4. Factoids/estimates I thought were interesting independent of the rest of the report:
  1. 56% of life expectancy increases are attributed to science effects in this paper (see S4.8), which, together with average increases in life expectancy means that a year of science increases our lifespans by around 0.261% (i.e. multiply by ~1.0026 every year). (For the impacts of science on utility via increased incomes, the paper estimates a per-year increase of 0.125%.)
  2. “In 2020, for example, roughly 1 in 20-25 papers published was related to Covid-19 in some way (from essentially none in 2019).”
  3. You conclude that: “Roughly half the value comes from technologies that make these people just a tiny bit richer, in every year, for a long time to come. The other half comes from technologies that make people just a tiny bit healthier, again in every year, for a long time to come.” I don’t know if I would have expected the model to predict such an even split.
  4. “Omberg and Tabarrok (2022) [examine] the efficacy of different methods of preparing for a biocatastrophe; specifically, the covid-19 pandemic. The study takes as its starting point the Global Health Security Index, which was completed in 2019, shortly before the onset of the pandemic. This index was designed by a large panel of experts to rate countries on their capacity to prevent and mitigate epidemics and pandemics. Omberg and Tabarrok examine how the index, and various sub-indices of it, are or are not correlated with various metrics of success in the covid-19 pandemic, mostly excess deaths per capita. The main conclusion is that almost none of the indices were correlated with covid-19 responses, whether those metrics related to disease prevention, detection, response, or the capacity of the health system.”
My rough understanding of the basic setup of the main(?) model
(Please let me know if you see an error! I didn’t check this carefully.)
1. Broad notes:
  1. “Time of (biological) perils” is a distinct period, which starts when the annual probability a biocatastrophe jumps.
  2. The primary benefits of science that are considered are income and health boosts (more income means more utility per person per year, better health means lower mortality rates).
  3. I think the quality of science that happens or doesn’t happen at different times is assumed to be ~always average.
2. The “more sophisticated model” described in S3.1 compares total utility (across everyone on earth, from now until eternity) under two scenarios: “status quo” and a year-long science “pause.”
3. Future utility is discounted relative to present/near-future utility, mostly for epistemic reasons (the further out we’re looking, the more uncertainty we have about what the outcomes will be). This is modeled by assuming a constant annual probability that the world totally stops being predictable (it enters a “new epistemic regime”); utility past that point does not depend on whether we accelerate science or not and can be set aside for the purpose of this comparison— see S3.2 and 4.2^[6]
4. Here’s how outcomes differ in the two scenarios, given that context:
  1. In the status quo scenario, current trends continue (this is the baseline).
  2. In the “pause science” scenario, science is “turned off” for a year, which has a delayed impact by:
    Delaying the start of the time of perils by a year (without affecting the end-date of the time of perils, if there is one)
    Slowing growth for the duration of a year starting time T
    I.e. after the pause things continue normally for a while, then slow down for a year, and then go back up after the year has passed. In the model, pausing science slows the decline in mortality for a year but doesn’t affect birth rates. Given some assumptions, this means that pausing science pushes the decline in mortality permanently behind where it should be, so population growth rates slow (forever, with the discount rate p as a caveat) after the effects kick in.
    Note that T is also taken to be the time at which the time of perils would start by default — there’s a section discussing what happens if the time of perils starts earlier or later than T that argues that we should expect pausing science to look worse in either scenario.
1. ^
  S 2.2. “Synthetic biology is not the only technology with the potential to destroy humanity—a short list could also include nuclear weapons, nanotechnology, and geoengineering. But synthetic biology appears to be the most salient at the moment. [...]”
2. ^
  Although I might not be able to respond much/fast in the near future
3. ^
  Most actions targeted at some goal A affect a separate goal B way less than an action that was taken because it targeted goal B would have affected B. (I think this effect is probably stronger if you filter by “top 10% most effective actions targeted at these goals, assuming we believe that there are huge differences in impact.) If you want to spend 100 resource units for goals A and B you should probably just split the resources and target the two things separately instead of trying to find things that look fine for both A and B.
  (I think the “barbell strategy” is a related concept, although I haven’t read much about it.)
4. ^
  (for some reason the thing that comes to mind is this SSC post about marijuana legalization from 2014 — I haven’t read it in forever but remember it striking a chord)
5. ^
  Seems like around the year 2038, the superforecasters expect a doubling of annual mortality rates from engineered pandemics from 0.0021% to 0.0041% — around 1 COVID every 48 years — and a shift from ~0% to ~0.0002%/year extinction risk. The increases are assumed to persist (although there were only forecasts until 2100?).
6. ^
  4.2 is a cool collection of different approaches to identifying a discount rate. Ultimately the author assumes p=0.98, which is on the slightly lower end and which he flags will put more weight on near-term events.
  I think p can also be understood to incorporate a kind of potential “washout” aspect of scientific progress today (if we don’t discover what we would have in 2024, maybe we still mostly catch up in the next few years), although I haven’t thought carefully about it.

Lizka

A couple of questions /​ follow-up discussions

Some things that were highlights to me

My rough understanding of the basic setup of the main(?) model

A couple of questions / follow-up discussions