Research scholar @ FHI and assistant to Toby Ord. Philosophy student before that. I do a podcast about EA called Hear This Idea.
finm
Karma: 1,519
Sounds to me like that would count! Perhaps you could submit the entire sequence but highlight the critical posts.
Replying in personal capacity:
I hope the contest will consider lower effort but insightful or impactful submissions to account for this?
Yes, very short submissions count. And so should “low effort” posts, in the sense of “I have a criticism I’ve thought through, but I don’t have time to put together a meticulous writeup, so I can either write something short/scrappy, or nothing at all.” I’d much rather see unpolished ideas than nothing at all.
Secondly, I’d expect people with the most valuable critiques to be more outside EA since I would expect to find blindspots in the particular way of thinking, arguing and knowing EA uses. What will the panelists do to ensure they can access pieces using a very different style of argument? Have you considered having non-EA panelists to aid with this?
Thanks, I think this is important.
We (co-posters) are proactively sharing this contest with non-EA circles (e.g.), and others should feel welcome and encouraged to do the same.
Note the incentives for referring posts from outside the Forum. This can and should include writing that was not written with this contest in mind. It could also include writing aimed at some idea associated with EA that doesn’t itself mention “effective altruism”.
It obviously shouldn’t be a requirement that submissions use EA jargon.
I do think writing a post roughly in line with the Forum guidelines (e.g. trying to be clear and transparent in your reasoning) means the post will be more likely to get understood and acted on. As such, I do think it makes sense to encourage this manner of writing where possible, but it’s not a hard requirement.
To this end, one idea might be to speak to someone who is more ‘fluent’ in modes of thinking associated with effective altruism, and to frame the submission as a dialogue or collaboration.
But that shouldn’t be a requirement either. In cases where the style of argument is unfamiliar, but the argument itself seems potentially really good, we’ll make the effort — such as by reaching out to the author for clarifications or a call. I hope there are few really important points that cannot be communicated through just having a conversation!
I’m curious which non-EA judges you would have liked to see! We went with EA judges (i) to credibly show that representatives for big EA stakeholders are invested in this, and (ii) because people with a lot of context on specific parts of EA seem best placed to spot which critiques are most underrated. I’m also not confident that every member of the panel would strongly identify as an “effective altruist”, though I appreciate connection to EA comes in degrees.
Thirdly, criticisms from outside of EA might also contain mistakes about the movement but nonetheless make valid arguments. I hope this can be taken into account and such pieces not just dismissed.
Yes. We’ll try to be charitable in looking for important insights, and and forgiving of innacuracies from missing context where they don’t affect the main argument.
That said, it does seem straightforwardly useful to avoid factual errors that can easily be resolved with public information, because that’s good practice in general.
What plans do you have in place to help prevent and mitigate backlash[?]
My guess is that the best plan is going to be very context specific. If you have concerns in this direction, you can email criticism-contest@effectivealtruism.com, and we will consider steps to help, such as by liaising with the community health team at CEA. I can also imagine cases where you just want to communicate a criticism privately and directly to someone. Let us know, and we can arrange for that to happen also (“we” meaning myself, Lizka, or Joshua).
Announcing a contest: EA Criticism and Red Teaming
Resource for criticisms and red teaming
Just commenting to say this was a really useful resource summarising an important topic — thanks for the time you put into it!
This (and your other comments) is incredibly useful, thanks so much. Not going to respond to particular points right now, other than to say many of them stick out as well worth pursuing.
Thanks for this, I think I agree with the broad point you’re making.
That is, I agree that basically all the worlds in which space ends up really mattering this century are worlds in which we get transformative AI (because scenarios in which we start to settle widely and quickly are scenarios in which we get TAI). So, for instance, I agree that there doesn’t seem to be much value in accelerating progress on space technology. And I also agree that getting alignment right is basically a prerequisite to any of the longer-term ‘flowthrough’ considerations.
If I’m reading you right I don’t think your points apply to near-term considerations, such as from arms control in space.
It seems like a crux is something like: how much precedent-setting or preliminary research now on ideal governance setups doesn’t get washed out once TAI arrives, conditional on solving alignment? And my answer is something like: sure, probably not a ton. But if you have a reason to be confident that none of it ends up being useful, it feels like that must be a general reason for scepticism that any kind of efforts at improving governance, or even values change, are rendered moot by the arrival of TAI. And I’m not fully sceptical about those efforts.
Suppose before TAI arrived we came to a strong conclusion: e.g. we’re confident we don’t want to settle using such-and-such a method, or we’re confident we shouldn’t immediately embark on a mission to settle space once TAI arrives. What’s the chance that work ends up making a counterfactual difference, once TAI arrives? Notquite zero, it seems to me.
So I am indeed on balance significantly less excited about working on long-term space governance things than on alignment and AI governance, for the reasons you give. But not so much that they don’t seem worth mentioning.
Ultimately, I’d really like to see [...] More up-front emphasis on the importance of AI alignment as a potential determinant.
This seems like a reasonable point, and one I was/am cognisant of — maybe I’ll make an addition if I get time.
(Happy to try saying more about any of above if useful)
Space governance—problem profile
I agree that fusion is feasible and will likely account for a large fraction (>20%) of energy supply by the end of the century, if all goes well. I agree that would be pretty great. And yeah, Helion looks promising.
But I don’t think we should be updating much on headlines about achieving ignition or breakeven soon. In particular, I don’t think these headlines should be significantly shifting forecasts like this one from Metaculus about timelines to >10% of energy supply coming from fusion. The main reason is that there is a very large gap between proof of concept and a cost-competitive supply of energy. Generally speaking, solar will probably remain cheaper per kWh than fusion for a long time (decades), so I don’t expect the transition to be very fast.
It’s also unclear what this should all mean for EA. One response could be: “Wow, a world with abundant energy would be amazing, we should prioritise trying to accelerate the arrival of that world.” But, I don’t know, there’s already a lot of interested capital flying around — it’s not like investors are naive to the benefits. On the government side, the bill for ITER alone was something in the order of $20 billion.
Another response could be: “Fusion is going to arrive sooner than we expected, so the world is soon going to look different from what we expected!” And I’d probably just dispute that the crowd (e.g. the Metaculus forecast above) is getting it especially wrong here in any action-relevant way. But I’d be delighted to be proved wrong.
Thanks, that’s a very good example.
I don’t think this actually describes the curve of EA impact per $ overall
For sure.
Concave and convex altruism
Just wanted to comment that this was a really thoughtful and enjoyable post. I learned a lot.
In particular, I loved the way the point about how the relative value of trajctory change should depend on the smoothness of your probability distribution over the value of the long-run future.
I’m also now curious to know more about the contingency of the caste system in India. My (original) impression was that the formation of the caste system was somewhat gradual and not especially contingent.
For what it’s worth I think I basically endorse that comment.
I definitely think an investigation that starts with a questioning attitude, and ends up less negative than the author’s initial priors, should count.
That said, some people probably do already just have useful, considered critiques in their heads that they just need to write out. It’d be good to hear them.
Also, presumably (convincing) negative conclusions for key claims are more informationally valuable than confirmatory ones, so it makes sense to explicitly encourage the kind of investigations that have the best chance of yielding those conclusions (because the claims they address look under-scrutinised).
Thank you, this is a really good point. By ‘critical’ I definitely intended to convey something more like “beginning with a critical mindset” (per JackM’s comment) and less like “definitely ending with a negative conclusion in cases where you’re critically assessing a claim you’re initially unsure about”.
This might not always be relevant. For instance, you might set out to find the strongest case against some claim, whether or not you end up endorsing it. As long as that’s explicit, it seems fine.
But in cases where someone is embarking on something like a minimal-trust investigation — approaching an uncertain claim from first principles — we should be incentivising the process, not the conclusion!
We’ll try to make sure to be clear about that in the proper announcement.
Yes, totally. I think a bunch of the ideas in the comments on that post would be a great fit for this contest.
Thanks, great points. I agree that we should only be interested in good faith arguments — we should be clear about that in the judging criteria, and clear about what counts as a bad faith criticism. I think the Forum guidelines are really good on this.
Of course, it is possible to strongly disagree with a claim without resorting to bad faith arguments, and I’m hopeful that the best entrants can lead by example.
Pre-announcing a contest for critiques and red teaming
The downweighting of AI in DGB was a deliberate choice for an introductory text.
Thanks, that’s useful to know.
I guess that kind of confirms the complaint that there isn’t an obvious, popular book to recommend on the topic!
Thanks very much for writing this — I’m inclined to agree that results from the happiness literature are often surprising and underrated for finding promising neartermist interventions and thinking about the value of economic growth. I also enjoyed hearing this talk in person!
The “aren’t people’s scales adjusting over time?” story (‘scale norming’) is most compelling to me, and I think I’m less sure that we can rule it out. For instance — if I’m reading you right, you suggest that one reason to be skeptical that people are adjusting their scales over time is that people mostly agree on which adjectives like “good” correspond with which numerical scores of wellbeing. This doesn’t strike me as strong evidence that people are not scale norming, since I wouldn’t be surprised if people adjust the rough meaning of adjectives roughly in line with numbers.
I don’t see a dichotomy between “people use the same scales across time and context for both words and adjectives” and “people view this task as meaningless”.
You also suggest a story about what people are doing when they come up SWB scores, which if true leaves little room for scale norming/adjustment. And since (again, if I’m reading you right) this story seems independently plausible, we have an independently plausible reason to be skeptical that scale norming is occurring. Here’s the story:
I think I don’t find this line of argument super compelling, and not even because I strongly disagree with that excerpt. Rather: the excerpt underdetermines what function you use to project from an extremely wide space onto a bounded scale, and there is no obvious such ‘Schelling’ function (i.e. I don’t even know what it would mean for your function to be linear). And indeed people could change functions over time while keeping those 0 and 10 pegs fixed. Another thing that could be going on is that people might be considering how to make their score informationally valuable, which might involve imagining what kind of function would give a relatively even spread across 0–10 when used population-wide. I don’t think this is primarily what is going on, but to the extent that it is, such a consideration would make a person’s scale more relative to the population they understand themselves to be part of[1], and as such to re-adjust over time.
Two extra things: (i) in general I strongly agree that this question (about how people’s SWB scales adjust across time or contexts) is important and understudied, and (ii) having spoken with you and read your stuff I’ve become relatively less confident in scale-norming as a primary explanation of all this stuff.
I would change my mind more fully that scale norming is not occuring if I saw evidence that experience-sampling type measures of affect also did not change over the course of decades as countries become/became wealthier (and earned more leisure time etc). I’d also change my mind if I saw some experiment where people were asked to rate how their lives were going in relation to some shared reference point(s), such as other people’s lives descibed in a good amount of detail, and where people’s ratings of how their lives were going relative to those reference points also didn’t change as countries became significantly wealthier.
(Caveat to all of above that I’m writing in a hurry!)
If almost everyone falls between 6–7 on the widest scale I can imagine, maybe the scale I actually use should significantly zoom in on that region.