I am curious, though, about what makes you view capacity building (CB) in a more positive light compared to other interventions within AI safety. As you point out, CB also has the potential to backfire. I would even argue that the downside risk of CB might be higher than that of other interventions because it increases the number of people taking the issue seriously and taking proactive action—often with limited information.
Yeah, just to clarify, CB is not necessarily better than other interventions. However, CB with low backfire risks could be promising. This does not necessarily mean doing community building, since community building could backfire depending on how it is done (for example maybe if it is done in a very expansive non-careful way it could more easily backfire). I think the PauseAI example that you gave is a good example of potentially non robust intervention, or at least I would not count it as a low backfire risk capacity building intervention.
One of the motivation of CB would be to put ourselves in a better position to pursue some intervention if we end up less clueless. It might be that we don’t in fact end up less clueless, and that while we have done CB, there are still no robust interventions that we can pursue after some time. In that case, it would be better to pursue determinately good short-term interventions even after doing CB (but then we have to pay the opportunity cost of the resources spent doing CB rather than doing the interventions good in the short term directly).
I am still uncertain about low backfire CB interventions (that are better than doing something good directly), perhaps some way of increasing capital or well targeted community building could be good examples, but it seems like an open question to me.
Bracketing is the most interesting recent idea in altruistic decision-making.
However, from the point of view of a sequential decision problem, an EV-maxing learner is more willing to make guesses about long-term, high-stakes effects and to learn from them, whereas a bracketing agent restricts itself to actions it is non-clueless about and may never learn about domains it brackets out–including the value of information itself. In this sense, bracketing seems to build a form of risk-aversion: it avoids speculative bets and forgoes opportunities to learn and adapt.
In practice, the main downside I see is that bracketers risk moving away from gaining influence over high-stakes uncertain problems and instead prioritize low-stakes neartermist interventions because of cluelessness about higher-stakes ones. As a result, bracketers may be overly conservative and lose influence relative to EV-maxers, who are more willing to make provisional guesses, risk being wrong, and update along the way.