jackva

Karma: 3,223

jackva 23 Apr 2024 12:44 UTC
13 points
3 ∶ 1
in reply to: Matthew Rendall’s comment on: If You’re Going To Eat Animals, Eat Beef and Dairy
@Vasco Grillo would be well-placed to do the math here, but I have the strong intuition that under most views giving some weight to animal welfare the marginal climate damage from additional beef consumption will be outweighed by animal suffering reduction by a large margin.

jackva 12 Apr 2024 16:31 UTC
7 points
0 ∶ 0
on: Increasing risks of GCRs due to climate change
My sense is that it is not a big priority.
However, I would also caution against the view that expected climate risk has increased over the past years.

Even if impacts are faster than predicted, most GCR-climate risk does probably not come from developments in the 2020s, but on emissions paths over this century.

And the big story there is that the expected cumulative emissions have much decreased (see e.g. here).

As far as I know no one has done the math on this, but I would expect that the decrease in likelihood of high warming futures dominates somewhat higher-than-anticipated warming at lower level of emissions.

jackva 4 Apr 2024 19:44 UTC
16 points
0 ∶ 2
on: Let’s Fund: Impact of our $1M crowdfunded grant to the Center for Clean Energy Innovation
Even if one is skeptical of the detailed numbers of a cost effectiveness analysis like this (as I am), I think it is nonetheless pretty clear that this 1M spent was a pretty great bet:
1. When I talked to ITIF in 2020, they were pretty clear how transformative the Let’s Funds campaign had been for their fundraising.
2. Given the amount of innovation-related decision making that occurred in the run-up to and early Biden administration—what became the IIJA, CHIPS, and IRA, probably the largest expansion of energy innovation activity in decades—significantly strengthening one of the most respected voices on energy innovation seemed clearly very good.
3. ITIF literally co-authored the most detailed blueprint for the Biden energy innovation agenda (Energizing America) and had clear ties into the White House so, conditional on them being funding-constrained (which they perceived themselves to be, see (1)) it seems hard to think there wasn’t a pretty useful way to spend this additional funding.
4. Even if one thinks ITIF shifted zero dollars towards innovation (from other areas), just marginally improving a single decision would quickly make this a great investment.
5. We have lots of evidence from other areas that this kind of philanthropy works and often has large impacts via legislative subsidy and other mechanisms.
6. 2000 smallish donors would not have spent their money better otherwise given how most small climate donors allocate their funds (Big Green etc).
I think there’s a failure mode of looking at a cost-effectiveness model like this and rightly thinking—this is really crude and unbelievable! -- while, in this case, wrongly concluding that this wasn’t a great bet even though it is hard to put into a credible BOTEC.

jackva 1 Apr 2024 12:57 UTC
10 points
1 ∶ 0
on: Illuminatea—A Proposal for EA Reform
“Pyramid scheme” has a new meaning.

jackva 29 Mar 2024 19:26 UTC
5 points
0 ∶ 0
in reply to: Zershaaneh Qureshi’s comment on: Timelines to Transformative AI: an investigation
I am also just beginning to think about this more, but some initial thoughts:
- Path dependency from self-ampliying processes—Thinking about model generations as forks where significant changes in the trajectory become possible (e.g. crowding in a lot more investment, as has happened with ChatGPT/GPT4, but also, as has also happened, a changed Overton window). I think overall this introduces a dynamic where the extremes of the scenario space become more likely, with social dynamics such as strong increase in investment or, on the other side, stricter regulation after a warning shot, having self-amplifying dynamics. As the sums get larger and the public and policy makers pay way more attention, I think the development process will become a lot more contingent (my sense is that you are already thinking about these things at Convergence).
- Modeling domestic and geopolitics—e.g. the Biden and Trump AI policies probably look quite different, as does the outlook for race dynamics (essentially all mentions of artificial intelligence by Project 2025, a Heritage-backed attempt to define priorities for an incoming Republican administration, are about science dominance and/or competition with China, there is no discussion of safety at all).
- Modeling more direct AI progress > AI politics > AI policy > AI progress feedback loops, i.e. based on what we know from past examples or theory, what kind of labor displacement would one need to see to expect serious backlash? what kind of warning shots would likely lead to serious regulation? and similar questions.

jackva 28 Mar 2024 9:46 UTC
17 points
3 ∶ 0
in reply to: MatthewDahlhausen’s comment on: [Linkpost] Leif Wenar’s The Deaths of Effective Altruism
I agree with you that the 2018 report should not have been used as primary evidence for CATF cost-effectiveness for WWOTF (and, IIRC, I advised against it and recommended an argument more based on landdscaping considerations with leverage from advocacy and induced technological change). But this comment is quite misleading with regards to FP’s work as we have discussed before:
1. I am not quite sure what is meant with “referencing it”, but this comment from 2022 in response to one of your earlier claims already discusses that we (FP) have not been using that estimate for anything since at least 2020. This was also discussed in other places before 2022.
2. As discussed in my comment on the Rethink report you cite, correcting the mistakes in the REDD+ analysis was one of the first things I did when joining FP in 2019 and we stopped recommending REDD+ based interventions in 2020. Indeed, I have been publicly arguing against treating REDD+ as cost-effective ever since and the thrust of my comment on the RP report is that they were still too optimistic.

jackva 27 Mar 2024 17:32 UTC
6 points
0 ∶ 0
on: An Overview of the AI Safety Funding Situation
It seems like that this number will increase by 50% once FLI (Foundation) fully comes online as a grantmaker (assuming they spend 10%/year of their USD 500M+ gift)

https://www.politico.com/news/2024/03/25/a-665m-crypto-war-chest-roils-ai-safety-fight-00148621

jackva 26 Mar 2024 22:11 UTC
4 points
0 ∶ 0
in reply to: Arepo’s comment on: EA “Worldviews” Need Rethinking
Interesting, thanks for clarifying!

Just to fully understand—where does that intuition come from? Is it that there is a common structure to high impact? (e.g. if you think APs are good for animals you also think they might be good for climate, because some of the goodness comes from the evidence of modular scalable technologies getting cheap and gaining market share?)

jackva 26 Mar 2024 19:56 UTC
10 points
2 ∶ 1
in reply to: Arepo’s comment on: EA “Worldviews” Need Rethinking
I don’t think these examples illustrate that “bewaring of suspicious convergence” is wrong.
For the two examples I can evaluate (the climate ones), there are co-benefits, but there isn’t full convergence with regards to optimality.

On air pollution, the most effective intervention for climate are not the most effective intervention for air pollution even though decarbonization is good for both.
See e.g. here (where the best intervention for air pollution would be one that has low climate benefits, reducing sulfur in diesel; and I think if that chart were fully scope-sensitive and not made with the intention to showcase co-benefits, the distinctions would probably be larger, e.g. moving from coal to gas is a 15x improvement on air pollution while only a 2x on emissions):

And the reason is that different target metrics (carbon emissions, reduced air pollution mortality) are correlated, but do not map onto each other perfectly and optimizing for one does not maximize the other.

Same thing with alternative proteins, where a strategy focused on reducing animal suffering would likely (depending on moral weights) prioritize APs for chicken, whereas a climate-focused strategy would clearly prioritize APs for beef.
(There’s a separate question here whether alternative proteins are an optimal climate strategy, which I think is not really established).

I think what these examples show is that we often have interventions with correlated benefits and it is worth asking whether one should optimize for both metrics jointly (given that there isn’t really an inherent reason to separate, say, lives lost from climate from lives lost from air pollution it could make sense to prioritize interventions which do not optimize either dimension), but if one decides to optimize for a single dimension not expecting to also accidentally optimize the other dimension (i.e. “bewaring of suspicious convergence”) continues to be good advice (or, more narrowly, is at least not discredited by these examples).

jackva 26 Mar 2024 15:00 UTC
6 points
1 ∶ 0
on: Timelines to Transformative AI: an investigation
Fascinating stuff!

I am curious how you think about integrating social and political feedback loops into timeline forecasts.

Roughly speaking, (a) when we remain in the paradigm of relatively predictable progress (in terms of amount of progress, not specific capabilities) enabled by scaling laws, (b) we put significant probability on being fairly close to TAI, e.g. within 10 years, (c) it remains true that model progress is clearly observable by the broader public,

then it seems that social and political factors might drive a large degree in the variance of expectable timelines (by affecting your (E2)).

E.g. things like (i) Sam Altman seeking to discontinously increase chip supply (ii) How the next jump in capabilities will be perceived, e.g. if it is true that GPT-5 will be another similarly sized jumped compared to what we’ve seen from GPT-3 to GPT-4 what policy and investment responses will this cause? (iii) whether there will be a pre-TAI warming shot that will lead to a significant shift in the Overton Window and more serious regulation.

It seems to me that those dynamics should take up a larger part of the variance the closer we get so I am curious how you think about this and whether you will include this in your upcoming work on short timelines.

jackva 26 Feb 2024 19:32 UTC
4 points
0 ∶ 0
in reply to: Vasco Grilo’s comment on: Nuclear war tail risk has been exaggerated?
but that a priori we should assume diminishing returns in the overall spending, otherwise the government would fund the philanthropic interventions.
I think this is fundamentally the crux—many of the most valuable philanthropic actions in domains with large government spending will likely be about challenging / advising / informationally lobbying the government in a way that governments cannot self-fund.

Indeed, when additional government funding does not reduce risk (does not reduce the importance of the problem) but is affectable, there can probably be cases where you should get more excited about philanthropic funding to leverage as public funding increases.

jackva 26 Feb 2024 9:12 UTC
2 points
0 ∶ 0
in reply to: Ramiro’s comment on: Estimates on expected effects of movement/pressure group/field building?
Yeah, that’s true, though in Luke’s treatment both are discussed and described as roughly equal—there’s no indication given that either should be more promising on priors and, as you say, they will often overlap.

jackva 26 Feb 2024 9:03 UTC
8 points
2 ∶ 1
in reply to: Vasco Grilo’s comment on: Nuclear war tail risk has been exaggerated?
(Last comment from me on this for time reasons)
- I think if you look at philanthropic neglectedness, the total sums across types of capital are not a good proxy. E.g., as far as I understand the nuclear risk landscape, it is both true that government spending is quite large but also that there is almost no civil society spending. This means that additional philanthropic funding should be expected to be quite effective on neglectedness grounds. Many obvious things are not done.
- The numbers on nuclear risk spending by 80k are entirely made up and not described otherwise (e.g. they do not cite a source and make no effort justifying the estimate, this is clearly a wild guess).
- If one constructed a similar number for AI risk, it could also be in the billions given it would presumably include stuff like the costs of government bureaucracies involved in tech regulation, emerging legislation etc.
I am fairly convinced your basic point will stand, but it seems important to not overplay the degree to which nuclear risk is not neglected, and to not underplay the degree to which government actors and others are now paying attention to AI risk (obviously, this also needs to be quality discounted, but this discounting does not reduce the value much for nuclear in your estimate).

jackva 25 Feb 2024 19:50 UTC
5 points
1 ∶ 0
in reply to: Vasco Grilo’s comment on: Nuclear war tail risk has been exaggerated?
I can’t open the GDoc on AI safety research.
But, in any case, I do not think this works, because philanthropic, private, and government dollars are not fungible, as all groups have different advantages and things they can and cannot do.

If looking at all resources, then 80M for AI safety research also seems an underestimate as this presumably does not include the safety and alignment work at companies?

jackva 25 Feb 2024 18:09 UTC
5 points
1 ∶ 1
on: Nuclear war tail risk has been exaggerated?
Nuclear risk philanthropy is about 30M/y, it seems you are comparing overall nuclear risk effort to philanthropic effort for AI?

In terms of philanthropic effort AI risk strongly dominates nuclear risk reduction.

jackva 21 Feb 2024 9:30 UTC
2 points
0 ∶ 0
in reply to: Linch’s comment on: Linch’s Shortform
Sorry for not being super clear in my comment, it was hastily written. Let me try to correct:
I agree with your point that we might not need to invest in govt “do something” under your assumptions (your (1)).
I think the point I disagree with is the implicit suggestion that we are doing much of what would be covered by (1). I think your view is already the default view.
- In my perception, when I look at what we as a community are funding and staffing, > 90% of this is only about (2) -- think tanks and other Beltway type work that is focused on make actors do the right thing, not just salience raising, or, alternatively having these clear conversations.
- Somewhat casually but to make the point, I think your argument would change more if Pause AI sat on 100m to organize AI protests, but we would not fund CSET/FLI/GovAI etc.
- Note that even saying “AI risk is something we should think about as an existential risk” is more about “what to do” than “do something”, it is saying “now that there is this attention to AI driven by ChatGPT, let us make sure that AI policy is not only framed as, say, consumer protection or a misinformation in elections problem, but also as an existential risk issue of the highest importance.”
This is more of an aside, but I think by default we err on the side of too much of “not getting involved deeply into policy, being afraid to make mistakes” and this itself seems very risky to me. Even if we have until 2030 until really critical decisions are to be made, the policy and relationships built now will shape what we can do then (this was laid out more eloquently by Ezra Klein in his AI risk 80k podcast).

jackva 19 Feb 2024 18:20 UTC
3 points
0 ∶ 0
in reply to: Jamie_Harris’s comment on: Estimates on expected effects of movement/pressure group/field building?
Thanks, Jamie! Indeed quite helpful to know that there’s nothing obvious I am missing.

Yes, agree on the last point—I am just surprised this has not been done as EA grant makers frequently face the decision, I think.

jackva 19 Feb 2024 16:48 UTC
2 points
0 ∶ 0
on: Movement Grant applications are now open!
Is there a process for more time-sensitive grants (where a decision would be needed earlier)?

jackva 19 Feb 2024 14:30 UTC
3 points
1 ∶ 0
in reply to: Linch’s comment on: Linch’s Shortform
This seems right to me on labs (conditional on your view being correct), but I am wondering about the government piece—it is clear and unavoidable that government will intervene (indeed, already is) and that AI policy will emerge as a field between now and 2030 and that decisions early on likely have long-lasting effects. So wouldn’t it be extremely important also on your view to now affect how government acts?

jackva 16 Feb 2024 10:18 UTC
2 points
0 ∶ 0
in reply to: Matt_Sharp’s comment on: Estimates on expected effects of movement/pressure group/field building?
Thanks, good shout!

From what I’ve seen, their work does not quite fit what I am looking for—they are not comparative and they are also more narrowly focused on left-leaning protest movements, which is more narrow than what I am trying to get at here.