Vael Gates Jan 4, 2023, 11:07 PM
1 point
0 ∶ 0
in reply to: wANIEL’s comment on: What AI Safety Materials Do ML Researchers Find Compelling?
No, the same set of ~28 authors read all of the readings.

The order of the readings was indeed specified:
1. Concise overview (Stuart Russell, Sam Bowman; 30 minutes)
2. Different styles of thinking about future AI systems (Jacob Steinhardt; 30 minutes)
3. A more in-depth argument for highly advanced AI being a serious risk (Joe Carlsmith; 30 minutes)
4. A more detailed description of how deep learning models could become dangerously “misaligned” and why this might be difficult to solve with current ML techniques (Ajeya Cotra; 30 minutes)
5. An overview of different research directions (Paul Christiano; 30 minutes)
6. A study of what ML researchers think about these issues (Vael Gates; 45 minutes)
7. Some common misconceptions (John Schulman; 15 minutes)
Researchers had the option to read the transcripts where transcripts were available; we said that consuming the content in either form (video or transcript) was fine.

Vael Gates Dec 30, 2022, 3:50 AM
16 points
3 ∶ 0
on: EA Global 2023 applications are now open!
I would love a way to interface with EAGs where (I pay no money, but) I have access to the Swapcard interface and I talk only with people who request meetings with me. I often want to “attend” EAGs in this way, where I don’t interface with the conference but I’m available as a resource if people want to talk to me, for which I will schedule remote 1:1s over Zoom. It’d be nice to be helpful to people at a time where they’re available and can see I’m available on Swapcard. Are there any kind of “virtual, restricted” options like this?

Vael Gates Dec 29, 2022, 10:41 PM
7 points
0 ∶ 0
in reply to: Vasco Grilo🔸’s comment on: What AI Safety Materials Do ML Researchers Find Compelling?
Nice, yeah! I wouldn’t have expected a statistically significant difference between a mean of 5.7 and 5.4 with those standard errors, but it’s nice to see it here.

I considered doing a statistical test, and then spent some time googling how to do something like a “3-paired” ANOVA on data that looks like (“s” is subject, “r” is reading):

[s1 r1 “like”] [s1 r1 “agreement”] [s1 r1 “informative”]
[s2 r1 “like”] [s2 r1 “agreement”] [s2 r1 “informative”]
… [s28 r1 “like”] [s28 r1 “agreement”] [s28 r1 “informative”]
[s1 r2 “like”] [s1 r2″agreement”] [s1 r2 “informative”]
[s2 r2 “like”] [s2 r2 “agreement”] [s2 r2 “informative”]
...
because I’d like to do an ANOVA on the raw scores, rather than the means. I did not resolve my confusion about about what to do about the 3-paired data (I guess you could lump each subject’s data in one column, or do it separately by “like”, “agreement”, and “informative”, but I’m interested in how good each of the readings are summed across the three metrics). I then gave up and just presented the summary statistics. (You can extract the raw scores from the Appendix if you put some work into it though, or I could pass along the raw scores, or you could tell me how to do this sort of analysis in Python if you wanted me to do it!)

When I look at these tables, I’m also usually squinting at the median rather than mean, though I look at both. You can see the distributions in the Appendix, which I like even better. But point taken about how it’d be nice to have stats.

Vael Gates Dec 29, 2022, 10:25 PM
1 point
0 ∶ 0
in reply to: technicalities’s comment on: What AI Safety Materials Do ML Researchers Find Compelling?
The authors were not anonymized, no.

Vael Gates Dec 29, 2022, 7:19 AM
7 points
4 ∶ 0
on: EAGx application community norms we’d like to see
This is not directly related, but I would love a way to interface with EAGs where (I pay no money, but) I have access to the Swapcard interface and I talk only with people who request meetings with me. I often want to “attend” EAGs in this way, where I don’t interface with the conference (physically or virtually) but I’m available as a resource if people want to talk to me, for which I will schedule remote 1:1s over Zoom. It’d be nice to be helpful to people at a time where they’re available and can see I’m available on Swapcard, but I don’t want to pay to do this (nor make the event organizers pay for my non-physical presence if I do end up applying—I usually want a “virtual only” ticket).
What links here?
- Proposed improvements to EAG(x) admissions process by ES (Jan 30, 2023, 6:10 AM; 62 points)

Vael Gates Dec 29, 2022, 1:02 AM
1 point
0 ∶ 0
in reply to: CEvans’s comment on: What AI Safety Materials Do ML Researchers Find Compelling?
I’m not going to comment too much here, but if you haven’t seen my talk (“Researcher Perceptions of Current and Future AI” (first 48m; skip the Q&A) (Transcript)), I’d recommend it! Specifically, you want the timechunk 23m-48m in that talk, when I’m talking about the results of interviewing ~100 researchers about AI safety arguments. We’re going to publish much more on this interview data within the next month or so, but the major results are there, which describes some AI researchers cruxes.

Vael Gates Dec 29, 2022, 12:47 AM
3 points
0 ∶ 0
in reply to: aog’s comment on: What AI Safety Materials Do ML Researchers Find Compelling?
(in response to the technical questions)

Mostly n=28 for each document, some had n =29 or n= 30; you can see details in the Appendix, quantitative section.

The Carlsmith link is to the Youtube talk version, not the full report—we chose materials based on them being pretty short.

What AI Safety Materials Do ML Researchers Find Compelling?

Vael GatesDec 28, 2022, 2:03 AM

130 points

12 comments EA link

Vael Gates Dec 6, 2022, 10:04 PM
1 point
0 ∶ 0
on: Announcing the AI Safety Field Building Hub, a new effort to provide AISFB projects, mentorship, and funding
Keeping a running list of field-building posts I personally want to keep track of:

Project ideas:
- Akash’s: https://forum.effectivealtruism.org/posts/yoP2PN5zdi4EAxdGA/ai-safety-field-building-projects-i-d-like-to-see
- Ryan’s: https://www.lesswrong.com/posts/v5z6rDuFPKM5dLpz8/probably-good-projects-for-the-ai-safety-ecosystem

Survey analysis:
- Ash’s: https://forum.effectivealtruism.org/posts/SuvMZgc4M8FziSvur/analysis-of-ai-safety-surveys-for-field-building-insights
- Daniel Filan’s: https://www.lesswrong.com/posts/rXSBvSKvKdaNkhLeJ/takeaways-from-a-survey-on-ai-alignment-resources
- https://arxiv.org/abs/2208.12852 (Bowman NLP Survey)
- AI Impacts surveys generally, 2022 and 2016, also GovAI has some

Presentation of ideas: (though this should also be updated in https://www.lesswrong.com/posts/gdyfJE3noRFSs373q/resources-i-send-to-ai-researchers-about-ai-safety )
- Marius’s: https://forum.effectivealtruism.org/posts/8JazqnCNrkJtK2Bx4/why-eas-are-skeptical-about-ai-safety
- Lukas’s: https://forum.effectivealtruism.org/posts/8JazqnCNrkJtK2Bx4/why-eas-are-skeptical-about-ai-safety
- Mine: https://forum.effectivealtruism.org/posts/q49obZkQujkYmnFWY/vael-gates-risks-from-advanced-ai-june-2022 // https://www.lesswrong.com/posts/LfHWhcfK92qh2nwku/transcripts-of-interviews-with-ai-researchers

Vael Gates Nov 22, 2022, 5:02 AM
6 points
0 ∶ 0
in reply to: Falk Lieder’s comment on: Open Phil is seeking applications from grantees impacted by recent events
The application form is actually really restrictive once you open it—when I filled it out, it explicitly instructed not to write any new material and only attach old material that was sent to FTXFF, and only had a <20 word box and <150 word box for grant descriptions. Today when I open the form even those boxes have disappeared. I think it’s meant to be a quite quick form, where they’ll reach out for more details later.

Vael Gates Nov 22, 2022, 4:44 AM
1 point
0 ∶ 0
on: What is the best article to introduce someone to AI safety for the first time?
Just updated my post on this: https://forum.effectivealtruism.org/posts/8sAzgNcssH3mdb8ya/resources-i-send-to-ai-researchers-about-ai-safety

I have different recommendations for ML researchers / the public / proto-EAs (people are more or less skeptical to begin with, rely on different kinds of evidence, and are willing to entertain weirder or more normative hypotheses), but that post covers some introductory materials.

If they’re a somewhat skeptical ML researcher and looking for introductory material, my top recommendation at the moment is “Why I Think More NLP Researchers Should Engage with AI Safety Concerns” by Sam Bowman (2022), 15m (Note: stop at the section “The new lab”)

Vael Gates Nov 2, 2022, 10:20 PM
1 point
0 ∶ 0
on: Vael Gates’s Shortform
(quick reply to a private doc on interaction effects vs direct effects for existential risks / GCR. They’re arguing for more of a focus on interaction effects overall, I’m arguing for mostly work on direct effects. Keeping for my notes.)

In addition to direct effects from AI, bio, nuclear, climate...

...there are also mitigating / interaction effects, which could make these direct effects better or worse. For each of the possible direct risks, mitigating / interaction effects are more or less important.

For AI, the mainline direct risks that are possibly existential (whether or not the risks occur) are both really bad and also roughly independent of things that don’t reduce the risk of AGI being developed, and it’s possible to work directly on the direct risks (technical and governance). e.g. Misinformation or disinformation won’t particularly matter if we get an unaligned AGI developed at one of the big companies, except to the extent that mis/disinformation contributed to the unaligned AGI getting developed (which I think is probably more influenced by other factors, but it’s debatable), so I think efforts should be focused on solving the technical alignment problem. Misinformation and disinformation are more relevant to the governance problem, but working on them still seems worse to me than a focus on specifically trying to make AI governance go well, given the goal of trying to reduce existential risk (and not solve other important problems associated with misinformation and disinformation). (I expect one will make more progress trying to directly reduce xrisk from AI than working on related things.)

(Bio has excellent interaction effects with AI in terms of risk though, so that’s going to be one of the best examples. )

Just to quickly go over my intuitions here about the interactions:

AI x bio <-- re: AI, I think AI direct effects are worse
AI x nuclear <-- these are pretty separate problems imo
AI x climate <-- could go either way; I expect AI could have substantial improvements on climate depending on how advanced our AI gets. AI doesn’t contribute that much to climate change compared to other factors I think
Bio x AI <-- bio is SO MUCH WORSE with AI, this is an important interaction effect
Bio x nuclear <-- these are pretty separate problems imo
Bio x climate <-- worse climate will make pandemics worse for sure
Nuclear x AI <-- separate problems
Nuclear x bio <-- separate problems
Nuclear x climate <-- if climate influences war then climate makes nuclear worse
Climate x AI <-- could go either way I think but I think probably best to work directly on climate if you don’t think we’ll get advanced AI systems soon
Climate x nuclear <-- nuclear stuff certainly does mess up climate a LOT, but then we’re thinking more about nuclear
Climate x bio <-- pandemics don’t influence climate that much I think

----

Feedback from another EA (thank you!)

> I think there are more interaction effects than your shortform is implying, but also most lines of inquiry aren’t very productive. [Agree in the general direction, but object-level disagree]

I think this is true and if presented arguments I’d agree with them and have a fairer / more comprehensive picture.

Vael Gates Oct 19, 2022, 3:35 AM
4 points
0 ∶ 0
on: Vael Gates’s Shortform
Continual investment argument for why AGI will probably happen, absent major societal catastrophes, written informally, for my notes:

We’ve been working on AI since ~1950s, in an era of history that feels normal to us but in fact develops technologies very very very fast compared to most of human existence. In 2012, the deep learning revolution of AI started with AlexNet and GPUs. Deep learning has made progress even faster than the current very fast rate of progress: 10 years later, we have unprecedented and unpredicted progress in large language model systems like GPT-3, which have unusual emergent capabilities (text generation, translation, coding, math) for being trained on the next token of a language sequence. One can imagine that if we continue to pour in resources like training data and compute (as many companies are), continue to see algorithmic improvements at the rate we’ve seen, and continue to see hardware improvements (e.g. optical computing), then maybe humanity develops something like AGI or AI at very high levels of capabilities.
Even if we don’t see this progress with deep learning and need a paradigm shift, there’s still an immense amount of human investment being poured into AI in terms of talent, money from private investors + government + company profits, and resources. There’s international competition to develop AI fast, there’s immense economic incentives to make AI products that make our lives ever more convenient along with other benefits, and some of the leading companies (DeepMind, OpenAI) are explicitly aiming at AGI. Given that we’ve only been working on AI since the 1950s, and the major recent progress has been in the last 10 years, and the pace of technological innovation seems very fast or accelerating with worldwide investment, it seems likely we will alive at advanced AI someday, and that someday could be well within our lifetimes, pending major societal disruption.

Vael Gates Oct 12, 2022, 4:38 AM
2 points
0 ∶ 0
on: Levelling Up in AI Safety Research Engineering
Thanks Gabriel—super useful step-by-step guide, and also knowledge/skill clarification structure! I usually gesture around vaguely when talking about my skills (I lose track of how much I know compared to others—the answer is I clearly completed Levels 1-3 then stopped) and trying to hire other people with related skills. It feels useful to be able to say to someone e.g. “For this position, I want you to have completed Level 1 and have a very surface level grasp of Levels 2-4”!

Vael Gates Oct 4, 2022, 10:30 PM
3 points
0 ∶ 0
in reply to: Joseph Bloom’s comment on: Transcripts of interviews with AI researchers
Still in progress as always, but this talk covers a lot of it! https://forum.effectivealtruism.org/posts/q49obZkQujkYmnFWY/vael-gates-risks-from-advanced-ai-june-2022

(Unfortunately the part about the insights isn’t transcribed—the first 20m is introduction, next ~30m is the description you want, last 10m is questions)

Vael Gates

Ret­ro­spec­tive on the AI Safety Field Build­ing Hub

“AI Risk Dis­cus­sions” web­site: Ex­plor­ing in­ter­views from 97 AI Researchers

Pre­dict­ing re­searcher in­ter­est in AI alignment

Re­sources I send to AI re­searchers about AI safety

What AI Safety Ma­te­ri­als Do ML Re­searchers Find Com­pel­ling?

Retrospective on the AI Safety Field Building Hub

“AI Risk Discussions” website: Exploring interviews from 97 AI Researchers

Predicting researcher interest in AI alignment

Resources I send to AI researchers about AI safety

What AI Safety Materials Do ML Researchers Find Compelling?