Vael Gates

Karma: 1,151

Vael Gates 4 Sep 2023 21:46 UTC
27 points
6 ∶ 1
on: How much do EAGs cost (and why)?
This is sideways to the main point in the post, but I’m interested in a ticket type that’s just “Swapcard / unsupported virtual attendee” where accepted people just get access to Swapcard, which lets them schedule 1-1 online videoconferencing, and that’s it.

I find a lot of the value of EAG is in 1-1s, and I’d hope that this would be an option where virtual attendees can get potentially lots of networking value for very little cost.

(Asking because I don’t want to pay a lot of money to attend an EAG where I’d mostly be taking on a mentor role, but I would potentially be happy to do some online 1-1s with people during a Schelling time.)

Update: Just learned about EAGxVirtual, which seems very relevant!

Vael Gates 19 Apr 2022 22:37 UTC
18 points
0 ∶ 0
on: What psychological traits predict interest in effective altruism?
I just did a fast-and-dirty version of this study with some of the students I’m TAing for, in a freshman class at Stanford called “Preventing Human Extinction”. No promises I got all the details right, in either the survey or the analysis.
—————————————————————————————————
QUICK SUMMARY OF DATA FROM https://forum.effectivealtruism.org/posts/7f3sq7ZHcRsaBBeMD/what-psychological-traits-predict-interest-in-effective
MTurkers (n=~250, having a hard time extracting it from 1-3? different samples):
- expansive altruism (M = 4.4, SD = 1.1)
- effectiveness-focus scale (M = 4.4, SD = 1.1)
- 49% of MTurkers had a mean score of 4+ on both scales
- 14% had a mean score of 5+ on both scales
- 3% had a mean score of 6+ on both scales
NYU students (n=96)
- expansive altruism (M = 4.1, SD = 1.1)
- effectiveness-focus (M = 4.3, SD = 1.1)
- 39% of NYU students had a mean score of 4+ on both scales
- 6% had a mean score of 5+ on both scales
- 2% had a mean score of 6+ on both scales
EAs (n=226):
- expansive altruism (M = 5.6, SD = 0.9)
- effectiveness-focus (M = 6.0, SD = 0.8)
- 95% of effective altruist participants had a mean score of 4+ on both scales
- 81% had a mean score of 5+ on both scales
- 33% had a mean score of 6+ on both scales
——————————————————————————————————
VAEL RESULTS:
Vael personally:
- Expansive altruism: 4.2
- Effectiveness-focus: 6.3
Vael sample (Stanford freshman taking a class called “Preventing Human Extinction” in 2022, n=27 included, removed one for lack of engagement)
- expansive altruism (M = 4.2, SD = 1.0)
- effectiveness-focus (M = 4.3, SD = 1.0)
- 48% of Vael sample participants had a mean score of 4+ on both scales,
− 4% had a mean score of 5+ on both scales,
− 0% had a mean score of 6+ on both scales
——————————————————————————————————
Survey link is here: https://docs.google.com/forms/d/e/1FAIpQLSeY-cFioo7SLMDuHx1w4Rll6pwuRnenvjJOfi1z8WCNNwCBiA/viewform?usp=sf_link
Data is here: https://drive.google.com/file/d/1SFLH4bGC-j0nGuy315z_HH4LwdNAiusa/view?usp=sharing
And Excel apparently didn’t decide to save the formulas, gah. Formulas at the bottom are: =AVERAGE(K3:K29), =STDEV(K3:K29), =AVERAGE(R3:R29), =STDEV(R3:R29), =COUNTIF(V3:V29, TRUE)/COUNTA(V3:V29), =COUNTIF(W3:W29, TRUE)/COUNTA(W3:W29), =COUNTIF(X3:X29, TRUE)/COUNTA(X3:X29) and the other formulas are: =AND(K3>4,R3>4), =AND(K3>5,R3>5), =AND(K3>6,R3>6) dragged down through the rest of the columns

Vael Gates 8 Nov 2021 2:44 UTC
18 points
0 ∶ 0
on: Vael Gates’s Shortform
(How to independent study)

Stephen Casper (https://stephencasper.com/) was giving advice today in how to upskill in research, and suggested doing a “deep dive”.

Deep dive: read 40-50 papers in a specific research area you’re interested in going into (e.g. adversarial examples in deep NNs). Take notes on each paper. You’ll then have comparable knowledge to people working in the area, after which you do a synthesis project at the end where you write something up (could be lit review, could be more original than that).

He said he’d trade any class he’d ever taken for one of these deep dives, and they’re worth doing even if it takes like 4 months.

*cool idea

Vael Gates 30 Dec 2022 3:50 UTC
16 points
3 ∶ 0
on: EA Global 2023 applications are now open!
I would love a way to interface with EAGs where (I pay no money, but) I have access to the Swapcard interface and I talk only with people who request meetings with me. I often want to “attend” EAGs in this way, where I don’t interface with the conference but I’m available as a resource if people want to talk to me, for which I will schedule remote 1:1s over Zoom. It’d be nice to be helpful to people at a time where they’re available and can see I’m available on Swapcard. Are there any kind of “virtual, restricted” options like this?

Vael Gates 8 Nov 2021 23:09 UTC
9 points
0 ∶ 0
in reply to: Miranda_Zhang’s comment on: Vael Gates’s Shortform
I think classes are great given they’re targeting something you want to learn, and you’re not uncommonly self-motivated. They add a lot of structure and force engagement (i.e. homework, problem sets) in a way that’s hard to find time / energy for by yourself. You also get a fair amount of guidance and scaffolding information, plus information presented in a pedagogical order! With a lot of variance due to the skill and time investment of the instructor, size of class and quality of the curriculum etc.

But if you DO happen to be very self-driven, know what you want to learn, and if in a research context if you’re the type of person who is capable of generating novel insights without much guidance, then heck yes classes are inefficient. Even if you’re not all of these things, it certainly seems worth trying to see if you can be, since self-learning is so accessible and one learns a lot by being focusedly confused. I like how neatly presented the above deep dives idea is: it feels like it gives me enough structure to have a handle on it and makes it feel unusually feasible to do.

But yeah, for the people who are best at deep dives, I imagine it’s hard for any class to match, even with how high-variance classes can be :).

Vael Gates 15 Feb 2024 23:56 UTC
8 points
0 ∶ 1
on: Offering AI safety support calls for ML professionals
FAQ
This is cool! Why haven’t I heard of this?
Arkose has been in soft-launch for a while, and we’ve been focused on email outreach more than public comms. But we’re increasingly public, and are in communication with other AI safety fieldbuilding organizations!

How big is the team? 
3 people: Zach Thomas and Audra Zook are doing an excellent job in operations, and I’m the founder.
How do you pronounce “Arkose”? Where did the name come from?
 I think whatever pronunciation is fine, and it’s the name of a rock. We have an SEO goal for arkose.org to surpass the rock’s Wikipedia page.

Where does your funding come from?
The Survival and Flourishing Fund.

 Are you kind of like the 80,000 Hours 1-1 team?
Yes, in that we also do 1-1 support calls, and that there are many people for whom it’d make sense to do a call with both 80,000 Hours and Arkose! One key difference is that Arkose is aiming to specifically support mid-career people interested in getting more involved in technical AI safety.

I’m not a mid-career person, but I’d still be interested in a call with you. Should I request a call?
Regretfully no, since we’re currently focusing on professors, PhD students, or industry researcher or engineers who have AI / ML experience. This may expand in the future, but we’ll probably still be pretty focused on mid-career folks.

Is Arkose’s Resource page special in any way?
Generally, our resources are selected to be most helpful to professors, PhD students, and industry professionals, which is a different focus than most other resource lists. We also think arkose.org/papers is pretty cool: it’s a list of AI safety papers that you can filter by topic area. It’s still in development and we’ll be updating it over time (and if you’d like to help, please contact Vael!)

How can I help?
• If you know someone who might be a good fit for a call with Arkose, please pass along arkose.org to them! Or fill out our referral form.
• If you have machine learning expertise and would like to help us review our resources (for free or for pay), please contact vael@arkose.org.

Thanks everyone!

Vael Gates 15 Mar 2023 0:06 UTC
8 points
1 ∶ 0
on: “AI Risk Discussions” website: Exploring interviews from 97 AI Researchers
Small update: Two authors gave me permission to publish their transcripts non-anonymously!
Interview with Michael L. Littman
Interview with David Duvenaud

Vael Gates 29 Dec 2022 22:41 UTC
7 points
0 ∶ 0
in reply to: Vasco Grilo’s comment on: What AI Safety Materials Do ML Researchers Find Compelling?
Nice, yeah! I wouldn’t have expected a statistically significant difference between a mean of 5.7 and 5.4 with those standard errors, but it’s nice to see it here.

I considered doing a statistical test, and then spent some time googling how to do something like a “3-paired” ANOVA on data that looks like (“s” is subject, “r” is reading):

[s1 r1 “like”] [s1 r1 “agreement”] [s1 r1 “informative”]
[s2 r1 “like”] [s2 r1 “agreement”] [s2 r1 “informative”]
… [s28 r1 “like”] [s28 r1 “agreement”] [s28 r1 “informative”]
[s1 r2 “like”] [s1 r2″agreement”] [s1 r2 “informative”]
[s2 r2 “like”] [s2 r2 “agreement”] [s2 r2 “informative”]
...
because I’d like to do an ANOVA on the raw scores, rather than the means. I did not resolve my confusion about about what to do about the 3-paired data (I guess you could lump each subject’s data in one column, or do it separately by “like”, “agreement”, and “informative”, but I’m interested in how good each of the readings are summed across the three metrics). I then gave up and just presented the summary statistics. (You can extract the raw scores from the Appendix if you put some work into it though, or I could pass along the raw scores, or you could tell me how to do this sort of analysis in Python if you wanted me to do it!)

When I look at these tables, I’m also usually squinting at the median rather than mean, though I look at both. You can see the distributions in the Appendix, which I like even better. But point taken about how it’d be nice to have stats.

Vael Gates 29 Dec 2022 7:19 UTC
7 points
4 ∶ 0
on: EAGx application community norms we’d like to see
This is not directly related, but I would love a way to interface with EAGs where (I pay no money, but) I have access to the Swapcard interface and I talk only with people who request meetings with me. I often want to “attend” EAGs in this way, where I don’t interface with the conference (physically or virtually) but I’m available as a resource if people want to talk to me, for which I will schedule remote 1:1s over Zoom. It’d be nice to be helpful to people at a time where they’re available and can see I’m available on Swapcard, but I don’t want to pay to do this (nor make the event organizers pay for my non-physical presence if I do end up applying—I usually want a “virtual only” ticket).
What links here?
- Proposed improvements to EAG(x) admissions process by Elika (30 Jan 2023 6:10 UTC; 62 points)

Vael Gates 6 Apr 2023 18:26 UTC
6 points
2 ∶ 0
in reply to: Linda Linsefors’s comment on: Vael Gates: Risks from Highly-Capable AI (March 2023)
I didn’t give a disagreement vote, but I do disagree on aisafety.training being the “single most useful link to give anyone who wants to join the effort of AI Safety research”, just because there’s a lot of different resources out there and I think “most useful” depends on the audience. I do think it’s a useful link, but most useful is a hard bar to meet!

Vael Gates 22 Nov 2022 5:02 UTC
6 points
0 ∶ 0
in reply to: Falk Lieder’s comment on: Open Phil is seeking applications from grantees impacted by recent events
The application form is actually really restrictive once you open it—when I filled it out, it explicitly instructed not to write any new material and only attach old material that was sent to FTXFF, and only had a <20 word box and <150 word box for grant descriptions. Today when I open the form even those boxes have disappeared. I think it’s meant to be a quite quick form, where they’ll reach out for more details later.

Vael Gates 18 May 2022 3:16 UTC
6 points
0 ∶ 0
on: Transcripts of interviews with AI researchers
I’ve been finding “A Bird’s Eye View of the ML Field [Pragmatic AI Safety #2]” to have a lot of content that would likely be interesting to the audience reading these transcripts. For example, the incentives section rhymes with the type of things interviewees would sometimes say. I think the post generally captures and analyzes a lot of the flavor / contextualizes what it was like to talk to researchers.

Vael Gates 9 May 2022 22:04 UTC
6 points
0 ∶ 0
in reply to: hb574’s comment on: Transcripts of interviews with AI researchers
Indeed! I’ve actually found that in most of my interviews people haven’t thought about the 50+ year future much or heard of AI alignment, given that my large sample is researchers who had papers at NeurIPS or ICML. (The five researchers who were individually selected here had thought about AI alignment uncommonly much, which didn’t particularly surprise me given how they were selected.)
A nice followup direction to take this would be to get a list of common arguments used by AI researchers to be less worried about AI safety (or about working on capabilities, which is separate), counterarguments, and possible counter-counter arguments. Do you plan to touch on this kind of thing in your further work with the 86 researchers?
Yes. With the note that the arguments brought forth are generally less carefully thought-through than the ones shown in the individually-selected-population, due to the larger population. But you can get a sense for some of the types of arguments in the six transcripts from NeurIPS / ICML researchers, though I wouldn’t say it’s fully representative.

Vael Gates 6 Apr 2023 0:48 UTC
5 points
0 ∶ 0
in reply to: Mau’s comment on: Deliberate practice for research?
Not directly relevant to the OP, but another post covering research taste: An Opinionated Guide to ML Research (also see Rohin Shah’s advice about PhD programs (search “Q. What skills will I learn from a PhD?”) for some commentary.

Vael Gates 15 Mar 2023 0:03 UTC
5 points
0 ∶ 0
on: Transcripts of interviews with AI researchers
Two authors gave me permission to publish their transcripts non-anonymously! Thus:
- Interview with Michael L. Littman
- Interview with David Duvenaud

Vael Gates 10 Aug 2023 16:17 UTC
4 points
2 ∶ 0
on: Alignment Grantmaking is Funding-Limited Right Now [crosspost]
“For those applying for grants, asking for less money might make you more likely to be funded”

My guess is that it’s good to still apply for lots of money, and then you just may not be funded the full amount? And one can say what one would do with more or less money granted, so that the grantmakers can take that into account in their decision.

Vael Gates 19 Oct 2022 3:35 UTC
4 points
0 ∶ 0
on: Vael Gates’s Shortform
Continual investment argument for why AGI will probably happen, absent major societal catastrophes, written informally, for my notes:

We’ve been working on AI since ~1950s, in an era of history that feels normal to us but in fact develops technologies very very very fast compared to most of human existence. In 2012, the deep learning revolution of AI started with AlexNet and GPUs. Deep learning has made progress even faster than the current very fast rate of progress: 10 years later, we have unprecedented and unpredicted progress in large language model systems like GPT-3, which have unusual emergent capabilities (text generation, translation, coding, math) for being trained on the next token of a language sequence. One can imagine that if we continue to pour in resources like training data and compute (as many companies are), continue to see algorithmic improvements at the rate we’ve seen, and continue to see hardware improvements (e.g. optical computing), then maybe humanity develops something like AGI or AI at very high levels of capabilities.
Even if we don’t see this progress with deep learning and need a paradigm shift, there’s still an immense amount of human investment being poured into AI in terms of talent, money from private investors + government + company profits, and resources. There’s international competition to develop AI fast, there’s immense economic incentives to make AI products that make our lives ever more convenient along with other benefits, and some of the leading companies (DeepMind, OpenAI) are explicitly aiming at AGI. Given that we’ve only been working on AI since the 1950s, and the major recent progress has been in the last 10 years, and the pace of technological innovation seems very fast or accelerating with worldwide investment, it seems likely we will alive at advanced AI someday, and that someday could be well within our lifetimes, pending major societal disruption.

Vael Gates 4 Oct 2022 4:39 UTC
4 points
0 ∶ 0
on: Announcing the AI Safety Field Building Hub, a new effort to provide AISFB projects, mentorship, and funding
Suggestion for a project from Jonathan Yan:
Given the Future Fund’s recent announcement of the AI Worldview Prize, I think it would be a good idea if someone could create an online group of participants. Such a group can help participants share resources, study together, co-work, challenge each other’s views, etc. After the AI Worldview Prize competition ends, if results are good, a global AI safety community can be built and grown from there.

Vael Gates 15 Sep 2022 3:04 UTC
4 points
0 ∶ 0
on: Vael Gates’s Shortform
(“AI can have bad consequences” as a motivation for AI safety--> Yes, but AI can have bad consequences in meaningfully different ways!)

Here’s some frame confusion that I see a lot, that I think leads to confused intuitions (especially when trying to reason about existential risk from advanced AI, as opposed to today’s systems):

1. There’s (weapons) -- tech like nuclear weapons or autonomous weapons that if used correctly involve people dying. (Tech like this exists)

2. There’s (misuse) -- tech was intentioned to be anywhere from beneficial <> neutral <> seems high on offense-defense balance, but it wasn’t designed for harm. Examples here include social media, identification systems, language models, surveillance systems. (Tech like this exists)

3. There’s (advanced AI pursuing instrumental incentives --> causing existential risk), which is not about misuse, it’s about the *system itself* being an optimizer and seeking power (humans are not the problem here, the AI itself is, once the AI is sufficiently advanced). (Tech like this does not exist)

You can say “AI is bad” for all of them, and they’re all problems, but they’re different problems and should be thought of separately. (1) is a problem (autonomous weapons is the AI version of it) but is pretty independent from (3). Technical AI safety discussion is mostly about the power-seeking agent issue (3). (2) is a problem all the time for all tech (though some tech lends itself more to this than others). They’re all going to need to get solved, but at least (1) and (2) are problems humanity has any experience with (and so we have at least some structures in place to deal with them, and people are aware these are problems).

Vael Gates 21 Nov 2021 20:15 UTC
4 points
0 ∶ 0
on: We need alternatives to Intro EA Fellowships
Just wanted to mention that if you were planning on standardizing an accelerated fellowship retreat, it seems definitely worth reaching out to CFAR folks (as mentioned), since they spent a lot of time testing models, including for post-workshop engagement, afaik! Happy to provide names / introductions if desired.

Vael Gates

FAQ