Michaël Trazzi
On a related note, has someone looked into the cost-effectiveness of funding new podcasts vs. convincing mainstream ones to produce more impactful content, similarly to how OpenPhil funded Kurzgesagt?
For instance, has anyone tried to convince people like Lex Fridman who has already interviewed MacAskill and Bostrom, to interview more EA-aligned speakers?
My current analysis gives roughly an audience of 1-10M per episode for Lex, and I’d expect that something around $20-100k per episode would be enough of an incentive.
In comparison, when giving $10k to start a podcast, the potential reach is maybe 100-10k per episode after 10 episodes, but maybe the EV is higher because most of the impact is after those 10 first episodes. Also, the core audience is more willing to update their models and benefit from the podcast than eg the average Lex Fridman listener.
Another counterargument woud be that Lex already interviews people like MacAskill and Bostrom so the marginal impact of an additional one is low, and EA-aligned impactful speakers already manage to get on mainstream media to do outreach anyway (eg Will going on cable cable TV for WWOTF).
I’m flattered for The Inside View to be included here among so many great podcasts. This is an amazing opportunity and I am excited to see more podcasts emerge, especially video ones.
If anyone is on the edge of starting and would like to hear some of the hard lessons I’ve learned and other hot takes I have on podcasting or video, feel free to message me at michael.trazzi at gmail or (better) comment here.
Note: if you want to discuss some of the content of this episode, or one of the above quotes, I’ll be at EAG DC this weekend chatting about AI Governance–feel free to book a meeting!
Agreed!
As Zach pointed out below there might be some mistakes left in the precise numbers, for any quantitative analysis I would suggest reading AI Impacts’ write-up: https://aiimpacts.org/what-do-ml-researchers-think-about-ai-in-2022/
Thanks for the corrections!
Can you tell me exactly which numbers I should change and where?
Sorry about that! The AI generating the transcript was not conscious of the pain created by his terrible typos.
Thanks for the quotes and the positive feedback on the interview/series!
Re Gato: we also mention it as a reason why training across multiple domains does not increase performance in narrow domains, so there is also evidence against generality (in the sense of generality being useful). From the transcript:
“And there’s been some funny work that shows that it can even transfer to some out-of-domain stuff a bit, but there hasn’t been any convincing demonstration that it transfers to anything you want. And in fact, I think that the recent paper… The Gato paper from DeepMind actually shows, if you look at their data, that they’re still getting better transfer effects if you train in domain than if you train across all possible tasks.”
I think he would agree with “we wouldn’t have GPT-3 from an economical perspective”. I am not sure whether he would agree with a theoretical impossibility. From the transcript:
“Because a lot of the current models are based on diffusion stuff, not just bigger transformers. If you didn’t have diffusion models [and] you didn’t have transformers, both of which were invented in the last five years, you wouldn’t have GPT-3 or DALL-E. And so I think it’s silly to say that scale was the only thing that was necessary because that’s just clearly not true.”
To be clear, the part about the credit assignment problem was mostly when discussing the research at his lab, and he did not explicitly argue that the long-term credit assignment problem was evidence that training powerful AI systems is hard. I included the quote because it was relevant, but it was not an “argument” per se.
Thanks for the reminder on the open-minded epistemics ideal of the movement. To clarify, I do spend a lot of time reading posts from people who are concerned about AI Alignment, and talking to multiple “skeptics” made me realize things that I had not properly considered before, learning where AI Alignment arguments might be wrong or simply overconfident.
(FWIW I did not feel any pushback in suggesting that skeptics might be right on the EAF, and, to be clear, that was not my intention. The goal was simply to showcase a methodology to facilitate a constructive dialogue between the Machine Learning and AI Alignment community.)
LessWrong has been A/B testing for a voting system separate from karma for “agree/disagree”. I would suggest contacting the LW team to know 1) the results from their experiments 2) how easy it would be to just copy the feature to the EAF (since codebases used to be the same).
Thanks for the thoughtful post. (Cross-posting a comment I made on Nick’s recent post.)
My understanding is that people were mostly speculating on the EAF about the rejection rate for the FTX future fund’s grants and distribution of $ per grantee. What might have caused the propagation of “free-spending” EA stories:the selection bias at EAG(X) conferences where there was a high % of grantees.
the fact that the FTX future fund did not (afaik) released their rejection rate publicly
other grants made by other orgs happening concurrently (eg. CEA)
This post helped me clarify my thoughts on this. In particular, I found this sentence useful to shed light on the rejection rate situation:
“For example, Future Fund is trying to scale up its giving rapidly, but in the recent open call it rejected over 95% of applications”
My understanding is that people were mostly speculating on the EAF about the rejection rate and distribution of $ per grantee. What might have caused the propagation of “free-spending” EA stories:
the selection bias at EAG(X) conferences where there was a high % of grantees.
the fact that FTX did not (afaik) release their rejection rate publicly
other grants made by other orgs happening concurrently (eg. CEA)
I found this sentence in Will’s recent post “For example, Future Fund is trying to scale up its giving rapidly, but in the recent open call it rejected over 95% of applications” useful to shed light on the rejection rate situation.
Note: the probabilities in the above quotes and in the podcast are the result of armchair forecasting. Please do not quote Peter on this. (I want to give some space for my guests to give intuitions about their estimates without having to worry about being extra careful.)
To make that question more precise, we’re trying to estimate xrisk_{counterfactual world without those people} - xrisk_{our world}, with xrisk_{our world}~1/6 if we stick to The Precipice’s estimate.
Let’s assume that the x-risk research community completely vanishes right now (including the past outputs, and all the research it would have created). It’s hard to quantify, but I would personally be at least twice as worried about AI risk that I am right now (I am unsure about how much it would affect nuclear/climate change/natural disasters/engineered pandemics risk and other risks).
Now, how much of the “community” was actually funded by “EA $”? How much of those researchers would not be capable of the same level of output without the funding we currently have? How much of the x-risk reduction is actually done by our impact in the past (e.g. new sub-fields of x-risk research being created, where progress is now (indirectly) being made by people outside of the x-risk community) vs. researcher hours today? What fraction of those researchers would still be working on x-risk on the side even if their work wasn’t fully funded by “EA $”?
I don’t know what to do for the url not to break on EA Forum by default.
Last time I tried the https without www and there was the same problem. Adding the www solved it.
I believe it’s a bug in how urls are validated by EAF (because it doesn’t break on LW and the urls are valid).
Not sure how to tag EAF devs but this is quite annoying.