I lead a small think tank dedicated to accelerating the pace of scientific advancement by improving the conditions of science funding. As well, I’m a senior advisor to the Social Science Research Council. Prior to these roles, I spent some 9 years at Arnold Ventures (formerly the Laura and John Arnold Foundation) as VP of Research.
Stuart Buck
Karma: 1,246
I don’t believe the $10m claim. Indeed, I don’t even see how it would be possible to spend that much without buying a Super Bowl ad. At $12k a month, you would have to hire nearly 140 PR firms for 6 months to add up to $10m. Perhaps someone added an extra zero or two . . .
My reaction was to Google “diamondoid bacteria” and then wonder why no one on the Internet has uttered that phrase other than Eliezer or someone quoting him.
All of that seems question-begging. If we define “true AGI” as that which knows how to rewrite its own code, then that is indeed what a “true AGI” would be able to do.
“But a true AGI could not only transform the world, it could also transform itself.”
Is there a good argument for this point somewhere? It doesn’t seem obvious at all. We are generally intelligent ourselves, and yet existed for hundreds of thousands of years before we even discovered that there are neurons, synapses, etc., and we are absolutely nowhere near the ability to rewire our neurons and glial cells so as to produce ever-increasing intelligence. So too, if AGI ever exists, it might be at an emergent level that has no idea it is made out of computer code, let alone knows how to rewrite its own code.
One issue for me is just that EA has radically different standards for what constitutes “impact.” If near-term: lots of rigorous RCTs showing positive effect sizes.
If long-term: literally zero evidence that any long-termist efforts have been positive rather than negative in value, which is a hard enough question to settle even for current-day interventions where we see the results immediately . . . BUT if you take the enormous liberty of assuming a positive impact (even just slightly above zero), and then assume lots of people in the future, everything has a huge positive impact.
Open Phil doesn’t necessarily owe anyone an explanation, but the website seems fairly vacuous, and most of their “projects” are just mentioning times when they have invested in someone else’s company. Strong vibe of “all hat, no cattle.”
Definitely agree with this post.
That said, I suspect that the underlying concern here is one of imbalance of power. As I’ve seen from the funder’s side, there are a number of downsides when a grantee is overly dependent on one particular funder: 1) the funder might change direction in a way that is devastating to the grantee and its employees; 2) the grantee is incentivized to cater solely to that one funder while remaining silent about possible criticisms, all of which can silently undermine the effectiveness of the work.
I wonder to what extent this happens to the EA movement broadly, given the dominance of Open Phil as a funder (more so due to the implosion of SBF).
To be sure, Open Phil does a better job than just about any philanthropy at being transparent and welcoming of critique!
Even so, folks are still intimidated (hence anonymity for that long “Doing EA Better” post), and it often strikes me that some possible lines of questioning aren’t raised at all.
For example, while I might have missed something, I haven’t seen anyone ask why the Regranting Challenge gave $70 million to the Gates Foundation that has a 2021 endowment of$55 billion, with Gates’ net worth over $100 billion and many more billions on the way from Warren Buffett.
Why is this cause “neglected”? It may well be, of course—perhaps the folks working on a TB vaccine are doing great work but don’t have the internal political capital at Gates to get even 1/1000th of the Gates wealth allocated to their team. And it may just be a matter of framing—a “collaboration” or “joint funding effort” with Gates wouldn’t come across the same way as “a grant to Gates,” which I guess is irrational on my part. So maybe there’s nothing here at all.
Still....”giving money to Gates” is the kind of thing that would attract some questioning if anyone else did it, and the only criticism of the Regranting Challenge I’ve seen is from another funder who isn’t dependent on future funding here: https://twitter.com/mulagostarr/status/1613911821857230848
In other words, I suspect that some folks have the internal feeling, “I think Open Phil made questionable decisions in one case or another, but it seems too high-risk to speak out despite Open Phil’s willingness to consider critiques. I wish there were a way to capture the kinds of insights that are currently being stifled.”
The relative lack of truly honest feedback is something that bothered me as a funder, and it seemed really hard to address! Even anonymous surveys won’t tell you what grantees and others really think (most people that have a very specific insight or critique can’t say very much without making it obvious which grant they’re talking about and who is speaking).
SBF Has a Substack!
FTX and Alameda definitely needed more bureaucracy—as in, doing stuff in a way that doesn’t resemble a scene from Idiocracy. https://docs.house.gov/meetings/BA/BA00/20221213/115246/HHRG-117-BA00-Wstate-RayJ-20221213.pdf “Although our investigation is ongoing and detailed findings will have to await its conclusion, the FTX Group’s collapse appears to stem from the absolute concentration of control in the hands of a very small group of grossly inexperienced and unsophisticated individuals who failed to implement virtually any of the systems or controls that are necessary for a company that is entrusted with other people’s money or assets.”
Thanks! There’s a very natural reason for all of this: https://twitter.com/stuartbuck1/status/1595254445683654657
My post from 4 months ago linked to a story about the lawyer, which is why I said I merely hinted at this point. The post from 2 months ago didn’t expressly mention it, but a followup post definitely did in detail (I deleted the post soon thereafter because I got a few downvotes and I got nervous that maybe it was over the line).
Over a year ago, I thought it was completely inexplicable that SBF hired a Chief Regulatory Officer who was a lawyer known only for involvement in online fraud. There is no legitimate reason to hire such a person. And even apart from the fraud, his resume was not the typical profile of someone who a legitimate multi-billionaire would hire at a high level. An actual multi-billionaire could have poached away the general counsel from a Fortune 100 company. Why settle for someone like this? https://coingeek.com/tether-links-to-questionable-market-makers-yet-another-cause-for-concern/
I finally worked up the courage to hint at this point 4 months ago ( https://forum.effectivealtruism.org/posts/KBw6wKDbvmqacbB5M/crypto-markets-ea-funding-and-optics?commentId=wcvYZtw7b4xvrdetL ), and then was a little more direct 2 months ago ( https://forum.effectivealtruism.org/posts/YgbpxJmEdFhFGpqci/winners-of-the-ea-criticism-and-red-teaming-contest?commentId=odHG4hhM2FSGiXXFQ).
Lo and behold, that guy was probably in on the whole thing—he also served as general counsel to Alameda! https://www.nbcnews.com/news/ftxs-regulatory-chief-4-job-titles-2-years-was-really-rcna57965
Is this Sam in disguise? You’re literally the only person in existence who seems to think it was somehow unfair to be suspicious (and correctly so!) of SBF for having hired a chief compliance officer with a long history of fraud, and of his pattern of trying to buy up other people’s frauds/scams.
Thanks for this post. Not sure what it would take to get it onto the front page? The algorithm seems very opaque to me.
The only flaw in my earlier comment is that I was too charitable towards SBF in suggesting that there might be some plausible excuse for the multiple red flags I noticed.
My criticism of Binance was not “it’s cryptocurrency.” My criticism of Binance was that at the very time that that SBF allied with Binance, it was a “hub for hackers, fraudsters and drug traffickers.” Apparently your defense of SBF is that “everyone knows” crypto is good for little else . . . but perhap if someone enters a field that is mostly or entirely occupied by criminal activity, that isn’t actually an excuse?
As for backstopping other scams and frauds, that isn’t a way to make sure that the “crypto sector stays healthy” (barring very unusual definitions of the word “healthy”), and in actuality, we’re now seeing evidence that FTX was just trying to extract assets from other companies in a desperate attempt to shore up their own malfeasance and fraud. https://twitter.com/AutismCapital/status/1591569275642589184
This admirably honest statement deserves more emphasis. As we know from medicine and international development and anywhere that does RCTs, it is really, really hard—even when the results of your actions are right in front of you—to know whether you have helped someone or harmed them. There are just too many confounding factors, selection bias, etc.
The long-termist AGI stuff has always struck me as even worse off in this respect. How is anyone supposed to know that the actions they take today will have a beneficial impact on the world decades from now, rather than making things worse? And given the premises of AGI alignment, making things worse would be utterly catastrophic for humanity.