Benjamin_Todd

Karma: 9,152

Benjamin_Todd Apr 9, 2023, 9:00 AM
3 points
1 ∶ 0
in reply to: Miquel Banchs-Piqué (prev. mikbp)’s comment on: My updates after FTX
If you’re tracking the annual change in wealth between two periods, you should try to make sure the start at the end point are either both market peaks or both market lows.
e.g. from 2017 to 2021, or 2019 to Nov 2022 would be valid periods for tracking crypto.
If you instead track from e.g. 2019 to 2021, then you’re probably going to overestimate.
Another option would be to average over periods significantly longer than a typical market cycle (e.g. 10yr).

Benjamin_Todd Apr 9, 2023, 8:57 AM
4 points
1 ∶ 0
in reply to: Miquel Banchs-Piqué (prev. mikbp)’s comment on: My updates after FTX
Thanks!
My social life is pretty much only people who aren’t in the EA community at this point.
Small comment on this:
actively fighting against EA communities to become silos and for EA enterprises to have workers outside EA communities would be of great value
It depends on the org, but for smaller orgs that are focused on EA community building, I still think it could make sense for them to pretty much only people who are very interested in EA. I wouldn’t say the same about e.g. most biorisk orgs though.

Benjamin_Todd Apr 7, 2023, 9:42 AM
1 point
0 ∶ 2
in reply to: AnonymousEAForumAccount’s comment on: My updates after FTX
Yes, I’d basically agree – he didn’t influence the thinking that much but he did impact what you could get paid to do (and that could also have long term impacts on the structure of the community).
Though, given income inequality, the latter problem seems very hard to solve.

Benjamin_Todd Apr 5, 2023, 11:49 AM
2 points
0 ∶ 0
in reply to: John G. Halstead’s comment on: My updates after FTX
That’s useful—my ‘naive optimizing’ thing isn’t supposed to be the same thing as naive utilitarianism, but I do find it hard to pin down the exact trait that’s the issue here, and those are interesting points about confidence maybe not being the key thing.

Benjamin_Todd Apr 3, 2023, 12:18 PM
3 points
0 ∶ 0
in reply to: Greg S’s comment on: My updates after FTX
Just a small clarification, I’m not saying we should abandon the practical project, but it could make sense to (relatively speaking) focus on more tractable areas / dial down ambitions / tilt more towards the intellectual project until we’ve established more operational competence.

I also agree dissociating has significant costs that need to be weighed against the other reasons.

Benjamin_Todd Apr 3, 2023, 12:08 AM
9 points
1 ∶ 2
in reply to: Cullen 🔸’s comment on: My updates after FTX
I’d agree a high degree of confidence + strong willingness to act combined with many other ideologies leads to bad stuff.
Though I still think some ideologies encourage maximisation more than others.
Utilitarianism is much more explicit in its maximisation than most ideologies, plus it (at least superficially) actively undermines the normal safeguards against dangerous maximisation (virtues, the law, and moral rules) by pointing out these can be overridden for the greater good.
Like yes there are extreme environmentalists and that’s bad, but normally when someone takes on an ideology like environmentalism, they don’t also explicitly & automatically say that the environmental is all that matters and that it’s in principle permissible to cheat & lie in order to benefit the environment.
nor do I think it would be a good counterargument if it was.
Definitely not saying it has any bearing on the truth of utilitarianism (in general I don’t think recent events have much bearing on the truth of anything). My original point was about who EA should try to attract, as a practical matter.

Benjamin_Todd Apr 2, 2023, 11:06 PM
4 points
3 ∶ 3
in reply to: Michelle_Hutchinson’s comment on: My updates after FTX
I basically agree and try to emphasize personality much more than ideology in the post.

That said, it doesn’t seem like a big leap to think that confidence in an ideology that says you need to maximise a single value to the exclusion of all else could lead to dangerously optimizing behaviour...

Having more concern for the wellbeing of others is not the problematic part. But utilitarianism is more than that.

Moreover it could still be true that confidence in utilitarianism is in practice correlated with these dangerous traits.

I expect it’s the negative component in the two factor model that’s the problem, rather than the positive component you highlight. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5900580/

Benjamin_Todd Apr 2, 2023, 4:42 PM
4 points
1 ∶ 0
in reply to: Jason’s comment on: My updates after FTX
Yes, something like that: he of course had an influence on what you could get paid for (which seems hard to avoid given some ppl have more money than others) but I don’t think he had a big influence on people’s thinking about cause pri.

Benjamin_Todd Apr 2, 2023, 4:38 PM
5 points
2 ∶ 0
in reply to: Henri Thunberg 🔸’s comment on: My updates after FTX
Hmm that does seem worse than I expected.

I wonder if it’s because gwwc has cut back outreach or is getting less promotion by other groups (whereas 80k continued it’s marketing as before, plus a lot of 80k’s reach is passive), or whether it points to outreach actually being harder now.

Benjamin_Todd Apr 1, 2023, 9:40 AM
4 points
0 ∶ 0
in reply to: Richard Y Chappell🔸’s comment on: My updates after FTX
I had you in mind as a good utilitarian when writing :)
Good point that just saying ‘naively optimizing’ utilitarians is probably clearest most of the time. I was looking for other words that would denote high-confidence and willingness to act without qualms.

Benjamin_Todd Apr 1, 2023, 9:38 AM
8 points
2 ∶ 1
in reply to: Ubuntu’s comment on: My updates after FTX
Thank you!
Yes I think if you make update A due to a single data point, then you realise you shouldn’t have updated on a single data point, you should undo update A. Like your original reasoning was wrong.
That aside, in the general case I think it can sometimes be justified a lot to update on a single datapoint. E.g. if you think an event was very unlikely, and then that event happens, your new probability estimate for the event will normally go up a lot.
In other cases, if you already have lots of relevant points, then adding a single extra one won’t have much impact.
One extra point is that I think people have focused too much on SBF. The other founders also said they supported EA. So if we’re just counting up people, it’s more than one.

Benjamin_Todd Apr 1, 2023, 9:32 AM
14 points
1 ∶ 0
in reply to: Ubuntu’s comment on: My updates after FTX
Some of the ones I’ve seen:
80k’s metrics seems unaffected so far, and it’s one of the biggest drivers of community growth.
I’ve also heard that EAG(x) applications didn’t seem affected.
GWWC pledgers were down, though a lot of that is due to them not doing a pledge drive in Dec. My guess is that if they do a pledge drive next Dec similar to previous ones, the results will be similar. The baseline of monthly pledges seems ~similar.
What links here?
- How has FTX’s collapse impacted EA? by AnonymousEAForumAccount (Oct 17, 2023, 5:02 PM; 248 points)

Benjamin_Todd Apr 1, 2023, 9:28 AM
4 points
0 ∶ 0
in reply to: alex lawsen’s comment on: Whether you should do a PhD doesn’t depend much on timelines.
Good point there are reasons why work could get more valuable the closer you are – I should have mentioned that.
Also interesting points about option value.

Benjamin_Todd Mar 31, 2023, 11:25 PM
11 points
1 ∶ 0
on: Whether you should do a PhD doesn’t depend much on timelines.
I agree with many of the points, especially that personal fit is a big deal and that doing a PhD is also in part useful research (rather than pure career capital), and what matters is time until the x-risk rather than random definitions of AGI, but I’m worried this bit understates the reasons for urgency quite a bit:
you might then conclude that delaying your career by 6 years would cause it to have ⁴¹⁄₉₁ = 45% of the value. If that’s the case, if the delay increased the impact you could have by a bit more than a factor of 2, the delay would be worth it.

This is on a model in which work becomes moot after a transition point. But it’s assuming work before the transition is equally valuable no matter the year.
However, the AI safety community is probably growing at 40%+ per year, and (if timelines are short) it’ll probably still be growing at 10-20%+ when the potential existential risk arrives. This roughly means that moving a year of labour invested in AI safety community building one year earlier makes it 10-20% more valuable. This would mean an extra year of labour now is worth 3-10x one in 10 years, all else equal.
Or to turn to direct work, there are serial dependencies i.e. 100 people working for 1 year won’t achieve anywhere near as much as 10 people working for 10 years. This again could make extra labour on alignment now many times more valuable work in 10 years.
Another argument is that since the community can have more impact in world with short timelines, people should act as if they’re shorter than they are.
This could mean, for instance, if your best guess is 33% timelines under 10yr, 33% medium timelines and 33% longer timelines, it might be optimal for people to allocate something like 70%, 15%, 15%. Yes in this world some people focus on long-term career capital, but it would be less than normal.
Estimating the size of these effects is hard – my main point is that they can be very large, especially as timelines get short. (Many of these effects feel like they go up non-linearly as timelines shorten.)
So, while I agree that if someone’s median timeline estimate changes from, say, 25 years to 20 years, that’s not going to have much effect on the question; I think how much to focus on career capital could be pretty sensitive to, say, your probability on <10 year timelines.
What links here?
- AI safety technical research—Career review by Benjamin Hilton (Jul 17, 2023, 3:34 PM; 50 points)
- AI safety technical research—Career review by Benjamin Hilton (LessWrong; Jul 17, 2023, 3:34 PM; 14 points)

Benjamin_Todd Mar 31, 2023, 11:10 PM
9 points
3 ∶ 2
in reply to: Ben_West🔸’s comment on: My updates after FTX
First to clarify, yes most of the general public haven’t heard of EA, and many haven’t made the connection with FTX.
I think EA’s brand has mainly been damaged among what you could call the chattering classes. But I think that is a significant cost.
It’s also damaged in the sense that if you search for it online you find a lot more negative stuff now.
On the question about comparisons, unfortunately I don’t have a comprehensive answer.
Part of my thinking is that early on I thought EA wasn’t a good public facing brand. Then things went better than I expected for a while. But then after EA actually got serious negative attention, it seemed like the old worries were correct after all.
My impression is that many (almost all?) face large scandals eventually, but it doesn’t always stick in the way it was sticking with EA.
I’m also sympathetic to Holden’s comments.

Benjamin_Todd Mar 31, 2023, 11:03 PM
4 points
0 ∶ 0
in reply to: Ben_West🔸’s comment on: My updates after FTX
From copying suggest changes :(

Benjamin_Todd Mar 31, 2023, 10:29 PM
2 points
0 ∶ 0
in reply to: pseudonym’s comment on: My updates after FTX
Thanks fixed!

Benjamin_Todd Mar 31, 2023, 8:49 PM
4 points
1 ∶ 0
in reply to: akash 🔸’s comment on: My updates after FTX
I was trying to do that :) That’s why I opened with naive optimizing as the problem. The point about gung-ho utilitarians was supposed to be an example of a potential implication.

Benjamin_Todd Mar 31, 2023, 8:10 PM
27 points
4 ∶ 0
on: My updates after FTX
Appendix: some other comments that didn’t make it into the main post
I made some mistakes in tracking EA money
These aren’t updates but rather things I should have been able to figure out ex ante:
- Mark down private valuations of startups by a factor of ~2 (especially if the founders hold most of the equity)
- Measure growth from peak-to-peak or trough-to-trough rather than trough-to-peak – I knew crypto was in a huge bull market
- I maybe should have expected more correlation between Alameda and FTX, and a higher chance of going to zero, based on the outside view
- If your assets are highly volatile and the community is risk-averse, that’s a good reason to discount the value of those assets and/or delay spending until you have more confidence in how much money you have.
Taken together, I think we should have planned on the basis of the portfolio being 2-3x smaller than my estimates, and expected less future growth. I think some of these effects were taken into account in planning (it was obvious that crypto could fall a lot), perhaps correctly in some cases, but not enough in others.
Moreover, even with these adjustments, I think spending would have still been significantly below target, so there would have still been efforts to grow spending significantly, but maybe slower in some areas, especially the FF.

Another mistake
I think I made a mistake to publicly affiliate 80,000 Hours with SBF as much as we did – just based on good PR sense and messaging strategy – and not to investigate it more. (80,000 Hours has written about a similar issue on their mistakes page.)

Epistemic virtues
I was unsure whether to say anything extra about epistemic virtues, like honesty, integrity and truthseeking. I agree that having more of these virtues would have given us led us to put a higher probability on what was happening (e.g. via more people sharing concerns even if might make the community look bad), though it doesn’t feel like a big update about the value of these traits, since we already knew they’re valuable. Someone would have also needed to end up with a pretty high probability to do much about it. I agree (as covered above) that these events are an update against the idea EA has unusually good epistemics, and that stronger epistemics virtues seem more important in order to constrain dangerous actors. Finally, I think epistemic virtues have already had a lot of emphasis relatively speaking, so it seems hard to get large gains here.

Degree of criticism in the culture
I feel worried that the updates listed are going to lead towards a more critical, non-trusting culture in which people’s character and minor actions are publicly scrutinised, and there’s more gossip and less friendship. Personally I already find EA culture often pretty negative and draining, so feel worried about moving more in that direction. One option is to accept it’s what’s needed to prevent dangerous actors, and that EA is going to be a less fun place to be, and therefore have less potential. However, I suspect and hope it’s also possible to find ways to make improvements in the directions I gesture at that don’t make the culture more negative.

Membership growth
I expect EA membership growth to take a hit. 2022 showed strong growth (20-60%), and I already expected 2023 to be lower (unless we maintain accelerating effort into growth). But FTX will reduce that because (i) a lot of the 2022 growth seems to be driven by personal connections, and that could be pretty sensitive to positive buzz (ii) lower funding & investment (iii) decent chance that a wave of people leaves (and this could start a negative feedback loop). I don’t think increased (negative) media coverage will lead to much growth, since even positive coverage hasn’t caused much growth in the past. I wouldn’t be shocked if the community were overall smaller in a couple of years. On the positive side, many measures of community growth seem unaffected so far, so it seems likely the longer term trend of 10-30% growth continues after the next 1-2 years.

Benjamin_Todd Mar 31, 2023, 8:07 PM
17 points
8 ∶ 3
in reply to: akash 🔸’s comment on: My updates after FTX
It’s a big topic, but the justification is supposed to be the part just before. I think we should be more worried about attracting naive optimizers, and I think people who are gung-ho utilitarians are one example of a group who are more likely to have this trait.
I think it’s notable that SBF was a gung-ho utilitarian before he got into EA.
It’s maybe worth clarifying that I’m most concerned about people who a combination of high-confidence in utilitarianism and a lack of qualms about putting it into practice.
There are lots of people who see utilitarianism as their best guess moral theory but aren’t naive optimizers in the sense I’m trying to point to here.
See more in Toby’s talk.