I am an independent research and programmer working at my own consultancy, Shapley Maximizers ÖU. I like to spend my time acquiring deeper models of the world, and generally becoming more formidable. I’m also a fairly good forecaster: I started out on predicting on Good Judgment Open and CSET-Foretell, but now do most of my forecasting through Samotsvety, of which Scott Alexander writes:
Enter Samotsvety Forecasts. This is a team of some of the best superforecasters in the world. They won the CSET-Foretell forecasting competition by an absolutely obscene margin, “around twice as good as the next-best team in terms of the relative Brier score”. If the point of forecasting tournaments is to figure out who you can trust, the science has spoken, and the answer is “these guys”.
I used to post prolifically on the EA Forum, but nowadays, I post my research and thoughts at nunosempere.com / nunosempere.com/blog rather than on this forum, because:
I disagree with the EA Forum’s moderation policy—they’ve banned a few disagreeable people whom I like, and I think they’re generally a bit too censorious for my liking.
The Forum website has become more annoying to me over time: more cluttered and more pushy in terms of curated and pinned posts (I’ve partially mitigated this by writing my own minimalistic frontend)
The above two issues have made me take notice that the EA Forum is beyond my control, and it feels like a dumb move to host my research in a platform that has goals different from my own.
But a good fraction of my past research is still available here on the EA Forum. I’m particularly fond of my series on Estimating Value.
I used to do research around longtermism, forecasting and quantification, as well as some programming, at the Quantified Uncertainty Research Institute (QURI). At QURI, I programmed Metaforecast.org, a search tool which aggregates predictions from many different platforms, which I still maintain. I spent some time in the Bahamas as part of the FTX EA Fellowship, and did a bunch of work for the FTX Foundation, which then went to waste when it evaporated.
Previously, I was a Future of Humanity Institute 2020 Summer Research Fellow, and then worked on a grant from the Long Term Future Fund to do “independent research on forecasting and optimal paths to improve the long-term.” I used to write a Forecasting Newsletter which gathered a few thousand subscribers, but I stopped as the value of my time rose. I also generally enjoy winning bets against people too confident in their beliefs.
Before that, I studied Maths and Philosophy, dropped out in exasperation at the inefficiency, picked up some development economics; helped implement the European Summer Program on Rationality during 2017, 2018 2019, 2020 and 2022; worked as a contractor for various forecasting and programming projects; volunteered for various Effective Altruism organizations, and carried out many independent research projects. In a past life, I also wrote a popular Spanish literature blog, and remain keenly interested in Spanish poetry.
You can share feedback anonymously with me here.
Note: You can sign up for all my posts here: <https://nunosempere.com/.newsletter/>, or subscribe to my posts’ RSS here: <https://nunosempere.com/blog/index.rss>
I’m missing a lot of context here, but my impression is that this argument doesn’t go through, or at least is missing some steps:
We think that the best Redwood research is of similar quality to work by [Neel Nanda, Tom Lieberum and others, mentored by Jacob Steinhardt]
Work by those others doesn’t cost $20M
Therefore the work by Redwood shouldn’t cost $20M
Instead, the argument which would go through would be:
Open Philanthropy spent $20M on Redwood Research
That $20M produced [such and such research]
This is how you could have spent $20M to produce [better research]
Therefore, Open Philanthropy shouldn’t have spent $20M on Redwood Research, but instead on [alternatives]
(or spent $20M on [alternatives] and on Redwood Research, if the value of Redwood Research is still above the bar)
But you haven’t shown step 3, the tradeoff against the counterfactual. It seems likely that the situation is such that producing good AI safety research depends on somewhat idiosyncratic non-monetary factors. Sometimes you will find a talented independent researcher or a PhD student that will produce quality research for relatively small amounts of money, sometimes you will spend $20M to get an outcome of a similar quality. I could see that being the case if the bottleneck isn’t money, which seems plausible.
Also note that building an institution is potentially much more scalable than funding one-off independent researchers.
As I said, I’m missing lots of context (i.e., I haven’t read Redwood’s research, seems within the normal range of possibility that it wouldn’t be worth $20M), but I thought I’d give my two cents.