Studying behaviour and interactions of boundedly rational agents, AI alignment and complex systems.
Research fellow at Future of Humanity Institute, Oxford. Other projects: European Summer Program on Rationality. Human-aligned AI Summer School. Epistea Lab.
[comment I’m likely to regret writing; still seems right]
It seems lot of people are reacting by voting, but the karma of the post is 0. It seems to me up-votes and down-votes are really not expressive enough, so I want to add a more complex reaction.
It is really very unfortunate that the post is framed around the question whether Will MacAskill is or is not honest. This is wrong, and makes any subsequent discussion difficult. (strong down-vote) (Also the conclusion (“he is not”) is not really supported by the evidence.)
It is (and was even more in the blog version) over-zealous, interpreting things uncharitably, and suggesting extreme actions. (downvote)
At the same time, it seems really important to have an open and critical discussion, and culture where people can challenge ‘canonical’ EA books and movement leaders. (upvote)
Carefully going through the sources and checking if papers are not cherry-picked and represented truthfully is commendable. (upvote)
Having really good epistemics is really important, in particular with the focus on long-term. Vigilance in this direction seems good. (upvote)
So it seems really a pity the post was not framed as a question somewhere in the direction “do you thing this is epistemically good?”
If I try to imagine something like “steel-maned version of the post”, without questioning honesty, and without making uncharitable inferences, the reaction could have been some useful discussion.
It seems to me
“Doing Good Better” is sometimes more on the “explaining & advocacy of ideas” side than “dispassionate representation of research”.
Given the genre, I would bet the book is in the top quartile on the metric of representing research correctly.
In some of the examples, it seems adding more caveats and reporting in more detail would have been better for readers interested in precision. Likely at the cost of making the text more dry.
Some emotions sometimes creep in: In case of the somewhat uncharitable part about Charity Evaluator, I remembered their much more uncharitable / misrepresenting text attacking effective altruism and GiveWell. Also while they talk about importance of other things, what they actually measure is actually wrong, and criticized correctly. In case of the whole topic… well a lot of evidence points toward things like 100x multiplier being true, meaning that yes, actually it is possible to save many more people. It seems hard to not to have some passion.
Given that several books about long-term future are now written, the update I would take from that is the books mostly about long-term should err more the side of caveating and describing disagreements and explaining uncertainty more, but my feeling is shift in this direction already happened between 2014 and 2018.
For me, unfortunately, the discourse surrounding Wytham Abbey, seems like a sign of epistemic decline of the community, or at least on the EA forum.
The amount of attentions spent on this seems to be a textbook example of bikeshedding.
Quoting Parkinson :”The time spent on any item of the agenda will be in inverse proportion to the sum [of money] involved.” A reactor is so vastly expensive and complicated that an average person cannot understand it (see ambiguity aversion), so one assumes that those who work on it understand it. However, everyone can visualize a cheap, simple bicycle shed, so planning one can result in endless discussions because everyone involved wants to implement their own proposal and demonstrate personal contribution.
In case of EAs, there are complicated, high-stakes things, for example what R&D efforts to support around AI. This has scale of billions of dollars now, much higher stakes in the future, and there is a lot to understand.
In contrast, absolutely anyone can easily form opinions about appropriateness of a manor house purchase, based on reading a few tweets.
Repeatedly, the tone of the discussion is a bit like “I’ve read a tweet by Émile Torres, I got upset, and I’m writing on EA forum”. Well, tweets by Émile Torres are known to be an unreliable source of information, and in contrast are often optimized to create negative emotional response. (To not single out Torres, this is also true for many other tweets.)
The discussion often almost completely misses the direct, object-level, even if just at back-of-the-envelope estimate way. On this level venue purchases were sensible:
the amount of EA x-risk-reduction money per person in early 2022 was pretty high
organising events and setting up venues is labour intensive, often more difficult to delegate than people assume, and often constrained on time of people who have high opportunity costs
on the margin, you can often trade money, time, work … this trade seems to make sense
apparently, multiple people reached similar conclusion; apart from Wytham, there was for example a different venue purchased in the Bay area, and other near Prague (note: I’m leading the org fiscally sponsoring the later project)
In contrast, large fraction attention in the discussion seems spent on topics which are both two steps removed from the actual thing , and very open to opinions. Where by one step removed I mean e.g. “how was this announced” or “how was this decided”, and two steps removed is e.g. “what will be the impact of how was this announced on the sentiment of the twitter discussion”. While I do agree such considerations can have large effect, driving decisions by this type of reasoning in my view moves people and orgs into the sphere of pure PR, spin and appearance.
This is a bit tangential/meta, but looking at the comment counter makes me want to express gratitude to the Community Health Team at CEA.
I think here we see a ‘practical demonstration’ of the counterfactuals of their work:
- insane amount of attention sucked by this
- the court of public opinions on fora seems basically strictly worse at all relevant dimensions like fairness, respect of privacy or compassion to people involved
As ‘something like this’ would be quite often the counterfactual to CH to trying to deal with stuff …it makes it clear how much value they are creating by dealing with these problems, even if their process is imperfect
Both “EA translation service” and “EA spaces everywhere” seem like ideas which can take good, but also many bad or outright harmful forms.
A few years ago, I tried to describe how to to establish a robustly good “local effective altruism” in a new country or culture (other than Anglosphere).
The super brief summary is
1. it’s not about translating a book, but about “transmission of a tradition of knowledge”
2. what is needed is a highly competent group of people
3. who can, apart from other things, figure out what the generative question of EA means in the given context (which may be quite different from Oxford or Bay)
Point #2 is the bottleneck. Without it, efforts to have “country/language X EA” will be often unsuccessful or worse. Doing Good Better was translated to e.g. japanese. It is doing a bit worse on Amazon.co.jp than the english version on Amazon.co.uk, but not by many orders. Yet you aren’t seeing a lot of content and projects from EA Japan.
So: One good version of “translation service” seems to be basically giving functional national EA groups money to get as good translations as needed.
One bad version is a “centralized project” trying to translate centrally selected content to centrally selected languages by hiring professional contractors to do it.
Similarly EA spaces:
Hiring offices in 100 cities is easy, and you can do that centrally. Running a space with a good culture and gatekeeping is harder, and bottlenecked on “#2″.
Good version of the project seems to be basically giving functional city EA groups money to get good coworking spaces, which is probably doable via CEA group support.
Many bad versions are somewhere close to “centralized megaproject to run EA spaces everywhere”.
In other words: it isn’t easy to get around the bottleneck ‘you need people on the ground’.
Footnote: there are many variations on this theme. The basic pattern roughly is:
“notice that successful groups are doing activity X” (e.g. they have a coworking space, or materials in the local language, or they organize events,...).
the next step, in which this goes wrong, is, “let’s start a project P(X) that will try to make activity X happen everywhere”.
Also: The chances of bad or explicitly harmful outcomes increase roughly in proportion to the combination of cultural, network and geographic distance from the “centre” from which such activity should be directed. Eg project which will try to run spaces in Stanford from Berkeley seem fine
I wanted to provide some context on the recently purchased events venue near Prague that has been discussed here in one of the subthreads. The subthread contains some misleading content, and is not clearly visible, so I am posting this as a top level comment to provide more information on the project. It is extremely long, sorry.
1. My relationship to the project
2. Context for Forum readers
3. Context for outsiders
4. About the project
5. About the venue
6. Communication timelines
1. My relationship to the project
The project is led by Irena Kotikova and was something she started thinking about and working on multiple years ago, before the FTX Future Fund existed.
I am not the recipient of the grant, nor am I responsible for the funding decision: I’m involved in the project as chairman of ESPR, which is a Czech non-profit housing part of the Hostacov project. The ESPR association decided to fiscally sponsor the project because it aligned with our mission (improving epistemics) and we already had the necessary “equivalency determination” to receive tax-exempt funding from a US foundation. However, the Hostacov project is independent and the property is owned by a separate legal entity. The ESPR association is not involved in the operation of the property.
I personally believe the project is good and I’m happy to defend it in my private capacity. The following comments are mine and not made on behalf of the project or its funder. I expect Irena to comment on this as well. (Even more context: I am friends with Irena and have collaborated with her on many projects.)
2. Context for EAs
Irena and I were fairly involved with the Czech government’s Covid response. Our efforts were at least partially successful and likely contributed to preventing thousands of death and preventing some economic damage. However, it also made us quite unpopular among anti-vax and anti-lockdown activists. (In the US context, you can imagine these groups as falling somewhere between the Great Barrington Declaration and QAnon.)
Based on one online media report, these circles noticed the venue purchase and are trying to frame our involvement in a false negative light, ranging from “me and Irena were aware of fraud within FTX” to “both we and FTX are part of an Illuminati conspiracy, we likely knew about the pandemic before it happened, our covid response efforts were part of the plot, and the grant is an expression of gratitude from the Illuminati”. Obviously bizarre, but sadly not harmless.
As a result, please note that the Czech media and these Covid sceptics might well find this discussion. So, anything you write on this post could easily be selectively quoted or otherwise used against you or me. I am not trying to prevent people expressing their opinions, but would ask that any speculation (or, for example, digging for the physical addresses of vaguely related entities) be done with the above in mind.
3. Context for outsiders (you can skip if you have followed the forum in past month)
In case you’re not familiar with FTX and its ill-fated philanthropy: FTX was a large crypto exchange that went bankrupt in early November. Soon after it was revealed that its founder, Sam Bankman-Fried, lied about the state of their assets and may have used client money in a fraudulent way. I recommend Spencer Greenberg’s podcast for more details.
FTX founded a charity, the FTX Future Fund / Foundation. The foundation was small compared to FTX, but quickly became a significant funder in several areas that I find important—AI safety, pandemic preparedness, catastrophic risks, tools to support collective epistemology, teaching critical thinking, etc.
Regardless of what exactly happened at FTX, the result is extremely bad for everyone. First of all for FTX customers—it is unclear what fraction will be returned to them in the bankruptcy proceedings. But it is also bad for people who worked at the FTX Foundation, grant recipients, and those who were promised grants but received nothing.
According to its website, the FTX Foundation has given out hundreds of grants, more than $130M by summer 2022. If they continued their pace, by November this was likely well over $200M. The group of recipients is diverse, from research groups at about half of the top 10 universities in the relevant fields, to tiny non-profits and individuals. Personally, I think that most of the grants were beneficial.
What I want to emphasise is that the grant recipients have done nothing wrong and have nothing to be ashamed of or apologise for.
The implication “you got a grant from a foundation that was sponsored by FTX” ⇒ “what you are doing is suspicious” is nonsense.
Unfortunately, some parts of online space are attempting to paint grant recipients as themselves scandalous or immoral, and shaming them. From what I have seen, this is based on ignoring context or on a clearly wrong framing:
1. Suggestive framing like “they got a grant not long before bankruptcy”. The implication is that grantees should have done – in addition to basic due diligence about the FTX Foundation being a legitimate entity and usual anti-money laundering – some sort of brilliant investigation to discover that FTX was going to go bankrupt in a few months because of fraud inside it.
This is clearly an unreasonable demand. Ironically, if someone had reliable information proving the insolvency of FTX a few months ago, such information would be extremely valuable—it seems likely that the market worth of this info would be greater than total FTX Foundation donations.
2. Ignoring causality. For example, Irena was working on the discussed project for years. It isn’t the case the SBF conceived this.
3. Thinking the grant recipients can just send the money back now. Grants are governed by contracts; organizations are governed by laws and bylaws. What should someone who has received money from the FTX Foundation do? Intuitively, one might think it best to immediately send the money “back” to FTX creditors. In practice, at least for a Czech non-profit, this would be likely illegal: a non-profit governed under Czech law cannot just take a donation it received from the Foundation for a specific purpose and send it to a private entity in the Bahamas. The reasonable option seems to be to wait for how the court proceedings develop, and adhere to contracts and the law. (This is not a legal advice)
4. Liability. Both the law and commonsense protect you if you took an action in good faith, as the grantees did.
4. About the project
As I mentioned earlier, this project hasn’t been announced yet.
While I’m not speaking on behalf of the project, I can try to explain why it seems good, and something like this should exist, from my personal perspective.
I. First, I believe the venue should not be thought of as “EAs for EAs” but rather as something that aims to improve the world. How? By providing a space for topics that are adjacent to EA, such as good epistemics, sanity in times of rapid change, increasingly strong AI, and existential security.
A typical use I imagine is an ‘epistemics workshop for decision-makers’, with most of the guests not being EAs, but decision-makers or researchers. There are many possible events of the type ‘some people should meet and think about something’ or ‘some people should meet and learn something’ where the people meeting and learning is someone else than effective altruists, yet the whole event is quite impactful. In the above mentioned case, you can for example imagine someone working on e.g. regulation of AI in some non-EA capacity understanding better how to think in probability distributions and how to evaluate conflicting statements by various parties.
Overall, I think interventions of this type could be reasonably high-leverage.
II. Second, I believe that the space in which an event is held can make a significant impact on its success. It can easily contribute around 20% to some intuitive “goodness” metric. A stable and amazing venue can also create interesting feedback loops. For example, some people care more about research progress than money and if they have a positive experience at the venue, they may want to return even if monetary incentives point elsewhere.
III. Third, historically, EA and EA-adjacent organisations based in Prague have provided a lot of operational support for various events run by other organisations. This gives me data on the demand for this type of venue.
5. The venue
The venue is a castle located about an hour and a half from Vaclav Havel Airport. Previously, it was operated as a hotel and often hosted events such as weddings or corporate meetings. In its current state, it can host around 60 people, and there is potential for expansion.
The specific venue was selected after Irena scouted for venues for years and investigated dozens of alternatives. I have seen some of the alternatives and, in my opinion, the decisive factor was that the venue was ready to use.
Some of the alternatives were much cheaper but usually required years of expensive reconstruction. Running some of the events I would be excited to happen now seems much more impactful than doing them in 3 years.
Another question could be, why there, and not in some entirely different place. In my view, given the growth of the fields in question, there is a space for multiple venues in different locations, with different use cases.
Compared to the venues in Oxford and near Berkeley, this venue is significantly cheaper, somewhat larger and in better shape.
In terms of travel: the venue is well-located in continental Europe (roughly midway between Berlin and Vienna). Prague Airport has direct flights to most of Europe and travel time from almost any big European airport to the venue is less than 5 hours. In my view, this makes the venue viable for events in Europe longer than roughly 3 or 4 days, and basically globally, for week-long events or longer.
…also, to reiterate, contrary to rumour, the property is not owned by Irena Kotíková or Czech EA association or “EA Prague”. Also it’s probably worth understanding for people involved, the project is mostly a lot of work, not some sort of leisure spending.
6. Communication timelines
Buying a venue of this size takes time. In this case, it involved multiple checks by lawyers, accountants, and technicians. The grant agreement was signed in July, the transaction was not actually completed and visible in the registry until mid-October. It would have been odd to announce the purchase before the transaction was complete.
Nice post, but my rough take is there is
it’s relatively common markets are inefficient, but unexploitable; trading on “everyone dies” seems a clear case of hard-to-exploit inefficiency
markets are not magic; impacts of one-off events with complex consequences are difficult to price in, and what all the magical market aggregation boils down to are a bunch of human brains doing the trades; e.g. I was able to beat the market and get n-times return at point where markets were insane about covid; later, I talked about it with someone in one of the giant hedge funds, and the simple explanation is, while they were looking into it, at some point I knew more about covid than they were able to assemble
example of such hard-to-predict event are e.g. capabilities and impacts of a specific model
the dichotomy 30% growth / everyone dies is unrealistic for trading purposes
near term, there are various outcomes like “industry X gets disrupted” or “someone loses job due to automation” or “war”
if you anticipate fears of this type to dominate in next 10 years, you should price many people increasing their savings and borrowing less
FWIW … in my opinion, retaining the property might have been a more beneficial decision.
Also, I think some people working in the space should not make an update against plans like “have a permanent venue”, but plausibly should make some updates about the “major donors”. My guess this almost certainly means Open Philanthropy, and also likely they had most of the actual power in this decision.
Before delving further, it’s important to outline some potential conflicts of interest and biases:
- I did co-organize or participated at multiple events at Wytham. For example, in 2023, ACS organized a private research retreat aimed at increasing the surface area between Active Inference and AI Alignment communities. The event succeeded to attracts some of the best people from both sides and was pretty valuable for the direction of alignment research I do care about, and the Oxford location was very useful for that. I regret running events like that in future will be more difficult.
- I have friends in all orgs or sides involved—Wytham project, Open Phil, EV, EAs who disapproved the purchase,...
- I lead an org funded by Open Philanthropy—
I also lead an org which was fiscally sponsoring a different project of venue purchase, which was funded by FTX regrant (won’t comment on that for legal reasons)
Also, without more details published, my current opinion is personal speculation, partially based on my reading of the vibes.
My impression-from-a-distance is part of the decision is driven by a factor which I think should not be given undue weight, and a factor where I likely disagree
Factor where I possibly disagree is aesthetics. As far as I can tell, the current preferred EA aesthetics is something more similar to how recent EAG Bay looked like. At EAG Bay, my impression of the vibes of venue was… quite dystopian—the main space is a giant hall with unpleasant artificial lights, no natural light, no colours, endless rows of identical black tables utilized by people having endless rows of 1:1s. In some vague aesthetic space, nearby vibes vectors are faceless bureaucracies, borgs and the scifi portrayals of heart-less technocratic baddies. Also something about naive utilitarianism and army.
Wytham seemed to stand in stark contrast to this aesthetics: the building was old and full of quirks. The vibes were more like an old Oxford college.
Factor which I would guess was part of the decision and I suspect had weight was PR concerns. Wytham definitely got some negative media coverage both in traditional media, social media, and on this forum.
What I dislike about this, these concerns often seemed to be mostly on the Simulacra levels 3 and 4, detached from the reality of running events in Oxford, or actual concern about costs. (Why I do think so? Because of approximately zero amount of negative PR, forum criticisms etc. anyone and anything in the ecosystem is getting for renting properties, even if they are more expensive per day or per person.)
To be clear—
I don’t think these were the only or main(?) factors.
- * I would expect somewhere also exists some spreadsheet with some estimates of “value” of events at Wytham. If this is the case, I probably also disagree about some of the generative opinions about what’s valuable.
Still, given the amount of speculative criticism the purchase of Wytham generated on the forum, it seems good for transparency to also express critical view about the sale.
I. It might be worth reflecting upon how large part of this seem tied to something like “climbing the EA social ladder”.
E.g. just from the first part, emphasis mine
Replace “EA” by some other environment with prestige gradients, and you have something like a highly generic social climbing guide. Seek cool kids, hang around them, go to exclusive parties, get good at signalling.
II. This isn’t to say this is bad . Climbing the ladder to some extent could be instrumentally useful, or even necessary, for an ability to do some interesting things, sometimes.
III. But note the hidden costs. Climbing the social ladder can trade of against building things. Learning all the Berkeley vibes can trade of against, eg., learning the math actually useful for understanding agency.
I don’t think this has any clear bottom line—I do agree for many people caring about EA topics it’s useful to come to the Bay from time to time. Compared to the original post I would probably mainly suggest to also consult virtue ethics and think about what sort of person you are changing yourself to, and if you, for example, most want to become “a highly cool and well networked EA” or e.g. “do things which need to be done”, which are different goals.