I’m going to separate some of my responses into separate comment threads here, since there’s a lot to unpack.
First, and most importantly:
I also think that there are some errors in your post.
Can you enumerate what these are? It’s not really clear to me from your response, though I may have missed it. (I’m assuming we’re both operating on a model where “errors” are not the same thing as “disagreements”.)
I don’t have time to go into great depth for every one of your questions, but I’ll try to give quick replies to as many as I can.
It’s fair to separate errors from disagreements, and I focused mostly on disagreements in my original reply. (There are more things that should be classified as disagreements than should be classified as errors, and I think that the disagreements in this case mattered more to the discussion, which is why I focused on them. It’s possible I should have deleted the “some errors” line in my reply once I drafted it.) Things I think are errors on your part:
You write: “the fact that there was a large waiting list for this conference was not at all surprising; I think I would have given an ex ante probability of over 90%. I’d be very surprised if the events team wasn’t also expecting this.”
The Events Team did not expect this. (You phrase it as a prediction that you would be surprised to learn that this was the case, so I don’t have a way of knowing if this is in fact an error. But I think at the core, the statement is that you suspect that we expected this. We didn’t.)
You write: “As with most COVID-related decisions made by the CEA events team over the course of the pandemic, the change in policy was justified solely by an appeal to authority – namely, that they consulted with their COVID advisory board.”
This isn’t correct — neither for the recent event nor for EA Global: SF 2020 (the other conference where our plans were substantially changed by COVID).
In the email where we offered refunds due to the change of plans, we shared our reason for expanding the conference: “We’ve received hundreds of applications for the event, and we currently have over 300 exceptional candidates on our waiting list. We really want these people to have a chance to attend, but we can only invite them if we increase our capacity.”
We were also concerned about the potential increase in risk and understood that some of the people who had registered might not want to attend as a result, and thus offered refunds and other resources they could use to make their decision (e.g. microCOVID).
One sentence in our email read: “After consulting our COVID Advisory Board, we’ve decided to increase this cap to allow those 300 people to join us in London”. This doesn’t mean that the board was our _only _source of guidance — just that consulting them was one necessary step we took before deciding to increase the cap.
For reference, the Events Team posted this about the decision in 2020. (Note that the composition of the COVID Board has changed since then.) This post isn’t just summarizing what the advisory board said — it also includes a lot of the team’s reasoning.
You write: “Apart from those few people privileged to have close links with CEA or the Oxford office, most attendees didn’t have access to the reasoning used to make the decision to expand. The arguments and data used to make that consequential decision should have been made available to the community, so that we could evaluate whether we found them convincing, challenge them if not, and update our beliefs and plans accordingly. This was not done.”
To be clear: none of the CEA Events Team was based in Oxford when this decision was made.
It’s true that we did not publish the reasoning. We did this because writing something that requires that kind of polish and care would have taken bandwidth and hours our team did (and still does) not have. We did however write a memo about this decision, which we shared with 40 community leaders (with a request for feedback), and had conversations about this with around a dozen people who attended the EA Meta Coordination Forum. We received overwhelmingly positive feedback on this decision.
I’d also like to add that while in general, I agree that transparency is good, I think you and I have pretty different pictures of what info would be most useful to the community here.
The info we prioritized getting to the community was what we thought would help them make decisions: whether to still come to the conference given the expanded size, and later the breakdown of microCOVID estimates by activity, which people could use to think about having more meetings outdoors, etc.
It sounds like you think it’s particularly important for us to share the specifics of the reasoning behind expanding the conference, beyond the points we’ve already shared. I don’t fully understand what you think would be useful to the community about having these particular details. There’s a lot of reasoning we could write up about each conference: Why does it cost this much? Why hold events in these cities and not those cities? Why have two conferences and not more? Why is this kind of food served?
Discussions and debates on these topics can be useful; sometimes, pushback from the community has helped CEA (and other organizations) recognize things we should be doing differently. But to publish all the reasoning behind all these things as it changes over time would take time that we can’t then use to actually produce events. (Especially as this work has to involve the people running the events, whose time is scarcer and harder to replace than contractor time.)
I think some of the disconnect/confusion in the follow-on discussion here is being caused by my failing to update from a model of “CEA made a difficult COVID cost/benefit calculation” to one of “CEA didn’t think COVID was a major concern” sufficiently quickly/completely. (Most of my post was written under the former model.)
I need to think more about that before I try to clarify my position here, because I’m not 100% sure what it is.
I’m going to separate some of my responses into separate comment threads here, since there’s a lot to unpack.
First, and most importantly:
Can you enumerate what these are? It’s not really clear to me from your response, though I may have missed it. (I’m assuming we’re both operating on a model where “errors” are not the same thing as “disagreements”.)
I don’t have time to go into great depth for every one of your questions, but I’ll try to give quick replies to as many as I can.
It’s fair to separate errors from disagreements, and I focused mostly on disagreements in my original reply. (There are more things that should be classified as disagreements than should be classified as errors, and I think that the disagreements in this case mattered more to the discussion, which is why I focused on them. It’s possible I should have deleted the “some errors” line in my reply once I drafted it.) Things I think are errors on your part:
You write: “the fact that there was a large waiting list for this conference was not at all surprising; I think I would have given an ex ante probability of over 90%. I’d be very surprised if the events team wasn’t also expecting this.”
The Events Team did not expect this. (You phrase it as a prediction that you would be surprised to learn that this was the case, so I don’t have a way of knowing if this is in fact an error. But I think at the core, the statement is that you suspect that we expected this. We didn’t.)
You write: “As with most COVID-related decisions made by the CEA events team over the course of the pandemic, the change in policy was justified solely by an appeal to authority – namely, that they consulted with their COVID advisory board.”
This isn’t correct — neither for the recent event nor for EA Global: SF 2020 (the other conference where our plans were substantially changed by COVID).
In the email where we offered refunds due to the change of plans, we shared our reason for expanding the conference: “We’ve received hundreds of applications for the event, and we currently have over 300 exceptional candidates on our waiting list. We really want these people to have a chance to attend, but we can only invite them if we increase our capacity.”
We were also concerned about the potential increase in risk and understood that some of the people who had registered might not want to attend as a result, and thus offered refunds and other resources they could use to make their decision (e.g. microCOVID).
One sentence in our email read: “After consulting our COVID Advisory Board, we’ve decided to increase this cap to allow those 300 people to join us in London”. This doesn’t mean that the board was our _only _source of guidance — just that consulting them was one necessary step we took before deciding to increase the cap.
For reference, the Events Team posted this about the decision in 2020. (Note that the composition of the COVID Board has changed since then.) This post isn’t just summarizing what the advisory board said — it also includes a lot of the team’s reasoning.
You write: “Apart from those few people privileged to have close links with CEA or the Oxford office, most attendees didn’t have access to the reasoning used to make the decision to expand. The arguments and data used to make that consequential decision should have been made available to the community, so that we could evaluate whether we found them convincing, challenge them if not, and update our beliefs and plans accordingly. This was not done.”
To be clear: none of the CEA Events Team was based in Oxford when this decision was made.
It’s true that we did not publish the reasoning. We did this because writing something that requires that kind of polish and care would have taken bandwidth and hours our team did (and still does) not have. We did however write a memo about this decision, which we shared with 40 community leaders (with a request for feedback), and had conversations about this with around a dozen people who attended the EA Meta Coordination Forum. We received overwhelmingly positive feedback on this decision.
I’d also like to add that while in general, I agree that transparency is good, I think you and I have pretty different pictures of what info would be most useful to the community here.
The info we prioritized getting to the community was what we thought would help them make decisions: whether to still come to the conference given the expanded size, and later the breakdown of microCOVID estimates by activity, which people could use to think about having more meetings outdoors, etc.
It sounds like you think it’s particularly important for us to share the specifics of the reasoning behind expanding the conference, beyond the points we’ve already shared. I don’t fully understand what you think would be useful to the community about having these particular details. There’s a lot of reasoning we could write up about each conference: Why does it cost this much? Why hold events in these cities and not those cities? Why have two conferences and not more? Why is this kind of food served?
Discussions and debates on these topics can be useful; sometimes, pushback from the community has helped CEA (and other organizations) recognize things we should be doing differently. But to publish all the reasoning behind all these things as it changes over time would take time that we can’t then use to actually produce events. (Especially as this work has to involve the people running the events, whose time is scarcer and harder to replace than contractor time.)
I think some of the disconnect/confusion in the follow-on discussion here is being caused by my failing to update from a model of “CEA made a difficult COVID cost/benefit calculation” to one of “CEA didn’t think COVID was a major concern” sufficiently quickly/completely. (Most of my post was written under the former model.)
I need to think more about that before I try to clarify my position here, because I’m not 100% sure what it is.