EAGxNYC 2023: Retrospective
What is the purpose of this post?
Building on past EAGx retrospectives (1)(2)(3), this short post is intended to serve as a general recap/update about the event from the organizers (independent contractors who do not speak for CEA). We also hope to build on this series on EA event practices by offering suggestions and asking for further comments about how we can improve EA conferences in the future.
A long-form retrospective, intended primarily as advice for future EAGx planners and feedback for the CEA events team from our EAGx organizing team, includes more specifics about our takeaways from the event planning process.
Event Overview
We consider EAGxNYC, NYC’s first-ever EA conference, to be a definite success. The average number of connections per attendee, one of CEA’s primary metrics for conference value, was ~10.9, the highest reached at a US EAGx event. However, even with ~5,400 connections made, the total cost in dollars-per-connection was still high relative to most other EAGx events. This is almost entirely due to the expense of venue rental in NYC at a peak time of year, and on relatively short notice.
Most respondents of the attendee survey indicated a high likelihood of recommending a similar event (8.94/10), and many complaints were for impossible-to-optimize variables (e.g. a roughly balanced number of respondents indicated that talks were “too general” vs “too niche”). While impact of the conference is hard to evaluate shortly after the event, several attendees have reported important career connections, high chances of new hires, donation pledges taken, and research collaborations that would not have occurred counterfactually.
High-level facts and figures
Dates: August 18-20, 2023
Location: Convene, near the Financial District of Manhattan
~655 Applications --> ~540 accepted applications and 75 direct invites --> ~510 registered attendees (of whom 50 were awesome volunteers!)
70 Speakers
89 Sessions of programming (talks, panels, speed friending, meetups, and office hours)
20 Orgs represented at the Career Fair
~5400 Estimated connections made based on feedback survey, ~10.9 per person (6,954 connections on Swapcard, ~13.4 per person)
8 light-hearted awards bestowed (including “Most 1-1s held” and “Most 1-1s canceled”, to the same person!)
Things we think we did well:
Active matchmaking: We tried to aid organic networking by offering a large number of unstructured meetups, with facilitators when possible, for both demographic groups and cause-area focus, as well as providing stewardship (connecting those whose applications indicated they may be particularly capable of mentoring with those who seemed best positioned to benefit from a mentor). The feedback on these efforts indicate they were among the best uses of our organizing time.
Persistence: Both in getting the conference to happen in the first place, and in finding a suitable venue. Given the timing (one of the first conferences to be greenlit after the change in the funding landscape of late-2022), there were many points at which it seemed like EAGxNYC wouldn’t be able to happen. After hunting through well over one hundred potential venues, we were able to find and negotiate a high caliber venue in a convenient, central location in downtown Manhattan. Some attendees expressed surprise (and even concern) at the venue’s apparent luxuriousness. And while it was not inexpensive (little in Manhattan ever is), the conference location was actually the lowest cost-per-attendee venue available among the many options we found.
Things we hope others might learn from:
We found the advice in the Events in EA: Learnings & Critiques series invaluable, and want to direct more eyes towards it. Our own takeaways mostly supplement the information there.
Timing: The Learnings series advised us to start organizing early. We took that into consideration, and still came away wanting (cf. Hofstadter’s Law) to suggest that people start even earlier. This applies both to our own planning work but also to others involved in the process (e.g. presenters and applicants/attendees). Specifics of project management are included in the longform retrospective, but we think it is valuable to share anecdata here about the consequences of perceived EA-norms of short timelines[1].
For example, our conference was advised to set the application deadline eighteen days before the conference. This was considered fairly early by EAGx standards, which often have applications open up to two weeks (or less!) before the conference. Accepted applicants are given the option to register up until the day of the conference and registrants are fully reimbursed for their ticket if they cancel at any time. These protocols are based on well-intentioned, impact-focused rationale (i.e. making the conferences available and convenient to more people and thereby maximizing the number of good candidates who can attend; many good candidates do not know they are available until shortly before the event).
However, there are also significant costs as well. The largest is that it adversely affects organizers’ ability to calibrate the admissions bar. For example, without knowing the total number of applicants or their average fit for the conference, the admissions team may waitlist some candidates who deliberately applied early (e.g. in order to book travel or time off). By the time it becomes clear that these waitlisted applicants will clear the bar, they may no longer be able to come. In this case, the conscientious early-applier loses out to a norm that caters to those who don’t apply until just before the conference. Late applications/registrations also affect production elements like merchandise and name-tag printing, and can dramatically swing the cost of the event when the venue and/or catering charges by headcount[2].
In the end, ~24% of applications were received in the last four days of the application window, less than three weeks before the event. We have significant anecdotal evidence that many of these were applicants who knew well in advance that they intended to come. Many applicants who applied early and were initially waitlisted before being accepted did not end up attending; some were likely in the category described above. Final registration numbers changed up until the day of the conference (both late registrants and late cancellations, some of which canceled each other out but still left a >5% swing in headcount in the last two days). This led to merch supply issues, budget uncertainty, and likely inflated venue/catering costs[3].
Once it became clear that we were paying for more attendees than would show, we granted admission to some late applicants who had reasonable explanations for not applying on time. This choice was made based on the same “maximize marginal impact” rationale as the choice to leave applications and registrations open until shortly before the conference. We nonetheless regret contributing to the perception that deadlines are optional, and encourage future event organizers to consider this norm-setting element when considering late applicants.
We heard from many other organizers that these “last-minute” issues were not unique to our event. The organizing team has experience with non-EA events (academic, professional, and trade conferences) that indicate that short timelines and their attendant consequences are not essential to event planning. They are not unique to EA, but they are avoidable and we believe EA events could benefit from moving towards earlier confirmation by all parties involved. It’s not just that it’s recommended to cancel your EAGx ticket as early as possible if you can’t attend, our takeaway is that it’s beneficial if applications, registrations and cancellations are all made as early as possible, as well as aiming as early as possible for all organizing elements, especially venue choice and speaker confirmations.
Given CEA’s experience of experimenting with earlier deadlines (which were then extended to reach capacity), it is unlikely that this move can or will come solely from top-down changes. This advice is therefore a general nudge to the broader EA community to move towards earlier action. We included some of the specifics of our conference’s consequences and our understanding of why the deadlines are set so late to try to make the point that even if the deadline is shortly before the conference, you shouldn’t anchor on that date. You can help organizers and actually improve EA events by applying and registering (and canceling if necessary) as soon as you can.
Working on an EAGx: The EAG/EAGx paradigm, where EAG’s are managed by CEA’s dedicated events team but EAGx’s are one-off teams, means that EAGx organizers will typically be operating under timescales where thorough vetting and team-building are difficult, and turnover is especially hard to manage. Our team came together in a very short time and in a fairly ad hoc manner, (possibly more so than other EAGx teams), so we aim to offer some suggestions to mitigate the difficulties of the process.
We encourage those that run EAGx events to consider requiring an application round (even if you have a list of people ready and willing to contribute), and for those that apply to consider the pros and cons of a medium-term, project-based contract. We believe that events are more likely to go well, and the individuals involved to benefit professionally, if the uncertain job demands are considered in advance and it is understood that one’s ability to handle their given responsibilities could substantially impact, positively or negatively, their career in the EA ecosystem. While it is definitely possible to fill some organizing duties as a “side-hustle,” it is important to prepare for complications to arise from this.
Executing an EAGx event is an unpredictable job, as it’s likely that every event will be substantively different from other EAGx conferences. But in our experience, it can also be an opportunity for personally satisfying work, an important career step, and generate positive impact on the EA community and the world.
- ^
Not those short timelines.
- ^
Almost all catering charges by headcount, but when rented in full, most venues do not charge extra per person. The conference center booked for EAGxNYC charged per-person for both catering and venue occupancy as part of a “meeting package,” and therefore the financial uncertainty due to late confirmation was especially pronounced.
- ^
Because of the above “meeting package” pricing, the consequence of underestimating total attendance and having more people show than were declared would be that CEA would be charged full price for every attendee, but that the venue could not guarantee sufficient portions for everyone. So without a firm headcount, we erred on the side of overestimating.
Thanks for sharing!
I previously suggested the wait-list as a way to deal with the late applicant. Another strategy would be to accept late applications, and evaluate them normally, but let people know there is a $50/day (or whatever level) surcharge for late applicants, and no discounts available. This has a number of positive effects:
It deters people from applying late.
No hard cutoff, so people who just missed the deadline are still incentivized to be as early as they can.
Late applicants compensate you for the costs they impose rather than free-riding.
Deter late applicants who don’t value the conference that much.
It’s credible you might actually enforce this, in contrast to the difficulty in actually rejecting a high-value late applicant when the time comes.
That sounds great, kudos to the events team and the volunteers!
This is a nitpick, but it’s not clear to me that that example is impossible to optimize. For example, maybe you could increase the variance in how general/niche talks are. If 50% of talks are really general, and 50% of talks are super niche, maybe that would satisfy more people than the status quo!
Thanks for the kind words!
To address the nit: Before changing it to “impossible-to-optimize variables,” I had “things where it is impossible to please everyone.” I think that claim is straightforwardly true, and maybe I should have left it there, but it doesn’t seem to communicate everything I was going for. It’s not just that attendees come in with mutually exclusive preferences, but from the organizers perspective it is practically impossible to chase optimality. We don’t have control over everything in presenters’ talks, and don’t have intimate knowledge of every attendees’ preferences, so complaints are, IMHO, inevitable (and that’s what I wanted to communicate to future organizers).
That said, I think we could have done somewhat better with our content list, mostly via getting feedback from applicants earlier so we could try to match cause-area supply and demand. For content depth, we aimed for some spread but for the majority of talks to be clustered on the medium-to-high side of EA familiarity (i.e. if a “1” was “accessible to anyone even if they’ve never heard of EA” and “10” was “only useful to a handful of professional EA domain experts,” then we aimed for a distribution centered around 7. We only included talks at the low end if we considered them uniquely useful, like a “How to avoid burnout” talk that, while being geared towards EAs, did not require lots of EA context).
I think, given that we selected for attendees with demonstrated EA activity, that this heuristic was pretty solid. Nothing in the feedback data would have me change it for the next go-around or advise other organizers to use a different protocol (unless, of course, they were aiming for a different sort of audience). But I’m happy for anyone to offer suggestions for improvement!
Thanks guys! I support what you were saying about application timeliness, expectation management, etc. This seems like a super reasonable set of norms.
I overall thought you crushed it, no notes, etc. My literal only grievance was that there were a bajillion forlorn nametags on the table of people I specifically had important world-saving business to check in with on, so the no-shows definitely lowered my productivity at the conf. I ended up being surprised by great discussions that popped up in spite of not having the ones I had hoped to, so I’m not complaining.
I was one of the “not supposed to be on that coast that weekend” people who had a bunch of stuff fall apart / come together at the last minute, it literally was the wednesday or thursday of the week itself that I was confident I would not be tied up in california---- so I’m wondering, should I have applied on time with an annotated application “I’m 90% sure I can’t make it, but it’ll be easier for me to be in the system already if we end up in the 10% world”? At this point, it becomes important for me to update the team so I don’t impose the costs like the forlorn nametags or anything else discussed in post, but those updates themselves increase the overall volume of comms / things to keep track of for both me and the staff, which is also a cost.
Even if my particular case is too extreme and unusual to apply to others, I hope norms or habits get formed in the territory of “trying to be thoughtful at all” cuz it sounds like we’re at the stage where we only have to be directionally correct.
I really appreciate and agree with “trying to be thoughtful at all” and “directionally correct,” as the target group to be nudged is those who see a deadline and wait until the end of the window (to look at it charitably, maybe they don’t know that there’s a difference in when they apply. So we’re just bringing it to their attention.)
We appreciate that there are genuine cases where people are unsure. I think in your case, the right move would’ve been to apply with that annotation; you likely would have been accepted and then been able to register as soon as you were sure.