EAGxNYC 2023: Retrospective
What is the purpose of this post?
Building on past EAGx retrospectives (1)(2)(3), this short post is intended to serve as a general recap/âupdate about the event from the organizers (independent contractors who do not speak for CEA). We also hope to build on this series on EA event practices by offering suggestions and asking for further comments about how we can improve EA conferences in the future.
A long-form retrospective, intended primarily as advice for future EAGx planners and feedback for the CEA events team from our EAGx organizing team, includes more specifics about our takeaways from the event planning process.
Event Overview
We consider EAGxNYC, NYCâs first-ever EA conference, to be a definite success. The average number of connections per attendee, one of CEAâs primary metrics for conference value, was ~10.9, the highest reached at a US EAGx event. However, even with ~5,400 connections made, the total cost in dollars-per-connection was still high relative to most other EAGx events. This is almost entirely due to the expense of venue rental in NYC at a peak time of year, and on relatively short notice.
Most respondents of the attendee survey indicated a high likelihood of recommending a similar event (8.94/â10), and many complaints were for impossible-to-optimize variables (e.g. a roughly balanced number of respondents indicated that talks were âtoo generalâ vs âtoo nicheâ). While impact of the conference is hard to evaluate shortly after the event, several attendees have reported important career connections, high chances of new hires, donation pledges taken, and research collaborations that would not have occurred counterfactually.
High-level facts and figures
Dates: August 18-20, 2023
Location: Convene, near the Financial District of Manhattan
~655 Applications --> ~540 accepted applications and 75 direct invites --> ~510 registered attendees (of whom 50 were awesome volunteers!)
70 Speakers
89 Sessions of programming (talks, panels, speed friending, meetups, and office hours)
20 Orgs represented at the Career Fair
~5400 Estimated connections made based on feedback survey, ~10.9 per person (6,954 connections on Swapcard, ~13.4 per person)
8 light-hearted awards bestowed (including âMost 1-1s heldâ and âMost 1-1s canceledâ, to the same person!)
Things we think we did well:
Active matchmaking: We tried to aid organic networking by offering a large number of unstructured meetups, with facilitators when possible, for both demographic groups and cause-area focus, as well as providing stewardship (connecting those whose applications indicated they may be particularly capable of mentoring with those who seemed best positioned to benefit from a mentor). The feedback on these efforts indicate they were among the best uses of our organizing time.
Persistence: Both in getting the conference to happen in the first place, and in finding a suitable venue. Given the timing (one of the first conferences to be greenlit after the change in the funding landscape of late-2022), there were many points at which it seemed like EAGxNYC wouldnât be able to happen. After hunting through well over one hundred potential venues, we were able to find and negotiate a high caliber venue in a convenient, central location in downtown Manhattan. Some attendees expressed surprise (and even concern) at the venueâs apparent luxuriousness. And while it was not inexpensive (little in Manhattan ever is), the conference location was actually the lowest cost-per-attendee venue available among the many options we found.
Things we hope others might learn from:
We found the advice in the Events in EA: Learnings & Critiques series invaluable, and want to direct more eyes towards it. Our own takeaways mostly supplement the information there.
Timing: The Learnings series advised us to start organizing early. We took that into consideration, and still came away wanting (cf. Hofstadterâs Law) to suggest that people start even earlier. This applies both to our own planning work but also to others involved in the process (e.g. presenters and applicants/âattendees). Specifics of project management are included in the longform retrospective, but we think it is valuable to share anecdata here about the consequences of perceived EA-norms of short timelines[1].
For example, our conference was advised to set the application deadline eighteen days before the conference. This was considered fairly early by EAGx standards, which often have applications open up to two weeks (or less!) before the conference. Accepted applicants are given the option to register up until the day of the conference and registrants are fully reimbursed for their ticket if they cancel at any time. These protocols are based on well-intentioned, impact-focused rationale (i.e. making the conferences available and convenient to more people and thereby maximizing the number of good candidates who can attend; many good candidates do not know they are available until shortly before the event).
However, there are also significant costs as well. The largest is that it adversely affects organizersâ ability to calibrate the admissions bar. For example, without knowing the total number of applicants or their average fit for the conference, the admissions team may waitlist some candidates who deliberately applied early (e.g. in order to book travel or time off). By the time it becomes clear that these waitlisted applicants will clear the bar, they may no longer be able to come. In this case, the conscientious early-applier loses out to a norm that caters to those who donât apply until just before the conference. Late applications/âregistrations also affect production elements like merchandise and name-tag printing, and can dramatically swing the cost of the event when the venue and/âor catering charges by headcount[2].
In the end, ~24% of applications were received in the last four days of the application window, less than three weeks before the event. We have significant anecdotal evidence that many of these were applicants who knew well in advance that they intended to come. Many applicants who applied early and were initially waitlisted before being accepted did not end up attending; some were likely in the category described above. Final registration numbers changed up until the day of the conference (both late registrants and late cancellations, some of which canceled each other out but still left a >5% swing in headcount in the last two days). This led to merch supply issues, budget uncertainty, and likely inflated venue/âcatering costs[3].
Once it became clear that we were paying for more attendees than would show, we granted admission to some late applicants who had reasonable explanations for not applying on time. This choice was made based on the same âmaximize marginal impactâ rationale as the choice to leave applications and registrations open until shortly before the conference. We nonetheless regret contributing to the perception that deadlines are optional, and encourage future event organizers to consider this norm-setting element when considering late applicants.
We heard from many other organizers that these âlast-minuteâ issues were not unique to our event. The organizing team has experience with non-EA events (academic, professional, and trade conferences) that indicate that short timelines and their attendant consequences are not essential to event planning. They are not unique to EA, but they are avoidable and we believe EA events could benefit from moving towards earlier confirmation by all parties involved. Itâs not just that itâs recommended to cancel your EAGx ticket as early as possible if you canât attend, our takeaway is that itâs beneficial if applications, registrations and cancellations are all made as early as possible, as well as aiming as early as possible for all organizing elements, especially venue choice and speaker confirmations.
Given CEAâs experience of experimenting with earlier deadlines (which were then extended to reach capacity), it is unlikely that this move can or will come solely from top-down changes. This advice is therefore a general nudge to the broader EA community to move towards earlier action. We included some of the specifics of our conferenceâs consequences and our understanding of why the deadlines are set so late to try to make the point that even if the deadline is shortly before the conference, you shouldnât anchor on that date. You can help organizers and actually improve EA events by applying and registering (and canceling if necessary) as soon as you can.
Working on an EAGx: The EAG/âEAGx paradigm, where EAGâs are managed by CEAâs dedicated events team but EAGxâs are one-off teams, means that EAGx organizers will typically be operating under timescales where thorough vetting and team-building are difficult, and turnover is especially hard to manage. Our team came together in a very short time and in a fairly ad hoc manner, (possibly more so than other EAGx teams), so we aim to offer some suggestions to mitigate the difficulties of the process.
We encourage those that run EAGx events to consider requiring an application round (even if you have a list of people ready and willing to contribute), and for those that apply to consider the pros and cons of a medium-term, project-based contract. We believe that events are more likely to go well, and the individuals involved to benefit professionally, if the uncertain job demands are considered in advance and it is understood that oneâs ability to handle their given responsibilities could substantially impact, positively or negatively, their career in the EA ecosystem. While it is definitely possible to fill some organizing duties as a âside-hustle,â it is important to prepare for complications to arise from this.
Executing an EAGx event is an unpredictable job, as itâs likely that every event will be substantively different from other EAGx conferences. But in our experience, it can also be an opportunity for personally satisfying work, an important career step, and generate positive impact on the EA community and the world.
- ^
Not those short timelines.
- ^
Almost all catering charges by headcount, but when rented in full, most venues do not charge extra per person. The conference center booked for EAGxNYC charged per-person for both catering and venue occupancy as part of a âmeeting package,â and therefore the financial uncertainty due to late confirmation was especially pronounced.
- ^
Because of the above âmeeting packageâ pricing, the consequence of underestimating total attendance and having more people show than were declared would be that CEA would be charged full price for every attendee, but that the venue could not guarantee sufficient portions for everyone. So without a firm headcount, we erred on the side of overestimating.
Thanks for sharing!
I previously suggested the wait-list as a way to deal with the late applicant. Another strategy would be to accept late applications, and evaluate them normally, but let people know there is a $50/âday (or whatever level) surcharge for late applicants, and no discounts available. This has a number of positive effects:
It deters people from applying late.
No hard cutoff, so people who just missed the deadline are still incentivized to be as early as they can.
Late applicants compensate you for the costs they impose rather than free-riding.
Deter late applicants who donât value the conference that much.
Itâs credible you might actually enforce this, in contrast to the difficulty in actually rejecting a high-value late applicant when the time comes.
That sounds great, kudos to the events team and the volunteers!
This is a nitpick, but itâs not clear to me that that example is impossible to optimize. For example, maybe you could increase the variance in how general/âniche talks are. If 50% of talks are really general, and 50% of talks are super niche, maybe that would satisfy more people than the status quo!
Thanks for the kind words!
To address the nit: Before changing it to âimpossible-to-optimize variables,â I had âthings where it is impossible to please everyone.â I think that claim is straightforwardly true, and maybe I should have left it there, but it doesnât seem to communicate everything I was going for. Itâs not just that attendees come in with mutually exclusive preferences, but from the organizers perspective it is practically impossible to chase optimality. We donât have control over everything in presentersâ talks, and donât have intimate knowledge of every attendeesâ preferences, so complaints are, IMHO, inevitable (and thatâs what I wanted to communicate to future organizers).
That said, I think we could have done somewhat better with our content list, mostly via getting feedback from applicants earlier so we could try to match cause-area supply and demand. For content depth, we aimed for some spread but for the majority of talks to be clustered on the medium-to-high side of EA familiarity (i.e. if a â1â was âaccessible to anyone even if theyâve never heard of EAâ and â10â was âonly useful to a handful of professional EA domain experts,â then we aimed for a distribution centered around 7. We only included talks at the low end if we considered them uniquely useful, like a âHow to avoid burnoutâ talk that, while being geared towards EAs, did not require lots of EA context).
I think, given that we selected for attendees with demonstrated EA activity, that this heuristic was pretty solid. Nothing in the feedback data would have me change it for the next go-around or advise other organizers to use a different protocol (unless, of course, they were aiming for a different sort of audience). But Iâm happy for anyone to offer suggestions for improvement!
Thanks guys! I support what you were saying about application timeliness, expectation management, etc. This seems like a super reasonable set of norms.
I overall thought you crushed it, no notes, etc. My literal only grievance was that there were a bajillion forlorn nametags on the table of people I specifically had important world-saving business to check in with on, so the no-shows definitely lowered my productivity at the conf. I ended up being surprised by great discussions that popped up in spite of not having the ones I had hoped to, so Iâm not complaining.
I was one of the ânot supposed to be on that coast that weekendâ people who had a bunch of stuff fall apart /â come together at the last minute, it literally was the wednesday or thursday of the week itself that I was confident I would not be tied up in california---- so Iâm wondering, should I have applied on time with an annotated application âIâm 90% sure I canât make it, but itâll be easier for me to be in the system already if we end up in the 10% worldâ? At this point, it becomes important for me to update the team so I donât impose the costs like the forlorn nametags or anything else discussed in post, but those updates themselves increase the overall volume of comms /â things to keep track of for both me and the staff, which is also a cost.
Even if my particular case is too extreme and unusual to apply to others, I hope norms or habits get formed in the territory of âtrying to be thoughtful at allâ cuz it sounds like weâre at the stage where we only have to be directionally correct.
I really appreciate and agree with âtrying to be thoughtful at allâ and âdirectionally correct,â as the target group to be nudged is those who see a deadline and wait until the end of the window (to look at it charitably, maybe they donât know that thereâs a difference in when they apply. So weâre just bringing it to their attention.)
We appreciate that there are genuine cases where people are unsure. I think in your case, the right move wouldâve been to apply with that annotation; you likely would have been accepted and then been able to register as soon as you were sure.