Tl;DR: we basically agree. We think the number of connections is (one of!) our decent, measurable proxies for Good Things Happening but we could do better and we’re working on that.
Yeah, in the abstract, I’m skeptical of the way you are measuring this, because you are measuring quantity and not quality. You don’t just want “more connections”, you want more connections that lead somewhere
Yes, we agree. We’re working on ideas that actually capture the “lead somewhere” part. This might be impact-adjusted connections or, more crudely, “critical connections” (connections that actually cause something good to happen).
You can fix the reduced number of senior-senior connections by having separate senior-only events, but that then creates fewer senior-junior connections.
Our (pretty scrappy) data suggests that the proportion of attendees who made “professional connections” was about the same in 2021 as previous conferences (~80%). The question for this was an option to select “I made an important professional connection (e.g. a potential career or hiring opportunity, or maybe a potential funder or collaborator)”. This sounds like it captures a lot of what you’d call “senior-junior connections”, but includes some senior-senior connections (funder/collaborator).
the promisingness of the average participant was noticeably lower than in previous years, and that this effect made it harder for me specifically to find people that I got something out of talking with
I’m sorry you found the conference a bit less valuable yourself. We’re hoping to build out our events portfolio such that more experienced EAs can continue to get a lot of value out of conferences, even as EAGs grow. Do let me know (here or via ollie@eaglobal.org) if you have ideas for this.
You could ask a random number of participants to actually follow up on their reported connections, and see whether they respond, or you could ask in a few months whether people actually followed up (maybe you are already doing this, though).
I share your scepticism. We already have plans to do this but this is a helpful nudge to make sure it happens!
It’s also not clear to me whether this is an interim update (in which case I’m probably being unnecessarily harsh), or whether this is the extent of your evaluation (in which case I am somewhat worried, the same as I would be if 80,000 hours only measured “number of people we had calls with”, rather than “number of importance-adjusted career changes”).
It’s an interim update, reporting immediate event outcomes, not the extent of our evaluation :)
Thanks for your comment Nuno, we do really appreciate the constructive feedback!
Ollie here from CEA’s events team
Tl;DR: we basically agree. We think the number of connections is (one of!) our decent, measurable proxies for Good Things Happening but we could do better and we’re working on that.
Yes, we agree. We’re working on ideas that actually capture the “lead somewhere” part. This might be impact-adjusted connections or, more crudely, “critical connections” (connections that actually cause something good to happen).
Our (pretty scrappy) data suggests that the proportion of attendees who made “professional connections” was about the same in 2021 as previous conferences (~80%). The question for this was an option to select “I made an important professional connection (e.g. a potential career or hiring opportunity, or maybe a potential funder or collaborator)”. This sounds like it captures a lot of what you’d call “senior-junior connections”, but includes some senior-senior connections (funder/collaborator).
I’m sorry you found the conference a bit less valuable yourself. We’re hoping to build out our events portfolio such that more experienced EAs can continue to get a lot of value out of conferences, even as EAGs grow. Do let me know (here or via ollie@eaglobal.org) if you have ideas for this.
I share your scepticism. We already have plans to do this but this is a helpful nudge to make sure it happens!
It’s an interim update, reporting immediate event outcomes, not the extent of our evaluation :)
Thanks for your comment Nuno, we do really appreciate the constructive feedback!
Great to hear!