Yep, RCT was probably the wrong word. I completely agree with the power concerns. I was more thinking that if one can get some suggestive evidence that might be better than no evidence (even if power is low and the estimated effects aren’t significant at conventional levels).
MatthiasE
Karma: 29
Maybe I misunderstood, but if retreats and conferences/summits are complements, this argument should not apply? Two events being complements means that both together are more impactful than each alone. Retreats increase the cost-effectiveness of conferences, and conferences increase the cost-effectiveness of retreats. Hence, under the complementarity assumption, if resources allow running two events, one should run a summit and a retreat rather than an EAGx and a summit.
Take, for example, a recent intro fellowship graduate with impostor syndrome.
- An EAGx alone would provide them with information and shallow connections, which could be useful to get an impactful job. However, given the impostor-syndrome assumption, the fellowship graduate would not dare to contact relevant people on EAGs due to their seniority, and they would never apply or would not take the necessary steps to build career capital (they think they would never be good enough anyway).
- A retreat alone could provide them with deep connections, which could help to increase self-confidence.
- Neither event alone creates impact with this hypothetical person, but together they do. With the confidence and continued support from the retreat, the graduate might have the courage to apply for positions or feel motivated to build the necessary career capital.
Similar arguments could be made for retreats increasing motivation, commitment, the feeling of being part of a community, insights about personal fit in a cause area, etc. All of these could be gained on retreats and also increase the impact of subsequent events, such as conferences.
Whether retreats and conferences are complements or substitutes is an empirical question. I would expect substantial complementarities based on anecdotal evidence (approx. five stories).
If there is a dataset that tracks people across events, one could check if retreat+conference has larger effects than conference alone (however, interpretation might still be difficult due to self-selection).
Better data could be gained by setting up GSF retreat RCTs: After pre-selecting a pool of eligible participants (rejecting applicants who are unlikely to benefit from the event), one could randomise admission. To gain a sufficient sample size, one could aggregate data across multiple retreats. This way, one could measure causally the direct effects (people do impactful things because of having attended a retreat) and indirect effects (people profit more from conferences because of having attended a retreat).
Outside view: If I got WID data right: net personal wealth of US top percentile increased from $.59 Million in 1820 to $13.53 Million in 2024. For the bottom two deciles of India it increased from $58 to $228.
The industrial revolution made some people very rich, but not others. Why would transformative AI make everybody incredibly rich?
See also https://intelligence-curse.ai/
I used: Average net personal wealth, all ages, equal split, Dollar $ ppp constant (2024)
(I’m new to WID database and did not have time to read the data documentation. Let me know if I interpret data wongly.) Source: https://wid.world/