Interesting results, thanks for sharing! I think getting data from people who attend events is an important source of information about what’s working and what’s not.
I do worry a bit about what’s best for the world coming apart from what people report as being valuable to them. (This comment ended up a bit rambley, sorry.)
Two main reasons that might be the case:
If the event causes someone’s goals or motivations to change in a way that’s great for the world, my guess is that doesn’t feel valuable to the person compared to helping the person get or do things they already want.
Eg if someone isn’t that on board with the EA project, then them getting more enthusiastic about making it an important priority in their life could be very good for the world, but feel like only a small personal benefit from the event (maybe it feels like “I had a good time and felt more excited”—and it’s very hard to tell from a report like that whether the person is now much more likely to go on to do the EA project well, or whether it’s not really going to affect their actions).
Eg if someone is attached to a cause area or job type that they have existing connections with, but the event nudges them to seriously consider other possibilities that are in fact more valuable, then this could be a great outcome that again doesn’t feel that valuable to the individual.
If the people attending the event don’t have that good an understanding of what’s in fact counterfactually valuable for achieving their goals.
Eg they might report that the event caused something to happen, but that thing would likely have happened anyway. Eg learning about something, getting funding, getting a job.
Eg they might just overrate or underrate the importance of specific outcomes (specific relationships, changes to motivation, specific ideas).
I think these reasons are actually important enough for people professionally involved in community building to try to beat the baseline of “let’s do things that people report as valuable” by trying to build a detailed understanding of the mechanisms that cause someone to go on to do things that are valuable for the world (weighted by their value). How exactly do their different motivations, beliefs and experiences fit together? Is there a typical “journey”, or maybe several different journeys? Are there certain things that are necessary in order for people to go on to do great work, and in particular are there things that individuals are likely to underrate?
Of course this happens some amount, but I’d be keen to see more discussion of this in general among people doing EA meta work.
(Something like the 2020 OP longtermist survey but much more focused on understanding the mechanisms that caused the good work, rather than the categories of thing that the people interacted with. Rather than a survey, maybe more like in-depth user interviews. I think 80k may have done a bit of this, I’m not sure.)
Thanks Isaac, I agree relying on self-report is a key limitation here. In fact, when reviewing people’s stories I would often wish they expanded on something that seemed small to them but important to us (e.g. they’d write “became more interested in pursuing X path” as part of a list, but that stood out to me as something exciting from an impact perspective).
I didn’t mention this in the report, but I also do user interviews fairly regularly to get some more colour on things like this and followed up with several people whose stories seemed impactful. I wouldn’t say the events team are following the baseline of “let’s do things that people report as valuable”, and are just using that as one guiding light (albeit a significant one). I agree forming a clearer framework of how people arrive at impactful work would be exciting.
Interesting results, thanks for sharing! I think getting data from people who attend events is an important source of information about what’s working and what’s not.
I do worry a bit about what’s best for the world coming apart from what people report as being valuable to them. (This comment ended up a bit rambley, sorry.)
Two main reasons that might be the case:
If the event causes someone’s goals or motivations to change in a way that’s great for the world, my guess is that doesn’t feel valuable to the person compared to helping the person get or do things they already want.
Eg if someone isn’t that on board with the EA project, then them getting more enthusiastic about making it an important priority in their life could be very good for the world, but feel like only a small personal benefit from the event (maybe it feels like “I had a good time and felt more excited”—and it’s very hard to tell from a report like that whether the person is now much more likely to go on to do the EA project well, or whether it’s not really going to affect their actions).
Eg if someone is attached to a cause area or job type that they have existing connections with, but the event nudges them to seriously consider other possibilities that are in fact more valuable, then this could be a great outcome that again doesn’t feel that valuable to the individual.
If the people attending the event don’t have that good an understanding of what’s in fact counterfactually valuable for achieving their goals.
Eg they might report that the event caused something to happen, but that thing would likely have happened anyway. Eg learning about something, getting funding, getting a job.
Eg they might just overrate or underrate the importance of specific outcomes (specific relationships, changes to motivation, specific ideas).
I think these reasons are actually important enough for people professionally involved in community building to try to beat the baseline of “let’s do things that people report as valuable” by trying to build a detailed understanding of the mechanisms that cause someone to go on to do things that are valuable for the world (weighted by their value). How exactly do their different motivations, beliefs and experiences fit together? Is there a typical “journey”, or maybe several different journeys? Are there certain things that are necessary in order for people to go on to do great work, and in particular are there things that individuals are likely to underrate?
Of course this happens some amount, but I’d be keen to see more discussion of this in general among people doing EA meta work.
(Something like the 2020 OP longtermist survey but much more focused on understanding the mechanisms that caused the good work, rather than the categories of thing that the people interacted with. Rather than a survey, maybe more like in-depth user interviews. I think 80k may have done a bit of this, I’m not sure.)
Thanks Isaac, I agree relying on self-report is a key limitation here. In fact, when reviewing people’s stories I would often wish they expanded on something that seemed small to them but important to us (e.g. they’d write “became more interested in pursuing X path” as part of a list, but that stood out to me as something exciting from an impact perspective).
I didn’t mention this in the report, but I also do user interviews fairly regularly to get some more colour on things like this and followed up with several people whose stories seemed impactful. I wouldn’t say the events team are following the baseline of “let’s do things that people report as valuable”, and are just using that as one guiding light (albeit a significant one). I agree forming a clearer framework of how people arrive at impactful work would be exciting.