In my experience, many EAs help and support others in the community (e.g., by giving feedback, emotional support, or making connections etc).
These ‘helpful EAs’ often improve the impact of those who receive their help (e.g., because the receivers start new collaborations, or improve their productivity or career choice etc). I’ll call this impact ‘indirect interpersonal impact’.
Most helpful EA’s indirect interpersonal impacts are illegible (i.e., hard to capture/show). This means that many EAs who have high indirect interpersonal impact (e.g., via helping many others or being a good knowledge broker/connector etc) are undervalued relative to those who mostly focus on doing their own projects(but who may benefit from the help of many others).
I think that this is probably important to address. It seems important to acknowledge and recognize the contributions of individuals who may not necessarily have a tangible output or project to show for their efforts, but may still have had a significant positive impact on others.
With the above in mind, I am wondering if anyone has a form to capture indirect interpersonal impacts or similar, or some resources that they use or recommend using?
I am not aware of anything which exists. I would like to either adapt or make something to use myself and share with others. I think that 80,000′s evaluation model is probably the best template to work from, but I haven’t investigated that yet.
I’d also welcome any thoughts on the claims made above and whether they resonate or seem incorrect.
Hey! I want to give my perspective as a person who does meta work (software career coaching). I’m sharing this as another perspective, not as a disagreement.
So… I’m actually afraid of getting too much credit, not too little.
(Almost?) everyone I talk to says it’s super helpful and so on to the point that I think it’s a culture norm of complimenting the conversation partner, not a correlation to impact.
I do some things to make the impact measurement more objective (I can elaborate), but the reason is in order to make sure I’m having impact at all, and that I’m not deluding myself. I can say much more on this.
Another perspective on the “getting too much credit” problem:
For example, if I send someone to the 80k job board, and they take a job in GiveDirectly, and this causes $1M of extra donations.. is the impact “mine”? 80k’s? GiveDirectly’s? The developer’s? The individual donor? This problem of “double counting” impact is very common in meta work, I think, and it too makes people (like me) probably think we are helping more than we actually are (it “feels” like I made $1M of impact).
I don’t know, maybe I’m the only meta person worrying about this. Not saying this is the right answer, just that there’s another side to “getting too little credit”, and it is “getting too much”.
(I noticed I’ve been nerd sniped and will probably never stop typing here, so I’m just clicking “submit”, excuse my incomplete answer)
See also Triple counting impact in EA (and comments) on the problem of “double counting”.
I do think there’s much more to be said on this topic though, and it probably depends on things like your model of the distribution of impact and various counterfactuals. Shapley values: Better than counterfactuals could also be a useful way to think about these things (in some specific cases, I personally don’t think they’re usually better).
Thanks, Yonatan, this is helpful! I agree that the impact accounting is hard here. I still think that encouraging a norm of making imprecise & short-term impact measurements and reporting them with the right level of uncertainty is probably better than the current situation.
It’s similar to how I think it is good, on aggregate, that EA community organisers evaluate the impact of conferences and events with surveys, despite also believing that such feedback data is relatively inaccurate, not particularly suited to assess counterfactual impact and open to misinterpretation.
I may change my mind on all that, though! On that note, I am interested to know what your evaluation system involves and why you worry that you might be deluding yourself?
Things I do to measure myself:
I ask at the beginning of almost every conversation “what’s your default plan? what would you do if we wouldn’t talk today?”, and in most conversations I ask once or twice later “what’s your default plan now?”
To be clear, my standard of “made a difference” is something like “[expect to have] saved a few months of the career of the person”
Almost all my conversations are recorded and also in a Google Doc, which means
I can go over my own work (in practice I only did this once, when I had a small breakdown and thought I’m maybe deluding myself, so I went over lots of calls)
It’s ready for a 3rd party to do an impact analysis on my work
I think this has a psychological effect on me, somewhat thinking what someone else would say about this
I sometimes ask for (or spontaneously get) feedback after the call, mostly in the comments to one of my posts
In about half the calls I ask for feedback at the end of the call (and I don’t take anything vague like “this was useful” as an indication of impact, only things like “this is where I changed my plan”)
I think I “should” do (I’d do this if I had more resources, or a 3rd party interested)
Tidy impact analysis, going over my calls, contacting people I spoke to, more similar to what 80k do
The crazy version here is a RCT. I specifically expect my CV exercise might get impressive results in raising people’s salary if someone would run an RCT on it
The thing that changed my mind during the small-breakdown and got me to believe that yes, I’m useful:
Specific stories of pretty big impact that people told me about months after speaking to me. Here’s one I got yesterday by surprise and made me really happy, even trying hard to analyze counterfactuals which I really appreciate