In my experience, many EAs help and support others in the community (e.g., by giving feedback, emotional support, or making connections etc).
These āhelpful EAsā often improve the impact of those who receive their help (e.g., because the receivers start new collaborations, or improve their productivity or career choice etc). Iāll call this impact āindirect interpersonal impactā.
Most helpful EAās indirect interpersonal impacts are illegible (i.e., hard to capture/āshow). This means that many EAs who have high indirect interpersonal impact (e.g., via helping many others or being a good knowledge broker/āconnector etc) are undervalued relative to those who mostly focus on doing their own projects(but who may benefit from the help of many others).
I think that this is probably important to address. It seems important to acknowledge and recognize the contributions of individuals who may not necessarily have a tangible output or project to show for their efforts, but may still have had a significant positive impact on others.
With the above in mind, I am wondering if anyone has a form to capture indirect interpersonal impacts or similar, or some resources that they use or recommend using?
I am not aware of anything which exists. I would like to either adapt or make something to use myself and share with others. I think that 80,000ā²s evaluation model is probably the best template to work from, but I havenāt investigated that yet.
Iād also welcome any thoughts on the claims made above and whether they resonate or seem incorrect.
Hey! I want to give my perspective as a person who does meta work (software career coaching). Iām sharing this as another perspective, not as a disagreement.
Soā¦ Iām actually afraid of getting too much credit, not too little.
(Almost?) everyone I talk to says itās super helpful and so on to the point that I think itās a culture norm of complimenting the conversation partner, not a correlation to impact.
I do some things to make the impact measurement more objective (I can elaborate), but the reason is in order to make sure Iām having impact at all, and that Iām not deluding myself. I can say much more on this.
Another perspective on the āgetting too much creditā problem:
For example, if I send someone to the 80k job board, and they take a job in GiveDirectly, and this causes $1M of extra donations.. is the impact āmineā? 80kās? GiveDirectlyās? The developerās? The individual donor? This problem of ādouble countingā impact is very common in meta work, I think, and it too makes people (like me) probably think we are helping more than we actually are (it āfeelsā like I made $1M of impact).
I donāt know, maybe Iām the only meta person worrying about this. Not saying this is the right answer, just that thereās another side to āgetting too little creditā, and it is āgetting too muchā.
(I noticed Iāve been nerd sniped and will probably never stop typing here, so Iām just clicking āsubmitā, excuse my incomplete answer)
See also Triple counting impact in EA (and comments) on the problem of ādouble countingā.
I do think thereās much more to be said on this topic though, and it probably depends on things like your model of the distribution of impact and various counterfactuals. Shapley values: Better than counterfactuals could also be a useful way to think about these things (in some specific cases, I personally donāt think theyāre usually better).
Thanks, Yonatan, this is helpful! I agree that the impact accounting is hard here. I still think that encouraging a norm of making imprecise & short-term impact measurements and reporting them with the right level of uncertainty is probably better than the current situation.
Itās similar to how I think it is good, on aggregate, that EA community organisers evaluate the impact of conferences and events with surveys, despite also believing that such feedback data is relatively inaccurate, not particularly suited to assess counterfactual impact and open to misinterpretation.
I may change my mind on all that, though! On that note, I am interested to know what your evaluation system involves and why you worry that you might be deluding yourself?
Things I do to measure myself:
I ask at the beginning of almost every conversation āwhatās your default plan? what would you do if we wouldnāt talk today?ā, and in most conversations I ask once or twice later āwhatās your default plan now?ā
To be clear, my standard of āmade a differenceā is something like ā[expect to have] saved a few months of the career of the personā
Almost all my conversations are recorded and also in a Google Doc, which means
I can go over my own work (in practice I only did this once, when I had a small breakdown and thought Iām maybe deluding myself, so I went over lots of calls)
Itās ready for a 3rd party to do an impact analysis on my work
I think this has a psychological effect on me, somewhat thinking what someone else would say about this
I sometimes ask for (or spontaneously get) feedback after the call, mostly in the comments to one of my posts
In about half the calls I ask for feedback at the end of the call (and I donāt take anything vague like āthis was usefulā as an indication of impact, only things like āthis is where I changed my planā)
I think I āshouldā do (Iād do this if I had more resources, or a 3rd party interested)
Tidy impact analysis, going over my calls, contacting people I spoke to, more similar to what 80k do
The crazy version here is a RCT. I specifically expect my CV exercise might get impressive results in raising peopleās salary if someone would run an RCT on it
The thing that changed my mind during the small-breakdown and got me to believe that yes, Iām useful:
Specific stories of pretty big impact that people told me about months after speaking to me. Hereās one I got yesterday by surprise and made me really happy, even trying hard to analyze counterfactuals which I really appreciate