Underinvestment at the top: what I discovered coaching a dozen EA leaders

Tee1 Apr 2022 12:36 UTC
101 points
19 comments34 min readEA link

Table of Contents & Key Links

I’d like to give special thanks to the trial participants – those who were comfortable giving public testimonials include Angus Mercer, Michael Plant (Happier Lives Institute), Luke Freeman (Giving What We Can), Bridget Williams (Center for Population-Level Bioethics), Kat Woods (Nonlinear Fund), Nikita Patel (Fortify Health), Varsha Venugopal (Suvita), Irena Kotikova (EA Czechia) and Dominika Kropucin (Rethink Priorities). More thanks in this section.

Navigating this post

I’ve tried to make this rather lengthy report more ‘click-to-expand’ in order to help readers navigate it nonlinearly (more like Seinfeld, not Game of Thrones). While the quantified results carry interesting potential implications, I think pairing the Executive Summary & Overviewand Takeaways & Lessons Learned sections would provide for the reader a summary together with richer insights into what happened during the coaching trials, and what I think it could mean for the Effective Altruism community. The anticipated questions section is also pretty revealing of my experience and motivations in conducting the trial and producing the report.

Reach out at tee@teebarnett.com or my website if you’re interested in receiving my coaching, collaborating and/​or helping fund my efforts to continue building a professionalized support infrastructure in the EA community.

Executive Summary & Overview

I’m Tee – you can find more contextualizing sections about me, about my style of coaching and references for the general case for coaching towards the end of the report.

Many of the insights contained within this report are derived from November 2021 – February 2022, where I conducted a dozen free trials of my coaching with EA leaders that I feel went pretty well.[1] Across several methods of approximating value and effects, the feedback seems to suggest that the coaching is both desired and effective for those undertaking the complexities that come with doing scalable good. I think there could be something profound here for the community to consider.

Given how highly the Effective Altruism community appears to value certain forms of direct work, it feels obvious to say that, for important individuals doing this direct work, investing in the development of their capabilities could be quite impactful.

From my prior experience coaching other high-performers, I had good reason to think that coaching could be a potentially potent way to develop EA leaders (EDs/​CEOs, senior-level managers & thought leaders). Coaching is only beginning to catch on within the community it seems. However, what is currently on offer is almost exclusively reliant on opaque patterns of in-network endorsement, varies widely in quality and intended purpose, approximations of its value are almost completely anecdotal, and there are several nontrivial and very understandable barriers to access.

I thought a more rigorous trial of my coaching would be fairly revealing in regards to key questions about my own practice. But more importantly, I thought a structured trial would give me a better sense of how my coaching (and perhaps by extension, coaching in general) could meet the developmental needs of individuals who are quite important to EA.

What if, for example, key agents that we’re counting on to solve important problems can noticeably improve their capabilities? Leading an organization and/​or coordinating others to achieve scalable good is incredibly challenging. Basically nobody is natively skilled in all domains required to be a great leader.

To be clear, this report is not a commentary on the leadership fitness of the trial participants, but instead the reality that different people require many types of support. The trials made it clearer to me that investing in developing the capacities of our leaders is neglected and undervalued, and that having a vetted support infrastructure of high quality coaches for EA leaders could be hugely impactful.

Notable Results

What effects does the coaching have?

According to quantified survey data of a dozen EA leaders, the top categories of self-reported value are the following:

1 – “Strategy towards personal and professional—I have greater ability to navigate how my personal life interacts with my professional life.”

Concrete examples derived from qualitative answers

  • Discovering and working through how personal hang ups crucially affected communication with funders and co-workers

  • Self-doubt in leadership and highlighting its action-bearing effects and limitations (worthiness, insecurity about intelligence/​capability, imposter syndrome, meta-imposter syndrome, etc)

  • Making sense of emotional shear and fallout from past, ongoing and anticipated future interactions and dynamics with specific coworkers. Making plans to deal with it.

“Tee Barnett Coaching has been really helpful for me across a range of areas, from the practical to the theoretical. We’ve explored areas from the meaning of intelligence, to the role of a CEO, to building operational systems, to impostor syndrome. I leave each session with a whole lot of new insights, and an option to innovate and improve in my professional life.”

– Nikita Patel, Co-founder & CEO @ Fortify Health

2 – “Perception shift—the underlying paradigm that I use to interpret large areas of my perception have meaningfully shifted in a way that is beneficial to me.”

Concrete examples derived from qualitative answers

  • Reconceptualizing their interpretation of workplace roles in a scaling organization. Crucially this includes working through transitional friction from one role to another (e.g. shifting to prioritize collective output rather than individual output)

  • Undertaking management training as a more expansive self-development project

  • Recasting life and work roles and responsibilities in light of (desired) personal development and maturation (e.g. taking active responsibility for dysfunctional team dynamics, rather than passively avoiding and critiquing)

“[Tee] was careful in asking questions that would help both of us to understand how I perceive situations and how misperceptions are limiting my ability to do good work and have meaningful relationships. He was generous enough to go into some difficult issues and did so with care and compassion. The coaching has helped me to identify mental and behavioural patterns that are holding me back, and provided me with some strategies to correct these.”

– Bridget Williams, Postdoctoral Associate @ The Center for Population-Level Bioethics, Rutgers University

3 – “Skill & mental models—mental models that I received were directly useful to what I wanted to do and/​or sharpening a specific skill”

Concrete examples derived from qualitative answers

  • Ascertaining what funders/​managers/​co-workers actually care about and pay attention to, rather spreading too thinly by trying to optimize across too many points (i.e. ignorance- and insecurity-driven perfectionism)

  • Addressing emotionality in skill building. Specifically, working through insecurities that stymie communication skill development within an EA organization (e.g. not getting overwhelmed and disoriented by EA/​rationalist style reasoning and argumentation)

  • Structuring inbound information triaging systems. Tracking information flow inefficiencies and leakage

“Tee nudged me to consider new ways to approach several areas of my work- interaction with colleagues, thinking strategically on hiring and letting go, and fundraising. I can already see my changed approach paying dividends. Tee is brilliant at listening deeply and providing appropriate mental models to understand yourself and your approaches.”

– Varsha Venugopal, Co-founder @ Suvita

Notable results by the numbers

  • Recommendations for EA Leaders – 100% (12 of 12) reported that this coaching could be beneficial for EA leaders (see related public testimonials or jump to this section)

  • Post-trial demand – 11⁄12 feedback respondents indicated that they’d like to continue on with Tee Barnett coaching in one form or another after the trial. 10⁄12 indicated interest in continuing with a paid arrangement (column x), 4⁄12 signaled interest in non-subsidized (fully paid) sessions.

    • 6⁄11 have already begun fully paid or subsidized sessions [as of April 4, 2022]

  • Mean response to how much it would be worth to receive as a grant instead of receiving coaching trial: $5,879.17, which works out to $862.86 per hour

    • Median response to how much it would be worth to receive as a grant instead of receiving coaching trial: $2,000.00, which works out to $322.62 per hour

  • Mean response to how much coachees would pay out-of-pocket for the grant trial if making 2-5x more income: $2,618.75, which works out to $394.81 per hour over the average duration of a trial

    • Median response to how much it would be worth to pay out-of-pocket for grant trial if making 2-5x more income: $1,700.00, which works out to $264.29 per hour

  • Mean response to how much coaching trial improved how much coachees could accomplish: 13.33%

    • Median = 5%

Basic trial structure by the numbers

  • 12 trials were completed out of a planned 15. There’s no list of trial participants in this post to protect privacy, but nearly all trial participants gave public testimonials

  • 12 of 15 trials yielded valid feedback (more explanation in methodology notes section)

  • Nearly all trials spanned 4 sessions, at an average of 6.46 hours per trial

  • The cost of the trials conducted over 4 months was ~$27,000

Quantitative Results

Note: I do not take the numerical valuations of the coaching literally, nor do I think it would be wise for anyone interpreting this report to anchor too hard on this. They’re simply meant to be a snapshot of potential value. I am happily soliciting feedback on how these questions were constructed.

Spreadsheet of quantitative answers and methodology notes section

1) “If you could have received a cash grant instead of the trial package of coaching sessions, how much money would it take for you to be indifferent between the grant and coaching?”

(12 of 12 respondents answered)

  • Mean response: $5,879.17

  • Median response: $2,000.00

  • Range: $350 – $30,000.00

  • Range in # of hours in a trial: 5 − 10 hrs.

    • Average trial duration: 6.46 hrs.

  • Hourly total implications (participant valuation /​ participant trial duration)

    • Hourly mean = $862.86 per hour

    • Hourly median = $322.62 per hour

2) “If you were earning substantially more than you currently do (e.g 2 − 5x, how much would it have been worth to pay for the trial package of sessions if you were paying out of pocket?”

(12 of 12 respondents answered)

  • Mean response: $2,618.75

  • Median response: $1,700.00

  • Range: $125 – $10,000

  • Range in # of hours in a trial: 5 − 10 hrs.

    • Average trial duration: 6.46 hrs.

  • Hourly total implications (participant valuation /​ participant trial duration)

    • Hourly mean = $394.81 per hour

    • Hourly median = $264.29 per hour

3) “By what percent would you estimate coaching changed the amount you accomplished over the past month?”

(12 of 12 respondents answered)

  • Mean response: 13.33%

  • Median response: 5%

  • Range: 3% − 75%

4) [optional question later in the survey][2] “How much would it have been worth to pay for the trial package of sessions, if you were paying out of pocket? (per hour)”[2]

  • Mean response: $132.50 per hour

  • Median response: $150 per hour

  • Range: $60 – $150

Quantitative Visuals

These visualizations have been pulled directly from the data inputted by respondents via Typeform.

Satisfaction Matrix – Please rate the areas where you received value from the coaching

(See category labels below)

Category Labels

  • Perception shift—the underlying paradigm that I use to interpret large areas of my perception have meaningfully shifted in a way that is beneficial to me

  • Attentional changes—my attention tracks different aspects of reality than previously, resulting in greater awareness and/​or different actions

  • Relationship to mind—how I relate to my mind is healthier and more aligned with how I wished to be

  • Mental tools—I picked up valuable tools from the coaching that will meaningfully affect the way that I think about and approach situations in the future

  • Skill & mental models—mental models that I received were directly useful to what I wanted to do and/​or sharpening a specific skill

  • Discovery of self—I was able to uncover information about myself that I can now use to interpret meaningful things in the present

  • Improved relationships—one or more relationships improved. I’ve meaningfully benefitted after having worked through (an) aversive interaction(s).

  • Strategy towards personal and professional—I have greater ability to navigate how my personal life interacts with my professional life.

  • Deep work—I do more and/​or higher-quality focused deep work to advance my top priorities.

  • (Re)defining goals and/​or desired ways of being—I have a better handle on what goals/​ways of being are important to me and what I should prioritize towards that end.

Top 5 areas of value across trial respondents

  1. Strategy towards personal and professional—I have greater ability to navigate how my personal life interacts with my professional life.

  2. Perception shift—the underlying paradigm that I use to interpret large areas of my perception have meaningfully shifted in a way that is beneficial to me

  3. Skill & mental models—mental models that I received were directly useful to what I wanted to do and/​or sharpening a specific skill

  4. Discovery of self—I was able to uncover information about myself that I can now use to interpret meaningful things in the present

  5. Mental tools—I picked up valuable tools from the coaching that will meaningfully affect the way that I think about and approach situations in the future

How satisfied are you with Tee Barnett Coaching overall?

Did Tee Barnett Coaching meet your expectations?

Are there ways that Tee’s Coaching had a meaningful effect on you and/​or what you’re capable of doing?


Were there any situations where a shift generated by Tee’s coaching resulted in you taking different actions than you otherwise would have?

Do you feel Tee’s Coaching could be beneficial to other leaders in EA?

Would you like to recommend that someone try Tee’s coaching?

Would you like to continue being coached by Tee Barnett?

Within which compensation structure would you be open to continue being coached by Tee Barnett?

Takeaways & Lessons Learned

EA Leadership & Community Insights

Note: these are a collection of impressions derived from my experience coaching EA leaders, my experience as a co-founder of Rethink Charity and numerous informal conversations. It is not meant to come across as objective, conclusive or exhaustive by any means, though I would encourage community funders to consider directing resources toward further exploration and research in this area. Once again, these insights are not a commentary on the leadership fitness of the trial participants, but instead the reality that different people require many types of support.

  • Being unaware of the benefits – I’d say a fair characterization of many trial participants’ experience with coaching was that they hadn’t tried it because they weren’t aware of how or in what ways it could be valuable, including those that had had prior coaching relationships that were sub-par or broke down for one reason or another.

  • Perceived need for an professionalized support infrastructure – I’d firmly advocate for a vetted and professionalized support infrastructure consisting of coaches, therapists, consultants, etc. to provide crucial strategic assistance, tactical guidance and emotional support for those in positions of importance for the EA community.

    • Incredible complexity of demands on leaders – a fact which has not been lost in the private and military sectors for years now, leaders currently face formidable emotional and intellectual challenges as their organizations scale. Nobody is born with the ability to be fluidly competent in all of the domains that modern leaders are responsible for, nor could we confidently say that institutional educational experiences adequately equip leaders. On top of that, modern leaders are often spread quite thin across domains that demand a certain amount of attention and energy to be executed well.

      This line of reasoning yields that EA leaders need ongoing professional development and emotional support in order to expand their capabilities overall. For a community with organizations seeking to accomplish such difficult aims, these implications may even be more true.

  • Coaching quality control – it’s likely that sooner rather than later, EA leaders will increasingly look outward for assistance as they become better resourced (due to improved funding) and encounter troubling complexity in their work. Perhaps most of all, ambitious leaders want to experience growth. Relying on their native capabilities and implicit models can only go so far in satisfying this desire for perceived growth. Coordinated efforts to reduce the overhead of searching for coaching, determining coach-coachee fit and improving the likelihood of positive results via improvement to coaching quality on offer seems quite valuable.

    • Coordinating Monitoring & Evaluation (M&E) – in order to seriously address the idea of having some kind of standard for coaches within the community, coordinated efforts to establish commonly recognized and accepted ‘metrics’ that can reliably approximate the skill of coaches. Due to my personal epistemic belief system, and because the effects of coaching are often interpreted quite subjectively, something like ‘an informative collage of snapshots of value’ constructed among a plurality of paradigms for assessing coaching skill and expected value seems appropriate. It would be nice to somehow standardize this collage of measures.

    • Investment in M&E – properly coordinating to create and enforce an M&E infrastructure would likely require funders and grantmaking bodies to invest in the refinement of these ‘metrics’ of coaching skill, detection of effects, and determination of overall value. (A concrete example of this sort of time and attentional investment is investing in M&E research in this area. Yet another is directing attention to periodically revisit funding arrangements and outcomes.)

  • Need for referral options – some coaches are simply a better fit for certain coachees than others. I had an instance where I was able to refer one of the trial recipients to another skilled coach. This greatly enhanced the experience of the coachee because the other skilled coach was able to give alternative and complementary insight and assistance. (Feedback from this coachee was invalidated due to the successful inclusion of another coach)

  • Anecdotal projections on the value of coaching – in contrast to much of the rest of the world, the EA community prides itself on utilizing a wide variety of (comparatively more rigorous) methods to evaluate and make decisions. For reasons that might include the one-on-one and highly personalized nature of the coaching itself, my impression is that community tastemakers and funders tend to draw too heavily on their own experience with coaching and therapy in their assessments of its overall value.
    My experience has been that there’s also a tendency to over-index on their impressions of someone from previous contexts. Factoring prior interactions with an individual is both useful and unavoidable, but again, shouldn’t dictate decisions regarding the abilities of the coach or value of coaching in general. The net outcome tends to be something like, “I’ve not had any (useful/​meaningful) coaching and I also kind of know that person from when they had ‘X’ role, which colors my impressions of whether this combined package is valuable.”
    For many of the same reasons that the community wouldn’t want anecdotes to dominate decision-making in other domains, more should be done here to depersonalize and debias these evaluations. In a couple of the points above, I outline how investment into a professionalized support infrastructure could address this.

  • EA leaders are often at least partially in the dark regarding expectations from funders. This could be the case for many reasons, but a common reasons among leaders included the following:

    • Reputational fears – Reticence to reach out due to some (un)justifiable fear of reputational harm

    • Value system clash/​lack of familiarity – not wanting to waste the time of funders, usually due to lack of familiarity and fears of how they would be received, but also sometimes a principled decision about not wanting to bother important decision-makers

    • Not having considered reaching out to funders regarding expectations at a meaningful enough grain of detail

    • (Likely not always misplaced) concerns about arbitrariness of the evaluation process

    • Preparation overhead – not being ‘ready’ in various ways. In some cases, My outside view of the situation led me to believe that quite a bit of preparational overhead and perfunctory correspondence could be avoided if funders made it clearer that they care less about certain aspects of performative presentation

  • Out-of-pocket vs. professional development budgets – valuations of the coaching trials, while nonetheless higher than I expected, were noticeably lower when trial participants were asked their out-of-pocket valuation. It seems as if the perceived value of a session (captured by amount willing to pay) is highly contingent on local (not simply personal) financial circumstances. This indicates to me that a) trial participants here valued the coaching enough to pay at least subsidized rates to continue, and b) professional development budgets could expand the potential for leaders to seek and receive assistance that they seem to value highly.
    My hunch is that EA-related self-concept and social pressure issues also push this valuation lower. Nonetheless, I was impressed with the valuations given this is a charity-based community where several of the trial participants worked for organizations that are funding constrained.

  • Some of the implications regarding how community tastemakers and funders value coaching has led me to believe that having an impactful career via these means is not only about being skilled and causing positive effects for coachees. A huge amount of work will need to go into creating and buttressing a narrative frame that will get community tastemakers on board for a coaching and therapy support infrastructure. Thankfully, this need has been recognized by others, resulting in some early initial investment into this (examples include the EA Mental Health Navigator website)

  • Funders could do more to solicit projects – in some cases, considerable nudging was necessary to get project leaders to even inquire whether funders would be interested in their project, often for the reasons cited in the points above. This seems particularly salient for funders to check in on ‘known’ projects in the community, rather than assuming they will reach out if need be.

  • Funders could do more to prioritize fostering relationships – greater familiarity with project leaders reduces inefficiencies of all sorts, including performative and preparation overhead, miscommunication, missed opportunities, etc.
    In my opinion, this should also apply to unsuccessful projects. A common theme that I’ve seen from funders, partly due to bandwidth issues though not entirely, is aversion to giving constructive feedback to unsuccessful projects that nonetheless endure within the community. Given my firsthand experience with many clients who are fairly averse to interpersonal conflict, it wouldn’t surprise me if aversion to conflict + public relations considerations + legal issues (and other things) precluded funders from giving constructive feedback to failed applications. Funders would likely need to hold the belief that this feedback would meaningfully improve these projects prospects, and therefore the community overall, in order to put in the requisite effort to get through these blocks to this type of action. They’d also likely need to feel reassured that the feedback wouldn’t be excessively damaging reputationally (for both themselves and others), destabilize the community, or the integrity of community norms.

Coaching Insights

Tl;dr – somewhat surprisingly, what I outline below roughly corresponds with the quantified survey results on what seems to be most useful and potent about my style of coaching. In short, I gather that it is quite valuable to spend a dedicated period of time with a skilled guide who is well-versed in a major component of their worldview (EA and resulting effects on thinking) who also falls outside of their professional and social circles. The ‘personal strategy’ style of coaching traverses and analyzes the emotional connection between the personal and professional, likely making sessions feel more emotionally resonant (and therefore impactful) than professional coaching, and more real-world and action-oriented than some forms of talk therapy. And finally, we often address the ‘social layer’ in a tangible way that leads to benefits from addressing social dynamics in a way that isn’t as passively theoretical/​abstract as some rationalist community-derived practices (though I’m indebted to thought leaders in that area for their openly shared models and methods).

Below are some of those insights in greater detail:

  • Efficiencies in worldview understanding – trial participants seemed to greatly appreciate the extent to which their worldview was diagnosed, recognized, understood and utilized as the basis for finding solutions (e.g. “Tee had more context to my specific situation (leadership within an organisation working on the project of effective altruism”). Many informal conversations I’ve had about prior experiences with coaches and therapists include lamenting about how too much time and energy was wasted on ‘bringing [the coach/​therapist] up-to-speed’ and/​or simply not being understood enough to be helped in a meaningful way.

  • Using the landscape to orient themselves – EA leaders experiencing lostness, operating in a vacuum, lacking feedback, not having instincts about what to do – are often alleviated by putting them into more contact with relevant aspects of the EA landscape that would give them key information. One way to do that is for the coach (me) to simply hand them that information based on my experience, my impressions from other EA leaders, publicly published content and other conversations. Another way is to build plans for them to make contact with agents and content out in the landscape that would give them a better sense of what to do. (e.g. rather than building what you think funders might want, take the time to do a form of ‘market research’ calibrating the interest level across several funders)

  • Non-involved stakeholder dynamic – our coaching sessions seemed to be a very rare sort of interaction for nearly all of the trial participants, many of whom had never tried coaching before. Some in-session feedback that I received was that these sessions were a unique space for individuals to speak with someone who is both skilled and not personally invested – a dedicated outsider who was willing to spend 60 to 120 minutes at a time to consistently dive into whatever key issue they felt useful to speak about.

  • Internal relationship management – it became clear that this type of coaching was quite important for coachees to talk and think through what to do in certain relationship situations. Coachees got a lot of mileage out of talking through management uncertainties, but this type of coaching seemed most valuable in situations where the other parties involved were of equal or higher status.

  • Interactive and attentional bandwidth limits – provided that the coachees value the sessions highly enough (which seems to be the case), there is definitely an upward limit on attentional energy one can put into attentively listening, modeling what’s happening, contributing to strategizing, etc. within a given day and week. For reference, my upper limit is 7–8 coaching hours per day, and 25–30 per week.

    • Attentional expenditure variance – depending on rapport with the client, subject matter, energy levels, mood, etc., coaching sessions can fly by with ease, or feel effectively double the actual length of time. I’d imagine there’s such high variance because of the way differences in interactive experience warps one’s sense of time, for better or worse.

    • Importance of high hourly rates – common wisdom around skilled consultants applies here: the high hourly rate pays for many things beyond what is delivered within the hour, including consolidated expertise, tailored implementation of methods, and preparation and debriefing related to the session. In other words, while the hourly rate might seem high, there’s a pretty natural cap for what coaches can earn. For funders trying to model the value here, unless a coach is working exclusively with high-paid corporate clients, overall income will likely be comparable to other senior-level positions.

    • Importance of professional development for the coach – handling a high volume of clients with disparate needs is tricky for a single coach. Thankfully I do have my own coaches that I paid out-of-pocket for to help me think through (always anonymized) client issues and dynamics that I struggled with. Helping coaches with the complex task of helping others in every single session should be prioritized for a whole program to function.

    • Coaching ecosystem quality control – as mentioned in the section above, there are several upsides to the community coordinating to construct a support ecosystem, rather than having leaders disparately seek coaching (often later than) when it feels necessary. Concerted efforts to improve coaching quality with the community include:

      • Compiled directories of endorsed coaches (e.g. EA Mental Health Navigator)

      • Coaching coaching—forms of coaching training, supervision, skill-sharing, etc.

      • Investment into novel techniques and synergies in applied methods (i.e. funding for coaching research and application)

      • Anonymous feedback outlets

      • Improved public transparency regarding coaching outcomes and feedback

  • Generalized frustrations are often about one person – while much work can be done in examining the patternized constructs that coachees tend to project onto the present day (i.e. forms of transference), often this work would be best conducted while simultaneously diagnosing and addressing present-day issues within particularly problematic relationships.

  • Endorsed v. actual epistemics – my style of coaching seemed to mesh well with the personal epistemic standards of trial participants, which was surprising to me given the degree to which I derive methods and inspiration from spaces that don’t fall within the intuitive epistemic bounds of EA. (e.g. only sticking to methods rigorously validated by certain empirical paradigms and refusing to venture beyond that). I also noticed a difference between personal epistemic standards and those that some trial participants felt they needed to signal and project in order to capture and retain social benefits. This is par for the course in some respects, but closing the gap between internal standards and projected personas would drastically reduce energy and resource inefficiencies.

  • More is being transferred than the explicit – from a combination of my own sense of things, in-session feedback and the documented feedback in this report, it seems reasonable to conclude that coachees receive forms of care, reassurance and validation that can be very nourishing. Perhaps more importantly, some sessions are focused on bringing the desire to replicate these fuller and more nourished ways of being to their organizations and daily work lives.

  • The value in working with parts of self – the integration of several coaching methods and theories of mind seemed quite useful. In particular, what rationalists would call a multi-agent model of mind, which bears similarities to Internal Family Systems, at the very least, helped coachees disentangle narrative frames and conflicting emotion-level feelings in a way that was important for making sense of situations. Pairing this with relevant neuroscientific insights into motivation was helpful for talking through behavioral change and habit formation.

  • Out-of-range targets – in hindsight, emotional coaching targets for 1–2 of the participants appear to have been too ambitious, particularly for one individual. While this coaching produces substantial value in the emotional and perceptual spaces between the personal and professional, selection of emotional issues should be done with great care, especially within a trial setting.

  • Confidential information – there appeared to be a noticeable level of catharsis for coachees who were able to divulge information that couldn’t be revealed to any other involved parties. Herein lies potential landmines for coaches and organizations working with multiple people on a single team. In my previous experience and current work, this has not been an issue if managed carefully.

Trial & Operational Insights

  • Confirm trial commitment firmly – actual trial commitments may vary. Important people are busy – for coaches and/​or coaching programs looking to get some indication of the value of their coaching program, there are many reasons why a trial can get disrupted, including the fact that busy people often reschedule. Here are some thoughts about running trials like this given this reality:

    • More attention should be paid to ensuring that trial participants are willing, committed and able to follow through on the trial in a timely manner. The results suggest these trials are very valuable to give away, so I’ll need to be clear about getting assurances of following through.

    • Run several trials – making sense of this noisy data is tricky. Conducting several trials across many different types of people, across different cause areas and with different target issues is a great way to collect data in general, but it’s also wise to hedge against staking too many implications on the fit with any single person, not to mention that the trial experience can get disrupted by sessions getting postponed.

      • There’s a vague sense that trial feedback was higher when trials were completed with regularly spaced sessions within an agreed upon timeframe

      • My feeling is that the absolute best way for coachees to absorb the benefit of this type of coaching would be in a dedicated workshop format

Things to do differently (moving forward)

  • See the “Out-of-range targets” bullet of the Coaching Insights section

  • Substandard prep and forecasting on my part – in short, the amount of prep and client care that went into individual coaching relationships was below the standard I would expect of myself and others. This was partially a prioritization issue, but also between trying to bootstrap funding, run the trials, coach private clients and seek professional development on my own, I did far less preparation and thinking about the needs and trajectory of individuals than I would have preferred. Despite the trial results being quite positive, this essentially means that I wasn’t able to offer the highest quality version of the coaching that I believe is available. In some ways, this bodes well in cases where I don’t need to incur the bandwidth strain of bootstrapping for funding. On the other hand, I still have feelings of guilt about not being able to put my best foot forward regardless of the results.

  • Incentivize completion of trial feedback forms – while the results were encouraging and most trial participants wanted to continue, there was a fair amount of chasing needed in order for feedback surveys to be completed. This could be remedied by offering another free session in exchange for completion of the survey, along with other potential incentives.

  • In the impact survey, ask valuation questions about coaching in general prior to undertaking the trial and then again following completion of the trial. I’ve implemented this in the upcoming Training For Good leadership coaching trial program

  • Though this compounds the upfront costs to getting started, having a brief conversation about goals, expectations, coaching methods, etc. would be important for determining the frequency of sessions that would be best for the client, therefore allowing for better prediction of trial timelines

  • Possibly include question about whether promotional content affected expectations (such as this report for future coaching trials)

  • Increase total trial period in order to build rapport and have a better opportunity to cause large effects

  • Incentivizing the completion of forms with additional hours and/​or coaching discounts seems worthwhile

  • Solicit whether coachees would like more systems for accountability and implement those systems. Some feedback was that the coaching was too free-form

  • Have optional prompts prior to sessions that are somewhat tailored to the stated aims of the participants. Optional because some coachees dislike needing to put in the preparation time. Somewhat tailored because non-tailored can be onerous and irrelevant, but too tailored is less feasible for a coach trying to handle a large number of clients.

Selected Qualitative Answers

View appendix of qualitative answers for more

“Are there ways that Tee’s Coaching had a meaningful effect on you and/​or what you’re capable of doing?”

“I’ve had a number of profound breakthroughs during our sessions. These were crucial shifts that I was able to make regarding my workflow and I’m already seeing results from it.”

“Tee worked with me to decide upon and take actionable steps in regards to the development and management of my staff, my personal ways of thinking and behaviours, and improving my stakeholder engagement.”

“Tee helped me reach some conclusions about myself and what I bring to my current role that caused perceptual shifts which I believe have made me more valuable to my team. He also helped me discard at least two bad plans in favor of better plans.”

“Were there any situations where a shift generated by Tee’s coaching resulted in you taking different actions than you otherwise would have?”

View appendix of qualitative answers for more

“I adjusted the strategy for my organisation, my own management style, how I thought about fundraising, and how to restructure my own role.”

“I had some questions around the role of a CEO, as well as questions around feeling stuck on some overwhelmingly large tasks. Tee helped me realistically analyse what I was bringing to my role, and to critically analyse my own views around the meaning of intelligence. He also suggested practical action points on one task I was especially stuck on, around an operational systems overhaul, which helped me break down the process into bite sized pieces; from there, I’ve been able to make significant progress on this task, that previously felt bottlenecked. Tee also helped me build confidence in saying ‘no’ where it was needed, to prevent future instances of burnout; I’ve been a little more successful at setting boundaries and protecting my time since my coaching with Tee began.”

“In one case, I changed my approach to an important conversation with an employee in a direction that was much more positive and productive. In another case, I discarded a plan that Tee made me realize was likely to fail and crafted a better, more robust plan.”

EA Leadership Recommendation – ”Do you feel Tee’s coaching could be beneficial to other leaders in EA?”

(See publicly displayed testimonials on this question and view appendix of qualitative answers for more)

“I think Tee has a unique ability put people at ease, actively listen, and develop intelligent, nuanced feedback, even with little information. He knows how to ask the right questions, and to prompt his clients to think about things in ways they may not have before. This, combined with his experience of EA, puts him in a brilliant position to support others in the EA community.”


“He does an amazing job of skillfully using a wide array of psychological techniques without you even noticing them. And then he does the best part for an EA—he explains what he was doing so you can understand it—and yourself—better. Tee is a fantastic coach. I highly recommend him if you want to be happier and more productive.”

“Generally speaking, I think that working with a knowledgeable coach—in this case Tee, can be a valuable experience to all leaders and individuals working in roles that may affect others (for example People Managers).”

What are the main reasons why you hadn’t tried coaching before?

(Optional section – selected ‘no’ for having had coaching before)

“I wasn’t aware of a coach would work for my needs and my style.”

“I simply hadn’t considered myself a candidate for it.”

“I guess the main constraint is the cost/​fee. Paying out of pocket can get a bit expensive, especially if it is an ongoing commitment.”

About me

I’m Tee – a self-styled ‘personal strategist’ currently based in Prague. You can learn more about my coaching practice from this section of the report and my website. At this point in time, I’m still coaching clients that I believe to have high impact potential, issuing more trials of my coaching, launching a podcast, and exploring potential ways to collaborate on the creation of a professionalized support infrastructure to serve those trying to achieve scalable good.

I’m a co-founder, former executive director, and current board director of Rethink Charity (RC), a project collective that launched and/​or incubated several EA community building projects, including Rethink Priorities, RC Forward, the EA Hub, the EA Survey, EA Giving Tuesday, and fiscal sponsorship for numerous startup EA-aligned projects (Dao Foods, AI Safety Support, Effective Thesis, LEAP conference).

Big picture stuff – my ultimate aim is to utilize my life and work to substantially improve the flourishing of all sentient beings. I’m channeling these efforts through coaching, which seeks to cause deeply valuable perceptual shifts for influential individuals that have unusually high impact potential in the world (e.g. Executive Directors, CEOs, influencers, thought leaders)

The general case for coaching

Evaluating a more general case for coaching that invokes and analyzes several types of empirical data is outside the scope of this report. Additional recommended reading that originates from the EA community includes a section from Sebastian Schmidt’s post on further empirical support for the idea that skilled coaching can be materially valuable for coachees. I also found Lynette Bye’s cost-benefit analysis on her own coaching worth checking out as well.

What’s Tee’s coaching like?

Working with me is about having a ‘personal strategist’ that helps guide the discovery, navigation and refinement of deep perceptual constructs that meaningfully affect your personal and professional life. Clients report that ‘shifts in perception’ are often the most valuable result of the coaching. This is meant to be done in a way that is truth- and reality-tracking better than the ingrained and patternized ways that people typically interpret their situations.

In short, the idea is that working through surface-level issues will often lead to the exploration of deeper patterns and structures of sensemaking and interpretation. Addressing these deep structures causes important and lasting effects.

The subject matter of sessions and my employed methods are varied and specifically tailored for the individual. (Anonymized client story example).

This type of coaching is nonetheless distinct from therapy. While we may work through emotional issues associated with the relevant challenges, several areas (e.g. trauma processing) are outside of the scope of this coaching. What’s taken from any given session shouldn’t be misconstrued as therapy or the advice of a credentialed clinician.

What do I hope comes as a result of this report?

  • More individuals attempting to scalable good take seriously the prospect of getting a coach

  • EA Funders and tastemakers take seriously the need for a professionalized support infrastructure for EA leaders (and EAs more broadly)

    • Funders make a concerted effort to integrate the difficulties that project leaders face (e.g. by integrating these needs into their grant making evaluation training programs)

  • Others get inspired, reassured and/​or validated in ways that I’ve not yet anticipated

  • I can continue my plans for coaching in EA and building out a vetted and professionalized support infrastructure for the community

  • Feedback and input on the following:

    • The metrics and survey questions used in this trial. More specifically, how they could be better.

    • What other types of measures could be experimented with that would give interesting snapshots into the value and effects of coaching (e.g. third-party reports on the perceived changes regarding what it’s like to work with the coachee after they’ve received coaching for a certain period.)

Anticipated Questions, objections and inquiries

  • This report is way too damn long, Tee. C’mon.

    • In terms of content digestibility, I definitely hear you. My hope is that people will follow their nose and skip around to sections that pique their curiosity.

  • What was most surprising to you about this trial?

    • Positive – how well it seemed to go, actually. I really only expected something like 1⁄2 or 2⁄3 of participants to have had a notably positive experience. I’m also pretty pleased with the high monetary valuations placed on the trials by a somewhat random assortment of leaders in a largely non-profit and altruistic community.

    • Negative – in conversations around the trial, grappling with the disparity between how critical I believe coaching can be and how much it is recognized or valued by others. A relatable example here is a situation where one is fully sold on the idea that determining the most effective way possible to help people (e.g. EA) is of almost unassailable importance, then encountering those who are inclined to wave it away casually.

  • What is the best argument for how you could be mistaken about the positive results of your coaching trial?

    • Probably if it were the case that the means of gathering information about the effects and value of the coaching are way off base. I do believe that willingness to continue the coaching and refer others are action-based observations that seem fairly persuasive. I’d love to hear about alternative ways to ask questions or conceive of how to inquire about the experience of an individual. You can leave a comment on this post or feel free to reach out at tee@teebarnett.com.

  • What is the best argument for how your coaching could appear valuable, but actually isn’t?

    • The strongest argument I’ve come across is that some folks can get caught up in something like a perpetuating placebo effect regarding the impact of the coaching due to inorganic/​forced beliefs that they are/​want to/​must be improving. A classic example are ‘self-improvement conference junkies’ who seem to be addicted to cycles of getting riled up about what they could possibly be capable of.

      There’s a lot here to unpack. While this is always possible, I do try to hedge against that by bringing in various forms of reality- and truth-tracking into the sessions.

      You’ll also notice that the coach would need to be complicit in contributing to this dynamic, optimizing in the relationship for engagement ($$$) and not necessarily being guided by what’s best for the client. Rather than attempt to lay bare my intentions, I do think signals exist for clients that I’m dedicated to aligning my interests with theirs as much as possible.

      A few examples of visible indicators – I try to get clients to reflect on their best approximation of counterfactual differences in action and well-being, I have periodic feedback and sessions that go meta on how the relationship is going, I don’t charge for packages of sessions – only one-at-a-time that is cancelable at any time, and I try to diversify revenue streams so as not to stake how I make a living on perpetuating any single paid relationship.

  • What are some circumstances where your coaching would not be a good fit for someone?

    • In situations where the subject matter would be classified as therapy (e.g. trauma processing)

    • Where gendered, racial and/​or cultural insights would be highest leverage – for some people, lenses and models associated with identity could be most helpful in navigating their local context. While I am receiving “coaching training” myself in order to be more conscientious and useful in this regard, other (types of) coaches with insights derived from identity associations that are different from mine could be a more appropriate fit.

    • When the vibe isn’t right – when people are considering coaching, I often encourage them to speak with others besides myself for a variety of reasons. Interpersonal meshing is surprisingly crucial.

    • When clients seek very specific models or know-how transfer – my coaching is not the place to seek highly specialized models within very narrow domains. I would say that people often over-index on needing this type of know-how transfer, rather than getting coaching about how to uncover and formulate models for themselves (which I try to help facilitate).

    • Where scarcity strongly colors the coachees approach – while I firmly believe that my style of coaching is one of the best things that can help an individual make the most from their current circumstances, sessions are less useful if clients don’t already share that belief. I like to think that the model for how this coaching comes to fruition resembles something more like patient accumulation, rather than trying to extract value from every minute. Note this is different from ‘don’t try coaching if you’re in a place of scarcity’. It’s all about whether the client relates to things in a noticeably different way when being in a mindset of scarcity.

    • When the individual is not attempting to do ambitious things – I’m less certain of this, but I think my niche is working with the emotional aspects of, and fallout originating from, trying hard things ‘out there’ in the world. Something like coaching to deal with chronic depression (which sounds like therapy) for example seems like something I’d be less suited for.

  • The linkage between underpowered data and broader claims about how this could be important for leaders in this community seem tenuous. Why not (a) simply report on the results of the trial, or (b) conduct something like a literature review to make a persuasive empirical case for coaching?

    • I’d concede that the claims (i) the trial seems to have gone quite well, and (ii) therefore it could be profoundly important for the community, might be an inferential bridge too far.

      My first response to that is to take the humble academic rhetorical position that I believe more investigation seems warranted in order to determine whether a clearer linkage exists. I’d also agree that the structure of this post doesn’t follow the implicit community convention for the creation and propagation of knowledge.

      But also inklings about how to solve important problems, or provide a paradigm-shifting service, often come from somewhere deep. In my case, it has been hundreds of hours speaking directly with leaders across various industries, noticing patterns of need, deficiency and potential solutions that have been revealed as a result.

  • What does the world look like where your personal style of coaching is potent and valuable, but a professionalized support infrastructure is nonetheless not worth it for the EA community to invest considerable resources in?

    • Issues of scaling coaching and trickiness associated with monitoring & evaluation immediately come to mind. Those two things alone could make investment into a professionalized and unified support infrastructure perhaps too costly.

  • I have thoughts/​concerns about how your impact evaluation questions were constructed and how they may have influenced the responses.

    • Great! You can leave a comment on this post or feel free to reach out at tee@teebarnett.com. Though I did receive help from well-qualified individuals in formulating the questions, I’m sure they were far from perfect and I’d love to hear about alternative ways to ask questions or conceive of how to inquire about the experience of an individual. In particular, I’d love to get more input on the nuances associated with how people value a service like coaching. Not included within this report were many of my thoughts regarding how individuals calibrate value and what their relationship is to money.

  • How do you know that there’s enough widespread demand in the community for coaching?

    • Strictly speaking, I don’t. I’d say the turnout and enthusiasm for my trial and the Training For Good coaching trials are broadly positive, but I do think that a concerted effort to assess the demand for coaching in the community – likely via a survey – would be worth conducting.

  • I’m a funder. My current take is that the most skilled people in EA are already well-funded. If they wanted coaching, they’d easily be able to afford it themselves. The fact that they do not pursue it means that they probably wouldn’t find it valuable enough. Why should any funding go into the promotion of individual coaches and/​or the construction of a professionalized support infrastructure for the community?

    • [This sounds quite specific but I’ve run into this line of reasoning multiple times]. There’s also a lot to unpack here. I’d say that, in some cases, this could certainly be correct. I have a couple of main points for believing otherwise. First and foremost, this line of reasoning seems to rest on a set of assumptions that well-resourced individuals in the community a) have an accurate understanding of the service on offer, b) assess the value appropriately, and c) have no unjustifiable barriers to receiving this service if they wanted to. As this report tries to illustrate, coachees weren’t always sure what they wanted out of sessions, where the value would come from, and were surprised but what they found in these regards.

      There are actually quite a few addressable reasons for why well-resourced people wouldn’t pursue coaching. Without venturing into an exhaustive list, individuals can be precluded by beliefs about what others would think about them if they received coaching and/​or paid a certain rate for coaching, even if they personally believed it to be quite valuable. (i.e. “EAs will judge me harshly if I do this/​spend money on this”).

      And finally, I’d very much disagree with the basic premise that the most skilled and worthy people in the community are sufficiently resourced.

  • This seems like far too much time and energy to put into a report like this. Why did you feel the need to put so much into this? (It doesn’t seem well-calibrated. What’s happening here?)

    • If I’m being really real, I’d say fear of how this could be received (mostly by the EA community). What I’m calling for is currently not on the radar of priorities in the community, claims about the potential implications are big, and the presentation is unusual. In my last Forum post about three years ago, me and other folks that I deeply respect spent a lot of heart, sweat and reputational capital into a project that basically went nowhere. That one stung. You’ll notice this has a very on-point relation to how perception, ego and emotions can dictate work. That’s exactly the spaces where my coaching finds large margins for improvement.

      On a practical level, also thematically related to the value of coaching, the process of articulating, clarifying, constructing and organizing mental models from implicit experience is very valuable to me. I also intend to use this report as a resource to refer to across many contexts and mediums, including my website.

      I’m also hoping this post will cause certain things that I’d like to see happen, so it felt important to bring it up to a certain level of polish. I’d like to expand the horizons of tastemakers, nudge funders to take this seriously, and provide an example for aspiring coaches for how (not) to make the case for this type of work.

  • Your coaching basically sounds like therapy. That’s concerning to me. Should you be doing that?

    • This type of coaching is very distinct from therapy. While we may work through emotional issues associated with the relevant challenges, several areas (e.g. trauma processing) are outside of the scope of this coaching. What’s taken from any given session shouldn’t be misconstrued as therapy or the advice of a credentialed clinician.

Methodology Notes

  • First thing to note is that I’m very clearly not a quantitative savant by any means. However, I have opened up my calculations for others to view with anonymized data. Some well-qualified individuals from the EA community took time outside of their official roles to help formulate and refine the survey. All mistakes, shortcomings, and inaccuracies are mine alone and should not reflect on any of the reviewers.

    • It was suggested that I include the interquartile range, which I did not end up doing to keep reporting simple. Readers can get a sense of the range and even calculate it on their own with the data provided.

  • 15 free trials were planned, of which 13 trials were conducted, with only 12 yielding valid data. Among those yielding invalid or no data, the first offered trial never began, the second was interrupted, and the third featured the successful inclusion of another coach, which nonetheless invalidated the data.

  • Asking about how much trial participants would value a coaching trial from any coach prior to initiating sessions would illuminate how much they value coaching more generally, and how that compares with the value of receiving the trial from a specific coach. Decontextualizing this question might be tricky though (i.e. there’s a palpable difference between the valuation of a trial from any coach, rather than valuations of an upcoming trial with one that you have information about)

  • Unfortunately, I put the question about dollar value per hour (“How much would it have been worth to pay for the trial package of sessions, if you were paying out of pocket? (per hour)”) in the ‘optional’ section of the survey, so we’re not quite able to measure hourly valuation against trial package valuation directly. This seems like a silly oversight on my part. However, the limited responses we have suggest that people value a trial package more as a whole than when the question is framed in terms of hourly cost.

  • A case can be made that appraisals of value based on hourly units are often over-emphasized (probably because it’s commonly used), and that given the cumulative nature of how coaching benefits are realized and come to fruition, perhaps overall packages of sessions are better approximations.

  • Asking about productivity might not make sense for my style of coaching

  • Productivity ‘in the last month’ seems strangely arbitrary. It’s likely that asking about productivity since the trial began would be more appropriate.

  • In the satisfaction matrix, I have some concerns that the categorical labels for what trial participants could have experienced within the session are too abstract and don’t cleanly track with their own idiolect or native conceptions.

  • I chose to highlight average valuations of grant packages divided by average trial duration for questions (1) & (2) because I thought it was fairly illuminating.

Additional Thanks

My opening thanks to those who put the time and energy into reviewing this report, including Kerry Vaughan, Sebastian Schmidt, Luke Freeman, David Moss, Jamie Elsey, Mark Lee, Dominika Kropucin, Monica Chen, Emily Crotteau, Kristina Nemcova, Oliver Crook and Jess Smith.

These coaching trials and the resulting report felt like a big step for me putting myself out there. Innumerable people are to thank for this and hopefully I’ve not been shy in private about how important their contribution has been.

Most relevantly, the trial participants who were kind enough to part with their very valuable time to test out the coaching. These folks are dedicating the best working years of their lives (at least) to effective causes, which I greatly admire. The independent funders who believed in me and the potential for this trial to yield valuable information when institutional funders did not, or were not a great fit. From being afforded the opportunity to make a living conducting this trial, and my hardening experience fundraising for Rethink Charity, I cannot stress enough how important small- to medium-sized donors could be in helping new projects get off the ground.

And finally, to those who contributed in my efforts to gain still in coaching – I couldn’t be more thankful. Many individuals spent countless hours patiently training me in the art of helping others. It’s something that changed my life and I would never trade away.

  1. ^

    This period also included a handful of subsidized trials, though only the free trial feedback is reflected in the data.

  2. ^

    I mention in the Methodology Notes section how it was certainly an oversight to not have asked for trial package valuations side-by-side with hourly valuations. There’s good reason to believe the valuations would be different

Tee1 Apr 2022 12:36 UTC
101 points
19 comments34 min readEA link