One way to approach this would simply be to make a hypothesis (i.e. the bar for grants is being lowered, we’re throwing money at nonsense grants), and then see what evidence you can gather for and against it.
Another way would be to identify a hypothesis for which it’s hard to gather evidence either way. For example, let’s say you’re worried that an EA org is run by a bunch of friends who use their billionaire grant money to pay each other excessive salaries and and sponsor Bahama-based “working” vacations. What sort of information would you need in order to support this to the point of being able to motivate action, or falsify it to the point of being able to dissolve your anxiety? If that information isn’t available, then why not? Could it be made available? Identifying a concrete way in which EA could be more transparent about its use of money seems like an excellent, constructive research project.
Overall I like your post and think there’s something to be said for reminding people that they have power; and in this case, the power is to probe at the sources of their anxiety and reveal ground-truth. But there is something unrealistic, I think, about placing the burden on the individual with such anxiety; particularly because answering questions about whether Funder X is lowering / raising the bar too much requires in-depth insider knowledge which—understandably—people working for Funder X might not want to reveal for a number of reasons, such as:
they’re too busy, and just want to get on with grant-making
with distributed responsibility for making grants in an organisation, there will be a distribution of happiness across staff with the process, and airing such tensions in public can be awkward and uncomfortable
they’ve done a lot of the internal auditing / assessment they thought was proportional
they’re seeing this work as inherently experimental / learning-by-doing and therefore plan more post-hoc reviews the prior process crafting
I’m also just a bit averse, from experience, of replying to people’s anxieties with “solve it yourself”. I was on a graduate scheme where pretty much every response to an issue raised—often really systemic, challenging issues which people haven’t been able to solve for years, or could be close to whistle-blowing issues—was pretty much “well how can you tackle this?”* The takeaway mesage then feels something like “I’m a failure if I can’t see the way out of this, even if this is really hard, because this smart more experienced person has told me it’s on me”. But lots of these systemic issues do not have an easy solution, or taking steps towards action are either emotionally / intellectually hard or frankly could be personally costly.
From experience, this kind of response can be empowering, but it can also inculcate a feeling of desperation when clever and can-do attitude people (like most EAs) are advised to solve something without support or guidance, especially when this is near intractable. I’m not saying this is what the response of ‘research it yourself’ is—in fact, you very much gave guidance—but I think the response was not sufficiently mindful of the barriers to doing this. Specifically, I think it would be really difficult for a small group of capable people to research this a priori, unless there were other inputs and support like e.g. significant cooperation from Funder X they’re looking to scrutinise, or advice from other people / orgs who’ve done this work. Sometimes that is available, but it isn’t always and I’d argue it’s kind of a condition for success / not getting burned out trying to get answers on the issue that’s been worrying you.
Side-note: I’ve deliberately tried to make this commentary funder neutral because I’m not sure how helpful the focus on FTx is. In fairness to them, they may be planning to publish their processes / invite critique (or have done so in private?), or are planning to take forward rigorous evaluation of their grants like GiveWell did? So would rather frame this as an invitation to comment if they haven’t already, because it felt like the assumptions throughout this thread are “they ain’t doing zilch about this” which might not be the case.
*EDIT: In fact, sometimes a more appropriate response would have been “yes, this is a really big challenge you’ve encountered and I’m sorry you feel so hopeless over it—but the feeling reflects the magnitude of the challenge”. I wonder if that’s something relevant to the EA community as well; that aspects of moral uncertainty / uncertainty about whether what we’re doing is impactful or not is just tough, and it’s ok to sit with that feeling.
Overall I like your post and think there’s something to be said for reminding people that they have power; and in this case, the power is to probe at the sources of their anxiety and reveal ground-truth. But there is something unrealistic, I think, about placing the burden on the individual with such anxiety; particularly because answering questions about whether Funder X is lowering / raising the bar too much requires in-depth insider knowledge which—understandably—people working for Funder X might not want to reveal for a number of reasons, such as:
they’re too busy, and just want to get on with grant-making
with distributed responsibility for making grants in an organisation, there will be a distribution of happiness across staff with the process, and airing such tensions in public can be awkward and uncomfortable
they’ve done a lot of the internal auditing / assessment they thought was proportional
they’re seeing this work as inherently experimental / learning-by-doing and therefore plan more post-hoc reviews the prior process crafting
I’m also just a bit averse, from experience, of replying to people’s anxieties with “solve it yourself”. I was on a graduate scheme where pretty much every response to an issue raised—often really systemic, challenging issues which people haven’t been able to solve for years, or could be close to whistle-blowing issues—was pretty much “well how can you tackle this?”* The takeaway mesage then feels something like “I’m a failure if I can’t see the way out of this, even if this is really hard, because this smart more experienced person has told me it’s on me”. But lots of these systemic issues do not have an easy solution, or taking steps towards action are either emotionally / intellectually hard or frankly could be personally costly.
From experience, this kind of response can be empowering, but it can also inculcate a feeling of desperation when clever and can-do attitude people (like most EAs) are advised to solve something without support or guidance, especially when this is near intractable. I’m not saying this is what the response of ‘research it yourself’ is—in fact, you very much gave guidance—but I think the response was not sufficiently mindful of the barriers to doing this. Specifically, I think it would be really difficult for a small group of capable people to research this a priori, unless there were other inputs and support like e.g. significant cooperation from Funder X they’re looking to scrutinise, or advice from other people / orgs who’ve done this work. Sometimes that is available, but it isn’t always and I’d argue it’s kind of a condition for success / not getting burned out trying to get answers on the issue that’s been worrying you.
Side-note: I’ve deliberately tried to make this commentary funder neutral because I’m not sure how helpful the focus on FTx is. In fairness to them, they may be planning to publish their processes / invite critique (or have done so in private?), or are planning to take forward rigorous evaluation of their grants like GiveWell did? So would rather frame this as an invitation to comment if they haven’t already, because it felt like the assumptions throughout this thread are “they ain’t doing zilch about this” which might not be the case.
*EDIT: In fact, sometimes a more appropriate response would have been “yes, this is a really big challenge you’ve encountered and I’m sorry you feel so hopeless over it—but the feeling reflects the magnitude of the challenge”. I wonder if that’s something relevant to the EA community as well; that aspects of moral uncertainty / uncertainty about whether what we’re doing is impactful or not is just tough, and it’s ok to sit with that feeling.