Please leave your kind feedback on the essay (on writing, logic, relevance etc.) in the comments section or through private message. Any feedback is much appreciated.
Entry question
What is could be the lifetime estimated impact (in $
donated) of being a EA GiveWell researcher?
TL;DR
This is a fermi estimate and shall be treated like that. Everything is an
estimate, there are many uncertainties and geometric means have been
used to bring the limits down to one number.
Lifetime Impact of a GiveWell Researcher: $8m
Epistemic Status: Off by a factor 10 (#fermi-estimates)
Introduction
I am curious to know what sort of impact a EA GiveWell Researcher
has. I choose GiveWell, as there is a very “clear” dollar figure on how
much money they “influenced” (at least according to them). With this
essay, the lifetime impact of an entry level researcher starting in
2021, and working for another 35 years, is estimated. The following
components help put together the final figure:
Organization Level
Impact of GiveWell in 2021
Growth rate of GiveWell’s impact for the next 35 years
Counterfactual impact of GiveWell
Researcher Level
Impact attributed to an entry level researcher in the first year
(2021)
Growth rate of researcher’s impact
Counterfactual impact of a Researcher
The rest of the essay deals with the above components and the final
lifetime impact calculation.
Note1: Fermi estimates are in general factor 2 or even factor
10 off. In many cases this might just be enough.
Note2: This estimate does not consider several factors such as
probability of EA loosing its momentum, probability of getting fired
and not being able to contribute as a researcher etc.
Impact and Growth Rate of GiveWell in 2021
GiveWell estimates how much money they move every year and
publishes them on their website. For example, in the year 2019 GiveWell
estimated that they moved $172m. In order to estimate the
impact of GiveWell in 2021, a bit of extrapolation of data is
needed. For this extrapolation, the growth rate is used.
To predict the growth rate in the coming years I believe it would be
reasonable to base it on the money moved from 2015 to 2019 instead of
2007 to 2019. This is because it appears that in the recent years the
growth has sort of “plateaued” (see table below) and this might be
more representative of the future.
Thus, the compound annual growth rate for the money moved between
2015 and 2019 comes out to be 9.4% ((172/120)^(1/4)-1). It’s really
hard to predict the change in growth rate for the coming years. For
simplicity’s sake, a constant growth rate of 9.4% for the next 35
years, is assumed.
As for the impact of GiveWell in 2021 it is estimated to be
$205m (172*(1.094)^2), based on a growth rate of 9.4% and the
money moved in 2019.
Counterfactual Impact of GiveWell
The counterfactual impact of GiveWell is the difference in impact
between the “Actual Scenario” (world with GiveWell since 2007) and
the “Counterfactual Scenario” (world without GiveWell since 2007).
In the counterfactual scenario, it is expected that a GiveWell-like
organization would have formed anyway, 3-10 years later (than when
GiveWell was founded)[2]. It appears likely because, many “similar”
organizations (such as Charity Science Health, Rethink Priorities,
Open Philanthropy, The Life You Can Save etc.) have formed since then
(between 2010 and 2020), involving cost-effectiveness estimates and
philosophies such as “doing the most-good for a dollar”.
While comparing the actual and counterfactual scenario, we compare the
two scenarios with the same amount of money but only varying
cost-effectiveness depending on the scenario. As a result it is
expected that when money doesn’t go to GiveWell-like organizations, it
goes to some random organization which is guessed to be within 1 to
1/50th times as cost-effective as GiveWell.
The counterfactual impact for the year 2021 is calculated as follows:
As part of the actual scenario, GiveWell is estimated to move $206m
with cost-effectiveness of 1. As part of the counterfactual scenario
GiveWell-like organization is expected to move $118m [3] with
cost-effectiveness of 1, in that year. The remainder $88m (206m - 118m$) is used by “other” organizations at one-seventh (geometric
mean of 1 and 1⁄50) the cost-effectiveness of that of GiveWell. The
difference between the actual and the counterfactual situation comes
to be $75m. And finally, the counterfactual impact as a function of
the money moved in the actual scenario is 36%.
The below table summarizes the counterfactual impact for the different
years. For years beyond 2021 the counterfactual impact essentially
starts to become a constant value of 31%. This 31% is used in
the final calculation as a constant.
AS --> Actual scenario with cost-effectiveness 1
CS GW-like --> Counterfactual scenario with cost-effectiveness 1
CS non-GW-like --> Counterfactual scenario with cost-effectiveness 1⁄7
Note: Cost of operation of GiveWell is not taken into account as
it is a small sum. For example, it is estimated that the cost of
operation for 45 people ($110k per person) shall be $5m. Contrast
that to it’s 206m expected to be moved in 2021 (a mere 3%). So this is
skipped confidently.
Note: Also the effects of the displacement chain are not
considered for simplicity’s sake.
Impact Attributed to an Entry Level Researcher in the First Year
In the year 2015, GiveWell estimates that an entry level researcher
had 10% of the impact of a co-founder and that there were 9 co-founder
equivalents [4]. In other words an entry level researcher was
estimated to be responsible for 1.1% (10%/9) of the total impact (in
2015). But this is not the whole story.
With more people joining GiveWell, this percentage gets diluted. In
2015 there were 25 “equivalent staff members”, where as in 2021 there
are 45 “equivalent staff members” [5]. The resulting contribution
would thus be much less. Let’s assume this contribution is inversely
proportional to the total number of staff. Then, the impact
attributed to an entry level researcher in the first year (2021) is
0.6% (1.1%x25/45).
Growth Rate of a Researcher
The growth rate of a researcher is split into 2 factors. The first
being the growth rate associated with the ‘increase in experience’,
which leads to higher researcher’s contribution. And the second being
the growth rate associated with ‘increase in number of staff’ of
GiveWell, which decreases the researcher’s contribution as years pass.
For the “growth rate associated with increase in experience”, the
year-variable is held constant (and hence the number of staff is
constant) and the difference between entry level researcher and
co-founder is observed.
In 2015, a co-founder is estimated to be responsible for 11%[4:1] of
the impact that year and an entry level researcher at 1.1% (as
mentioned above). If it takes an entry level researcher 10-20 years
(geometric mean: 14 years) to reach co-founder level, then growth rate
per year due to increase in experience (alone) is a whopping 19%
((11%/1.1%)^(1/n)-1). (It is assumed further that the growth rate
falls sharply to 0% after reaching co-founder level.)
Item
Value
Entry level researcher
1.1%
Co-founder equivalent
11%
Number of years
14
Growth rate due to increase in experience
19%
For the “growth rate attributed to increase in staff”, the average
increase in the number of people per year in GiveWell is
estimated. This increase is assumed to be negatively proportional to
growth rate attributed to increase in staff.
From 2015 to 2021 the number of staff has increased from 25 to 45
[5:1]. This implies an increase per year of 10% per year
((45/25)^(1/6)-1). This implies a growth rate attributed to increase
in staff of -10%.
Counterfactual Impact of a Researcher
The researcher positions seem very competitive now (with ~1%
acceptance rate). If the entry level researcher manages to be 5-15%
(geometric mean: 9%) better than the next researcher in line (perhaps
by Deliberately Practicing), then that would be the factor multiplied
to the impact attributed to a researcher. This would then produce the
counterfactual impact of the researcher.
Note: I tried considering a “complex model” taking into account
displacement chains (explained in the appendix), but the
improvement was almost nothing (few $100k) compared to the final
value, so it has been moved to the appendix. Adding it here only made
the explanation unnecessarily complicated.
Bringing it all together for 2021 to 2056
The different factors computed so far are listed below:
Organization level
Impact of GiveWell in 2021 = $205m
Constant growth rate of GiveWell = 9.4%
Counterfactual impact = 31%
Researcher Level
Impact attributed to an entry level researcher in the first year
(2021) = 0.6%
Growth Rate due to increase in experience = 19% between 2021 and
2035, and 0 otherwise.
Growth Rate due to increase in staff members = −10%
Counterfactual contribution of Researcher = 9%
Taking them into account brings the counterfactual impact of a
researcher to $8m (could be off by a factor of 10). The following
table shows part of the Google Sheets document used to compute
the final outcome.
Year
Impact of GW (m$)
Counterfactual of GW (%)
Impact attributed to a researcher (%)
Counterfactual attributed to a researcher (%)
Total (m$)
2021
206
31%
0.60%
9%
0.03
2022
225
31%
0.64%
9%
0.04
2023
246
31%
0.69%
9%
0.05
2024
270
31%
0.74%
9%
0.06
2025
295
31%
0.79%
9%
0.07
…
…
…
…
…
…
2033
605
31%
1.37%
9%
0.24
2034
662
31%
1.46%
9%
0.28
2035
724
31%
1.57%
9%
0.32
2036
792
31%
1.41%
9%
0.32
…
…
…
…
…
…
2055
4,367
31%
0.19%
9%
0.25
2056
4,777
31%
0.17%
9%
0.25
Note: Milan Griffes estimates here that as an entry level
researcher he expects to make an impact of $90k (back in 2016). This
is one-third the estimate for 2021 as per the above calculations.
Why I can only be a factor 10 confident
First of all these are all forecasts, there are a ton of assumptions,
such as the constant growth rates, the entire counterfactual scenario
being “made up”, counterfactual impact of a researcher being 9%,
etc. I don’t think a more complex model is the solution. Even a really
complex model of the growth rate will have it’s own un-testable
assumptions by virtue of it being a forecast.
It is very hard to verify this on any level. For example, GiveWell
estimates that in 2015 an entry level researcher contributes 10% of a
co-founder. I don’t know how accurate this is and how they are testing
this assumption over time.
Many numbers are crazy sensitive. Just consider that the
counterfactual contribution of a researcher changes to 1% instead of
the 9%, then the final estimate falls from $8m to $800k. And we
will “never really know” what the “true” value is. The estimation gets
even harder with organizations like Rethink Priorities for example,
where they produce research and are not really “moving money”.
Appendix: Counterfactual Impact of a Researcher
The counterfactual impact of a researcher is the difference in impact
between the “Actual Scenario” where there is an extra researcher and
the “Counterfactual Scenario” which is without that extra
researcher.
A model table is shown below attempting to estimate the difference in
impact between the 2 scenarios. In the actual scenario we have
researchers $R_{1}$ to $R_{n}$ and people who Earn-To-Give:
$R_{n+1}$ followed by $E_{2}$ to $E_{n}$. In the counterfactual
scenario we are missing $R_{1}$ and the whole chain displaces
upwards accordingly.EA is the amount of impact associated with
being a researcher and ETGX is the Earning-To-Give impact of
different people. And the final counterfactual impact is given by
$R_{n}-R_{n+1}$ x EA + $R_{n+1}$x ETG1.
Actual
Counterfactual
Impact
Difference
$R_{1}$
$R_{2}$
1x EA
$R_{1}-R_{2}$ x EA
$R_{2}$
$R_{3}$
1x EA
$R_{2}-R_{3}$ x EA
…
…
…
…
$R_{n-1}$
$R_{n}$
1x EA
$R_{n-1}-R_{n}$x EA
$R_{n}$
$R_{n+1}$
1x EA
$R_{n}-R_{n+1}$x EA
$R_{n+1}$
0
ETG1
$R_{n+1} - 0$ x ETG1
$E_2$
$E_2$
ETG2
0
…
…
…
0
$E_n$
$E_n$
ETGN
0
-
-
Total
$R_{1}-R_{n+1}$x EA + $R_{n+1}$x ETG1
There are two components to the counterfactual impact of a
researcher. The difference between the extra ($R_{1}$) and the
displaced researcher is the first one: $R_{n+1}$) ($R_{n}-R_{n+1} x EA$. The researcher positions seem very competitive now (with ~1%
acceptance rate). If the extra researcher ($R_{1}$) manages to be
5-15% (geometric mean: 9%) better than the displaced researcher, then
the impact will be 0.09 x EA.
In addition, the displaced researcher’s Earning-To-Give ($R_{n+1}$ x ETG1) will also contribute to the counterfactual impact. Assuming the
person is European and is willing to donate 10% of his salary, this
would roughly mean an Earning-To-Give base of $4000 with a growth
rate of 5%.
The impact of GiveWell-like organization is simulated by shifting
GiveWell’s impact by 5 years (geometric mean of 3 and 10
years). So in 2021 GiveWell-like organizations’ impact is basically
GiveWell’s 2016 impact.
“(Co-founder time was given a score of 1 but time spent by newer
staff was scaled down significantly; for example, we multiplied
time spent by entry-level staff in their first year at GiveWell by
0.1.)”—GiveWell
“The table shows that our overall capacity has increased
significantly since 2012 (from approximately 3 co-founder
equivalents then to 9 today).”---
GiveWell
Lifetime Impact of a GiveWell Researcher?
Please leave your kind feedback on the essay (on writing, logic, relevance etc.) in the comments section or through private message. Any feedback is much appreciated.
Entry question
What
iscould be the lifetime estimated impact (in$
donated) of being aEAGiveWell researcher?TL;DR
This is a fermi estimate and shall be treated like that. Everything is an estimate, there are many uncertainties and geometric means have been used to bring the limits down to one number.
Lifetime Impact of a GiveWell Researcher:
$
8mEpistemic Status: Off by a factor 10 (#fermi-estimates)
Introduction
I am curious to know what sort of impact a
EAGiveWell Researcher has. I choose GiveWell, as there is a very “clear” dollar figure on how much money they “influenced” (at least according to them). With this essay, the lifetime impact of an entry level researcher starting in 2021, and working for another 35 years, is estimated. The following components help put together the final figure:Organization Level
Impact of GiveWell in 2021
Growth rate of GiveWell’s impact for the next 35 years
Counterfactual impact of GiveWell
Researcher Level
Impact attributed to an entry level researcher in the first year (2021)
Growth rate of researcher’s impact
Counterfactual impact of a Researcher
The rest of the essay deals with the above components and the final lifetime impact calculation.
Note1: Fermi estimates are in general factor 2 or even factor 10 off. In many cases this might just be enough.
Note2: This estimate does not consider several factors such as probability of EA loosing its momentum, probability of getting fired and not being able to contribute as a researcher etc.
Impact and Growth Rate of GiveWell in 2021
GiveWell estimates how much money they move every year and publishes them on their website. For example, in the year 2019 GiveWell estimated that they moved
$
172m. In order to estimate the impact of GiveWell in 2021, a bit of extrapolation of data is needed. For this extrapolation, the growth rate is used.To predict the growth rate in the coming years I believe it would be reasonable to base it on the money moved from 2015 to 2019 instead of 2007 to 2019. This is because it appears that in the recent years the growth has sort of “plateaued” (see table below) and this might be more representative of the future.
Thus, the compound annual growth rate for the money moved between 2015 and 2019 comes out to be 9.4% (
(172/120)^(1/4)-1
). It’s really hard to predict the change in growth rate for the coming years. For simplicity’s sake, a constant growth rate of 9.4% for the next 35 years, is assumed.$
) [1]Impact of GiveWell in 2021
As for the impact of GiveWell in 2021 it is estimated to be
$
205m (172*(1.094)^2
), based on a growth rate of 9.4% and the money moved in 2019.Counterfactual Impact of GiveWell
The counterfactual impact of GiveWell is the difference in impact between the “Actual Scenario” (world with GiveWell since 2007) and the “Counterfactual Scenario” (world without GiveWell since 2007).
In the counterfactual scenario, it is expected that a GiveWell-like organization would have formed anyway, 3-10 years later (than when GiveWell was founded)[2]. It appears likely because, many “similar” organizations (such as Charity Science Health, Rethink Priorities, Open Philanthropy, The Life You Can Save etc.) have formed since then (between 2010 and 2020), involving cost-effectiveness estimates and philosophies such as “doing the most-good for a dollar”.
While comparing the actual and counterfactual scenario, we compare the two scenarios with the same amount of money but only varying cost-effectiveness depending on the scenario. As a result it is expected that when money doesn’t go to GiveWell-like organizations, it goes to some random organization which is guessed to be within 1 to 1/50th times as cost-effective as GiveWell.
The counterfactual impact for the year 2021 is calculated as follows: As part of the actual scenario, GiveWell is estimated to move
$
206m with cost-effectiveness of 1. As part of the counterfactual scenario GiveWell-like organization is expected to move$
118m [3] with cost-effectiveness of 1, in that year. The remainder$
88m (206m - 118m$
) is used by “other” organizations at one-seventh (geometric mean of 1 and 1⁄50) the cost-effectiveness of that of GiveWell. The difference between the actual and the counterfactual situation comes to be$
75m. And finally, the counterfactual impact as a function of the money moved in the actual scenario is 36%.The below table summarizes the counterfactual impact for the different years. For years beyond 2021 the counterfactual impact essentially starts to become a constant value of 31%. This 31% is used in the final calculation as a constant.
AS --> Actual scenario with cost-effectiveness 1
CS GW-like --> Counterfactual scenario with cost-effectiveness 1
CS non-GW-like --> Counterfactual scenario with cost-effectiveness 1⁄7
Note: Cost of operation of GiveWell is not taken into account as it is a small sum. For example, it is estimated that the cost of operation for 45 people (
$
110k per person) shall be$
5m. Contrast that to it’s 206m expected to be moved in 2021 (a mere 3%). So this is skipped confidently.Note: Also the effects of the displacement chain are not considered for simplicity’s sake.
Impact Attributed to an Entry Level Researcher in the First Year
In the year 2015, GiveWell estimates that an entry level researcher had 10% of the impact of a co-founder and that there were 9 co-founder equivalents [4]. In other words an entry level researcher was estimated to be responsible for 1.1% (
10%/9
) of the total impact (in 2015). But this is not the whole story.With more people joining GiveWell, this percentage gets diluted. In 2015 there were 25 “equivalent staff members”, where as in 2021 there are 45 “equivalent staff members” [5]. The resulting contribution would thus be much less. Let’s assume this contribution is inversely proportional to the total number of staff. Then, the impact attributed to an entry level researcher in the first year (2021) is 0.6% (
1.1%x25/45
).Growth Rate of a Researcher
The growth rate of a researcher is split into 2 factors. The first being the growth rate associated with the ‘increase in experience’, which leads to higher researcher’s contribution. And the second being the growth rate associated with ‘increase in number of staff’ of GiveWell, which decreases the researcher’s contribution as years pass.
For the “growth rate associated with increase in experience”, the year-variable is held constant (and hence the number of staff is constant) and the difference between entry level researcher and co-founder is observed.
In 2015, a co-founder is estimated to be responsible for 11%[4:1] of the impact that year and an entry level researcher at 1.1% (as mentioned above). If it takes an entry level researcher 10-20 years (geometric mean: 14 years) to reach co-founder level, then growth rate per year due to increase in experience (alone) is a whopping 19% (
(11%/1.1%)^(1/n)-1
). (It is assumed further that the growth rate falls sharply to 0% after reaching co-founder level.)For the “growth rate attributed to increase in staff”, the average increase in the number of people per year in GiveWell is estimated. This increase is assumed to be negatively proportional to growth rate attributed to increase in staff.
From 2015 to 2021 the number of staff has increased from 25 to 45 [5:1]. This implies an increase per year of 10% per year (
(45/25)^(1/6)-1
). This implies a growth rate attributed to increase in staff of-10%
.Counterfactual Impact of a Researcher
The researcher positions seem very competitive now (with ~1% acceptance rate). If the entry level researcher manages to be 5-15% (geometric mean: 9%) better than the next researcher in line (perhaps by Deliberately Practicing), then that would be the factor multiplied to the impact attributed to a researcher. This would then produce the counterfactual impact of the researcher.
Note: I tried considering a “complex model” taking into account displacement chains (explained in the appendix), but the improvement was almost nothing (few
$
100k) compared to the final value, so it has been moved to the appendix. Adding it here only made the explanation unnecessarily complicated.Bringing it all together for 2021 to 2056
The different factors computed so far are listed below:
Organization level
Impact of GiveWell in 2021 =
$
205mConstant growth rate of GiveWell = 9.4%
Counterfactual impact = 31%
Researcher Level
Impact attributed to an entry level researcher in the first year (2021) = 0.6%
Growth Rate due to increase in experience = 19% between 2021 and 2035, and 0 otherwise.
Growth Rate due to increase in staff members = −10%
Counterfactual contribution of Researcher = 9%
Taking them into account brings the counterfactual impact of a researcher to
$
8m (could be off by a factor of 10). The following table shows part of the Google Sheets document used to compute the final outcome.Note: Milan Griffes estimates here that as an entry level researcher he expects to make an impact of
$
90k (back in 2016). This is one-third the estimate for 2021 as per the above calculations.Why I can only be a factor 10 confident
First of all these are all forecasts, there are a ton of assumptions, such as the constant growth rates, the entire counterfactual scenario being “made up”, counterfactual impact of a researcher being 9%, etc. I don’t think a more complex model is the solution. Even a really complex model of the growth rate will have it’s own un-testable assumptions by virtue of it being a forecast.
It is very hard to verify this on any level. For example, GiveWell estimates that in 2015 an entry level researcher contributes 10% of a co-founder. I don’t know how accurate this is and how they are testing this assumption over time.
Many numbers are crazy sensitive. Just consider that the counterfactual contribution of a researcher changes to 1% instead of the 9%, then the final estimate falls from
$
8m to$
800k. And we will “never really know” what the “true” value is. The estimation gets even harder with organizations like Rethink Priorities for example, where they produce research and are not really “moving money”.Appendix: Counterfactual Impact of a Researcher
The counterfactual impact of a researcher is the difference in impact between the “Actual Scenario” where there is an extra researcher and the “Counterfactual Scenario” which is without that extra researcher.
A model table is shown below attempting to estimate the difference in impact between the 2 scenarios. In the actual scenario we have researchers
$R_{1}$
to$R_{n}$
and people who Earn-To-Give:$R_{n+1}$
followed by$E_{2}$
to$E_{n}$
. In the counterfactual scenario we are missing$R_{1}$
and the whole chain displaces upwards accordingly.EA
is the amount of impact associated with being a researcher andETGX
is the Earning-To-Give impact of different people. And the final counterfactual impact is given by$R_{n}-R_{n+1}$ x EA
+$R_{n+1}$x ETG1
.$R_{1}$
$R_{2}$
1x EA
$R_{1}-R_{2}$ x EA
$R_{2}$
$R_{3}$
1x EA
$R_{2}-R_{3}$ x EA
$R_{n-1}$
$R_{n}$
1x EA
$R_{n-1}-R_{n}$x EA
$R_{n}$
$R_{n+1}$
1x EA
$R_{n}-R_{n+1}$x EA
$R_{n+1}$
ETG1
$R_{n+1} - 0$ x ETG1
$E_2$
$E_2$
ETG2
$E_n$
$E_n$
ETGN
$R_{1}-R_{n+1}$x EA
+$R_{n+1}$x ETG1
There are two components to the counterfactual impact of a researcher. The difference between the extra (
$R_{1}$
) and the displaced researcher is the first one:$R_{n+1}$
) ($R_{n}-R_{n+1} x EA$
. The researcher positions seem very competitive now (with ~1% acceptance rate). If the extra researcher ($R_{1}$
) manages to be 5-15% (geometric mean: 9%) better than the displaced researcher, then the impact will be0.09 x EA
.In addition, the displaced researcher’s Earning-To-Give (
$R_{n+1}$ x ETG1
) will also contribute to the counterfactual impact. Assuming the person is European and is willing to donate 10% of his salary, this would roughly mean an Earning-To-Give base of$
4000 with a growth rate of 5%.Footnotes
2014
2015
2016
2017
2018
2019
(80000hours also does something similar to estimate their counterfactual impact by assuming they changed someone’s career path by a few years).
The impact of GiveWell-like organization is simulated by shifting GiveWell’s impact by 5 years (geometric mean of 3 and 10 years). So in 2021 GiveWell-like organizations’ impact is basically GiveWell’s 2016 impact.
“(Co-founder time was given a score of 1 but time spent by newer staff was scaled down significantly; for example, we multiplied time spent by entry-level staff in their first year at GiveWell by 0.1.)”—GiveWell
“The table shows that our overall capacity has increased significantly since 2012 (from approximately 3 co-founder equivalents then to 9 today).”--- GiveWell
1⁄9 = 11%
Well in 2015 there were 21 people and 10 board members (not sure how much work they do). Let’s round it to 25 people.
In 2021 GiveWell has 41 staff and 6 board members. We round it up to 45 people.---GiveWell