EA Hotel Fundraiser 3: Estimating the relative Expected Value of the EA Hotel (Part 1)

[CEEALAR]

Executive summary

Here in Part 1, we will define a formula for the expected value (EV) of the EA Hotel, relative to donating to other EA organisations, called $rV$. We will then factor each of the elements of $rV$ as much as we can, and present our best estimate for each of these factors, using data gathered from current residents. In Part 2, we will use these estimates as an anchor for defining a plausible range of values for each factor, and do a Monte Carlo simulation on them. We aim to show 1) that under a wide range of assumptions, the hotel is higher EV than the alternative and 2) a few showcased values for $rV$ that will match different potential donors.

1. What is the relative EV of the EA Hotel, $rV$?

Since the hotel is a meta-charity that hosts people who are working on projects that range as broadly as EA itself, we will not try to calculate actual EV, as in QALYs or some other standardised measure. We will instead attempt to calculate relative EV, which is EV divided by counterfactual EV.

Let us first establish the counterfactual. We will assume that most readers have a donation budget, which they already intend to spend on EA. We will also assume that the EA organisation they would otherwise fund is funding-constrained, and would spend these additional funds on hiring. We imagine a hypothetical philanthropist P, who has $X to spend. This amount would allow either $\frac{X}{C_{hire}}$ hires to be made ($C_{hire}$ being the cost of a marginal hire at the counterfactual EA org), or it would allow $\frac{X}{C_{resident}}$ residents to live at the hotel (note that $C_{resident}$ is low—it costs us <£6000/​year all in to host someone at the EA Hotel).

Counterfactual EV would be the added value of these marginal hires, which is the impact of their work subtracted by the impact they would be making when not employed by an EA organisation. We call it $V_{org}=\frac{X}{C_{hire}}E[V_{hire}-V_{hire}^c]$.

Hotel EV would be the added value of these additional residents, which we will call $V_{hotel}=\frac{X}{C_{resident}}E[V_{resident}-V_{resident}^c]$. We will assume that the amount of residents we can host scales linearly with the amount of funding we receive. Since economies of scale apply, and since our funding gap arguably wouldn’t be closed until well into the hundreds of thousands, this is a conservative assumption.

Relative EV would be dividing the latter by the former. Simplifying, we get $rV=\frac{V_{hotel}}{V_{org}}=\frac{C_{hire}}{C_{resident}}\frac{E\left[V_{resident}\right]-E\left[V_{resident}^c\right]}{E\left[V_{hire}\right]-E\left[V_{hire}^c\right]}$.

In the rest of this post, we will be further elaborating and estimating all of these variables.

1.1 Making $rV$ more intuitive

In many of our estimates, we will defer to our intuition to estimate an answer. This is a necessary evil, for we are calculating the EV of a group of projects that is sufficiently heterogeneous to be beyond properly formalising.

We found that the terms in our equation are easier to intuit if we define each one of them as relative to the counterfactual. For each of our variables, we define a relative version:

$rV=\frac{C_{hire}}{C_{resident}}\frac{\frac{E\left[V_{resident}\right]}{E\left[V_{hire}\right]}-\frac{E\left[V_{resident}^c\right]}{E\left[V_{hire}\right]}}{1-\frac{E\left[V_{hire}^c\right]}{E\left[V_{hire}\right]}}=\frac{C_{hire}}{C_{resident}}\frac{r{V}_{resident}-r{V}_{resident}^c}{1-r{V}_{hire}^c}$

2 Defining $r{V}_{resident}$

Remember that $r{V}_{resident}$ stands for the value of the work of one hotel resident, divided by the value of the work of one counterfactual EA hire.

The value of the hotel comes from:

• Allowing more people to do charity work

• Allowing those people to do charity work more resourcefully, by giving them:

  1. Time (because the hotel does their housekeeping, and because they don’t have to travel much for social activities)

  2. Focus (by engineering an environment that boosts mental health and productivity)

  3. A network (by living with ~20 residents and an array of visitors passing through)

The question of EV of the hotel, per resident, can thus be factored as follows:

1) How resourceful does the hotel environment allow a resident to be, compared to a counterfactual EA hire?

2) If they were given the same resources as a counterfactual EA hire, how much value would the work of a hotel resident create?

We specify $r{V}_{resident}$ as follows:

$r{V}_{resident}=\frac{R_{resident}}{R_{hire}}\frac{W_{resident}}{W_{hire}}=rRrW$

• $rR$ being the resources that a hotel resident could invest into their work, relative to a counterfactual EA hire
• $rW$ being the value generated by their work, after controlling for resources.

2.1 What is $rW$, the relative value of a project enabled by the EA Hotel?

This variable has a very precise meaning, so allow us to describe it in detail.

$rW$ should capture:

• The quality of the project that a hotel resident wants to carry out

• The quality of the judgment of the resident in particular

It should not capture the amount of resources that this particular hotel resident can spend on their work. Any difference in $rW$ that doesn’t stem from their project, should stem from their general ability to carry out projects.

To have a good intuition for $rW$, imagine that we would have a hotel resident, and put them in a “normalised” working environment. That is: they have their own house to keep, access to an office at a normal distance, a social life that requires travel, a professional network of an average amount of loose ties, etc. Then the impact of their particular project, divided by the impact of a marginal EA hire, would be $rW$. This is perhaps the crux of the whole EV calculation.

We have found that formally estimating $rW$ is nearly impossible to do properly, because it involves factoring it into all the complexity of the various projects that EA Hotel guests are doing. These projects change over time, are often highly explorative and/​or entrepreneurial and cover a broad range of cause areas including those for which no established metrics exist yet.

Yet, we have an intuitive sense that most of these projects clearly meet EA standards, and aren’t necessarily less impactful than a marginal EA hire.

Now EA prides itself for doing the measurements to determine whether a strategy for impact is actually worthwhile, but we feel that this risks neglecting the black box of intuition that often gives better answers.

If you are considering the hotel as a recipient of your donation, we ask you to make your own intuitive estimate for $rW$. These are the steps:

1. Identify your counterfactual. Which charity would you donate to, if not the EA Hotel?

2. Imagine that your donation would be exactly enough to make the charity hire one additional employee. Intuit the amount of impact this employee would make.

3. Imagine that your donation would also allow exactly one additional guest to stay at the hotel. Look at our last post for reference. Intuit the amount of impact the work of this guest would make, if it were carried out by this guest with a normal level of motivation, in a regular office, with a network of normal size.

4. Now divide the former to the latter. This is your personal intuitive value for $rW$.

You may be more comfortable using bounds instead of numbers. Bar negative outcomes, it is obvious that $rW$ is more than 0.01. It is also obvious that $rW$ is less than 100. Where would you put the bounds of $rW$, so that you are still 90% certain that $rW$ is between them?

You may be thinking: what about hotel guests who are studying, or skilling up in a new area? In these cases we could need to factor out a discount rate, $D$, and a lead time in years, $T$, to working on object level work of expected relative quality $r{W}^{\prime }$:

$rW=\left(D^T\right)r{W}^{\prime }$

Discount rates should be set at a level you think is reasonable given the Haste Consideration (note that the Haste Consideration also favours investing in more people wanting to transition to full time direct EA work, as the hotel is doing). Perhaps $D$= 0.9, meaning a 10% yearly discount, is reasonable. Or maybe even $D$=0.8 (20%) if you think the urgency of direct EA work is particularly high.$T$ might typically be between 0.5 and 2 years.

2.2 What is $rR$, the factor by which the productivity of EA Hotel residents is multiplied?

Imagine a marginal EA hire would spend 30 focused hours on work. If a hotel resident finds the means to work 40 focused hours, due to less time spent on housekeeping and travel, and due to a motivating environment, that is an amount of added value equal to 33% of a marginal EA hire working full-time. With this example, $rR$ would be 1.33. Since the hotel environment often causes a considerable increase in productivity, we want to add this element to our calculations.

While residents report that they are able to do 2.2 times as much work at the hotel on average, this number does not map nicely to our analysis. For some residents, it compares their overall impact with the case in which they would self-fund. For others it takes into account that they’re doing different work. We are modelling $rR$ as a factor on productivity that comes from the hotel as compared to a regular job, in which the employee has to organise their own life to a much larger extent. Some factors that go into $rR$ are:

• Freed up time because housework is largely done by the hotel, and because less time is spent on travel

• Increased well-being, from an environment intentionally designed for it, and free access to counselling

• An improved professional and social network, from living with 20 EA’s and frequent visitors

We will generalise the first two into a variable we would like to call “personal productivity”, which is the amount of concentrated effort one can put into one’s day to day work. The latter will have an analysis on its own. We expect these factors to be roughly independent, meaning that they both feed into $rR$:

$rR=\frac{R_{resident}}{R_{hire}}=\frac{P_{resident}}{P_{hire}}\frac{N_{resident}}{N_{hire}}=rPrN$

$rP$ being the amount of productive hours worked relative to an EA org hire, and $rN$ being the value of one’s network relative to the counterfactual.

2.2.1 Estimating relative productivity, $rP$

Note that $rP=\frac{P_{resident}}{P_{hire}}$. The numerator being the amount of productive hours a hotel resident can work, and the denominator being the amount of productive hours a marginal EA hire can work.

Our hope is that a healthy and time-saving hotel environment causes an increase in the amount of focused hours worked.

We noticed that our culture strongly inclines us towards productivity. Simply sitting in the living room compels one to work. We expect that access to an environment like this leads to an increase in hours worked.

We asked residents to estimate their amounts of productive hours per week, and found an average of 40:

We also asked how many productive hours they would have worked otherwise, and found an average of 29.

While there is some reason to believe that the former will go up as we further develop the hotel, we will now use it as a conservative estimate for $P_{resident}$. While it may be tempting to use the latter as an estimate for $P_{hire}$, we don’t think this would be fair: 29 is the hours that our residents would otherwise work from home. Compared to a good office, working from home presumably lowers their productivity, so $P_{hire}$ might be higher. It’s also worth considering that EA organisation employees are selected for conscientiousness, hard work and dedication at a high rate.

So we conclude that $rP=\frac{40}{P_{hire}}$, leaving the value of $P_{hire}$ to our simulation in Part 2.

2.2.2 estimating network value, $rN$

Having more like-minded people around gives you more opportunities. According to the social version of Metcalfe’s law, the value of a network of $n$ nodes scales as $n\times \log n$, or $\log n$ per node. We take this to mean that, for each person in a network each increment in size will increase their number of opportunities by one, but since their bandwidth is limited and interests don’t always align, they will only be able to capitalise on the equivalent value of $\log n$ opportunities.

Observe that, in some abstract sense, all the value of the EA community can be stated as an accumulation of positive sum trades between people. Our $n\times \log n$ law states that, if one’s network size changes from $N_a$ to $N_b$, the value of their work is multiplied by $\frac{\log N_a}{\log N_b}$.

So we redefine $rN$ as follows:

$rN=\frac{\log S_{resident}}{\log S_{hire}}$

$S_{resident}$ being the professional network size of a hotel resident, and $S_{hire}$ being the professional network size of a counterfactual EA hire.

While for some, “professional network size” has an intuitive meaning, we found that this isn’t the case for everyone, so allow us to define that more precisely. What we mean is the set of people that you could easily interact with in a professional capacity, if incentives lined up that way. To approximate this number, take each person you know. Do the two of you have common knowledge about the rough outline of both of each other’s work? Then they’re in your professional network. Note that this means that people in a wildly different cause area, or outside of EA, can be in your professional network.

We can’t estimate the value of $S_{hire}$, because that depends on the counterfactual of the donor, but we have asked our residents to estimate their professional network size:

Note the logarithmic scale.

2.3 Conclusion for $r{V}_{resident}$

Putting it all together, we get $r{V}_{resident}=\frac{E\left[W_{resident}{R}_{resident}\right]}{E\left[W_{hire}{R}_{hire}\right]}=\frac{E\left[W_{resident}{P}_{resident}\log \right(S_{resident}\left)\right]}{E\left[W_{hire}{P}_{hire}\log \right(S_{hire}\left)\right]}$.

For each guest, we have gathered their answers for $P$ and $S$ and estimated their $W$ based on our intuition, given what we know about them.

The data suggests that $P_{resident}$ and $S_{resident}$ are mostly uncorrelated (r = 0.15), and so are $W_{resident}$ and $P_{resident}$ (r = 0.04), but $S_{resident}$ and $W_{resident}$ are strongly correlated (r = 0.6). This may be owed to our possibly biased assumption that the most value is in building infrastructure, which naturally requires a larger network.

We will assume that the same pattern of independencies will hold for hires. Simplifying further:

$r{V}_{resident}=\frac{E\left[P_{resident}\right]E\left[W_{resident}\log \right(S_{resident}\left)\right]}{E\left[P_{hire}\right]E\left[W_{hire}\log \right(S_{hire}\left)\right]}$

We will calculate $r{V}_{resident}$ under a range of assumptions in Part 2.

3. What are $r{V}_{resident}^c$ and $r{V}_{hire}^c$, the relative values of work done by people if they’re not housed or hired?

Both of these can be given a similar structure to $r{V}_{resident}$:

$r{V}_{resident}^c=\frac{E\left[P_{resident}^c\right]E\left[W_{resident}^c\log \right(S_{resident}^c\left)\right]}{E\left[P_{hire}\right]E\left[W_{hire}\log \right(S_{hire}\left)\right]}$

$r{V}_{hire}^c=\frac{E\left[P_{hire}^c\right]E\left[W_{hire}^c\log \right(S_{hire}^c\left)\right]}{E\left[P_{hire}\right]E\left[W_{hire}\log \right(S_{hire}\left)\right]}$

$W^c$, $P^c$, … being the values of these variables in the counterfactual situation where the resident or hire is doing something else with their career.

Properly estimating these variables will require gathering data from EA’s. This wasn’t an option for us given short runways, so we will have to make do with rough estimates instead.

3.1 Rough estimates for $r{V}_{resident}^c$

3.1.1 $W_{resident}^c$

15 of the hotel residents are doing work that they would otherwise self-fund by working part time. For these, we can define $W_{resident}^c=W_{resident}$.

Of those that are doing something else at the hotel, three would be working a normal job outside EA, one would continue doing promising AI capabilities research, and one would continue to scout for opportunities in the EA landscape. We estimate that their values for $W^c$ would be $0.1W_{resident}$, $-W_{resident}$ and $0.5W_{resident}$ respectively.

Using the average of our current guests as an estimate, we will guess that $E\left[W_{resident}^c\right]$ is around $\frac{5}{6}E\left[W_{resident}\right]$

3.1.2 $P_{resident}^c$

Since most of the hotel residents would self-fund, they would spend considerably less time working on their projects, with the rest of their time spent on working part-time instead. They report an average speedup of 2.2, which we will use to guess that $2.2E\left[P_{resident}^c\right]\approx E\left[P_{resident}\right]$.

3.1.3 $S_{resident}^c$

We would expect that everyone meets approximately the same amount of people when they move to the hotel, so we will define $S_{resident}$ to be a constant increase over $S_{resident}^c$. Our survey shows that this increase is an average of 20.6, which we will use to say $E\left[\log \right(S_{resident}^c\left)\right]\approx E\left[\log \right(S_{resident}-20.6\left)\right]$.

3.1.4 $r{V}_{resident}^c$

Putting the above together, we get:

$r{V}_{resident}^c=\frac{0.38E\left[P_{resident}\right]E\left[W_{resident}\log \right(S_{resident}^c\left)\right]}{E\left[W_{hire}\log \right(S_{hire}\left)\right]}$

3.2 Rough estimates for $r{V}_{hire}^c$

3.2.1 $W_{hire}^c$

We don’t have any data whatsoever about likely values for this, so we will leave it to the simulation in Part 2.

3.2.2 $P_{hire}^c$

Given the strong competition, we will assume that a marginal EA hire is highly conscientious. They might not have a project available to them that is as good as working at an EA org, but whatever project they will be working, they will work on it productively. Our best guess for $P_{hire}^c$ is that it equals $P_{hire}$.

3.2.3 $S_{hire}^c$

Again, we’re assuming that this number is some constant difference from $S_{hire}$, the size of which depends on the particular organisation that the counterfactual EA hire joins. If they’re doing remote work, it may be just a few, or only 1. If they get to work at an office of a major EA org, it may be as high as 50. Even more if they relocate to a large hub like the Bay or London.

3.2.4 $V_{hire}^c$

Putting the above together, we get:

$r{V}_{hire}^c=\frac{E\left[W_{hire}^c\log \right(S_{hire}^c\left)\right]}{E\left[W_{hire}\log \right(S_{hire}\left)\right]}$

3.3 Conclusions for $r{V}_{resident}^c$ and $r{V}_{hire}^c$

We will calculate $r{V}_{resident}^c$ and $r{V}_{hire}^c$ under a range of assumptions in Part 2.

4. Putting it all together

We get:

$rV=\frac{C_{hire}}{C_{resident}}\frac{r{V}_{resident}-r{V}_{resident}^c}{1-r{V}_{hire}^c}$

$=\frac{C_{hire}}{C_{resident}}\frac{\frac{E\left[P_{resident}\right]E\left[W_{resident}\log \right(S_{resident}\left)\right]}{E\left[P_{hire}\right]E\left[W_{hire}\log \right(S_{hire}\left)\right]}-\frac{0.38E\left[P_{resident}\right]E\left[W_{resident}\log \right(S_{resident}^c\left)\right]}{E\left[W_{hire}\log \right(S_{hire}\left)\right]}}{1-\frac{E\left[W_{hire}^c\log \right(S_{hire}^c\left)\right]}{E\left[W_{hire}\log \right(S_{hire}\left)\right]}}$

$=\frac{C_{hire}}{C_{resident}}\frac{E\left[P_{resident}\right](\frac{E\left[W_{resident}\log \right(S_{resident}\left)\right]}{E\left[P_{hire}\right]}-0.38E[W_{resident}\log \left(S_{resident}^c\right)\left]\right)}{E\left[W_{hire}\log \right(S_{hire}\left)\right]-E\left[W_{hire}^c\log \right(S_{hire}^c\left)\right]}$

---

In Part 2 of this post we will give some example calculations under a range of assumptions and scenarios, and attempt to draw some conclusions. We will also link to an online calculator for readers to calculate their own estimates.

Thanks to Toon Alfrink for developing the methodology and drafting the post, Greg Colbourn for editing, and Sasha Cooper and Florent Berthet for comments.

---

Our Ask

Do you like this initiative, and want to see it continue for longer, on a more stable footing? Do you want to cheaply buy time spent working full time on work relating to EA, whilst simultaneously facilitating a thriving EA community hub? Is your relative EV estimate for the project high? Then we would like to ask for your support.

We are getting critically low on runway. Our current shortfall is ~£5k/​month from May onward. We will have to start giving current guests notice in 1 month.

To donate, please see our GoFundMe or get in touch if you’d like to make a direct bank transfer.

If you’d like to give regular support, we also have a Patreon.

Previous posts in this series:

• EA Hotel Fundraiser 1: the story

• EA Hotel Fundraiser 2: current guests and their projects