I think the cost per hour of engagement is a good intuitive metric to assess the cost-effectiveness of running the EA Forum. From footnote 1, the daily cost to run the EA Forum is 1.48 k$ (= 1.3*10^6*2.5/ā6/ā365.25). There have been around 240 hours of engagement per day over the last 6 months or so. So the cost per hour of engagement has been roughly 6.17 $ (= 1.48*10^3/ā240). I suspect the engagement time would drop significantly if users had to pay 6.17 $ per hour they spend on the EA Forum. I believe this suggests the marginal cost-effectiveness of running the EA Forum is negative (trusting my guess for the usersā revealed preferences), and that there should be a reduction in the time spent running the EA Forum.
The iteration of September 2023 of The Introductory EA Program had 1.58 attendances per hour spent running the program. For 30 $ per hour spent running the program, and 4 h of engagement per attendance (3 h of preparation, plus 1 h of discussion), there would be 6.32 h (= 1.58*4) of engagement per hour spent running the program, and the cost per hour of engagement would be 4.75 $ (= 30ā6.32), 77.0 % (= 4.75/ā6.17) as much as for the EA Forum. So, considering uncertainty, it looks like the iteration of September 2023 of The Introductory EA Program had a cost per hour of engagement similar to that of the EA Forum nowadays.
You can try to make comparisons with other programs, or estimate which work you are doing which generates the most engagement time per $ (although figuring out the additional engagement time a given activity generated is tricky; for example, running an EA Forum event will of course cause engagement with the posts and comments related to the event, but will tend to decrease the engagement with other posts and comments).
Hmm, I think thereās some sense to your calculation (and thus I appreciate you doing+showing this calculation), but the $6.17 conclusionāspecifically, āengagement time would drop significantly if users had to pay 6.17 $ per hour they spend on the EA Forum, which suggests the marginal cost-effectiveness of running the EA Forum is negativeāāstrikes me as incorrect.
What matters is by how much engaging with the Forum raises altruistic impact, which, insofar as this impact can be quantified in dollars, is far, far higher than what one would be willing and able to pay out of oneās own pocket to use the Forum. @NunoSempere once estimated the (altruistic) value of the average EA project to be between 10 and 500 million dollars (see cell C4 of this spreadsheet; hereās the accompanying post). That is far higher than the actual dollar cost of running the average project. (Indeed, if one is funded by EA money, then oneās generation of altruistic dollars needs to outpace oneās consumption of actual dollarsāand by a large multiplier, if one is to meet the funding bar.)
Going back to NuƱoās spreadsheet: If I make the arrogant assumption that Iām within an order of magnitude of Ben Todd, impact-wise, then that means my lifetime impact is at least 10 million dollars. Assuming linearity (which isnāt a great assumption, but letās go with it for now) and a career length of 40 years, this means my impact over the past 4 years has been ā„1 million dollars.[1] In that time, Iāve spent maybe 500 hours on the EA Forum.[2] Meanwhile, Iād say that the Forum has contributed greatly to my intellectual development, i.e., added at least 20% to my impact. (The true percentage may in fact be much higher, because of crucial considerations that the Forum has helped me orient toward, but letās lowball things at 20%, for now.) This would imply that my impact has been amplified by at least $200,000/ā(500 hours) = $400 per hour spent on the Forum. ([Insert usual caveats about there being large error bars.]) Contrast with your $6.17.
(I did this calculation on myself not because Iām special, but because I know what the numbers are for myself. Iād guess that the per-hour bottom line for other Forum users would be ~similar.)
We can now go one step further, and estimate the Forumās āaltruistic dollar generated per actual dollar spentā multiplier to be at least 400ā6.17 ā 65. Embarrassingly, I donāt know how this compares against todayās funding bar,[3] but seems very plausible to me that itās above.
(Nonetheless, people may still not pay $6.17/āhour to use the Forum because $6.17/āhour is a non-trivial cost considering peopleās actual incomes. Additionally, people are just used to being able to browse the internet for free, and so I suspect many wouldnāt do the expected value calculations and reach the ārationalā conclusion that they should in fact pay.)
That is, 500 hours consuming the Forumās content. Iāve also spent time writing on the Forum, but if we model the Forum as a two-way market, with writers and consumers, and say that itās the consumers who benefit from being here, then it doesnāt make sense to include my writing time. (Alsoāand perhaps more relevantlyāI donāt think writing time gets counted by the Forumās analytics engine as engagement time if itās spent mostly in a Google doc.)
Further detail: What really matters is what the multiplier is on the margin (i.e., what it is for the last dollar being spent on a project), rather than what it is for the project as a whole.
Thanks for sharing your thoughts, Will! I agree what matters is the additional altruistic impact caused by engaging with the Forum. However, I think my point holds as long as people have accurate views about how to maximise their altruistic impact.
For example, if you believed āfactual impact of your marginal hour on the Forumāāācounterfactual impact of this hourā < āimpact of donating 100 $[1] to the organisation or project you consider the most cost-effectiveā, and using the Forum costed 100 $/āh, I think you would have a greater altruistic impact by your own lights by spending less time on the Forum, and donating the savings. Do you agree?
Analogously, if the user spending the marginal hour on the Forum believed āfactual impact of their marginal hour on the Forumāāācounterfactual impact of this hourā < āimpact of donating 6.17 $ to the organisations or projects they consider the most cost-effectiveā, and using the Forum costed 6.17 $, I think they would have a greater altruistic impact by their own lights by spending less time on the Forum, and donating the savings. In this case, if the marginal user-hour costed 6.17 $ to the Forum team[2], I believe they would also increase altruistic impact in the eyes of the user of the marginal hour by spending less time generating engagement on the Forum, and donating the savings to the organisations or projects that user considers the most cost-effective.
I think the cost per hour of engagement is a good intuitive metric to assess the cost-effectiveness of running the EA Forum. From footnote 1, the daily cost to run the EA Forum is 1.48 k$ (= 1.3*10^6*2.5/ā6/ā365.25). There have been around 240 hours of engagement per day over the last 6 months or so. So the cost per hour of engagement has been roughly 6.17 $ (= 1.48*10^3/ā240). I suspect the engagement time would drop significantly if users had to pay 6.17 $ per hour they spend on the EA Forum. I believe this suggests the marginal cost-effectiveness of running the EA Forum is negative (trusting my guess for the usersā revealed preferences), and that there should be a reduction in the time spent running the EA Forum.
The iteration of September 2023 of The Introductory EA Program had 1.58 attendances per hour spent running the program. For 30 $ per hour spent running the program, and 4 h of engagement per attendance (3 h of preparation, plus 1 h of discussion), there would be 6.32 h (= 1.58*4) of engagement per hour spent running the program, and the cost per hour of engagement would be 4.75 $ (= 30ā6.32), 77.0 % (= 4.75/ā6.17) as much as for the EA Forum. So, considering uncertainty, it looks like the iteration of September 2023 of The Introductory EA Program had a cost per hour of engagement similar to that of the EA Forum nowadays.
You can try to make comparisons with other programs, or estimate which work you are doing which generates the most engagement time per $ (although figuring out the additional engagement time a given activity generated is tricky; for example, running an EA Forum event will of course cause engagement with the posts and comments related to the event, but will tend to decrease the engagement with other posts and comments).
@Toby Tremlettš¹, you may be interested in this comment.
Hmm, I think thereās some sense to your calculation (and thus I appreciate you doing+showing this calculation), but the $6.17 conclusionāspecifically, āengagement time would drop significantly if users had to pay 6.17 $ per hour they spend on the EA Forum, which suggests the marginal cost-effectiveness of running the EA Forum is negativeāāstrikes me as incorrect.
What matters is by how much engaging with the Forum raises altruistic impact, which, insofar as this impact can be quantified in dollars, is far, far higher than what one would be willing and able to pay out of oneās own pocket to use the Forum. @NunoSempere once estimated the (altruistic) value of the average EA project to be between 10 and 500 million dollars (see cell C4 of this spreadsheet; hereās the accompanying post). That is far higher than the actual dollar cost of running the average project. (Indeed, if one is funded by EA money, then oneās generation of altruistic dollars needs to outpace oneās consumption of actual dollarsāand by a large multiplier, if one is to meet the funding bar.)
Going back to NuƱoās spreadsheet: If I make the arrogant assumption that Iām within an order of magnitude of Ben Todd, impact-wise, then that means my lifetime impact is at least 10 million dollars. Assuming linearity (which isnāt a great assumption, but letās go with it for now) and a career length of 40 years, this means my impact over the past 4 years has been ā„1 million dollars.[1] In that time, Iāve spent maybe 500 hours on the EA Forum.[2] Meanwhile, Iād say that the Forum has contributed greatly to my intellectual development, i.e., added at least 20% to my impact. (The true percentage may in fact be much higher, because of crucial considerations that the Forum has helped me orient toward, but letās lowball things at 20%, for now.) This would imply that my impact has been amplified by at least $200,000/ā(500 hours) = $400 per hour spent on the Forum. ([Insert usual caveats about there being large error bars.]) Contrast with your $6.17.
(I did this calculation on myself not because Iām special, but because I know what the numbers are for myself. Iād guess that the per-hour bottom line for other Forum users would be ~similar.)
We can now go one step further, and estimate the Forumās āaltruistic dollar generated per actual dollar spentā multiplier to be at least 400ā6.17 ā 65. Embarrassingly, I donāt know how this compares against todayās funding bar,[3] but seems very plausible to me that itās above.
(Nonetheless, people may still not pay $6.17/āhour to use the Forum because $6.17/āhour is a non-trivial cost considering peopleās actual incomes. Additionally, people are just used to being able to browse the internet for free, and so I suspect many wouldnāt do the expected value calculations and reach the ārationalā conclusion that they should in fact pay.)
Sanity check: 80,000 Hours says that impactful roles generate millions of dollars worth of altruistic impact per year.
That is, 500 hours consuming the Forumās content. Iāve also spent time writing on the Forum, but if we model the Forum as a two-way market, with writers and consumers, and say that itās the consumers who benefit from being here, then it doesnāt make sense to include my writing time. (Alsoāand perhaps more relevantlyāI donāt think writing time gets counted by the Forumās analytics engine as engagement time if itās spent mostly in a Google doc.)
Further detail: What really matters is what the multiplier is on the margin (i.e., what it is for the last dollar being spent on a project), rather than what it is for the project as a whole.
Thanks for sharing your thoughts, Will! I agree what matters is the additional altruistic impact caused by engaging with the Forum. However, I think my point holds as long as people have accurate views about how to maximise their altruistic impact.
For example, if you believed āfactual impact of your marginal hour on the Forumāāācounterfactual impact of this hourā < āimpact of donating 100 $[1] to the organisation or project you consider the most cost-effectiveā, and using the Forum costed 100 $/āh, I think you would have a greater altruistic impact by your own lights by spending less time on the Forum, and donating the savings. Do you agree?
Analogously, if the user spending the marginal hour on the Forum believed āfactual impact of their marginal hour on the Forumāāācounterfactual impact of this hourā < āimpact of donating 6.17 $ to the organisations or projects they consider the most cost-effectiveā, and using the Forum costed 6.17 $, I think they would have a greater altruistic impact by their own lights by spending less time on the Forum, and donating the savings. In this case, if the marginal user-hour costed 6.17 $ to the Forum team[2], I believe they would also increase altruistic impact in the eyes of the user of the marginal hour by spending less time generating engagement on the Forum, and donating the savings to the organisations or projects that user considers the most cost-effective.
Implying the marginal cost-effectiveness of your time on the Forum is 25 % (= 400ā100) of the past cost-effectiveness.
I guess it costs more due to increasing user-hours becoming more difficult as user-hours increase.