The estimated mQARPs per employee per month seems to differ substantially between sections. Is this based on something like dividing the posts/papers the org produced by the org’s total budget or number of FTE employees? (Your comment on ALLFED vs AI safety papers seems to indicate this? Note that I didn’t look closely at the Guesstimate model.)
“I have reason to believe that the amount of money influenced was large, and Larks’s writeup was further the only one available. My confidence interval is then $100,000 to $1M in moved funding”. That seems surprisingly high, but I have no specific knowledge on this. Could you share your reason to believe that? (But no problem if the reasoning is based on private info or is just hard to communicate concisely and explicitly.)
“You can see my first guesstimate model here, but I think that this was too low because it only took into account the value of papers, so here is a second model, which takes into account that the value which does not come from papers is 2 to 20x the magnitude of the value which does come from papers. I’m still uncertain, so my final estimate is even more uncertain than the Guesstimate model.”
Could you give a sense of where you see the rest of that value is coming from? (I’m not saying I disagree with you.)
Was that accounted for in the other evaluations too? My impression from your written description (without looking at the models) was that e.g. for ALLFED you estimated their impact as entirely coming from posts and papers?
Is there a reason you estimated the impact of the Giving Tuesday and EAGx things in terms of dollars moved, without converting that into mQARPS?
Part of why this confuses me is that:
I’d guess that dollars moved is actually not the primary value of EAGx (though you can still convert the value into dollars-moved-equivalents if you want)
Meanwhile, I’d guess that dollars moved is the primary value of the animal welfare commitments post and maybe some other posts (though you can still convert the value into mQARPs if you want)
“Similarly, while EAGx Boston 2018 and the EA Giving Tuesday Donation Matching Initiative might have taken similar amounts of time to organize, by comparably capable people, I prefer the second. This is in large part because EAGx events are scalable, whereas Giving Tuesdays are not.” Did you mean to say Giving Tuesday is scalable, whereas EAGx events are not?
In the case of ALLFED, this is based on my picturing of one employee going about its month, and asking myself how surprising it would be if they couldn’t produce 10 mQARPS of value per month, or how surprising it would be if they could produce 50 mQARPs per month. In the case of the AI safety organizations, this is based on estimating the value of each of the papers that Larks things are valuable enough to mention, and then estimating what fraction of the total value of an organization those are.
Private info
a) Building up researchers into more capable researchers, knowledge acquired that isn’t published, information value of trying out dead ends, acquiring prestige, etc. b) I actually didn’t estimate ALLFED’s impact, I estimated the impact of the marginal hires, per 1.
Personal taste, it’s possible that was the inferior choice. I found it more easy to picture the dollars moved than the improvement in productivity. In hindsight, maybe improving retention would be another main benefit which I didn’t consider.
I got that as a comment. The intuition here is that it would be really, really hard to find a project which moves as much money as Giving Tuesday and which you could do every day, every week, or every month. But if there are more than 52 local EA groups, an EAGx could be organized every week. If you think that EA is only doing projects at maximum efficiency (which it isn’t), and knowing only that Giving Tuesdays are done once a year and EAGx are done more often, I’d expect one EAGx to be less valuable than one Giving Tuesday.
Or, in other words, I’d expect there to be some tradeoff between quality and scalable.
I actually didn’t estimate ALLFED’s impact, I estimated the impact of the marginal hires, per 1.
So did that estimate of the impact of marginal hires also account for how much those hires would contribute to “Building up researchers [themselves or others] into more capable researchers, knowledge acquired that isn’t published, information value of trying out dead ends, acquiring prestige”?
Some specific things I was confused about
The estimated mQARPs per employee per month seems to differ substantially between sections. Is this based on something like dividing the posts/papers the org produced by the org’s total budget or number of FTE employees? (Your comment on ALLFED vs AI safety papers seems to indicate this? Note that I didn’t look closely at the Guesstimate model.)
“I have reason to believe that the amount of money influenced was large, and Larks’s writeup was further the only one available. My confidence interval is then $100,000 to $1M in moved funding”. That seems surprisingly high, but I have no specific knowledge on this. Could you share your reason to believe that? (But no problem if the reasoning is based on private info or is just hard to communicate concisely and explicitly.)
“You can see my first guesstimate model here, but I think that this was too low because it only took into account the value of papers, so here is a second model, which takes into account that the value which does not come from papers is 2 to 20x the magnitude of the value which does come from papers. I’m still uncertain, so my final estimate is even more uncertain than the Guesstimate model.”
Could you give a sense of where you see the rest of that value is coming from? (I’m not saying I disagree with you.)
Was that accounted for in the other evaluations too? My impression from your written description (without looking at the models) was that e.g. for ALLFED you estimated their impact as entirely coming from posts and papers?
Is there a reason you estimated the impact of the Giving Tuesday and EAGx things in terms of dollars moved, without converting that into mQARPS?
Part of why this confuses me is that:
I’d guess that dollars moved is actually not the primary value of EAGx (though you can still convert the value into dollars-moved-equivalents if you want)
Meanwhile, I’d guess that dollars moved is the primary value of the animal welfare commitments post and maybe some other posts (though you can still convert the value into mQARPs if you want)
“Similarly, while EAGx Boston 2018 and the EA Giving Tuesday Donation Matching Initiative might have taken similar amounts of time to organize, by comparably capable people, I prefer the second. This is in large part because EAGx events are scalable, whereas Giving Tuesdays are not.” Did you mean to say Giving Tuesday is scalable, whereas EAGx events are not?
In the case of ALLFED, this is based on my picturing of one employee going about its month, and asking myself how surprising it would be if they couldn’t produce 10 mQARPS of value per month, or how surprising it would be if they could produce 50 mQARPs per month. In the case of the AI safety organizations, this is based on estimating the value of each of the papers that Larks things are valuable enough to mention, and then estimating what fraction of the total value of an organization those are.
Private info
a) Building up researchers into more capable researchers, knowledge acquired that isn’t published, information value of trying out dead ends, acquiring prestige, etc. b) I actually didn’t estimate ALLFED’s impact, I estimated the impact of the marginal hires, per 1.
Personal taste, it’s possible that was the inferior choice. I found it more easy to picture the dollars moved than the improvement in productivity. In hindsight, maybe improving retention would be another main benefit which I didn’t consider.
I got that as a comment. The intuition here is that it would be really, really hard to find a project which moves as much money as Giving Tuesday and which you could do every day, every week, or every month. But if there are more than 52 local EA groups, an EAGx could be organized every week. If you think that EA is only doing projects at maximum efficiency (which it isn’t), and knowing only that Giving Tuesdays are done once a year and EAGx are done more often, I’d expect one EAGx to be less valuable than one Giving Tuesday.
Or, in other words, I’d expect there to be some tradeoff between quality and scalable.
Thanks for the clarifications :)
So did that estimate of the impact of marginal hires also account for how much those hires would contribute to “Building up researchers [themselves or others] into more capable researchers, knowledge acquired that isn’t published, information value of trying out dead ends, acquiring prestige”?
Oof, no it didn’t, good point.