Thanks for the pushback. I agree that a linear model will be importantly wrong, although if you approximate the impact from the conference using the number of connections people report and assume that stays roughly the same, it doesn’t seem wild as a first pass. (Please let me know if you disagree!)
[Half-formed thoughts below.]
On the other hand, I think 10-20% more valuable seems very off to me, especially in this case, given we were not “lowering the bar” for the second group of attendees. Setting this case aside, I can imagine a world in which someone is very confident in their ability to admit the people who will benefit the most from a conference (and the people who would be most useful for them to meet with), and in this world, you might be able to get 90% of the value with 50% of the size — but I don’t really think we’re in this world (especially in terms of identifying people who will benefit most from the event).
I’m not really sure how well people self-sort at conferences, which was a big uncertainty for me when I was thinking about these things more. I do think people will often identify (often with help) some of the people with whom it would be most useful to meet. If people are good at self-sorting (e.g. searching through swapcard and finding the most promising 10-15 meetings), and if those most-useful meetings over the whole conference aren’t somehow concentrated on meetings with a small number of nodes, then admitting double the people will likely lead to more than double the impact.[1] If people are not good at self-sorting, though, it seems more likely that we’d get closer to straightforward doubling, I think. (I’m fairly confident that people are better than random, though.)
It does seem possible that there are some “nodes” in the network — at a very bad first pass, you could imagine that everyone’s most valuable meetings are with the speakers. The speakers each meet with lots of people (say, they have lots of time and don’t get tired) and would be at the conference in any world (doubling or not). Then the addition of 500 extra people doesn’t significantly improve the set of possible meetings for the 500 first attendees, although 500 extra people get to meet with the speakers (which is nearly all that matters in this model).
I’m really unsure about the extent to which the “nodes” thing is true (and if it’s true I don’t really think that “speakers” are the right group), but there’s something here that seems like it could be right given what we hear. There’s also the added nuance that some nodes are probably in the second group of 500, and also that the size and capacity for meetings of the “nodes” group would matter.
Hmm, I do really think there is a very wrong intuition here. I think by-default in most situations the return to doubling a specific resource should be modeled as logarithmic (i.e. the first doubling is as valuable as the second doubling). I think in this model, it is very rare that doubling a thing along any specific dimension produces twice the value. I think the value of marginally more people in EA should likely also be modeled as having logarithmic returns (or I might argue worse than logarithmic returns, but I think logarithmic is the right prior).
I think you will get estimates wrong by many orders of magnitude if you do reasoning of the type “if I just double this resource that will double the whole value of the event”, unless you have a strong argument for network effects.
I wonder how much of your intuition comes from thinking that marginal (ex ante) impact of marginal EAG attendees is much lower than the existing average, vs normal logarithmic prior considerations vs how much of it comes from diseases of scale (e.g. higher population making things harder to coordinate, pressure towards conformity).
The first consideration is especially interesting to isolate, since:
I think the value of marginally more people in EA should likely also be modeled as having logarithmic returns (or I might argue worse than logarithmic returns, but I think logarithmic is the right prior
If you think doubling the quality-adjusted people in EA overall has logarithmic returns, you still get ~linear effects from doubling the output of one event or outreach project, since differential functions are locally linear.
Thanks for the pushback. I agree that a linear model will be importantly wrong, although if you approximate the impact from the conference using the number of connections people report and assume that stays roughly the same, it doesn’t seem wild as a first pass. (Please let me know if you disagree!)
[Half-formed thoughts below.]
On the other hand, I think 10-20% more valuable seems very off to me, especially in this case, given we were not “lowering the bar” for the second group of attendees. Setting this case aside, I can imagine a world in which someone is very confident in their ability to admit the people who will benefit the most from a conference (and the people who would be most useful for them to meet with), and in this world, you might be able to get 90% of the value with 50% of the size — but I don’t really think we’re in this world (especially in terms of identifying people who will benefit most from the event).
I’m not really sure how well people self-sort at conferences, which was a big uncertainty for me when I was thinking about these things more. I do think people will often identify (often with help) some of the people with whom it would be most useful to meet. If people are good at self-sorting (e.g. searching through swapcard and finding the most promising 10-15 meetings), and if those most-useful meetings over the whole conference aren’t somehow concentrated on meetings with a small number of nodes, then admitting double the people will likely lead to more than double the impact.[1] If people are not good at self-sorting, though, it seems more likely that we’d get closer to straightforward doubling, I think. (I’m fairly confident that people are better than random, though.)
It does seem possible that there are some “nodes” in the network — at a very bad first pass, you could imagine that everyone’s most valuable meetings are with the speakers. The speakers each meet with lots of people (say, they have lots of time and don’t get tired) and would be at the conference in any world (doubling or not). Then the addition of 500 extra people doesn’t significantly improve the set of possible meetings for the 500 first attendees, although 500 extra people get to meet with the speakers (which is nearly all that matters in this model).
I’m really unsure about the extent to which the “nodes” thing is true (and if it’s true I don’t really think that “speakers” are the right group), but there’s something here that seems like it could be right given what we hear. There’s also the added nuance that some nodes are probably in the second group of 500, and also that the size and capacity for meetings of the “nodes” group would matter.
Hmm, I do really think there is a very wrong intuition here. I think by-default in most situations the return to doubling a specific resource should be modeled as logarithmic (i.e. the first doubling is as valuable as the second doubling). I think in this model, it is very rare that doubling a thing along any specific dimension produces twice the value. I think the value of marginally more people in EA should likely also be modeled as having logarithmic returns (or I might argue worse than logarithmic returns, but I think logarithmic is the right prior).
I think you will get estimates wrong by many orders of magnitude if you do reasoning of the type “if I just double this resource that will double the whole value of the event”, unless you have a strong argument for network effects.
I wonder how much of your intuition comes from thinking that marginal (ex ante) impact of marginal EAG attendees is much lower than the existing average, vs normal logarithmic prior considerations vs how much of it comes from diseases of scale (e.g. higher population making things harder to coordinate, pressure towards conformity).
The first consideration is especially interesting to isolate, since:
If you think doubling the quality-adjusted people in EA overall has logarithmic returns, you still get ~linear effects from doubling the output of one event or outreach project, since differential functions are locally linear.