Some possible setups:
Say an intervention costs $10,000,000.00, and speeds up the field of information theory by 1% per year for 10 years.
Say a great paper is written in information theory that gets N citations.
What are some reasonable approaches to estimating the expected values of such setups?
Some low-effort thoughts (I am not an economist so I might be embarrassing myself!):
My first inclination is something like “find the average output of the field per unit time, then find the average growth rate of a field, and then calculate the ‘extra’ output you’d get with a higher growth rate.” In other words: (1) what is the field currently doing of value? (2) how much more value would that field produce if they did whatever they’re currently doing faster?
It would be interesting to see someone do a quantitative analysis of the history of progress in some particular field. However, because so much intellectual progress has happened in the last ~300 years by so few people (relatively speaking), my guess is we might not have enough data in many cases.
The more something like the “great man theory” applies to a field (i.e. the more stochastic progress is), the more of a problem you have with this model. [Had an example here, removed it because I no longer think it’s appropriate.]
With regard to that latter question (also your second set-up), I wonder how reliably we could apply heuristics for determining the EV of particular contributions (i.e. how much value do we usually get from papers in field Y with ~X citations?).
Thanks! This is interesting, will spend some time thinking about.
Please don’t worry much about embarrassing yourself! It’s definitely a challenge with forums like this, but it would be pretty unreasonable for anyone to expect that post/comment authors have degrees in all the possibly relevant topics.
Low-effort thoughts can be pretty great, they may be some of the highest value-per-difficulty work.
It’s worth keeping in mind that it could actually be net negative if it pulls enough attention of other (potential) researchers away from better things to work on.
Agreed. I’d add that the EV of intellectual progress in certain fields could also be net negative for other reasons. What resonates with me most is concerns about increasing existential risk (e.g., because some dangerous technological development is accelerated; see also differential progress). But there could also be other downsides, such as risks of increasing meat consumption or environmental damage (e.g., via economic and population growth).
But this will vary from field to field, and based on many other factors, and there are of course many benefits to progress in many areas as well. I just raise this as a possibility/consideration.
Yea, I think there’s a similar concern any time you make other fields more well run. That said, as a rule of thumb, this seems a lot safer than making many other fields less well run. It would be great to be able to apply intellectual abilities selectively, but when that’s too hard, doing it generally seems fairly good to me.
Definitely agreed.
Is it too naive / basic to just suggest value of information calculations?
I think so. While the main value of research lies in it’s value of information, the problem here seems to be about how to go about estimating the impact and not so much about the modeling.
Agreed, though the suggestions are appreciated!
VOI calculations in general seem like a good approach, but figuring out how to best apply them seems pretty tough.
But how do we estimate the EV of estimating the EV of general intellectual progress?
On a less facetious note, it’s about the average effect of intellectual progress on innovation right? What EV comes from general intellectual progress that is not a result of innovation?
So you try to causally estimate the effect of innovation on things you value (e.g. GDP), and you try to create measures of general intellectual progress to see how those causally impact innovation. That’s obviously easier said than done.
That’s a great question! I don’t have any good answer, but I’ve looked online and found some interesting papers so I’ll just post some stuff I’ve got so far.
It seems like there is recently a shift toward “societal-impact focused research”, as opposed to “quality-focused”, driven mostly by the need to calculate Return On Investment. I think that this biases the current metrics/evaluators to be more short-termed and focused on health/security/tech-innovations.
Here, the authors ask research evaluators how they think about assessing societal impact. They have identified 5 dimension -
1. The Importance of the Underpinning Research in Evaluating Impact.
2. The Value of the Impact Versus the Value of the “Right” Impact.
3. Impact as Linear, Controllable or Serendipitous
4. Push Factors and Assessing Impact
5. Measurable Impact Outcomes Versus Unmeasurable Impact Journeys
seems relevant, and I want to look into more deeply—Back to Basics: Basic Research Spillovers, Innovation Policy and Growth
Nice find! This seems like a useful step, though of course likely considerably different than what I imagine consequentialists would aim for.