For my own benefit I thought I’d write down examples of markets that I can see are inadequate yet inexploitable. Not all of these I’m sure are actually true, some just fit the pattern.
I notice that most charities aren’t cost effective, but if I decide to do better by making a super cost-effective charity I shouldn’t expect to be more successful than the other charities.
I notice that most people at university aren’t trying to learn but get good signals for their career, I can’t easily do better in the job market by stopping trying to signal and just learn better
I notice most parenting technique books aren’t helpful (because genetics), but I probably can’t make money by selling a shorter book that tells you the only parenting techniques that do matter.
If I notice that politicians aren’t trying to improve the country very much, I can’t get elected over them by just optimising for improving the country more (because they’re optimising for being elected).
If most classical musicians spend a lot of money on high-status instruments and spend time with high-status teachers that don’t correlate with quality, you can’t be more successful by just picking high quality instruments and teachers.
If most rocket companies are optimising for getting the most money out of government, you probably can’t win government contracts by just making a better rocket company. (?)
If I notice that nobody seems to be doing research on the survival of the human species, I probably can’t make it as an academic by making that my focus
If I notice that most music recommendation sites are highly reviewing popular music (so that they get advance copies) I can’t have a more successful review site/magazine by just being honest about the music.
Correspondingly, if these models are true, here are groups/individuals who it would be a mistake to infer strong information about if they’re not doing well in these markets:
Just because a charity has a funding gap doesn’t mean it’s not very cost-effective
Just because someone has bad grades at university doesn’t mean they are bad at learning their field
Just because a parenting book isn’t selling well doesn’t mean it isn’t more useful than others
Just because a politician didn’t get elected doesn’t mean they wouldn’t have made better decisions
Just because a rocket company doesn’t get a government contract doesn’t mean it isn’t better at building safe and cheap rockets than other companies
Just because an academic is low status / outside academia doesn’t mean they’re views aren’t true
Just because a band isn’t highly reviewed in major publications doesn’t mean it isn’t innovative/great
Some of these seem stronger to me than others. I tend to think that academic fields are more adequate at finding truth and useful knowledge than music critics are adequate at figuring out which bands are good.
As an academic in existential risk, I thought I would comment. In my experience, it is challenging getting interdisciplinary papers published, which is why I think it would be great if someone started an interdisciplinary existential/global catastrophic risk journal. But I would say that mentions of “global catastrophic risk” and “existential risk” appeared to be growing about 40% per year when I tried to analyze Google scholar. This growth is good for citations, and my paper citations have not been bad.
Good list! Makes me wonder whether there’s some way to model the expected level of adequacy in a field. Factors we’d have to consider:
How much money is available within the field?
How much prestige is available within the field?
How many people are there in the field?
How much do participants care about the field for non-monetary, non-prestige reasons? That is, how inherently fun is it to work within this field?
How hard is it to work within the field? That is, to what extent do skills other than “having good ideas” matter? (Some scientific fields invest hundreds of hours of grunt work in every paper, while a philosophy paper requires little effort outside of writing.)
How fast does new information enter the field, compared to the amount of information that exists within the field already?
How many competing ideas/groups exist within the field, and how easy is it for a new idea/group to get started?
As a toy example, you could look at the metagame for the Modern format of Magic: The Gathering, which consists of roughly one hundred thousand players (maybe ten thousand of whom are really serious about winning). Rewards for being in the top 1% of the serious group equate to a few thousand dollars in profit per year and a few dozen fans; for the top 0.1%, a few tens of thousands of dollars in profit per year and a few thousand fans, plus a solid chance at a steady job if you want it (producing Magic-related media, working as a designer, etc.).
It’s possible to generate a good burst of fame and profit by creating a new deck that matches up well against current popular decks (the “metagame”).
Information enters the field rapidly, at a pace of a few hundred new cards (into a pool of 11,000) every three months. Building a new deck and thoroughly testing it against the metagame might take a few hundred dollars and a few dozen hours, but the cost is balanced by the fact that playing Magic: The Gathering is a lot of fun. The competitors you need to worry about are people who spend about 50 hours/week playing, and who are as skilled or a little more skilled than you are at the “basics” of the game. Maybe 5% of new decks that are tested this thoroughly turn out to generate any kind of positive return, relative to playing a deck someone else designed already. And so on.
...at the end, we can observe that a new, good Modern deck (one capable of winning a 500-player, $5000 tournament) comes out once every couple of months, and that cards from sets that are more than one year old are almost never at the center of “new, good decks”, relative to newer sets. Magic: The Gathering seems to be an adequate market; any new ideas are absorbed quickly, and few people really get a chance to profit off of them. The evaluation system for cards and decks optimizes more slowly than the evaluation system for stocks, but more quickly than the evaluation system for Major League Baseball players circa 1990.
I’d be interested in viewing examples for other fields, with better data collection, perhaps gathered into some kind of “adequacy” database.
For my own benefit I thought I’d write down examples of markets that I can see are inadequate yet inexploitable. Not all of these I’m sure are actually true, some just fit the pattern.
I notice that most charities aren’t cost effective, but if I decide to do better by making a super cost-effective charity I shouldn’t expect to be more successful than the other charities.
I notice that most people at university aren’t trying to learn but get good signals for their career, I can’t easily do better in the job market by stopping trying to signal and just learn better
I notice most parenting technique books aren’t helpful (because genetics), but I probably can’t make money by selling a shorter book that tells you the only parenting techniques that do matter.
If I notice that politicians aren’t trying to improve the country very much, I can’t get elected over them by just optimising for improving the country more (because they’re optimising for being elected).
If most classical musicians spend a lot of money on high-status instruments and spend time with high-status teachers that don’t correlate with quality, you can’t be more successful by just picking high quality instruments and teachers.
If most rocket companies are optimising for getting the most money out of government, you probably can’t win government contracts by just making a better rocket company. (?)
If I notice that nobody seems to be doing research on the survival of the human species, I probably can’t make it as an academic by making that my focus
If I notice that most music recommendation sites are highly reviewing popular music (so that they get advance copies) I can’t have a more successful review site/magazine by just being honest about the music.
Correspondingly, if these models are true, here are groups/individuals who it would be a mistake to infer strong information about if they’re not doing well in these markets:
Just because a charity has a funding gap doesn’t mean it’s not very cost-effective
Just because someone has bad grades at university doesn’t mean they are bad at learning their field
Just because a parenting book isn’t selling well doesn’t mean it isn’t more useful than others
Just because a politician didn’t get elected doesn’t mean they wouldn’t have made better decisions
Just because a rocket company doesn’t get a government contract doesn’t mean it isn’t better at building safe and cheap rockets than other companies Just because an academic is low status / outside academia doesn’t mean they’re views aren’t true
Just because a band isn’t highly reviewed in major publications doesn’t mean it isn’t innovative/great
Some of these seem stronger to me than others. I tend to think that academic fields are more adequate at finding truth and useful knowledge than music critics are adequate at figuring out which bands are good.
As an academic in existential risk, I thought I would comment. In my experience, it is challenging getting interdisciplinary papers published, which is why I think it would be great if someone started an interdisciplinary existential/global catastrophic risk journal. But I would say that mentions of “global catastrophic risk” and “existential risk” appeared to be growing about 40% per year when I tried to analyze Google scholar. This growth is good for citations, and my paper citations have not been bad.
Good list! Makes me wonder whether there’s some way to model the expected level of adequacy in a field. Factors we’d have to consider:
How much money is available within the field?
How much prestige is available within the field?
How many people are there in the field?
How much do participants care about the field for non-monetary, non-prestige reasons? That is, how inherently fun is it to work within this field?
How hard is it to work within the field? That is, to what extent do skills other than “having good ideas” matter? (Some scientific fields invest hundreds of hours of grunt work in every paper, while a philosophy paper requires little effort outside of writing.)
How fast does new information enter the field, compared to the amount of information that exists within the field already?
How many competing ideas/groups exist within the field, and how easy is it for a new idea/group to get started?
As a toy example, you could look at the metagame for the Modern format of Magic: The Gathering, which consists of roughly one hundred thousand players (maybe ten thousand of whom are really serious about winning). Rewards for being in the top 1% of the serious group equate to a few thousand dollars in profit per year and a few dozen fans; for the top 0.1%, a few tens of thousands of dollars in profit per year and a few thousand fans, plus a solid chance at a steady job if you want it (producing Magic-related media, working as a designer, etc.).
It’s possible to generate a good burst of fame and profit by creating a new deck that matches up well against current popular decks (the “metagame”).
Information enters the field rapidly, at a pace of a few hundred new cards (into a pool of 11,000) every three months. Building a new deck and thoroughly testing it against the metagame might take a few hundred dollars and a few dozen hours, but the cost is balanced by the fact that playing Magic: The Gathering is a lot of fun. The competitors you need to worry about are people who spend about 50 hours/week playing, and who are as skilled or a little more skilled than you are at the “basics” of the game. Maybe 5% of new decks that are tested this thoroughly turn out to generate any kind of positive return, relative to playing a deck someone else designed already. And so on.
...at the end, we can observe that a new, good Modern deck (one capable of winning a 500-player, $5000 tournament) comes out once every couple of months, and that cards from sets that are more than one year old are almost never at the center of “new, good decks”, relative to newer sets. Magic: The Gathering seems to be an adequate market; any new ideas are absorbed quickly, and few people really get a chance to profit off of them. The evaluation system for cards and decks optimizes more slowly than the evaluation system for stocks, but more quickly than the evaluation system for Major League Baseball players circa 1990.
I’d be interested in viewing examples for other fields, with better data collection, perhaps gathered into some kind of “adequacy” database.