I was in the process of writing a comment trying to debunk this. My counterexample didn’t work so now I’m convinced this is a pretty good post. This is a nice way of thinking about ITN quantitatively.
The counterexample I was trying to make might still be interesting for some people to read as an illustration of this phenomenon. Here it is:
Scale “all humans” trying to solve “all problems” down to “a single high school student” trying to solve “math problems”. Then tractability (measured as % of problem solved / % increase in resources) for this person to solve different math problems is as follows:
A very large arithmetic question like “find 123456789123456789^2 by hand” requires ~10 hours to solve
A median international math olympiad question probably requires ~100 hours of studying to solve
A median research question requires an undergraduate degree (~2000 hours) and then specialized studying (~1000 hours) to solve
A really tough research question takes a decade of work (~20,000 hours) to solve
A way ahead of its time research question (maybe, think developing ML theory results before there were even computers) I could see taking 100,000+ hours of work
Here tractability varies by 4 orders of magnitude (10-100,000 hours) if you include all kinds of math problems. If you exclude very easy or very hard things (as Thomas was describing) you end up with 2 orders of magnitude (~1000-100,000 hours).
You may already have this in mind but—if you are re-running this program in summer 2023, I think it would be a good idea to announce this further in advance.