Some other things that come to mind: 1. I’m nervous about the “when will X labor be automated”, as a lot of jobs just become more demanding. 2. Similarly, not sure how valuable patent rates are. 3. I really want better indicators of “how quickly is AI progress happening?” My guess now is that the most reliable ones are things like, “How much effective computation is happening over time?” 4. “Long-term autonomous LLM operation when?” → I’m nervous about this, as I expect that most long-running processes will have some percent human oversight/intervention, so it’s hard to define.
A lot of these concepts are slippery, and I’d generally expect a lot of human+AI hybrid systems for a long time, making it less clear how to measure the AI part specifically.
Some other things that come to mind:
1. I’m nervous about the “when will X labor be automated”, as a lot of jobs just become more demanding.
2. Similarly, not sure how valuable patent rates are.
3. I really want better indicators of “how quickly is AI progress happening?” My guess now is that the most reliable ones are things like, “How much effective computation is happening over time?”
4. “Long-term autonomous LLM operation when?” → I’m nervous about this, as I expect that most long-running processes will have some percent human oversight/intervention, so it’s hard to define.
A lot of these concepts are slippery, and I’d generally expect a lot of human+AI hybrid systems for a long time, making it less clear how to measure the AI part specifically.
Thanks Ozzie