Just looking at this recently, for thoughts I’ve been having here.
Some super quick thoughts: 1. Overall, really like this sort of work. Want to see more like it. 2. I like that you’ve helped prioritize these questions. 3. I find the questions right now very difficult to parse. There are so many of them, they cover many different topics, it’s hard for me to feel like I grasp them or keep many in my head at once. 4. I think that scorable function definitions would help here.
I’m currently working on planning some potential scorable function definitions around this issue, might make a post on that later.
Some other things that come to mind: 1. I’m nervous about the “when will X labor be automated”, as a lot of jobs just become more demanding. 2. Similarly, not sure how valuable patent rates are. 3. I really want better indicators of “how quickly is AI progress happening?” My guess now is that the most reliable ones are things like, “How much effective computation is happening over time?” 4. “Long-term autonomous LLM operation when?” → I’m nervous about this, as I expect that most long-running processes will have some percent human oversight/intervention, so it’s hard to define.
A lot of these concepts are slippery, and I’d generally expect a lot of human+AI hybrid systems for a long time, making it less clear how to measure the AI part specifically.
Just looking at this recently, for thoughts I’ve been having here.
Some super quick thoughts:
1. Overall, really like this sort of work. Want to see more like it.
2. I like that you’ve helped prioritize these questions.
3. I find the questions right now very difficult to parse. There are so many of them, they cover many different topics, it’s hard for me to feel like I grasp them or keep many in my head at once.
4. I think that scorable function definitions would help here.
I’m currently working on planning some potential scorable function definitions around this issue, might make a post on that later.
Some other things that come to mind:
1. I’m nervous about the “when will X labor be automated”, as a lot of jobs just become more demanding.
2. Similarly, not sure how valuable patent rates are.
3. I really want better indicators of “how quickly is AI progress happening?” My guess now is that the most reliable ones are things like, “How much effective computation is happening over time?”
4. “Long-term autonomous LLM operation when?” → I’m nervous about this, as I expect that most long-running processes will have some percent human oversight/intervention, so it’s hard to define.
A lot of these concepts are slippery, and I’d generally expect a lot of human+AI hybrid systems for a long time, making it less clear how to measure the AI part specifically.
Thanks Ozzie