We’re aiming to shift the balance towards supporting high-quality reasoning. Every tool has some non-zero usefulness for non-central use cases, but seems unlikely that it will be as useful as tools that were made for those use cases.
Every tool has some non-zero usefulness for non-central use cases, but seems unlikely that it will be as useful as tools that were made for those use cases.
I agree!
supporting high-quality reasoning
This sounds to me like almost the most generic-problem-solving thing someone could aim for, capable of doing many things without going outside the general use case.
As a naive example, couldn’t someone use “high quality reasoning” to plan how to make military robotics? (though the examples I’m actually worried about are more like “use high quality reasoning to create paperclips”, but I’m happy to use your one)
In other words, I’m not really worried about a chess robot being used for other things [update: wait, Alpha Zero seems to be more general purpose than expected], but I wouldn’t feel as safe with something intentionally meant for “high quality reasoning”
[again, just sharing my concern, feel free to point out all the ways I’m totally missing it!]
I agree that misuse is a concern. Unlike alignment, I think it’s relatively tractable because it’s more similar to problems people are encountering in the world right now.
To address it, we can monitor and restrict usage as needed. The same tools that Elicit provides for reasoning can also be used to reason about whether a use case constitutes misuse.
This isn’t to say that we might not need to invest a lot of resources eventually, and it’s interestingly related to alignment (“misuse” is relative to some values), but it feels a bit less open-ended.
We’re aiming to shift the balance towards supporting high-quality reasoning. Every tool has some non-zero usefulness for non-central use cases, but seems unlikely that it will be as useful as tools that were made for those use cases.
I agree!
This sounds to me like almost the most generic-problem-solving thing someone could aim for, capable of doing many things without going outside the general use case.
As a naive example, couldn’t someone use “high quality reasoning” to plan how to make military robotics? (though the examples I’m actually worried about are more like “use high quality reasoning to create paperclips”, but I’m happy to use your one)
In other words, I’m not really worried about a chess robot being used for other things [update: wait, Alpha Zero seems to be more general purpose than expected], but I wouldn’t feel as safe with something intentionally meant for “high quality reasoning”
[again, just sharing my concern, feel free to point out all the ways I’m totally missing it!]
I agree that misuse is a concern. Unlike alignment, I think it’s relatively tractable because it’s more similar to problems people are encountering in the world right now.
To address it, we can monitor and restrict usage as needed. The same tools that Elicit provides for reasoning can also be used to reason about whether a use case constitutes misuse.
This isn’t to say that we might not need to invest a lot of resources eventually, and it’s interestingly related to alignment (“misuse” is relative to some values), but it feels a bit less open-ended.
[debugging further]
Do you think misuse is a concern—to the point that if you couldn’t monitor and restrict usage—you’d think twice about this product direction?
Or is this more “this is a small issue, and we can even monitor and restrict usage, but even if we couldn’t then we wouldn’t really mind”?
What are your views on whether speeding up technological development is, in general, a good thing?
I’m thinking of arguments like https://forum.effectivealtruism.org/posts/gB2ad4jYANYirYyzh/a-note-about-differential-technological-development, that make me wonder if we should try to slow research instead of speeding it up.
Or do you think that Elicit will not speed up AGI capabilities research in a meaningful way? (Maybe because it will count as misuse)
It’s something I’m really uncertain about personally, that’s going to heavily influence my decisions/life, so I’m really curious about your thoughts!