Correct me if I’m wrong but my understanding is most everything ChatGPT can do was already possible with GPT3 (especially post InstructGPT) but it just took more intentional wrangling. What ChatGPT seems to be offering is a much more accessible interface.
text-davinci-003 (which is effectively ChatGPT) was a bit better than text-davinci-002 anecdotally and when I benchmarked it on TriviaQA. It was only released about a week before ChatGPT so it’s not necessarily unreasonable to lump them together. If you do, then the interface isn’t the only change one might associate with ChatGPT.
This is probably a stupid question, but: do we actually know if ChatGPT uses text-davinci-003?
When I talk to ChatGPT with the Network tab of Chrome DevTools open, filter for the name “conversation,” and look at any request payload, I see that it has the key-value pair
model: “text-davinci-002-render”
Which seems to indicate that it might not be using text-davinci-003.
The blog post says ChatGPT is trained with proximal policy optimization. This documentation says text-davinci-003 was trained with PPO, but not text-davinci-002.
However, it is interesting what you’re saying about the request payloads, because this seems to be contradictory. So I’m not quite sure anymore. It’s possible that ChatGPT was trained with PPO on top of the non-PPO text-davinci-002.
Yeah, it didn’t update my timeline number much since I’d seen other language models, but it started to make short-timeline intuitions feel a lot more real as the capabilities are a lot more obvious now.
Correct me if I’m wrong but my understanding is most everything ChatGPT can do was already possible with GPT3 (especially post InstructGPT) but it just took more intentional wrangling. What ChatGPT seems to be offering is a much more accessible interface.
That sounds accurate. The key difference with ChatGPT is that there’s a LOT more public attention to the underlying capabilities of GPT-3.
text-davinci-003 (which is effectively ChatGPT) was a bit better than text-davinci-002 anecdotally and when I benchmarked it on TriviaQA. It was only released about a week before ChatGPT so it’s not necessarily unreasonable to lump them together. If you do, then the interface isn’t the only change one might associate with ChatGPT.
This is probably a stupid question, but: do we actually know if ChatGPT uses text-davinci-003?
When I talk to ChatGPT with the Network tab of Chrome DevTools open, filter for the name “conversation,” and look at any request payload, I see that it has the key-value pair
Which seems to indicate that it might not be using text-davinci-003.
The blog post says ChatGPT is trained with proximal policy optimization. This documentation says text-davinci-003 was trained with PPO, but not text-davinci-002.
However, it is interesting what you’re saying about the request payloads, because this seems to be contradictory. So I’m not quite sure anymore. It’s possible that ChatGPT was trained with PPO on top of the non-PPO text-davinci-002.
Yeah, it didn’t update my timeline number much since I’d seen other language models, but it started to make short-timeline intuitions feel a lot more real as the capabilities are a lot more obvious now.