For onlookers, note that the “instruct models” have been available since Dec 2020, and been progressively improved with training by “prompts” and “completions” by users.
It might be interesting to note, that some believe that these instruct models were created by fine tuning with minute amounts of data. Using a few hundred prompts and completions of users, could have achieved a lot of the functionality improvement we see.
The instruct models greatly improve the economics of using Open AI. This happens through two channels:
OpenAI charges $ by length (of input + output). So long “prompts”, long “completions”, or unsatisfactory results are expensive—instruct models improves each of these factors.
The performance of “instruct models” allows you to use smaller, less expensive models, e.g. “Curie” instead of “Davinci” (a 10x savings itself), to achieve the same functionality.
In total, the savings could be very large. Maybe 10x to 50x for someone savvy.
What’s important is that these cost savings opens up the economics to new business uses. Before, using Davinci can cost hundreds of dollars per user per month. That’s not workable for many business cases. These current savings, as well as expected future development and access to other language models like Eleuther’s, opens up many more products, including several “niches” that aren’t being exploited.
As mentioned above, fine tuning is powerful. This further improves cost efficiency (e.g. more “zero shot” completions) and gets you subtle, or otherwise hard to achieve quality improvements.
This is a great article!
For onlookers, note that the “instruct models” have been available since Dec 2020, and been progressively improved with training by “prompts” and “completions” by users.
It might be interesting to note, that some believe that these instruct models were created by fine tuning with minute amounts of data. Using a few hundred prompts and completions of users, could have achieved a lot of the functionality improvement we see.
The instruct models greatly improve the economics of using Open AI. This happens through two channels:
OpenAI charges $ by length (of input + output). So long “prompts”, long “completions”, or unsatisfactory results are expensive—instruct models improves each of these factors.
The performance of “instruct models” allows you to use smaller, less expensive models, e.g. “Curie” instead of “Davinci” (a 10x savings itself), to achieve the same functionality.
In total, the savings could be very large. Maybe 10x to 50x for someone savvy.
What’s important is that these cost savings opens up the economics to new business uses. Before, using Davinci can cost hundreds of dollars per user per month. That’s not workable for many business cases. These current savings, as well as expected future development and access to other language models like Eleuther’s, opens up many more products, including several “niches” that aren’t being exploited.
As mentioned above, fine tuning is powerful. This further improves cost efficiency (e.g. more “zero shot” completions) and gets you subtle, or otherwise hard to achieve quality improvements.
(There is a large, like absurdly large, amount of unstated considerations for the thoughts below.)
But basically, these niches and business uses are relevant to posts like this quasi-series for billion dollar EA companies, this 80kh post, this recent startup idea post (where language models don’t appear?!) and the FTX idea #2, and others, which by the way, accepts for-profit and equity investments.
If we don’t see a lot of (public) activity in this area, it’s worth writing about that.