[Linkpost] Alpaca 7B release | Budget ChatGPT for everybody?

[Felix Wolf 🔸]

Link post

The new YouTube Channel AI-Explained gives a summary about the new Alpaca 7B model. It is an instruction-following language model, you can find the blog post here.

I will now quote the video description, which has a short summary and lists its sources, and give my own two cents.

8 years of cost reduction in 5 weeks: how Stanford’s Alpaca model changes everything, including the economics of OpenAI and GPT 4. The breakthrough, using self-instruct, has big implications for Apple’s secret large language model, Baidu’s ErnieBot, Amazon’s attempts and even governmental efforts, like the newly announced BritGPT.
I will go through how Stanford put the model together, why it costs so little, and demonstrate in action versus Chatgpt and GPT 4. And what are the implications of short-circuiting human annotation like this? With analysis of a tweet by Eliezer Yudkowsky, I delve into the workings of the model and the questions it rises.
Web Demo: https://​​alpaca-ai0.ngrok.io/​​
Alpaca: https://​​crfm.stanford.edu/​​2023/​​03/​​13/​​…
Ark Forecast: https://​​research.ark-invest.com/​​hubfs…
Eliezer Tweet: https://​​twitter.com/​​ESYudkowsky/​​statu… https://​​twitter.com/​​ESYudkowsky/​​statu…
Self-Instruct: https://​​arxiv.org/​​pdf/​​2212.10560.pdf
InstructGPT: https://​​openai.com/​​research/​​instructi…
OpenAI Terms: https://​​openai.com/​​policies/​​terms-of-use
MMLU Test: https://​​arxiv.org/​​pdf/​​2009.03300.pdf
Apple LLM: https://​​www.nytimes.com/​​2023/​​03/​​15/​​te…
GPT 4 API: https://​​openai.com/​​pricing
Llama Models: https://​​arxiv.org/​​pdf/​​2302.13971.pdf
BritGPT: https://​​www.theguardian.com/​​technolog…
Amazon: https://​​www.businessinsider.com/​​amazo…
AlexaTM: https://​​arxiv.org/​​pdf/​​2208.01448.pdf
Baidu Ernie: https://​​www.nytimes.com/​​2023/​​03/​​16/​​wo…
PaLM API: https://​​developers.googleblog.com/​​202…
patreon.com/​AIExplained

Using a second AI to multiply the self-instruct training data and the resulting feedback loop at such an early stage could lead to a drastic improvement in cost efficiency. I have to reevaluate my AI timeline again.

• human annotated training data multiplication → cost reduction

• cost-effective variant, as only fine-tuning of existing pre-trained models is performed

• smaller model is used more efficiently due to better tuning

• available to all and can be specialised for specific applications

What are your thoughts about Alpaca 7B and Stanford’s CRFM publication of the new model? Are the presented terms and conditions enough to permit misuse?

An alpaca sitting in front of a computer, depicted by DALL-E