Good call, I just did some more investigating and I would agree that EleutherAI’s Pythia is Open Source, I’ll update the post with a new image and wording shortly.
As a side note, the extra research I did as a result of your comment led me to find another Open Source model from the Allen Institute for AI (OLMo), and the Model Openness Framework, which I will also be adding.
Thanks!
Jacob-Haimes
Karma: 49
Of course, glad I could help!
Hi CAISID, thanks for letting me know that you think the podcast is interesting!
(For context to others, the original comment is referring to an aside that I give at 27:04 about “foundation models”)
I definitely did have a difficult time finding concrete answers regarding training costs of current cutting edge foundation models. The only numbers I could find from the big companies that were primary sources, i.e. from people within the company/press releases, were very loose numbers that included salaries, which basically makes them meaningless (at least, for the information I want out of them).
There has been some work done on estimating training costs (e.g., Epoch or the AI Index Report), but it seemed that I would need to spend a significant amount of time collecting data and even doing some forecasting to actually get approximations for current state of the art.
Would love to hear your thoughts on this either here or whatever messaging format you prefer.
Hey Chris, thanks for the comment!
I am slightly confused about it though, as I am able to control the playback speed of the player on my website (at intervals of .25 from .25x to 2x). Additionally, I know that on Spotify the episode can be played back at up to 3.5x.
If anyone is still out there thinking: “I really can’t stand people’s voices unless they are sped up by more than 3.5x,” you can also download the mp3 file directly from my podcast website.
Let me know if I am missing something here, as I want to make sure my content is maximally accessible to everyone :)
The post has now been updated appropriately, please let me know if you don’t think the modifications were sufficient so that I can fix them appropriately.
Now to make the changes on the other platforms!