Also happy to help on a more local level: eazurich.org/join
If you’re not already in contact with EA Zürich, just sent us a mail and we will get back to you: info@eazurich.org .
lennart
Karma: 350
Transformative AI and Compute [Summary]
What is Compute? - Transformative AI and Compute [1/4]
I have been wondering the same. However, given that OpenAI’s “AI and Compute” inclusion criteria are also a bit vague, I’m having a hard time which of our data points would fulfill their criteria.
In general, I would describe our dataset matching the same criteria because:
“relatively well known” equals our “lots of citations”.
“used a lot of compute for their time” equals our dataset if we exclude outliers from efficient ML models.
There’s a recent trend in efficient ML models that achieve similar performance by using less compute for inference and training (those models are then used for e.g., deployment on embedded systems or smartphones).
“gave enough information to estimate the compute”: We also rely on estimates from us or the community based on the information available in the paper. For a source of the estimate see the note on the cell in our dataset.
We’re working on gathering more compute data by directly asking researchers (next target
n=100) .
I’d be interested in discussing more precise inclusion criteria. As I say in the post:
Also, it is unclear on which models we should base this trend. The piece AI and Compute also quickly discusses this in the appendix. Given the recent trend of efficient ML models due to emerging fields such as Machine Learning on the Edge, I think it might be worthwhile discussing how to integrate and interpret such models in analyses like this — ignoring them cannot be the answer.
Thanks, Michael.
nis counting the number of ML systems in the analysis at the point of writing. (We have added more systems in the meantime). An example for such a system is GPT-3, AlphaFold, etc. - basically a row in our dataset.Right, good point. I’ll add the number of systems for the given time period.
That’s hard to answer. I don’t think OpenAI misinterpreted anything. For the moment, I think it’s probably a mixture of:
the inclusion criteria for the systems on which we base this trend
actual slower doubling times for reasons which we should figure out Nonetheless, as outlined in Part 1 - Section 2.3, I did not interpret those trends yet but I’m interested in a discussion and trying to write up my thoughts on this in the future.
Thanks, I’ve edited it.
Forecasting Compute—Transformative AI and Compute [2/4]
Thanks, Sammy. Indeed this is related and very interesting!
For “Semiconductor industry amortize their R&D cost due to slower improvements” the decreased price comes from the longer innovation cycles, so the R&D investments spread out over a longer time period. Competition should then drive the price down.
While in contrast “Sale price amortization when improvements are slower” describes the idea that the sale price within the company will be amortized over a longer time period given that obsolescence will be achieved later.
Those ideas stem from Cotra’s appendices: “Room for improvements to silicon chips in the medium term”.
The described doubling time of 6.2 months is the result when the outliers are excluded. If one includes all our models, the doubling time was around ≈7 months. However, the number of efficient ML models was only one or two.
Compute Governance and Conclusions—Transformative AI and Compute [3/4]
Thanks, Nuño.
Compute Research Questions and Metrics—Transformative AI and Compute [4/4]
I’m still holding the same view that (a) we will probably see a switch in funding distribution and (b) if this does not happen those groups won’t be able to compete with SOTA models.
we will and should see a switch in funding distribution at publicly funded AI research groups
I would change my mind if we find more evidence towards algorithmic innovation being a stronger or the significant driver.
Some recent updates in regards to providing more funding for infrastructure include The National AI Research Cloud which is currently being investigated by the US government or Compute Canada.
The Cooperative AI Foundation works on an agenda relevant to s-risks.
Thanks for writing this, Akash! :) I’ve been following a similar paradigm for quite a while, some things I did:
Meetings only in the afternoon—the less important/demanding the further into the evening
Focused work time is scarce and, therefore my morning time is a resource to protect
Taking my location into consideration. As you outline, when I’m already somewhere I should make use of it.
Stopping to work when I feel like I don’t make progress. Trading this time against my future “leisure time” where I’ll hopefully be more energetic.
Glad to hear you’re now more excited. :)
Regarding:
I know there were a few meet-ups for other groups such as religious EA’s, it would be great if there had been something like ‘lonely and new meetup’?
I remember that at EAG London 2021 there was an event for newcomers and you were even matched with a mentor. Maybe we should copy this or make it a group effort for all the conferences.
You can implement them easily in Ghost by using the HTML embed. You find this when you click on the bottom right corner the share button and click “embed”.
Great post Trevor! I share your message. :)
Especially:
The retreat features lots of people who are already “on board” with EA. At least a few should be at least moderately charismatic people for whom EA is a major consideration in how they make decisions.
That’s also my experience. While “some buy-in” already helps a lot, people with more experience provide in my experience even “more value”—sharing their EA story, their connected struggles, and maybe how they managed to work on EA-adjacent stuff.
You bring something along these lines up later by saying “The retreat includes “professional EAs.””.Therefore, I’d also encourage the more “senior people” to join retreats from time to time. You can provide an enormous value. And to all the organizers, reach out to them!
Is there any chance to get a hold of the material which you used for this workshop?