Seems like there’s room in the ecosystem for a weekly update on AI that does a lot of contextualization / here’s where we are on ongoing benchmarks. I’m familiar with:
a weekly newsletter on AI media (that has a section on important developments that I like)
Jack Clark’s substack which I haven’t read much of but seems more about going in depth on new developments (though does have a “Why this matters” section. Also I love this post in particular for the way it talks about humility and confusion.
Doing Westminster Better on UK politics and AI / EA, which seems really good but again I think goes in depth on new stuff
I could imagine spending time on aggregation of prediction markets for specific topics, which Metaculus and Manifold are doing better and better over time.
I’m interested in something that says “we’re moving faster / less fast than we thought we would 6 months ago” or “this event is surprising because” and kind of gives a “you are here” pointer on the map. This Planned Obsolescence post called “Language models surprised us” I think is the closest I’ve seen.
Seems hard, also maybe not worth it enough to do, also maybe it’s happening and I’m not familiar with it, would love to hear, but it’s what I’d personally find most useful and I suspect I’m not alone.
Wout Schellart, Jose Hernandez-Orallo, and Lexin Zhou have started an AI evaluation digest, which includes relevant benchmark papers etc. It’s pretty brief, but they’re looking for more contributers, so if you want to join in and help make it more comprehensive/contextualised, you should reach out! https://groups.google.com/g/ai-eval/c/YBLo0fTLvUk
Seems like there’s room in the ecosystem for a weekly update on AI that does a lot of contextualization / here’s where we are on ongoing benchmarks. I’m familiar with:
a weekly newsletter on AI media (that has a section on important developments that I like)
Jack Clark’s substack which I haven’t read much of but seems more about going in depth on new developments (though does have a “Why this matters” section. Also I love this post in particular for the way it talks about humility and confusion.
Doing Westminster Better on UK politics and AI / EA, which seems really good but again I think goes in depth on new stuff
I could imagine spending time on aggregation of prediction markets for specific topics, which Metaculus and Manifold are doing better and better over time.
I’m interested in something that says “we’re moving faster / less fast than we thought we would 6 months ago” or “this event is surprising because” and kind of gives a “you are here” pointer on the map. This Planned Obsolescence post called “Language models surprised us” I think is the closest I’ve seen.
Seems hard, also maybe not worth it enough to do, also maybe it’s happening and I’m not familiar with it, would love to hear, but it’s what I’d personally find most useful and I suspect I’m not alone.
I think I agree, but also want to flag this list in case you (or others) haven’t seen it: List of AI safety newsletters and other resources
Another newsletter(?) that I quite like is Zvi’s
Wout Schellart, Jose Hernandez-Orallo, and Lexin Zhou have started an AI evaluation digest, which includes relevant benchmark papers etc. It’s pretty brief, but they’re looking for more contributers, so if you want to join in and help make it more comprehensive/contextualised, you should reach out!
https://groups.google.com/g/ai-eval/c/YBLo0fTLvUk
Less directly relevant, but Harry Law also has a new newsletter in the Jack Clark style, but more focused on governance/history/lessons for AI:
https://learningfromexamples.substack.com/p/the-week-in-examples-3-2-september
Thanks!