General comment: Huge fan of the newsletter, and think it’s awesome you’re doing this sort of review. I should also caveat that I’m not an AIS researcher, so not exactly target audience.
My first guess is that there’s significant value in someone maintaining an open, exhaustive database of AIS research. My main uncertainty is whether you are the best positioned to do this as things ramp up. It is plausible to me that an org with a safety team (e.g. DeepMind/OpenAI) is already doing this in-house, or planning to do so. It’s less clear that they would be willing to maintain a public resource. I’d want to verify this, and make sure that you’re coordinating with them to avoid any unnecessary duplication. More broadly, these labs might have some good systems in place for maintaining databases of new research in areas with a much higher volume than AIS, so could potentially share some best-practices.
My first guess is that there’s significant value in someone maintaining an open, exhaustive database of AIS research.
Yeah, I agree. But there’s also significant value in doing more AIS research, and I suspect that on the current margin for a full-time researcher (such as myself) it’s better to do more AIS research compared to writing summaries of everything.
Note that I do intend to keep adding all of the links to the database, it’s the summaries that won’t keep up.
It is plausible to me that an org with a safety team (e.g. DeepMind/OpenAI) is already doing this in-house, or planning to do so.
I’m 95% confident that no one is already doing this, and if they were seriously planning to do so I’d expect they would check in with me first. (I do know multiple people at all of these orgs.)
More broadly, these labs might have some good systems in place for maintaining databases of new research in areas with a much higher volume than AIS, so could potentially share some best-practices.
You know, that would make sense as a thing to exist, but I suspect it does not. Regardless that’s a good idea, I should make sure to check.
My intuition is that this would be a good time to formalize the structure of the newsletter somewhat, especially given that there are multiple contributors & you are starting to function more as an editor.
Could do by incorporating a small publishing organization that produces the newsletter, or by housing the newsletter in an existing organization. The former would be more work, but also seems better (less worry that we’re getting DeepMind (or whoever’s) spin if it’s coming out of an independent org).
Plausibly it’s fine to keep it as an informal research product, but I’d guess that “AI alignment newsletter editor” could basically be (or soon become) a full-time job.
incorporating a small publishing organization that produces the newsletter...
Could get a grant to fund this, or could do a pay-per-subscription model (a la Ben Thompson’s Stratechery, which I believe has > $1 million in annual revenue entirely from $10/month subscribers).
My intuition is that this would be a good time to formalize the structure of the newsletter somewhat, especially given that there are multiple contributors & you are starting to function more as an editor.
Certainly more systems are being put into place, which is kind of like “formalizing the structure”. Creating an organization feels like a high fixed cost for not much benefit—what do you think the main benefits would be? (Maybe this is combined with paying content writers and editors, in which case an organization might make more sense?)
Plausibly it’s fine to keep it as an informal research product, but I’d guess that “AI alignment newsletter editor” could basically be (or soon become) a full-time job.
If I were to make this my full-time job, the newsletter would approximately double in length (assuming I found enough content to cover), and I’d expect that people wouldn’t read most of it. (People already don’t read all of it, I’m pretty sure.) What do you think would be the value of more time put into the newsletter?
(Maybe this is combined with paying content writers and editors, in which case an organization might make more sense?)
Right, that’s what I was gesturing towards.
What do you think would be the value of more time put into the newsletter?
This was in response to “the growing amount of AI safety research.”
Presumably as there is more research, it takes more time to read & assess the forthcoming literature to figure out what’s important / worth including in the newsletter.
This was in response to “the growing amount of AI safety research.”
Yeah, I think I phrased that question poorly. The question is both “should all of it be summarized” and “if yes, how can that be done”.
Presumably as there is more research, it takes more time to read & assess the forthcoming literature to figure out what’s important / worth including in the newsletter.
I feel relatively capable of that—I think I can figure out for any given reading whether I want to include it in ~5 minutes or so with relatively high accuracy. It’s actually reading and summarizing it that takes time.
Comment thread for the question: How should I deal with the growing amount of AI safety research?
General comment: Huge fan of the newsletter, and think it’s awesome you’re doing this sort of review. I should also caveat that I’m not an AIS researcher, so not exactly target audience.
My first guess is that there’s significant value in someone maintaining an open, exhaustive database of AIS research. My main uncertainty is whether you are the best positioned to do this as things ramp up. It is plausible to me that an org with a safety team (e.g. DeepMind/OpenAI) is already doing this in-house, or planning to do so. It’s less clear that they would be willing to maintain a public resource. I’d want to verify this, and make sure that you’re coordinating with them to avoid any unnecessary duplication. More broadly, these labs might have some good systems in place for maintaining databases of new research in areas with a much higher volume than AIS, so could potentially share some best-practices.
Yeah, I agree. But there’s also significant value in doing more AIS research, and I suspect that on the current margin for a full-time researcher (such as myself) it’s better to do more AIS research compared to writing summaries of everything.
Note that I do intend to keep adding all of the links to the database, it’s the summaries that won’t keep up.
I’m 95% confident that no one is already doing this, and if they were seriously planning to do so I’d expect they would check in with me first. (I do know multiple people at all of these orgs.)
You know, that would make sense as a thing to exist, but I suspect it does not. Regardless that’s a good idea, I should make sure to check.
My intuition is that this would be a good time to formalize the structure of the newsletter somewhat, especially given that there are multiple contributors & you are starting to function more as an editor.
Could do by incorporating a small publishing organization that produces the newsletter, or by housing the newsletter in an existing organization. The former would be more work, but also seems better (less worry that we’re getting DeepMind (or whoever’s) spin if it’s coming out of an independent org).
Plausibly it’s fine to keep it as an informal research product, but I’d guess that “AI alignment newsletter editor” could basically be (or soon become) a full-time job.
Could get a grant to fund this, or could do a pay-per-subscription model (a la Ben Thompson’s Stratechery, which I believe has > $1 million in annual revenue entirely from $10/month subscribers).
Certainly more systems are being put into place, which is kind of like “formalizing the structure”. Creating an organization feels like a high fixed cost for not much benefit—what do you think the main benefits would be? (Maybe this is combined with paying content writers and editors, in which case an organization might make more sense?)
If I were to make this my full-time job, the newsletter would approximately double in length (assuming I found enough content to cover), and I’d expect that people wouldn’t read most of it. (People already don’t read all of it, I’m pretty sure.) What do you think would be the value of more time put into the newsletter?
Right, that’s what I was gesturing towards.
This was in response to “the growing amount of AI safety research.”
Presumably as there is more research, it takes more time to read & assess the forthcoming literature to figure out what’s important / worth including in the newsletter.
Yeah, I think I phrased that question poorly. The question is both “should all of it be summarized” and “if yes, how can that be done”.
I feel relatively capable of that—I think I can figure out for any given reading whether I want to include it in ~5 minutes or so with relatively high accuracy. It’s actually reading and summarizing it that takes time.