From your comment, I just learned that Distill.pub is shutting down and this is sad.
The site was beautiful. The attention to detail, and attention to the reader and presentation were amazing.
Their mission seems relevant to AI safety and risk.
Relevant to the main post and the comment above, the issues with Distill.pub seem not to be structural/institutional/academic/social—but operational, related to resources and burnout.
This seems entirely fixable by money, maybe even a reasonable amount compared to other major interventions in the AI/longtermist space?
To explain, consider the explanation on the page:
But it is not sustainable for us to continue running the journal in its current form...We also think that it’s a lot healthier for us and frees up our energy to do new projects that provide value to the community.
...
We set extremely high standards for ourselves: with early articles, volunteer editors would often spend 50 or more hours improving articles that were submitted to Distill and bringing them up to the level of quality we aspired to. This invisible effort was comparable to the work of writing a short article of one’s own. It wasn’t sustainable, and this left us with a constant sense that we were falling short. A related issue is that we had trouble setting well-defined boundaries of what we felt we owed to authors who submitted to us.
...
But the truth is that, with us being quite burnt out, our review process has become much slower and more similar to a typical journal. It’s unclear to us whether the value added by our present review process is worth the time costs we impose on authors.
It seems that 50 hours of a senior editor’s time to work on a draft is pretty wild.
The use of senior staff time like this doesn’t seem close to normal/workable with the resources and incentives on the publication market.
But this can be fixed by hiring junior/middle level ML practitioners and visualization designers/frontend engineers. The salaries are going to be higher than most non-profits but there seems like there is a thick market for these skills.
What doesn’t seem fungible is some of the prestige and vision of the people associated with the project.
As just one example, Mike Bostock is the creator of D3.js and in visualization is a “field leader” by any standard.
Maybe this can interest someone way smarter than me to consider funding/rebuilding/restoring Distill.pub as an AI safety intervention.
I am more bullish about this.
I think for distill to succeed it needs to have at least two full time editors committed to the mission.
Managing people is hard. Managing people, training them and making sure the vision of the project is preserved is insanely hard—a full time job for at least two people.
Plus the part Distill was bottlenecked on is very high skilled labour, which needed a special aesthetic sensitivity and commitment.
50 senior hours per draft sounds insane—but I do believe the Distill staff when they say it is needed.
This wraps back to why new journals are so difficult : you need talented researchers with additional entrepreneurial skills to push it forward. But researchers by and large would much rather just work on their research than manage a journal.
Note: I think what was meant here was “bearish”, not bullish.
I am more bullish about this. I think for distill to succeed it needs to have at least two full time editors committed to the mission.
I think what you’re saying is you’re bearish or have a lower view of this intervention because the editor/founders have a rare combination of vision, aesthetic view and commitment.
You point out this highly skilled management/leadership/labor is not fungible—we can’t just hire 10 AI practitioners and 10 designers to equal the editors who may have left.
You point out this highly skilled management/leadership/labor is not fungible
Yes, exactly.
I think what I am pointing towards is something like “if you are one such highly skilled editor, and your plan is to work on something like this part time delegating work to more junior people, then you are going to find yourself burnt out very soon. Managing a team of junior people / people who do not share your aesthetic sense to do highly skilled labor will be, at least for the first six months or so, much more work than if you do it on your own.”.
I think an editor will be ten times more likely to succeed if:
They have a high skilled co-founder who shares their vision
They have a plan to work on something like this full time, at least for a while
They have a plan for training aligned junior people on skills OR to teach taste to experts
On hindsight I think my comment was too negative, since I would still be excited about someone retrying a distill-like experiment and throwing money at it.
I think you can still publish in conferences, and I have seen that at least AAAI has the topic of safety and trustworthiness between their areas of interest. I would say then that this is not the main issue?
Creating a good journal seems like a good thing to do, but I think it addresses a bit different problem, “how to align researchers incentive with publishing quality results”, not necessarily getting them excited about AIS.
I think it’s more of a comment that one would find the number of academics ‘excited’ about AIS would increase as the number of venues for publication grew.
Create a journal of AI safety, and get prestigious people like Russell publishing on them.
Basically many people in academia are stuck chasing publications. Aligning that incentive seems important.
The problem is that journals are hard work, and require a very specific profile to push it forward.
Here is a post mortem of a previous attempt: https://distill.pub/2021/distill-hiatus/
From your comment, I just learned that Distill.pub is shutting down and this is sad.
The site was beautiful. The attention to detail, and attention to the reader and presentation were amazing.
Their mission seems relevant to AI safety and risk.
Relevant to the main post and the comment above, the issues with Distill.pub seem not to be structural/institutional/academic/social—but operational, related to resources and burnout.
This seems entirely fixable by money, maybe even a reasonable amount compared to other major interventions in the AI/longtermist space?
To explain, consider the explanation on the page:
It seems that 50 hours of a senior editor’s time to work on a draft is pretty wild.
The use of senior staff time like this doesn’t seem close to normal/workable with the resources and incentives on the publication market.
But this can be fixed by hiring junior/middle level ML practitioners and visualization designers/frontend engineers. The salaries are going to be higher than most non-profits but there seems like there is a thick market for these skills.
What doesn’t seem fungible is some of the prestige and vision of the people associated with the project.
As just one example, Mike Bostock is the creator of D3.js and in visualization is a “field leader” by any standard.
Maybe this can interest someone way smarter than me to consider funding/rebuilding/restoring Distill.pub as an AI safety intervention.
I am more bullish about this. I think for distill to succeed it needs to have at least two full time editors committed to the mission.
Managing people is hard. Managing people, training them and making sure the vision of the project is preserved is insanely hard—a full time job for at least two people.
Plus the part Distill was bottlenecked on is very high skilled labour, which needed a special aesthetic sensitivity and commitment.
50 senior hours per draft sounds insane—but I do believe the Distill staff when they say it is needed.
This wraps back to why new journals are so difficult : you need talented researchers with additional entrepreneurial skills to push it forward. But researchers by and large would much rather just work on their research than manage a journal.
Hi, this is another great comment, thank you!
Note: I think what was meant here was “bearish”, not bullish.
I think what you’re saying is you’re bearish or have a lower view of this intervention because the editor/founders have a rare combination of vision, aesthetic view and commitment.
You point out this highly skilled management/leadership/labor is not fungible—we can’t just hire 10 AI practitioners and 10 designers to equal the editors who may have left.
Oops yes 🐻
Yes, exactly.
I think what I am pointing towards is something like “if you are one such highly skilled editor, and your plan is to work on something like this part time delegating work to more junior people, then you are going to find yourself burnt out very soon. Managing a team of junior people / people who do not share your aesthetic sense to do highly skilled labor will be, at least for the first six months or so, much more work than if you do it on your own.”.
I think an editor will be ten times more likely to succeed if:
They have a high skilled co-founder who shares their vision
They have a plan to work on something like this full time, at least for a while
They have a plan for training aligned junior people on skills OR to teach taste to experts
On hindsight I think my comment was too negative, since I would still be excited about someone retrying a distill-like experiment and throwing money at it.
I think you can still publish in conferences, and I have seen that at least AAAI has the topic of safety and trustworthiness between their areas of interest. I would say then that this is not the main issue?
Creating a good journal seems like a good thing to do, but I think it addresses a bit different problem, “how to align researchers incentive with publishing quality results”, not necessarily getting them excited about AIS.
I think it’s more of a comment that one would find the number of academics ‘excited’ about AIS would increase as the number of venues for publication grew.