My commendations on another detailed and thoughtful review. A few reactions (my views, not GCRI’s):
The only case I can think of where scientists are relatively happy about punitive safety regulations, nuclear power, is one where many of those initially concerned were scientists themselves.
Actually, a lot of scientists & engineers in nuclear power are not happy about the strict regulations on nuclear power. Note, I’ve been exposed to this because my father worked as an engineer in the nuclear power industry, and I’ve had other interactions with it through my career in climate change & risk analysis. Basically, widespread overestimation of the medical harms from radiation has caused nuclear power to be held to a much higher standard than other sources, especially fossil fuels.
A better example would be recombinant DNA—see Katja Grace’s very nice study of it. The key point is the importance of the scientists/engineers buying into the regulation. This is consistent with other work I’m familiar with on risk regulation etc., and with work I’ve published, e.g. this and this.
My impression is that policy on most subjects, especially those that are more technical than emotional is generally made by the government and civil servants in consultation with, and being lobbied by, outside experts and interests
More precisely, the distinction is between issues that matter to voters in elections (plus campaign donors etc.) and issues that fly more under the radar. For now at least, AI still flies under the radar, creating more opportunity for expert insiders (like us) to have significant impact, as do most other global catastrophic risks. The big exception is climate change. (I’m speaking in terms of US politics/policy. I don’t know about other countries.)
Without expert (e.g. top ML researchers in academia and industry) consensus, no useful policy will be enacted. Pushing directly for policy seems if anything likely to hinder expert consensus. Attempts to directly influence the government to regulate AI research seem very adversarial
This depends on the policy. A lot of policy is not about restricting AI, but instead about coordination, harmonizing standards, ensuring quality applications, setting directions for the field, etc. That said, it is definitely important to factor the reactions of AI communities into policy outreach efforts. (As I have been pushing for in e.g. the work referenced above.)
With regard to published research, in general I think it is better for it to be open access, rather than behind journal paywalls, to maximise impact. Reducing this impact by a significant amount in order for the researcher to gain a small amount of prestige does not seem like an efficient way of compensating researchers to me.
It varies from case to case. For a lot of research, the primary audience is other researchers/experts in the field. They generally have access to paywall journals and place significant weight on journal quality/prestige. Also open access journals typically charge author publication fees, generally in the range of hundreds to thousands of dollars. That raises the question of whether it’s a good use of funds. I’m not at all against open access (I like open access!); I only mean to note that there are other factors that may make it not always the best option.
it seems a bit of a waste to have to charge for books
Again it depends. Mass-market books typically get a lot more attention when they’re from a major publisher. These books are more than just books—they are platforms for a lot of attention and discussion. If e.g. Bostrom had self-published Superintelligence, it probably wouldn’t have gotten nearly the same attention. Also good publishers have editors who improve the books, and that costs money. I see a stronger case for self-publishing technical reports that have a narrower audience, especially if the author and/or their organization have the resources to do editing, page layout, promotion, etc.
More prosaically, organisations should make sure to upload the research they have published to their website
Yes, definitely! I for one frequent the websites of peer organizations, and often wish they were more up to date.
in general I do not give full credence to charities saying they need more funding because they want much more than a 18 months or so of runway in the bank
I might worry that this could bias the field away from more senior people who may have larger financial responsibilities (family, mortgage, etc.) and better alternative opportunities for income. There’s no guarantee that future donations will be made, which creates a risk for the worker even if they’re doing excellent work.
the conventional peer review system seems to be extremely bad at dealing with this issue
Peer review should filter out bad/unoriginal research, sort it by topic (journal X publishes on topic X etc.), and improve papers via revision requests. Good journals do this. Not all journals are good. Overall I for one find significantly better quality work in peer reviewed journals (especially good journals) than outside of peer review.
The Bay Area
I can’t speak to concerns about the Bay Area, but I can say that GCRI has found a lot of value in connecting with people outside the usual geographic hubs, and that this is something ripe for further investment in (whether via GCRI or other entities). See e.g. this on GCRI’s 2019 advising/collaboration program, which we’re continuing in 2020.
My commendations on another detailed and thoughtful review. A few reactions (my views, not GCRI’s):
Actually, a lot of scientists & engineers in nuclear power are not happy about the strict regulations on nuclear power. Note, I’ve been exposed to this because my father worked as an engineer in the nuclear power industry, and I’ve had other interactions with it through my career in climate change & risk analysis. Basically, widespread overestimation of the medical harms from radiation has caused nuclear power to be held to a much higher standard than other sources, especially fossil fuels.
A better example would be recombinant DNA—see Katja Grace’s very nice study of it. The key point is the importance of the scientists/engineers buying into the regulation. This is consistent with other work I’m familiar with on risk regulation etc., and with work I’ve published, e.g. this and this.
More precisely, the distinction is between issues that matter to voters in elections (plus campaign donors etc.) and issues that fly more under the radar. For now at least, AI still flies under the radar, creating more opportunity for expert insiders (like us) to have significant impact, as do most other global catastrophic risks. The big exception is climate change. (I’m speaking in terms of US politics/policy. I don’t know about other countries.)
This depends on the policy. A lot of policy is not about restricting AI, but instead about coordination, harmonizing standards, ensuring quality applications, setting directions for the field, etc. That said, it is definitely important to factor the reactions of AI communities into policy outreach efforts. (As I have been pushing for in e.g. the work referenced above.)
It varies from case to case. For a lot of research, the primary audience is other researchers/experts in the field. They generally have access to paywall journals and place significant weight on journal quality/prestige. Also open access journals typically charge author publication fees, generally in the range of hundreds to thousands of dollars. That raises the question of whether it’s a good use of funds. I’m not at all against open access (I like open access!); I only mean to note that there are other factors that may make it not always the best option.
Again it depends. Mass-market books typically get a lot more attention when they’re from a major publisher. These books are more than just books—they are platforms for a lot of attention and discussion. If e.g. Bostrom had self-published Superintelligence, it probably wouldn’t have gotten nearly the same attention. Also good publishers have editors who improve the books, and that costs money. I see a stronger case for self-publishing technical reports that have a narrower audience, especially if the author and/or their organization have the resources to do editing, page layout, promotion, etc.
Yes, definitely! I for one frequent the websites of peer organizations, and often wish they were more up to date.
I might worry that this could bias the field away from more senior people who may have larger financial responsibilities (family, mortgage, etc.) and better alternative opportunities for income. There’s no guarantee that future donations will be made, which creates a risk for the worker even if they’re doing excellent work.
Peer review should filter out bad/unoriginal research, sort it by topic (journal X publishes on topic X etc.), and improve papers via revision requests. Good journals do this. Not all journals are good. Overall I for one find significantly better quality work in peer reviewed journals (especially good journals) than outside of peer review.
I can’t speak to concerns about the Bay Area, but I can say that GCRI has found a lot of value in connecting with people outside the usual geographic hubs, and that this is something ripe for further investment in (whether via GCRI or other entities). See e.g. this on GCRI’s 2019 advising/collaboration program, which we’re continuing in 2020.