Thank you for making this interesting post. It’s certainly something that pops up in forum discussions so it’s useful to see in a single post. Obviously without concrete examples it’s hard to delve into the details but I think it’s worth engaging on the discussion on an, ironically, more abstract level.
I think a lot of this comes down to how individual people define ‘impact’, which you do mention in your post. For some, increasing academic knowledge of a niche topic is impact. Other people might perceive citations as impact. For others, publishing a research paper that only gets read by other EA orgs but increases their social standing and therefore likelihood for further funding or work is impact. For some career capital is the intended impact. Some people measure impact only by the frontline change it elicits. This seems like the focus of your post unless I am mistaken, so it sounds like your post boils down to ‘EA-centric research doesn’t cause real-world, measurable change often enough’.
If that is the measure of impact you think is important, I think your post has some merit. That’s not to say the other two are any lesser, or deserve less attention, but I think you are correct that there’s an ‘impact gap’ near the end of the research-to-change pipeline.
I can only speak to AI Governance as that is my niche. As fortune would have it, my career is in AI Governance within organisational change—that is to say my role is to enter private or public sector organisations to a greater or lesser extent and then help create new AI governance and policy on either a project or org-wide basis. So my feedback/thoughts here come with that experience but also that bias. I’ll also take the opportunity to point out that AI governance isn’t just about lobbying politicians but there’s lots of wider organisational work there too, though I understand the oversight was likely word-count related.
Generally I think the problem you describe isn’t so much one within EA as it is one within wider academia. During my PhD I got declined travel funding to present my research to government decision-makers at a government-sponsored event because it wasn’t an ‘academic conference’ and therefore failed their ‘impact’ criteria. I was accepted by the same fund the year previous to that to present a (hardly ground-breaking) poster at a 35-person conference. I was very upset at the time because I had to miss that meeting and that opportunity passed me by, and I was frustrated that they gave me money to attend a conference that changed nothing and didn’t give me the money I needed to make a big impact the year later. It was only later that I realised they just wanted different outcomes than me.
The problem there was that the university’s definition of ‘impact’ differed from mine, so by their metric presenting a poster at an academic conference to 35 people was more impactful to their criteria than my meeting with government officials to show my research. It’s a handy example of the fact that impact maps to goals.
So I think what it boils down to is how much this concept of goal-related impact bleeds into EA.
There is also a difference between research that you think should change the behaviour of decision makers, and what will actually influence them. While it might be clear to you that your research on some obscure form of decision theory has implications for the actions that key decision makers should take, if there is a negligible chance of them seeing this research or taking this on board then this research has very little value.
This point features partly in a post I am currently writing for Draft Amnesty Week, but essentially I think you’re correct that in my work in more ‘frontline’ AI governance I’ve found that anecdotally roughly 0% of decision-makers read academic research. Or know where it is published. Or how to access it. That’s a real problem when it comes to using academic research as an influence lever. That’s not to say the research is pointless, it’s just that there’s extra steps between research and impact that are woefully neglected. If end-user change is something that is important to you as a researcher, it would be understandably frustrating for this hurdle to reduce that impact.
This isn’t an EA issue but a field issue. There’s plenty of fantastic non-EA AI governance research which lands like a pin-drop, far from the ears of decision-makers, because it wasn’t put in the right hands at the right time. The problem is many decision-makers where it counts (particularly in industry) get their knowledge from staff, consultants, dedicated third-party summary organisations, or field-relevant newsletters/conferences. Not directly from academia.
One caveat here is that some fields, like Law, have a much greater overlap of ‘people reading/publishing’ and ‘decision-makers’. This is partly because publishing and work in many legal sectors are designed for impact in this way. So the above isn’t always ironclad, but it largely tracks for general decision-making and AI governance. I find the best EA orgs at generating real-world impact are the orgs in the legal/policy because of the larger than normal amount of legal and policy researchers there coupled with the fact they are more likely to measure their success by policy change.
A further complicating factor that I think contributes to the way you feel is that unfortunately some AI Governance research is undertaken and published by people who don’t always have lots of experience in large organisations. Perhaps they spent their entire career in academia, or have worked only in start-ups, or via different paths, but that’s where you see different ‘paths to impact’ which don’t translate well to larger-scale impact like the type you describe in your post. Again the reason here is that each of these spheres have their own definition of what constitutes ‘impact’ and it doesn’t always translate well.
As a partial result of this I’ve seen some really good AI governance ideas pitched really badly, and to the wrong gatekeeper. Knowing how to pitch research to an organisation is a skillset curated by experience, and the modern academic pathway doesn’t give people the opportunity to gain much of that experience. Personally, I just learned it by failing really hard a lot of times early in my career. For what it’s worth, I’d 100% recommend that strategy if there’s any early careers folks reading this.
I will disagree with you on one point here:
Soon after the initial ChatGPT launch probably wasn’t the right time for governments to regulate AI, but given the amount of funding that has gone into AI governance research it seems like a bad sign that there weren’t many (if any) viable AI governance proposals that were ready for policymakers to take off-the-shelf and implement.
I’ll be pedantic and point out that governments already do regulate AI, just to different extents than some would like, and that off-the-shelf governance proposals don’t really exist because of how law and policy works. So not sure this is a good metric to use for your wider point. Law and policy of AI is literally my career and I couldn’t create an off-the-shelf policy that was workable just because of how many factors are required to be considered.
It seems like EA think tanks are becoming more savvy and gradually moving in the direction of action-guiding research and focusing on communicating to decision makers, especially in AI governance.
Taking a leaf from your vagueness book, I’ll say that in my experience some of the EA or EA-adjacent AI governance orgs are really good at engaging external stakeholders, and some are less good. I say this as an outsider because I don’t work for and nor have I ever worked for an EA org, but I do follow their research. So take this with appropriate pinches of salt.
I think part of the disparity is that some orgs recruit people with experience in how internal government decision-making works—ie people who have worked in the public sector or have legal or policy backgrounds. Some others don’t. I think that translates largely to their goal. It’s not random that some are good at it and some not so much, it’s just some value that and some don’t—therefore effort is invested in change impact or it isn’t.
If an EA research org defines ‘impact’ as increasing research standing within EA, or amount of publications per year, or amount of conferences attended, then why would they make effort to create organisational change? Likewise, I don’t publish that much because it’s just not directly related to how effective my measurements of my own impact are. Neither is better, it just relates to how goals are measured.
If, as I think your post details, your criticism is that EA research doesn’t create more frontline change often enough, then I think that there are some relatively simple fixes.
EA research has something of a neglect of involving external stakeholders which I think links back to the issues you explore in your post. Stakeholder engagement can be quite easily and well integrated into AI Governance research as this example shows, and that’s quite an easy (and often non-costly) methodology to pick up that can result in frontline impact.
Stakeholder-involved research must always be done carefully, so I don’t blame EA funding orgs or think tanks for being very careful in approaching it, but they need to cultivate the right talent for this kind of work and use it because it’s very important.
I think a solution would be to offer grants or groups for this specific kind of work. Even workshops for people might work. I’d volunteer some of my experience for that, if asked to do so. Just something to give researchers who want the kind of impact you describe, but don’t know how to do it, a head-start.
I think impact-centric conferences would also be a good idea. Theoretical researchers do fantastic work, and many of us more involved in the change side of things couldn’t do our jobs without them, so creating a space where those groups can exchange ideas would be awesome. EAGs are good for that, I find. I often get a lot of 1-1s booked, and I get a lot from them too.
Thank you for making this interesting post. It’s certainly something that pops up in forum discussions so it’s useful to see in a single post. Obviously without concrete examples it’s hard to delve into the details but I think it’s worth engaging on the discussion on an, ironically, more abstract level.
I think a lot of this comes down to how individual people define ‘impact’, which you do mention in your post. For some, increasing academic knowledge of a niche topic is impact. Other people might perceive citations as impact. For others, publishing a research paper that only gets read by other EA orgs but increases their social standing and therefore likelihood for further funding or work is impact. For some career capital is the intended impact. Some people measure impact only by the frontline change it elicits. This seems like the focus of your post unless I am mistaken, so it sounds like your post boils down to ‘EA-centric research doesn’t cause real-world, measurable change often enough’.
If that is the measure of impact you think is important, I think your post has some merit. That’s not to say the other two are any lesser, or deserve less attention, but I think you are correct that there’s an ‘impact gap’ near the end of the research-to-change pipeline.
I can only speak to AI Governance as that is my niche. As fortune would have it, my career is in AI Governance within organisational change—that is to say my role is to enter private or public sector organisations to a greater or lesser extent and then help create new AI governance and policy on either a project or org-wide basis. So my feedback/thoughts here come with that experience but also that bias. I’ll also take the opportunity to point out that AI governance isn’t just about lobbying politicians but there’s lots of wider organisational work there too, though I understand the oversight was likely word-count related.
Generally I think the problem you describe isn’t so much one within EA as it is one within wider academia. During my PhD I got declined travel funding to present my research to government decision-makers at a government-sponsored event because it wasn’t an ‘academic conference’ and therefore failed their ‘impact’ criteria. I was accepted by the same fund the year previous to that to present a (hardly ground-breaking) poster at a 35-person conference. I was very upset at the time because I had to miss that meeting and that opportunity passed me by, and I was frustrated that they gave me money to attend a conference that changed nothing and didn’t give me the money I needed to make a big impact the year later. It was only later that I realised they just wanted different outcomes than me.
The problem there was that the university’s definition of ‘impact’ differed from mine, so by their metric presenting a poster at an academic conference to 35 people was more impactful to their criteria than my meeting with government officials to show my research. It’s a handy example of the fact that impact maps to goals.
So I think what it boils down to is how much this concept of goal-related impact bleeds into EA.
This point features partly in a post I am currently writing for Draft Amnesty Week, but essentially I think you’re correct that in my work in more ‘frontline’ AI governance I’ve found that anecdotally roughly 0% of decision-makers read academic research. Or know where it is published. Or how to access it. That’s a real problem when it comes to using academic research as an influence lever. That’s not to say the research is pointless, it’s just that there’s extra steps between research and impact that are woefully neglected. If end-user change is something that is important to you as a researcher, it would be understandably frustrating for this hurdle to reduce that impact.
This isn’t an EA issue but a field issue. There’s plenty of fantastic non-EA AI governance research which lands like a pin-drop, far from the ears of decision-makers, because it wasn’t put in the right hands at the right time. The problem is many decision-makers where it counts (particularly in industry) get their knowledge from staff, consultants, dedicated third-party summary organisations, or field-relevant newsletters/conferences. Not directly from academia.
One caveat here is that some fields, like Law, have a much greater overlap of ‘people reading/publishing’ and ‘decision-makers’. This is partly because publishing and work in many legal sectors are designed for impact in this way. So the above isn’t always ironclad, but it largely tracks for general decision-making and AI governance. I find the best EA orgs at generating real-world impact are the orgs in the legal/policy because of the larger than normal amount of legal and policy researchers there coupled with the fact they are more likely to measure their success by policy change.
A further complicating factor that I think contributes to the way you feel is that unfortunately some AI Governance research is undertaken and published by people who don’t always have lots of experience in large organisations. Perhaps they spent their entire career in academia, or have worked only in start-ups, or via different paths, but that’s where you see different ‘paths to impact’ which don’t translate well to larger-scale impact like the type you describe in your post. Again the reason here is that each of these spheres have their own definition of what constitutes ‘impact’ and it doesn’t always translate well.
As a partial result of this I’ve seen some really good AI governance ideas pitched really badly, and to the wrong gatekeeper. Knowing how to pitch research to an organisation is a skillset curated by experience, and the modern academic pathway doesn’t give people the opportunity to gain much of that experience. Personally, I just learned it by failing really hard a lot of times early in my career. For what it’s worth, I’d 100% recommend that strategy if there’s any early careers folks reading this.
I will disagree with you on one point here:
I’ll be pedantic and point out that governments already do regulate AI, just to different extents than some would like, and that off-the-shelf governance proposals don’t really exist because of how law and policy works. So not sure this is a good metric to use for your wider point. Law and policy of AI is literally my career and I couldn’t create an off-the-shelf policy that was workable just because of how many factors are required to be considered.
Taking a leaf from your vagueness book, I’ll say that in my experience some of the EA or EA-adjacent AI governance orgs are really good at engaging external stakeholders, and some are less good. I say this as an outsider because I don’t work for and nor have I ever worked for an EA org, but I do follow their research. So take this with appropriate pinches of salt.
I think part of the disparity is that some orgs recruit people with experience in how internal government decision-making works—ie people who have worked in the public sector or have legal or policy backgrounds. Some others don’t. I think that translates largely to their goal. It’s not random that some are good at it and some not so much, it’s just some value that and some don’t—therefore effort is invested in change impact or it isn’t.
If an EA research org defines ‘impact’ as increasing research standing within EA, or amount of publications per year, or amount of conferences attended, then why would they make effort to create organisational change? Likewise, I don’t publish that much because it’s just not directly related to how effective my measurements of my own impact are. Neither is better, it just relates to how goals are measured.
If, as I think your post details, your criticism is that EA research doesn’t create more frontline change often enough, then I think that there are some relatively simple fixes.
EA research has something of a neglect of involving external stakeholders which I think links back to the issues you explore in your post. Stakeholder engagement can be quite easily and well integrated into AI Governance research as this example shows, and that’s quite an easy (and often non-costly) methodology to pick up that can result in frontline impact.
Stakeholder-involved research must always be done carefully, so I don’t blame EA funding orgs or think tanks for being very careful in approaching it, but they need to cultivate the right talent for this kind of work and use it because it’s very important.
I think a solution would be to offer grants or groups for this specific kind of work. Even workshops for people might work. I’d volunteer some of my experience for that, if asked to do so. Just something to give researchers who want the kind of impact you describe, but don’t know how to do it, a head-start.
I think impact-centric conferences would also be a good idea. Theoretical researchers do fantastic work, and many of us more involved in the change side of things couldn’t do our jobs without them, so creating a space where those groups can exchange ideas would be awesome. EAGs are good for that, I find. I often get a lot of 1-1s booked, and I get a lot from them too.