Thank you for taking the time to write this valuable report.
Do you happen to be familiar with systematic attempts to estimate the impact of think tanks on policy, or to identify the main paths to impact? I am aware of various anecdotes and case studies—you discuss the Center for Budget and Policy Priorities, Rob mentions the Center for Global Development and others, and I would add the Institute of Economic Affairs—, but it’s unclear to me what can be inferred from this type of impressionistic evidence.
I think the basic inference we can draw from those anecdotes (and lived experience in DC) is this line from the post:
Whereas the potential for impact is widely accepted, the average level of think tank impact is far more uncertain.
The best-known studies on the question of impact (to my knowledge) are included in Appendix A, but I don’t know that they’d fully satisfy you (they don’t fully satisfy me either).
Questions about average impact (or “systemic estimates”) are tough to answer, in part because all the concepts involved are rather underspecified. “Think tank work” involves many things, and varies widely in quality. Some types of work and some think tanks could be quite effective, whereas others may not be. “Impact” can also mean many things, most of which are by definition hard to observe and measure (I myself work at a think tank and probably >50% of the impact I’ve had is talking policymakers or government staff out of bad ideas, which means 0 observed action in the world).
Even if you had months to collect data, or had the ability to run an RCT, I’m not sure (a) what variables you would include in a regression model or (b) even if you did manage to come up with something, what useful things we could learn from the coefficients/average relationships. Due to the wide variety of think tank work that exists, the relevant reference class for any given think tank-related decision (e.g. if you are a funder assessing a specific proposal or an applicant considering whether to take a job) is going to be rather narrow (much narrower than “think tank projects” or “think tank jobs” writ large).
By the way, I relied on your report and some other literature to write a brief EA Wiki entry on think tanks. If you decide to read it and have any critical feedback, feel free to leave a comment or get in touch.
I weakly suggest that Twitter is also underrated by EAs interested in policy. I’m happy to write something fully if people upvote this.
The sketch of my argument is that lots of top staffers are on Twitter and it’s not hard to get access to them. Many EAs are followed by big academics, pubic figures etc, I’ve had several chats to movers and shakers (Noah Smith, David Shor, Tyler Cowen). I imagine someone who sought to talk to staffers in a particular area could after about 6 months of posting for 3 hours a week.
I agree Twitter is probably underrated in the EA community. However, I would disagree with (or at least strongly caveat) this statement: “I imagine someone who sought to talk to staffers in a particular area could after about 6 months of posting for 3 hours a week.”
One meeting with a staffer is not difficult to get if you’ve been in DC for a while. The question is what you do in that meeting and what you want to get out of it. Twitter is a tool that you can use to promote ideas/research/etc., so you need those pieces before Twitter is useful. Staffers won’t find it very helpful if you meet with them and you can’t go deeper than Twitter-level conversation. Typically, real impact from staffer engagement comes only after the third or fourth meeting, and those follow-ups require trust and deep analysis.
So I think Twitter is useful, but I don’t think it’s a substitute for more “traditional” think tank work, and I would discourage people who don’t have much of a policy background from using Twitter in order to get a meeting with a staffer — I’m not sure if you were suggesting that, but it’s possible to interpret your statement that way, so I just want to flag that before people start rushing to Twitter en masse. :)
I think that’s correct. The same arguments apply if a non-US citizen has a plausible path to becoming a US citizen, but, if not, much of the career capital (DC networks, understanding the US policy process, etc.) does not really transfer across borders. (Though I can also imagine some exceptions, for example a European who wants to work on transatlantic issues from Brussels or their national capital and who could leverage their time in DC for those purposes.)
How would you categorize the schedule flexibility at think tanks? Do you believe it varies by the three categories you’ve mentioned or by seniority levels? My well-being and productivity are much higher with a later start time
I can think of one factor that encourages a rigid schedule. Government work starts at 8am or 9am, sometimes by mandate. Think tanks will have their workday earlier to maximize overlap with bureaucrat schedules
But I can also think of another factor that encourages a flexible schedule. “Ideas industry” work have deliverables that may not be time-sensitive. This means less hard deadlines, less need for “putting out fires”, and less need for everyone to be on the exact same schedule
Overall, I would say there’s a fair bit of flexibility in the think tank world (certainly more than in government). But it does vary across think tanks, and across teams within think tanks.
As far as I know, the variance isn’t strongly correlated with the type of think tank. The nature of the work/team may be more predictive — if you’re in an external-facing communications role, you may have to be available/monitoring your email at certain times. Research and writing roles tend to be more flexible, but you could still end up on a team with a strong 9-to-5 culture.
If this is important to you and you’re considering think tank jobs, I’d encourage you to ask around — former or current employees of specific think tanks should at least be able to tell you about the norms at that think tank, even if they can’t give blanket answers.
Thank you for taking the time to write this valuable report.
Do you happen to be familiar with systematic attempts to estimate the impact of think tanks on policy, or to identify the main paths to impact? I am aware of various anecdotes and case studies—you discuss the Center for Budget and Policy Priorities, Rob mentions the Center for Global Development and others, and I would add the Institute of Economic Affairs—, but it’s unclear to me what can be inferred from this type of impressionistic evidence.
I think the basic inference we can draw from those anecdotes (and lived experience in DC) is this line from the post:
The best-known studies on the question of impact (to my knowledge) are included in Appendix A, but I don’t know that they’d fully satisfy you (they don’t fully satisfy me either).
Questions about average impact (or “systemic estimates”) are tough to answer, in part because all the concepts involved are rather underspecified. “Think tank work” involves many things, and varies widely in quality. Some types of work and some think tanks could be quite effective, whereas others may not be. “Impact” can also mean many things, most of which are by definition hard to observe and measure (I myself work at a think tank and probably >50% of the impact I’ve had is talking policymakers or government staff out of bad ideas, which means 0 observed action in the world).
Even if you had months to collect data, or had the ability to run an RCT, I’m not sure (a) what variables you would include in a regression model or (b) even if you did manage to come up with something, what useful things we could learn from the coefficients/average relationships. Due to the wide variety of think tank work that exists, the relevant reference class for any given think tank-related decision (e.g. if you are a funder assessing a specific proposal or an applicant considering whether to take a job) is going to be rather narrow (much narrower than “think tank projects” or “think tank jobs” writ large).
Thanks for the answer.
By the way, I relied on your report and some other literature to write a brief EA Wiki entry on think tanks. If you decide to read it and have any critical feedback, feel free to leave a comment or get in touch.
Cool, great to see, thanks!
I weakly suggest that Twitter is also underrated by EAs interested in policy. I’m happy to write something fully if people upvote this.
The sketch of my argument is that lots of top staffers are on Twitter and it’s not hard to get access to them. Many EAs are followed by big academics, pubic figures etc, I’ve had several chats to movers and shakers (Noah Smith, David Shor, Tyler Cowen). I imagine someone who sought to talk to staffers in a particular area could after about 6 months of posting for 3 hours a week.
I agree Twitter is probably underrated in the EA community. However, I would disagree with (or at least strongly caveat) this statement: “I imagine someone who sought to talk to staffers in a particular area could after about 6 months of posting for 3 hours a week.”
One meeting with a staffer is not difficult to get if you’ve been in DC for a while. The question is what you do in that meeting and what you want to get out of it. Twitter is a tool that you can use to promote ideas/research/etc., so you need those pieces before Twitter is useful. Staffers won’t find it very helpful if you meet with them and you can’t go deeper than Twitter-level conversation. Typically, real impact from staffer engagement comes only after the third or fourth meeting, and those follow-ups require trust and deep analysis.
So I think Twitter is useful, but I don’t think it’s a substitute for more “traditional” think tank work, and I would discourage people who don’t have much of a policy background from using Twitter in order to get a meeting with a staffer — I’m not sure if you were suggesting that, but it’s possible to interpret your statement that way, so I just want to flag that before people start rushing to Twitter en masse. :)
I assume this makes much more sense for US folks than others. If this isn’t the case correct me.
I think that’s correct. The same arguments apply if a non-US citizen has a plausible path to becoming a US citizen, but, if not, much of the career capital (DC networks, understanding the US policy process, etc.) does not really transfer across borders. (Though I can also imagine some exceptions, for example a European who wants to work on transatlantic issues from Brussels or their national capital and who could leverage their time in DC for those purposes.)
How would you categorize the schedule flexibility at think tanks? Do you believe it varies by the three categories you’ve mentioned or by seniority levels? My well-being and productivity are much higher with a later start time
I can think of one factor that encourages a rigid schedule. Government work starts at 8am or 9am, sometimes by mandate. Think tanks will have their workday earlier to maximize overlap with bureaucrat schedules
But I can also think of another factor that encourages a flexible schedule. “Ideas industry” work have deliverables that may not be time-sensitive. This means less hard deadlines, less need for “putting out fires”, and less need for everyone to be on the exact same schedule
Overall, I would say there’s a fair bit of flexibility in the think tank world (certainly more than in government). But it does vary across think tanks, and across teams within think tanks.
As far as I know, the variance isn’t strongly correlated with the type of think tank. The nature of the work/team may be more predictive — if you’re in an external-facing communications role, you may have to be available/monitoring your email at certain times. Research and writing roles tend to be more flexible, but you could still end up on a team with a strong 9-to-5 culture.
If this is important to you and you’re considering think tank jobs, I’d encourage you to ask around — former or current employees of specific think tanks should at least be able to tell you about the norms at that think tank, even if they can’t give blanket answers.