Help, Please: Integrating EA Ideas into Large Research Organization

Lauren Zitney30 Oct 2021 1:23 UTC
37 points
11 comments4 min readEA link

Greetings!

I’m an Economist (read: low-level analyst) at RTI International, a non-profit research institute whose mission and vision (pasted below) seem very EA-aligned. In fact, I joined RTI because of this language. However, I’ve been here now for over two years, and am growing increasingly convinced that our actions don’t align with our words. We may say we’re addressing the world’s most critical problems, but much of our research and recommendations seem to just sit on dusty shelves or in untouched directories of governmental agencies once we pass our research off to our clients. Or worse, sometimes we work with clients whose missions seem antithetical to our “make the world a better place” ideals (e.g., the U.S. Department of Defense, ExxonMobil).

I’m currently volunteering my time to try to better align RTI’s actions with their mission and vision in order to significantly increase RTI’s level of positive impact.

The short of my request: do you all have any connections to folks who might be helpful to talk to as I work to integrate EA ideas and metrics into this large research organization? Are you one of those people yourself? (If so, I would love to talk.) I would additionally be interested in any resources you think might be helpful.

I’ve provided much more context below. Thank you for any thoughts, connections, or recommendations you’re able to send my way!


Warmest of wishes,

Lauren Zitney (she/​her/​hers)

------------------

RTI Mission and Vision:

Mission: To improve the human condition by turning knowledge into practice


Vision: We address the world’s most critical problems with science-based solutions in pursuit of a better future. Through innovation and technology, we deliver exemplary outcomes for our partners. We support one another in an environment grounded in integrity and respect.

---

Potential Labor Hour Contributions to High-Impact Causes:

  • In 2020, RTI employed 5,881 people.

  • Short/​medium term goal (within 5-10 years): I think it is somewhere between 10% and 60% likely that RTI could contribute 903,321.6 labor hours per year to high-impact causes. (Meeting this goal would probably take 5-10 years, but then should be sustainable every year after.)

  • This means that every year (after the 5-10 year achievement window), RTI could contribute the labor-hour equivalent of 11.3 people spending entire, 80,000 hour careers in high-impact problem areas.

  • If RTI achieves this goal, that would increase the likelihood that RTI could contribute even more high-impact hours than the short/​medium term goal. The biggest part of the challenge is going to be finding funding for high-impact work. But if we can scale it, I think RTI would be very receptive to doing as much high-impact work as we can find funding for.

Here is a more detailed breakdown of that prediction:

---

If you’re curious about RTI’s financial scale:

RTI’s Revenue:

Fiscal Year 2018: $957 Million

Fiscal Year 2019: $963 Million

Fiscal Year 2020: $912 Million – I think this drop in revenue was due, at least in part, to COVID-19

Average Revenue over Past 3 years: $944 Million

---

These are specific areas I’d love help thinking through:

  1. Evaluating the level of criticality of projects within our portfolio: I would love thoughts on how granular one should be when evaluating a given problem’s level of criticality. (For now, I’m using the scale + level of neglect + tractability as a definition for what I mean by “critical.”)

    • RTI is currently very interested in pursuing “climate change” work because the Biden administration seems to be poised to spend a lot of money on “climate change.” But, it’s really hard to evaluate how critical climate change work is without being more specific. (Are we talking about carbon tax policies? Energy production? Accommodating expected climate-caused migration? etc...)

    • But at the same time, if we get too granular, then trying to evaluate RTI’s portfolio by level of criticality starts feeling very time-consuming and unwieldy. (When we talk about energy production, do we mean solar, hydro-electric, nuclear? And then within each of those, are we talking about making those types of energy production more efficient, or are we trying to subsidize them to make them more easily accessible, or are we doing public relations campaigns to try to make them more politically popular?)

  2. Impact Evaluation: How can we measure the impact of any given project or deliverable to ensure our work has a life after the research phase ends?

    • RTI’s work spans a wide variety of fields. (International Development, Medicine, Tobacco Control, and Education are just a few examples…) Based on my experience, and the experiences of colleagues, our research is very rarely turned into practice. But, if we can quantify the types of projects and clients where our research is or is not impactful, then there may be steps we can take to work with clients more likely to generate impact, or to design deliverables to encourage implementation.

    • We often use “number of peer-reviewed journal articles published” as a metric for how impactful we are. However, most peer-reviewed research is published behind paywalls and are cited by few people in a niche subject area. (case in point: Suleski, 2009)

    • Maybe we should measure media attention per published article. It would be great if getting media attention for our research was the rule, rather than the exception. Or if it already is the rule, it’d be great to know that.

  3. Impact Strategy: There are probably some very easy ways we can increase the impact of the work we’re already doing.

    • For instance, we could write better. Here, Helen Sword, author of Stylish Academic Writing talks about how awful most academic writing is, and how we can make it better. I imagine having a rubric based off of Helen Sword’s work could be really valuable both for researchers writing academic literature and editors within RTI.

    • We could also invest a bit of time in distribution campaigns. (e.g., print research along with a summary of key takeaways and send them to policymakers we think may be interested)

  4. Institutional Decision-Making: I’m aware of flash-forecasting as a norm for organizational communication.

    • Are there people who could teach me how to pilot flash-forecasting on a small scale? Or who may be able to give me some tips and tricks based off their experience trying to integrate it into their organizations?

    • Are there other similar ideas that could improve institutional decision-making that I should be learning or championing within my organization?

  5. Randomized Control Trials: The most compelling evidence I could have for convincing RTI to adopt some of these ideas is scientific research.

    • Are there studies that have tried to measure the effect of any of the above interventions?

    • If not, it seems like RTI might be a good place to start. Are there any organizations that might be interested in funding this type of research?

      • For instance, I think a Randomized Control Trial measuring the impact of flash-forecasting on the amount of fruitful high-risk organizational spending could be really interesting. Or maybe measuring the citation rates of academics who use a writing-style-guide rubric during the preparation of academic articles vs. academics who don’t.

Lauren Zitney30 Oct 2021 1:23 UTC
37 points
11 comments4 min readEA link