Are alignment researchers devoting enough time to improving their research capacity?

(Epistemic Status: Anecdotal)

If we want to reduce AGI x-risk, it seems pretty intuitive to me that alignment researchers should be regularly dedicating time to improving their research capacity. But I’m suspicious that many of them don’t do this.

Am I wrong? I would love to know if I am.

(I’m using the phrase “research capacity” vaguely, to mean both research skill and productivity.)

Over the last 3 − 4 years, I’ve had something like 12 − 20 informal conversations with researchers at various alignment orgs on the subject of how they improve their research capacity. In these conversations, I’d ask questions like “How do you personally go about getting better at research?”, or “What have you done recently to improve your research process?” or “What is one thing you could do to get better at your job?”. And more than half of the responses are one of the following:

1. a blank stare
2. a long, thoughtful pause followed by no actual answer
3. an argument that “this kind of messy, abstract work doesn’t lend itself to direct improvement in the way other skills do”
4. an argument that “the best way to improve at research is to simply do the work and keep your eyes peeled for opportunities to improve as you go.”

To be clear, some researchers do have different responses. But these answers were surprisingly common, and… they seem wrong?

My Response To #3 :

“this kind of messy, abstract work doesn’t lend itself to direct improvement in the way other skills do”

I’m not a researcher, but I can’t think of any skill I’ve learned, whether intellectual or physical, that doesn’t benefit from some amount of regular and intentional focus on improving it.

Additionally, over the last couple months, I’ve started having debugging conversations with researchers who want help thinking through how to improve, and in these conversations most of them generate lots of ideas and claim the discussion is quite productive. That’s not what I would expect if doing explicit skill improvement just didn’t work for alignment research.

Here are a few examples of opportunities for improvement that researchers have identified in talking with me:

improving their ability to find research collaborators
improving at prioritizing research tasks (e.g. should they spend the next hour reading a textbook, or spend this time writing out their current ideas)
improving their ability to deconstruct larger problems into subproblems.

My Response To #4:

the best way to improve at research is to simply do the work, and keep your eyes open for opportunities to improve as you go.

Once again, I’m not a researcher, but as far as I can tell the above quote has never been true for me with respect to any skill I’ve developed in the past.

To be fair, I do think it’s important to keep your skill improvement approach grounded in real problems you’re solving. It’s bad to lose track of the object-level and get lost in endless meta-level considerations. But I would be really surprised if just doing the work and dedicating zero time to improving your capacity directly was actually “the best way”.

In my experience, most skills contain lots of parts, many of which will seem small and insignificant if I’m looking at them from the perspective of a single problem right in front of my face. It’s only when I intentionally zoom out and consider how these parts affect my performance across many problems that I notice how valuable it would be to improve them.

For any researcher who adheres to the “just do the work” approach, I’m curious if you’ve run an experiment like this with yourself:
- Spend a week “just doing the work and keeping your eyes open”, and see how many improvement ideas you generate and how valuable they seem.
- Then spend 20 minutes specifically trying to think of ways to improve, and see how many you generate and how valuable they seem.
- Compare Results.
If you don’t generate more valuable ideas in the 20 minutes than during the week, that would be interesting to hear.

Two More Objections I’ve Heard

“Alignment researchers already spend lots of time learning in the pursuit of their research. They read the textbooks of esoteric technical fields, they learn from experts and from other researchers, and they try to keep up with developments in their field. Isn’t that enough time devoted to improving their research capacity?”

No, I don’t think so.

I view those activities as part of the work of research, not as something extra on top of the work. In fact, those are great examples of activities at which I’d think a researcher would want to improve.

Researchers are more effective when they’re allowed to follow their curiosity. Prescribing some standard idea of what good (or better) research looks like never works and actually harms their capacity.

Yep, that sounds right to me. So it’s a good thing I didn’t claim otherwise.

Seeking improvements doesn’t have to mean letting go of curiosity as a crucial driving force in your research.

And I’m certainly not implying that I have some specific idea of how research is best done. I’m just making the claim that there should be some way of getting better at it. I assume that will look different for each researcher.

Closing Thoughts

So what’s the deal? Have I spoken to an unrepresentative sample of researchers? Is there some miscommunication happening?

Or are we actually lacking a culture of constant improvement among alignment researchers?