Aaron Gertler 🔸 comments on Why scientific research is less effective in producing value than it could be: a mapping

[Aaron Gertler 🔸]

“Lack of connection with end-user” reminds me of this essay from a psychology professor who recently left academia.

As someone who wanted to be a psychology professor for a year or so, I felt a spark of recognition when I read this section (emphasis mine):

The stuff we do doesn’t matter

The thing that I’ll probably most miss about academia is getting to research whatever I’m curious about. I’m not missing it just yet, however. While on the tenure track, I didn’t think I was discovering hidden truths about the human mind by studying the American college undergraduate and Prolific.co participant.

Honestly, I’d felt pretty discouraged about research for a while. The things we study tend to have small effects, and when we can’t detect those small effects a second time, it can be hard to tell why. (Possible explanations include noise, publication bias, errors in methods, differences in populations due to culture or even the passage of time.) It’s why we spend so much time arguing about the fidelity of replication methods and hidden moderators.

Because the things we study have small, purportedly delicate effects, it’s rare that we expect to see them applied and working in the real world. It’s unpleasant to say it, but I feel that a lot of the research that we do doesn’t matter. It’s because it doesn’t matter that we were able to get all the way into the 2010s before having a replication crisis. If we had screwed up our basic science in physics or biology or chemistry, we would notice pretty quickly when the engineers told us their bridges were collapsing or the crops were dying or the soda pop was going flat. By comparison, very little in social psychology seems to be applied or expected to work in any routinely detectable way.

The lackadaisical response I’ve sometimes received when raising concerns about papers has further convinced me that most social psych research does not matter. When I email a journal to say “none of these statistics add up” or “these effect sizes are ridiculously big,” I often get no reply. Compare this to the sort of all-hands-on-deck response we might get if we found poison in the dog food. It doesn’t matter that the product is no good—we produce it for the sake of producing it, quality irrelevant.

In comparison, the stuff I’m doing as a data scientist isn’t glamorous, but it’s useful. Some of our projects save the company millions of dollars a year in shipping costs. That’s a lot of gasoline and traffic and cardboard and dry ice that we’re able to save. Reducing the amount of oil and packaging that gets used up might be the most useful thing I’ve done in years.