Please take a survey on the quality/​impact of things I’ve written

[MichaelA]

If you’ve read anything I’ve written on the EA Forum or LessWrong, I’d really appreciate you taking this brief, anonymous survey. Your feedback is useful whether your opinion of my work is positive, mixed, lukewarm, meh, or negative. And remember what mama always said: If you’ve got nothing nice to say, self-selecting out of the sample for that reason will just totally bias Michael’s impact survey.

Don’t Panic!^[1]

I plan to use people’s responses as inputs—rather than definitive answers—in my ongoing efforts to plan my career and improve in various ways. And I’ll combine these inputs with a lot of other inputs.

Thus, you shouldn’t feel that this is uncomfortably high-stakes, nor that you should only take the survey if you’re really confident in what you’d say. You can just provide any tentative thoughts you have, and I can be responsible for working out how much weight I should give them, whether and how they should affect my decisions, etc.

(This is a good division of labour, as I know more about myself and the context of my work than you do, but you have the advantage of existing outside of my swirling vortex of alternating imposter syndrome and overconfidence.)

I’ve also added a comment below with more info on why I’m running this survey, but you’re very welcome to answer the survey without reading that comment.

Update: See also my reflections on the survey responses I’ve gotten so far, and Should surveys about the quality/​impact of research outputs be more common?

---

1. ↩︎

   The Douglas Adams reference, not the Coldplay song.

[MichaelA]

Why I’m running this survey

I think that getting clear feedback on how well one is doing, and how much one is progressing, tends to be somewhat hard in general, but especially when it comes to:

• Research

  • And especially relatively big-picture/​abstract research, rather than applied research

• Actually improving the world compared to the counterfactual

  • Rather than, e.g., getting students’ test scores up, meeting an organisation’s KPIs, or publishing a certain number of papers

• Longtermism

And I’ve primarily been doing big-picture/​abstract research aimed at improving the world, compared to the counterfactual, from a longtermist perspective. So, yeah, I’m a tad in the dark about how it’s all been going…[1]

I think some of the best metrics by which to judge research are whether people:

• are bothering to pay attention to it

• think it’s interesting

• think it’s high-quality/​rigorous/​well-reasoned

• think it addresses important topics

• think it provides important insights

• think they’ve actually changed their beliefs, decisions, or plans based on that research

• etc.

I think this data is most useful if these people have relevant expertise, are in positions to make especially relevant and important decisions, etc. But anyone can at least provide input on things like how well-written or well-reasoned some work seems to have been. And whoever the respondents are, whether the research influenced them probably provides at least weak evidence regarding whether the research influenced some other set of people (or whether it could, if that set of people were to read it).

This year, I’ve gathered a decent amount of data about the above-listed metrics. But more data would be useful. And the data I’ve gotten so far has usually been non-anonymous, and often resulted from people actively reaching out to me. Both of those factors likely bias the responses in a positive direction.

So I’ve created this survey in order to get additional—and hopefully less biased—data, as an input into my thinking about:

1. whether EA-aligned research and/​or writing is my comparative advantage (as I’m also actively considering a range of alternative pathways)

2. which topics, methodologies, etc. within research and/​or writing are my comparative advantage

3. specific things I could improve about my research and/​or writing (e.g., topic choice, how rigorous vs rapid-fire my approach should be, how concise I should be)

But there’s also another aim of this survey. The idea of doing this survey, and many of the questions, was inspired partly by Rethink Priorities’ impact survey. But I don’t recall seeing evidence that individual researchers/​writers (or even other organisations) run such surveys.[2] And it seems plausible to me that they’d benefit from doing so.

So this is also an experiment to see how feasible and useful this is, to inform whether other people should run their own surveys of this kind. I plan to report back here in a couple weeks with info like how many responses I got and how useful this seemed to be.

Feel free to comment regarding what I’ve said above, whether you expect it to be worthwhile for more people/​orgs to run this sort of survey, or anything else!

[1] I’m not necessarily saying that that type of research is harder to do than e.g. getting students’ test scores up. I’m just saying it’s harder to get clear feedback on how well one is doing.

[2] Though I have seen various EAs provide links to forms for general anonymous feedback. I think that’s also a good idea, and I’ve copied the idea in my own forum bio.

[MichaelDickens]

Off-topic, but how was this comment posted in August 29 if the post wasn’t published until September 1?

[MichaelA]

There’s been some uncertainty among the moderators about whether this post is broadly relevant enough to EA to be promoted to the frontpage. The post was briefly promoted, at which point its date of posting was reset so it would actually appear on the first page of results for a bit. (Though it was then demoted quickly, so it didn’t really spend any time there.)