Fantastic! I feel as though nearly any project founded with the basis of those 7 principles is bound to be pretty amazing.
I can’t wait to see how you’ll tackle these challenges and uncertainties. You’ve got great question along with a great idea.
I had a few thoughts pop up throughout the read, but I’ll just stick to 2 to post in this comment:
#1.) I’m curious to hear what people in the EA forum think about the idea of ESH running its own research from time to time to help fill in any gaps or further test any ideas. If ESH is truly struggling to find good research on a particular topic, then could the ESH team conduct its own studies? For example, Clearer Thinking runs its own studies for almost every article it writes. Thoughts?
#2.) Since your feedback request is for potential flaws, I’ll briefly mention a risk that I’ve seen in self-help that is adjacent to two of the points you mentioned in the “Downside risks” section (“ESH gives advice that proves to be of net harm” and “Individual differences in benefit significantly outweigh the general differences in value between interventions”) ::
Some self-help interventions can be wonderful for resolving a personal problem for X% of people in a particular set of circumstances yet also exacerbate the problem for Y% of people with a particular factor that changes the conditions. (And without getting into details, I’ll just say that the cascading consequences tend to result in a lot of suffering for the people in group Y.)
So how can this be prevented? In the section on “differences in value between interventions” you mentioned the idea of using “a screening quiz for prioritising recommendations based on the individual.” I think that’s a great idea. Maybe there are other solutions to come up with as well.
I just wanted to call this out since sometimes interventions aren’t just a positive “difference in value” between people — sometimes they’re helpful for most yet harmful to a few. And with any given medium/format for the resources ESH will provide, it will need to consider how to communicate this (if there is that risk of harm). In the form of an article, sometimes all you can do is empower the reader with the tools they need to evaluate their own conditions/circumstances to determine which intervention makes sense for them personally (or how to customize an intervention, or whether or not they should even consider the intervention for themselves at all). Beyond articles, it would be terrific to incorporate more interactive elements (like the screening idea) to not only improve the effectiveness of the ESH resources but also prevent potential harms/risks. (And I think these ideas fall nicely into “Practicality”, “Comprehensiveness/Breadth”, “Presentation”, “Research Rigor”, and [personalized] “Prioritization”.)
(Overall, I’m in love with the idea and am surprised it didn’t come into being 10 years ago. Thanks for the thorough introduction!)
Depending on the study design, it can actually be relatively cheap! Just use Amazon Mechanical Turk or Positly to get participants. Of course, some study designs would be 20x harder than others. I imagine that testing different self help interventions would be rather hard but smaller surveys would be really easy.
Great question! I don’t actually know. (Although I do know that Spark Wave, the parent organization, is also the “parent” of Positly [both founded by Spencer Greenberg], so they probably have a deal worked out haha. Who knows.)
It’s rarely wise to do as many different types of things as Spencer Greenberg does and usually best to focus more, at least until you’re ~excelling at each thing
I think Spencer’s approach is going very well for him, but that it’s not good as standard advice for most people or even most entreprneurial EAs
I think it’s plausible it’d make sense for ESH to eventually sometimes do quite small, quick, simple, and cheap primary empirical research, like a small quick survey, as one small additional input into recommendations. But I think it’s easy to underestimate how much survey work benefits from methodological expertise and effort, and I think that sort of work is very different from what’s proposed in this post. So I’d guess it shouldn’t be one of ESH’s primary focuses or something ESH tries to get really good at (just in order to focus).
Fantastic! I feel as though nearly any project founded with the basis of those 7 principles is bound to be pretty amazing.
I can’t wait to see how you’ll tackle these challenges and uncertainties. You’ve got great question along with a great idea.
I had a few thoughts pop up throughout the read, but I’ll just stick to 2 to post in this comment:
#1.) I’m curious to hear what people in the EA forum think about the idea of ESH running its own research from time to time to help fill in any gaps or further test any ideas. If ESH is truly struggling to find good research on a particular topic, then could the ESH team conduct its own studies? For example, Clearer Thinking runs its own studies for almost every article it writes. Thoughts?
#2.) Since your feedback request is for potential flaws, I’ll briefly mention a risk that I’ve seen in self-help that is adjacent to two of the points you mentioned in the “Downside risks” section (“ESH gives advice that proves to be of net harm” and “Individual differences in benefit significantly outweigh the general differences in value between interventions”) ::
Some self-help interventions can be wonderful for resolving a personal problem for X% of people in a particular set of circumstances yet also exacerbate the problem for Y% of people with a particular factor that changes the conditions. (And without getting into details, I’ll just say that the cascading consequences tend to result in a lot of suffering for the people in group Y.)
So how can this be prevented? In the section on “differences in value between interventions” you mentioned the idea of using “a screening quiz for prioritising recommendations based on the individual.” I think that’s a great idea. Maybe there are other solutions to come up with as well.
I just wanted to call this out since sometimes interventions aren’t just a positive “difference in value” between people — sometimes they’re helpful for most yet harmful to a few. And with any given medium/format for the resources ESH will provide, it will need to consider how to communicate this (if there is that risk of harm). In the form of an article, sometimes all you can do is empower the reader with the tools they need to evaluate their own conditions/circumstances to determine which intervention makes sense for them personally (or how to customize an intervention, or whether or not they should even consider the intervention for themselves at all). Beyond articles, it would be terrific to incorporate more interactive elements (like the screening idea) to not only improve the effectiveness of the ESH resources but also prevent potential harms/risks. (And I think these ideas fall nicely into “Practicality”, “Comprehensiveness/Breadth”, “Presentation”, “Research Rigor”, and [personalized] “Prioritization”.)
(Overall, I’m in love with the idea and am surprised it didn’t come into being 10 years ago. Thanks for the thorough introduction!)
Wouldn’t that be extremely expensive?
Depending on the study design, it can actually be relatively cheap! Just use Amazon Mechanical Turk or Positly to get participants. Of course, some study designs would be 20x harder than others. I imagine that testing different self help interventions would be rather hard but smaller surveys would be really easy.
Great question! I don’t actually know. (Although I do know that Spark Wave, the parent organization, is also the “parent” of Positly [both founded by Spencer Greenberg], so they probably have a deal worked out haha. Who knows.)
FWIW, I’d guess that:
It’s rarely wise to do as many different types of things as Spencer Greenberg does and usually best to focus more, at least until you’re ~excelling at each thing
I think Spencer’s approach is going very well for him, but that it’s not good as standard advice for most people or even most entreprneurial EAs
I think it’s plausible it’d make sense for ESH to eventually sometimes do quite small, quick, simple, and cheap primary empirical research, like a small quick survey, as one small additional input into recommendations. But I think it’s easy to underestimate how much survey work benefits from methodological expertise and effort, and I think that sort of work is very different from what’s proposed in this post. So I’d guess it shouldn’t be one of ESH’s primary focuses or something ESH tries to get really good at (just in order to focus).
But those are just some quick thoughts.