Derek

Karma: 863

Derek Mar 26, 2020, 6:39 PM
4 points
0 ∶ 0
in reply to: Aaron Gertler 🔸’s comment on: What posts you are planning on writing?
Hah! I was working on them before getting sidelined with covid stuff.
I can send you the drafts if you send me a PM. The content is >80% done (I’ve decided to add more, so the % complete has dropped) but they need reorganising into ~10 manageable posts rather than 3 massive ones.

Derek Mar 19, 2020, 10:00 PM
2 points
0 ∶ 0
in reply to: AidanGoth’s comment on: Founders Pledge Charity Recommendation: Action for Happiness
Thanks Aidan! Hope you’re feeling better now.
Most of your comments sound about right.
On retention rates: Your general methods seem to make sense, since one would expect gradual tapering off of benefits, but your inputs seem even more optimistic than I originally thought.
I’m not sure Strong Minds is a great benchmark for retention rates, partly because of the stark differences in context (rural Uganda vs UK cities), and partly because IIRC there were a number of issues with SM’s study, e.g. a non-randomised allocation and evidence of social desirability bias in outcome measurement, plus of course general concerns related to the fact it was a non-peer-reviewed self-evaluation. Perhaps retention rates of effects from UK psychotherapy courses of similar duration/intensity would be more relevant? But I haven’t looked at the SM study for about a year, and I haven’t looked into other potential benchmarks, so perhaps yours was a sensible choice.
Also not a great benchmark in a UK context, but Haushofer and colleagues recently did a study* of Problem Management+ in Uganda that found no benefits at the end of a year (paper forthcoming), even though it showed effectiveness at the 3 month mark in a previous study in Kenya.
*Haushofer, J., Mudida, R., & Shapiro, J. (2019). The Comparative Impact of Cash Transfers and Psychotherapy on Psychological and Economic Well-being. Working Paper. Available upon request.

Derek Mar 19, 2020, 3:43 PM
3 points
0 ∶ 0
on: AMA: Elie Hassenfeld, co-founder and CEO of GiveWell
Do you think GiveWell top charities are the best of all current giving opportunities? If so, what is the next best opportunity?

Derek Mar 19, 2020, 1:01 PM
15 points
0 ∶ 0
on: AMA: Elie Hassenfeld, co-founder and CEO of GiveWell
Do you think adopting subjective wellbeing as your primary focus would materially affect your recommendations?
In particular:
(a) Would using SWB as the primary outcome measure in your cost-effectiveness analysis change the rank ordering of your current top charities in terms of estimated cost-effectiveness?
(b) If it did, would that affect the ranking of your recommendations?
(c) Would it likely cause any of your current top charities to no longer be recommended?
(d) Would it likely cause the introduction of other charities (such as ones focused on mental health) into your top charity list?

Derek Mar 18, 2020, 9:46 PM
7 points
0 ∶ 0
on: AMA: Elie Hassenfeld, co-founder and CEO of GiveWell
How likely is it that GiveWell will ultimately (e.g. over a 100-year or 10,000-year period) do more harm than good? If that happens, what is the most likely explanation?

Derek Mar 18, 2020, 9:37 PM
2 points
0 ∶ 0
on: AMA: Elie Hassenfeld, co-founder and CEO of GiveWell
A recent post on this forum (one of the most upvoted of all time) argued that “randomista” development projects like GiveWell’s top charities are probably less cost-effective than projects to promote economic growth. Do you have any thoughts on this?

Derek Mar 7, 2020, 4:51 PM
24 points
0 ∶ 0
on: Founders Pledge Charity Recommendation: Action for Happiness
I like your general approach to this evaluation, especially:
- the use of formal Bayesian updating from a prior derived in part from evidence for related programmes
- transparent manual discounting of the effect size based on particular concerns about the direct study
- acknowledgement of most of the important limitations of your analysis and of the RCT on which it was based
- careful consideration of factors beyond the cost-effectiveness estimate.
I’d like to see more of this kind of medium-depth evaluation in EA.

I don’t have time at the moment for a close look at the CEA, but aside from limitations acknowledged in your text, 3 aspects stand out as potential concerns:

1. The “conservative” and “optimistic” results are quite extreme. This seems to be in part because “conservative” and “optimistic” values for several parameters are multiplied together (e.g. DALYs gained, yearly retention rate of benefits, % completing the course, discount rates...). As you’ll know, it is highly improbable that even, say, three independent parameters would simultaneously obtain at, say, the 10th percentile: 0.1*0.1*0.1 = 0.001. Did you consider making a probabilistic model in Guesstimate, Causal, Excel (with macros for Monte Carlo simulation), R, etc in order to generate confidence intervals around the final results? (I appreciate there are major advantages to using Sheets, but it should be fairly straightforward to reproduce at least the “Main CEA” and “Subjective CEA inputs” tabs in, for example, Guesstimate. This would also enable a rudimentary sensitivity analysis.)

2. The inputs for “Yearly retention rate of benefits” (row 10) seem pretty high (0.30, 0.50, and 0.73 for conservative, best guess, and optimistic, respectively) and the results seem fairly sensitive to this parameter. IIRC the study this was based on only had an 8-week follow-up, which would be about half your “conservative” figure (8/52 = 0.15). Even their “extended” follow-up (without a control group) was only for another 2 months. It is certainly plausible that the benefits endure for several months, but I would say that estimates of about 0.1, 0.3, and 0.7 are more reasonable. With those inputs, the cost per DALY increases to about $47,000, $4,500, or $196. That central figure is roughly on a par with CBT for depression in high-income countries, i.e. pretty good but not comparable with developing-country interventions. (And I wouldn’t take the “optimistic” figure seriously for the reasons given in (1) above.)

3. I haven’t seen the “growth model” on which the cost estimates are based, but my guess is that it doesn’t account for the opportunity cost of facilitators’ (or participants’) time. IIRC each course is led by two “skilled” volunteers who may otherwise do another pro-social activity.
What links here?
- EA Forum Prize: Winners for March 2020 by Aaron Gertler 🔸 (May 13, 2020, 10:09 AM; 26 points)

Derek Mar 5, 2020, 10:36 PM
7 points
0 ∶ 0
on: Founders Pledge Charity Recommendation: Action for Happiness
There is also evidence that health problems have a much smaller effect on subjective well-being than one might imagine.
This is only the case for (some) physical health problems, especially those associated with reduced mobility. People tend to underestimate the SWB impact of (at least some) mental health problems. See e.g. Gilbert & Wilson, 2000; De Wit et al., 2000; Dolan & Kahneman, 2007; Dolan 2008; Pyne et al., 2009; Karimi et al., 2017

Derek Feb 12, 2020, 1:26 AM
1 point
0 ∶ 0
on: Poverty in Depression-era England: Excerpts from Orwell’s “Wigan Pier”
You might want to mention the publication date (1937)

Derek Jan 31, 2020, 11:05 PM
2 points
0 ∶ 0
in reply to: cole_haus’s comment on: A Local Community Course That Raises Mental Wellbeing and Pro-Sociality
Thanks—I missed that on my skim. But the “extended” follow-up is only for another two months. It does seem to indicate that effects persist for at least that period, without any trend towards baseline, which is promising (though without a control group the counterfactual is impossible to establish with confidence). I wonder why they didn’t continue to collect data beyond this period.

Derek Jan 31, 2020, 10:45 PM
6 points
0 ∶ 0
in reply to: cole_haus’s comment on: A Local Community Course That Raises Mental Wellbeing and Pro-Sociality
Thanks—“trained facilitator” might be a bit misleading. Still, it looks like there were two volunteer course leaders for each course, selected in part for their unspecified “skills”, who were given “on-going guidance and support” to facilitate the sessions, and who have to arrange a venue etc themselves, then go through a follow-up process when it’s over. So it’s not a trivial amount of overhead for an average of 13 participants.

Derek Jan 31, 2020, 2:49 PM
34 points
0 ∶ 0
on: A Local Community Course That Raises Mental Wellbeing and Pro-Sociality
I don’t have much time to spend on this, but here are a few thoughts based on a quick skim of the paper.

The study was done by some of the world’s leading experts in wellbeing and the study design seems okay-ish (‘waitlist randomisation’). The main concern with internal validity, which the authors acknowledge, is that changes in the biomarkers, while mostly heading in the right direction, were far from statistically significant. This could indicate that the effects reported on other measures were due to some factor other than actual SWB improvement, e.g. social desirability bias. But biomarkers are not a great metric, and measures were taken to address these concerns, so I find it plausible that the effects in the study population were (nearly) as large as reported.

However:
- The participants were self-selected, largely from people who were already involved with Action for Happiness (“The charity aims to help people take action to create more happiness, with a focus on pro-social behaviour to bring happiness to others around them”), and largely in the UK. They also had to register online. It’s unclear how useful it would be for other populations.
- It’s quite an intensive program, involving weekly 2–2.5 hour group meetings with a trained facilitator two volunteer facilitators. (“Each of these sessions builds on a thematic question, for example, what matters in life, how to find meaning at work, or how to build happier communities.”) This may limit its scalability and accessibility to certain groups.
- Follow-up was only for 2 months, the duration of the course itself. (This limitation seems to be due to the study design: the control group was people taking the course 8 weeks later.)
- The effect sizes for depression and anxiety were smaller than for CBT, so it may still not be the best option for mental health treatment (though the CBT studies were done in populations with a diagnosed mental disorder, so direct comparison is hard; and subgroup analyses showed that people with lower baseline wellbeing benefited most from the program).
- For clarity, the average effect size for life satisfaction was about 1 point on a 10-point scale. This is good compared to most wellbeing interventions, but that might say more about how ineffective most other interventions are than about how good this one is.
So at the risk of sounding too negative: it’s hardly surprising that people who are motivated enough to sign up for and attend a course designed to make them happier do in fact feel a bit happier while taking the course. It seems important to find out how long these effects endure, and whether the course is suitable for a broader range of people.

Derek Jan 29, 2020, 6:05 PM
7 points
0 ∶ 0
in reply to: Greg_Colbourn ⏸️ ’s comment on: The EA Hotel is now the Centre for Enabling EA Learning & Research (CEEALAR)
But I really think the whole name should be reconsidered.

Derek Jan 29, 2020, 6:05 PM
6 points
0 ∶ 0
in reply to: Greg_Colbourn ⏸️ ’s comment on: The EA Hotel is now the Centre for Enabling EA Learning & Research (CEEALAR)
You could keep the name but drop the first ‘A’: CEELAR. Excluding the ‘A’ of Altruism isn’t great, but I think you’re allowed to take major liberties with acronyms. And really, almost anything is better than CEEALAR.

Derek Jan 28, 2020, 11:56 AM
3 points
0 ∶ 0
in reply to: RobM’s comment on: AMA: Rob Mather, founder and CEO of the Against Malaria Foundation
Thanks Rob!
As you’ve said, in addition to averting deaths it looks like AMF considerably improves lives, e.g. by improving economic outcomes and reducing episodes of illness. Have you considered collecting data on subjective wellbeing in order to help quantify these improvements? Could that be integrated into your program without too much expense/difficulty?
On the other side of the coin, one possible negative impact of programs that increase wealth and/or population size is the suffering of animals farmed for food (since better-off people tend to eat more meat). Do you have any data on dietary changes resulting from bed net distribution (or similar programs)? Would it be feasible to collect that data in future?

Derek Jan 24, 2020, 9:51 AM
22 points
0 ∶ 0
on: AMA: Rob Mather, founder and CEO of the Against Malaria Foundation
A recent post on this forum (the fourth most popular ever, at the time of writing) argued that “randomista” development projects like AMF are probably less cost-effective than projects to promote economic growth. Do you have any thoughts on this?

Derek Jan 24, 2020, 9:48 AM
9 points
0 ∶ 0
on: AMA: Rob Mather, founder and CEO of the Against Malaria Foundation
What are your thoughts on the indirect (“flow-through”) effects of AMF? For example:

1. What do you think are the main positive and negative indirect impacts of the program, both long- and short-term? (E.g. increasing productivity and economic growth, increasing/decreasing total population, strengthening health systems, greenhouse gas emissions, consumption of factory-farmed meat...) Do you have any data on these? Are you planning to gather data on any of them?

2. What proportion of the long-term benefit from the program is due to short-term direct effects such as saving lives and averting unpleasant episodes of malaria, relative to indirect benefits?

3. Do you hold a particular view of population ethics (totalism, averagism, person-affecting, etc)?

4. What is your response to critics who claim we are ultimately “clueless” about the long-run magnitude or even sign of interventions like this? (I think the basic argument is that e.g. averting deaths has a wide range of knock-on effects, both good and bad, and that we may not be justified in being confident that ultimately – say, over the next few hundred years—the impact will be net positive. See e.g. here, here, and here for a better explanation)
What links here?
- AMA: Founder and CEO of the Against Malaria Foundation, Rob Mather by Toby Tremlett🔹 (Dec 11, 2023, 6:16 PM; 73 points)

Derek Jan 24, 2020, 8:56 AM
3 points
0 ∶ 0
on: Katriel Friedman: The Benefits of Starting Your Own Charity
It’s a core part of the research ethics that they teach you when you’re being trained to run an RCT — whether you can run them if you have equipoise (i.e., are certain that an intervention works).
You might want to clarify this. Equipoise is uncertainty about whether the intervention works, and is often considered a pre-requisite for an RCT. I’m sure Katriel understands this but the phrasing here is misleading.

Derek Jan 22, 2020, 9:58 PM
18 points
0 ∶ 0
on: AMA: Rob Mather, founder and CEO of the Against Malaria Foundation
Can you explain your ’20 minute rule’?

Derek Jan 20, 2020, 3:26 PM
3 points
0 ∶ 0
on: Logarithmic Scales of Pleasure and Pain: Rating, Ranking, and Comparing Peak Experiences Suggest the Existence of Long Tails for Bliss and Suffering
Do you have any thoughts on whether valenced experience is asymmetrical, i.e. whether the most negative experiences (e.g. ¹⁰⁄₁₀ on some suitable pain scale) are more bad than the most positive ones (e.g. ¹⁰⁄₁₀ on some suitable pleasure scale) are good?
My hunch is that the worst experiences are more intense, at least if you exclude weird/rare things like Jhanas and 5-MeO-DMT trips, e.g. I’d give up days or weeks of ‘maximum happiness’ to avoid being burned alive for a minute. But not everyone shares this intuition, and I’m not sure how to settle the debate (at least until you prove and operationalise your symmetry theory of valence).