Rosie_Bettle

Karma: 651

Rosie_Bettle 5 Dec 2024 9:39 UTC
13 points
2 ∶ 1
in reply to: Aidan Whitfield🔸’s comment on: GWWC’s 2024 evaluations of evaluators
Hey Aidan,

I want to acknowledge my potential biases for any new comment thread readers (I used to be the senior researcher running the fund at FP, most or all of the errors highlighted in the report are mine, and I now work at GiveWell.) these are personal views.

I think getting further scrutiny and engagement into key grantmaking cruxes is really valuable. I also think this discussion this has prompted is cool. A few points from my perspective-
1. As Matt’s comment points out- there is a historical track record for many of these grants. Some have gone on the be GiveWell supported, or (imo) have otherwise demonstrated success in a way that suggests they were a ‘hit’. In fact, with the caveat that there are a good number of recent ones where it’s too early to tell, there hasn’t yet been one that I consider a ‘miss’. Is it correct to update primarily from 3 spot checks of early stage BOTECs (my read of this report) versus updating from what actually happened after the grant was made? Is this risking goodharting?
2. Is this really comparing like for like? In my view, small grants shouldn’t require as strong an evidence base as like, a multimillion grant, mainly due to the time expenditure reasons that Matt points out. I am concerned about whether this report is getting us further to a point where (due to the level of rigour and therefore time expenditure required) the incentives for grantmaking orgs are to only make really large grants. I think this systematically disadvantages smaller orgs, and I think this is a negative thing (I guess your view here partially depends on your view on point ‘3’ below.)
3. In my view, a really crucial crux here is really about the value of supporting early stage stuff, alongside other potentially riskier items, such as advocacy and giving multipliers. I am genuinely uncertain, and think that smart and reasonable people can disagree here. But I agree with Matt’s point- that there’s significant upside through potentially generating large future room for funding at high cost effectiveness. This kind of long term optionality benefit isn’t typically included in an early stage BOTEC (because doing a full VOI is time consuming) and I think it’s somewhat underweighted in this report.
4. I no longer have access to the BOTECs to check (since I’m no longer at FP) and again I think the focus on BOTECs is a bit misplaced. I do want to briefly acknowledge though that I’m not sure that all of these are actually errors (but I still think it’s true that there are likely some BOTEC errors, and I think this would be true for many/ most orgs making small grants).

Rosie_Bettle 27 Nov 2024 22:56 UTC
80 points
4 ∶ 1
in reply to: jackva’s comment on: GWWC’s 2024 evaluations of evaluators
Hey Vasco, these are my personal thoughts and not FP’s (I have now left FP, and anything FP says should take precedence). I have pretty limited capacity to respond, but a few quick notes—

First, I think it’s totally true that there are some BOTEC errors, many/ most of them mine (thank you GWWC for spotting them— it’s so crucial to a well-functioning ecosystem, and more selfishly, to improving my skills as a grantmaker. I really value this!)

At the same time—these are hugely rough BOTECs, that were never meant to be rigorous CEA’s: they were being used as decision-making tools to enable quick decision-making under limited capacity (i do not take the exact numbers seriously: i expect they’re wrong in both directions), with many factors beyond the BOTEC going into grantmaking decisions.

I don’t want to make judgments about whether or not the fund (while I was there) was surpassing GiveWell or not— super happy to leave this to others. I was focused on funders who would not counterfactually give to GW, meaning that this was less decision-relevant for me.

I think it’s helpful to look at the grant history from FP GHDF. Here’s all the grants that I think have been made by FP GHDF since Jan 2023, apologies if i’ve missed any:

* New Incentives, Sightsavers, Pure Earth, Evidence Action
* r.i.c.e, FEM, Suvita (roughly- currently scaling orgs, that were considerably smaller/ more early stage when we originally granted)
* 4 are ecosystem multiplier-y grants (Giving Multiplier, TLYCS, Effective Altruism Australia, Effektiv Spenden)
* 1 was an advocacy grant to 1DaySooner (malaria vax roll out stuff), 1 was an advocacy grant to LEEP
* 5 are recent young orgs, that we think are promising but ofc supporting young orgs is hit and miss (Ansh, Taimaka, Essential, IMPALA, HealthLearn)
* 1 was deworming research grant
* 1 was an anti-corruption journalism grant which we think is promising due to economic growth impacts (OCCRP)

I think it’s plausible that i spent too little time on these grant evals, and this probably contributed to BOTEC errors. But I feel pretty good about the actual decision-making, although I am very biased:
* At least 2 or 3 of these struck me as being really time-sensitive (all grants are time-sensitive in a way, but I’m talking about ‘might have to shut down some or all operations’ or ‘there is a time-sensitive scaling opportunity’).
* I think there is a bit of a gap for early-ish funding, and benefits to opening up more room for funding by scaling these orgs (i.e. funding orgs beyond seed funding, but before orgs can absorb or have the track record for multi million grants). Its still early days, but feel pretty good about the trajectories of the young orgs that FP GHDF supported.
* Having a few high EV, ‘big if true’ grants feels reasonable to me (advocacy, economic growth, R and D).

I hope this context is useful, and note that I can’t speak to FP’s current/ future plans with the FP GHDF. I value the engagement- thanks.
What links here?
- Matt_Lerner's comment on GWWC’s 2024 evaluations of evaluators by Giving What We Can (3 Dec 2024 20:18 UTC; 81 points)

Are Far-UVC Interventions Overhyped? [Founders Pledge]

christian.r9 Jan 2024 17:38 UTC

142 points

8 comments61 min readEA link

Rosie_Bettle 5 Jan 2024 14:26 UTC
13 points
0 ∶ 0
on: Malaria vaccines: how confident are we?
Thank you for posting this, really interesting! I am pretty excited for malaria vaccine roll-out, but think that posts which go against current thinking are valuable, and I appreciate this being posted.

Three things that I’d be super curious about (and i think might have the effect of being more optimistic);

(1) IIRC the WHO found a 13% reduction in all-cause mortality (!) with a vaccination coverage rate of ~65%. This was in Ghana/ Kenya and Malawi, I think. I assume these areas already had some availability of SMC/ bednets, and this makes me think that there’s likely quite a substantial impact beyond nets/ SMC?

(2) Beyond immediate impacts of the vaccine- will the malaria vaccine likely bring eradication forward beyond the impact of nets/ SMC? (my current understanding is yes, and I think that bringing eradication forward is super important.)
(3) As you already point out, where would money have otherwise been spent. My impression here is that the money which funds malaria vax roll-out would likely come from a mix of sources, rather than nescessarily pulling from SMC/ bednets (but I still have some concern/ uncertainty here).

Rosie_Bettle 15 Dec 2023 20:07 UTC
5 points
0 ∶ 0
in reply to: Vasco Grilo🔸’s comment on: My quick thoughts on donating to EA Funds’ Global Health and Development Fund and what it should do
correct: to be absolutely clear, CE and the orgs themselves definitely incubated and developed the first CEAs for the CE charities (not us!). Thanks for this, will edit my comment. What I meant was is that evaluating and supporting orgs in the ‘no longer a seed stage charity but also not yet at scale’ stage is a key role that I see for the FP GHD fund, and I think we’ve had previous success here.

Yep, 1DS’ pandemic preparedness has been supported by OP. And thanks, I will mention to our comms team. FYI that our impact report is upcoming onto the website, which will list all recent grantmaking.

(also cheers for this post, and making the subtleties & differences between different GHD funds etc clearer!)

Rosie_Bettle 15 Dec 2023 19:09 UTC
10 points
0 ∶ 0
in reply to: Vasco Grilo🔸’s comment on: My quick thoughts on donating to EA Funds’ Global Health and Development Fund and what it should do
Hey Vasco,
RE: most of the funding going to groups which are already in the EA ecosystem- yep, I think this is correct. I still think this is compatible with the statement of ‘being especially interested in stuff falling outside the main GHD ecosystem’ though, and that the grantmaking focus has changed somewhat in the last year or two.

* r.i.c.e we granted to an early stage (before they were GiveWell supported) when they were having a severe funding crunch.
* FEM, Suvita and LEEP we evaluated and granted to at an early stage. I think there’s value here in ‘identifying promising charities and getting up to scale asap’.
* Our most recent grants aren’t up on the website yet: I don’t think (? could be wrong) 1daysooner’s malaria stuff has gotten EA funding yet, Ubongo isn’t EA supported afaik, or Essential.

It’s true though that a lot of FP GHD grantmaking is stuff that’s already supported by the EA community. I could have been clearer here; our guiding principle is just ‘whatever we think is most cost-effective’ and many of the existing EA-supported orgs still have funding gaps that are highly cost-effective. Aka I wouldn’t want to prioritise something that sounds cool on a ‘this is new basis’ over something that is already known in EA, and is more cost-effective. Most of the decisions that I really agonise over are ‘does this really beat bednets? Most things don’t beat bednets, and I’m aware that most things seem less cost-effective as more researcher hours go into probing them’. I do think it is possible though to to find funding opportunities in the ‘falling outside the current GHD ecosystem and >10X in expectation’ space though, and these have been some of my favourite grants.

RE: the scope of Founders Pledge’s global health and development fund being different in the past. This was before my time (I joined FP maybe 1.5 years ago) but yes, I would say that our scope has changed a little to be somewhat more risk tolerant (while still focusing on evidence base) and figuring out where our comparative advantages are (speed being one of them).

RE: being more open with evaluations; yeah I totally agree with you, and think you’re right that explaining when we depart from GiveWell is esp important. (Some places where we’ve just started to do this: Vadim’s education work, my mass media stuff. But I haven’t published actual FO evals as often). IMO our evals are rigorous, without being as in-depth as GiveWell; there’s a process of understanding the area (some of our cause area reports are on the EA forum), then a ~25 page ish eval & CEA, and red-teaming. Our rapid grants are shorter, but still have an eval, CEA/ BOTEC + red-teaming.

Rosie_Bettle 15 Dec 2023 16:24 UTC
9 points
3 ∶ 0
on: My quick thoughts on donating to EA Funds’ Global Health and Development Fund and what it should do
I am obviously biased (I co-manage the fund), so please bear that in mind. But I’d like to throw in Founders Pledge GHD fund as another GHD option- it’s not the case that you have to be an FP member to donate to this fund.

We tend to fund GiveWell-style recs (e.g. AMF) alongside things that we think are higher risk but potentially more impactful/ younger orgs that we want to get up to scale quickly. Recent grants have included: 1DaySooner to try and speed up malaria vaccine roll-out, Essential for research into producing low-cost proteins in East Africa, Ubongo, Suvita, LEEP, r.i.c.e for kangeroo mothercare, Family Empowerment Media, Taimaka. Some stuff that we’re currently checking out include Pure Earth, other lead research-y things, and some anti-corruption stuff via my colleague Vadim.

I’m especially interested in things which could be very good, but might fall outside the bounds of the current GHD funding ecosystem (e.g. things that are not super amenable to RCTs, orgs that are too young to get mainstream funders but too old for seed funding, urgent things that require a very quick turnaround).

Full disclosure that we haven’t been the best at publicly publishing evaluations- due to capacity, but still. I hope to improve this/ generally provide more resources about how we do grantmaking from the fund over the next year or so.

Rosie_Bettle 5 Dec 2023 16:37 UTC
3 points
0 ∶ 0
on: Research Report: Using Mass Media to Increase Caregiver Awareness in Sub-Saharan Africa
Hullo, this report is fascinating! Upvoted. Just a quick note that Founders Pledge estimate of DMI’s child survival program is out of date (which totally makes sense, as we haven’t published our full eval of DMI’s child survival program).

We currently rate this program at 4x GD, not 12x GD- this is after applying a really strong adjustment (16%; internal valdity adjustment of 30%, external of 54%), to try and deal with the uncertainty from the Sarrassat et al. RCT. thanks!

Rosie_Bettle 5 Dec 2023 11:30 UTC
42 points
0 ∶ 0
on: Is The Freedom Fund still the best charity addressing human trafficking / modern slavery, and why is this cause area not of more interest to EA?
Hello, I’ve been researching modern slavery at Founders Pledge recently. (Also oops, pressed send before I meant to- hence the edit!)

I really respect the Freedom Fund for pushing forward with evaluative work, esp in an area where there aren’t many published evaluations (you can see one of their evaluations here). At the same time, I think the $657 figure is very optimistic. My understanding is that the figure GWWC are pulling from (the $657 amount) is based on weaker, observational evidence—and doesn’t really account for the marginal funding amount. By this, I mean that I expect that most high-quality programs would be funded within the next few years (so this is really the cost to speed this process up).

My general sense is that this is an area where the evidence basis for direct work is suprisingly weak; there are hardly any RCTs, or strong quasi-experiments. It’s true that modern slavery interventions are difficult to test, but nonetheless this is pretty striking given that modern slavery is an area that’s of increasing interest to many policy makers (e.g. movement towards an EU ban on products made with forced labor, SDG goal 8.7).

I suspect the lack of RCT-style evidence, and the sense that it’s not super neglected, might be why there hasn’t been much EA focus here. Nonetheless, I think focusing on interventions around advocacy, tech and research may be promising here (and I’d probably argue for prioritising this over direct work, given current funding patterns).

Advocacy in this area can push for things like strengthening existing Modern Slavery Acts (e.g. by adding sanctions, existing legislature is very weak) and bringing in ‘floor wages’ for workers (e.g. see this video from the Asia Floor Wage Alliance). Tech developments to actually be able to locate modern slavery seem critical to me, and like a key bottleneck in this area: to NGOs (who need to be able to prioritise their efforts, measure the impact of their work and so on), to multi-national companies (who need to be able to locate modern slavery within their supply chains, in order to choose contractors who do not use modern slavery), to consumers (to know which products are at risk of modern slavery and apply consumer pressure), and to strengthening legislation (it seems difficult to impose sanctions on multi-national companies, if companies can genuinely argue that they don’t have the ability to locate modern slavery in their supply chains). I’m also interested in tech work that helps workers stay safe as they migrate (e.g. apps that provide accurate job info, tell people their work rights, etc etc). So I’d probably lean to charities with a strong focus on this; afraid I don’t have a one-stop recommendation yet as work is still in progress, but you might want to check out the Asia Floor Wage Alliance, Global Fund to End modern Slavery, Anti-Slavery International and International Justice Mission as well as the Freedom Fund (for groups that also have an advocacy/ tech focus).

Rosie_Bettle 6 Sep 2023 14:03 UTC
2 points
0 ∶ 0
in reply to: Stan Pinsent’s comment on: A systematic framework for assessing discounts [Founders Pledge]
Thanks Stan! This is really helpful- agreed with you that they should be combined as multipliers rather than added together (I’ve now edited accordingly). I’m still mulling a bit over whether using the word ‘discount’ or ‘adjustment’ or something else might help improve clarity.

Rosie_Bettle 17 Aug 2023 11:31 UTC
2 points
0 ∶ 0
in reply to: Joel Tan🔸’s comment on: A systematic framework for assessing discounts [Founders Pledge]
Thank you Joel! Hope that you had a nice flight.

A systematic framework for assessing discounts [Founders Pledge]

Rosie_Bettle16 Aug 2023 10:49 UTC

83 points

5 comments26 min readEA link

Rosie_Bettle 7 Mar 2023 19:11 UTC
1 point
0 ∶ 0
on: Mass media interventions probably deserve more attention (Founders Pledge)
Note that I made an edit to this report (March 2023) to highlight that the per person reached cost estimated for DMI includes RCT costs; their per person costs to scale-up a program (aka when an RCT is not being run) are significantly cheaper. I am looking into this in more detail at present.

Rosie_Bettle 29 Nov 2022 14:56 UTC
13 points
1 ∶ 0
in reply to: JoelMcGuire’s comment on: Don’t just give well, give WELLBYs: HLI’s 2022 charity recommendation
Hi Joel, thanks for this detailed + helpful response! To put in context for anyone skimming comments, I found this report fascinating, and I personally think StrongMinds are awesome (and plan to donate there myself).
Yep, my primary concern is that I’m not sure the longterm effects of grief from the loss of a child have been accounted for. I don’t have access to the Clark book that I think the 5 year estimate comes from- maybe there is really strong evidence here supporting the 5 year mark (are they estimating for spouse loss in particular?). But 5 years for child loss intuitively seems wrong to me, for a few reasons:
1. Again, idk how good the Clark 5 year finding is so I could be wrong! But the data that I’m aware of suggests that the effect of childhood bereavement is considerably longer than this. Rogers et al (2008) finds that parents still report poorer wellbeing at the 18 year mark, Song et al. (2011) finds that older bereaved couples still had lower quality of lives in their 60s relative to unbereaved couples, etc. From a quick scan (just from looking up ‘long term effects child berevement’) the estimates that I’m seeing pop up are higher than 5 years.
2. People may develop depression/ anxiety etc in response to the loss of a child. From a quick look (there may well be a better data source than this) there’s an NEJM article from Li et al. 2005 that puts the relative risk ratio (for psychiatric hospitalisation, relative to unbereaved parents) at 1.67. I assume that’s going to be a longlasting effect, and I see an argument to ‘treat key sources of depression’.
I do realise there may be confounding variables in the analyses above (aka so they’re overestimating grief as a result)- this might be where i’m mistaken. However, this does fit with my general sense that people tend to view the death of a child as being *especially* bad.
My secondary concern is that I think the spillover effects here might go considerably beyond immediate household members. In response to your points;
1. Thanks, that makes sense to me! I do think there’s some way in which this is an overestimate (aka maybe v young children are less affected by the death of a sibling). However, idk if this enough to compensate for not having accounted for non-household spillover effects; my sense is that the friends of the kid would be affected, plus other members of the village, parents who get rightly scared that their kid will fall ill, and so on. Aka I could see effects along the line of ‘my childhood best friend died, this adverse childhood experience contributed to my adult battle with depression’.
2. That also makes sense to me, cheers. I don’t have a strong sense either way on this.
3. This would make sense to me if it was adult versus adult deaths, but it seems worse to me for a parent to lose a kid (relative to counterfactual). I can see how this is a super thorny issue! Maybe it ameliorates the problem somewhat?
I do want to highlight the potential ‘duration of effect’ plus ‘negative spillover might be higher than (positive) spillover effects from GD’ issues because I think those might change the numbers around a fair bit. I.e. if we assume that effects last 10 years rather than 5 (and I see an argument that child bereavement could be like 20+) , and spillover is maybe 1.5X as big as assumed here, that would presumably make AMF 3X as good.

Rosie_Bettle 25 Nov 2022 10:06 UTC
18 points
3 ∶ 0
on: Don’t just give well, give WELLBYs: HLI’s 2022 charity recommendation
Thanks for this post, fascinating stuff!
My quick-ish question: is it possible that you are underestimating the WELLBY effect of grief, for AMF? My understanding (from referring back to the ‘elephant in the bednet’ post, but totally possible that I’ve missed something) is that these estimates are coming from Oswald and Powdthavee (2008), and then assuming a 5 year duration from Clark et al. (2018). Hence getting an estimate of ~ 7 WELLBYs.
The reason I’m a little skeptical of this is first that it seems likely to me (disclaimer that I have not done a deep dive) that losing a child would increase the likelihood of depression and other mental illnesses, alongside other things like marriage disruption (e.g. Rogers et al. 2008, which highlights effects lasting to the 18 year mark). I don’t think these effects will be accounted for by pulling out the estimate coming from Oswald & Powdthavee according to the Clark paper.
Along similar lines, I also think the spillover effects of AMF might be underestimated; my intuition is that losing a child is inherently especially shocking, and that the spillover effects might larger than the spillover effects from things like cash transfers/ therapy- e.g. everyone in the community feels sad (to some degree) about it. Am I correct that the spillover for AMF is calculated only for family members, not for friends and other members of the community?

Rosie_Bettle 18 Nov 2022 14:16 UTC
1 point
0 ∶ 0
in reply to: Tim Hua’s comment on: Mass media interventions probably deserve more attention (Founders Pledge)
Interesting, thanks for sharing! I checked out the slides and am now curious about the cultural effects of Fox News...

Rosie_Bettle 18 Nov 2022 14:08 UTC
10 points
2 ∶ 0
in reply to: Davidmanheim’s comment on: Mass media interventions probably deserve more attention (Founders Pledge)
Hi David, thanks making these points. I totally agree that there’s likely to be a lot of variation between campaigns, and that examining this is a critical step before making funding decisions- I don’t think (for instance) we should just fund mass media campaigns in general.
I did find it helpful to focus upon mass media campaigns (well, global health related mass media campaigns) as a whole to start with. This is because I think that there are methodological reasons to expect that the evidence for mass media will be somewhat weak (even if these interventions work) relative to the general standard of evidence that we tend to expect for global health interventions- namely, RCTs. This is because of problems in randomising, and of achieving sufficiently high power, for an RCT examining a mass media campaign. I think this factor is generally true of mass media campaigns (and perhaps not especially well-known), hence the fairly broad focus at the start of this report.
I agree with you though that ascertaining which programs tend to work is hugely important. I’ve pointed to a few factors (cultural relevance, media coverage etc), but this section is currently pretty introductory. The examples I’ve focused upon here are the ones where there is existing RCT evidence in LMICs (e.g. family planning is Glennerster et al., child survival from recognition of symptoms is Sarassat et al., HIV prevention is Banerjee et al.) Some things that stand out to me as being crucial (note that I’m focusing upon global health mass media campaigns in LMIC) include the communities at hand having the resources to successfully change their behavior, there being a current ‘information gap’ that people are motivated to learn about (e.g. the Sarassat one focuses on getting parents to recognise particular symptoms of diseases that could effect their children, and the Glennerster one provides info about the availability and usage of modern contraceptives), cultural relevance (i.e. through the design of the media) and media coverage.