Thanks for the link. So the PMA survey shows a significant increase in modern contraceptive prevalence rate starting in Feb ’21 - a full 7 months BEFORE the start of the FEM radio campaign—without any appreciable change on slope of the increase after the beginning of the FEM campaign (shaded area on the graph). How are you attributing ANY increase directly due to the FEM campaign itself? And any thoughts about what led to the increased rate preceding the start of the campaign?
I think you’re reading the graph wrong. There are two data points, one with data collection “Dec 2020- Feb 2021” (before the intervention) and one with data collection “Dec 2021- Jan 2022″ (after the intervention).[1] There are no in-between measures that would give you a “slope” from Feb ’21 to the start of the intervention.
Thanks. In reviewing the survey methodology more carefully, I see the data comes from cross-sectional survey over 3 months at yearly time points Therefore impossible to say when during the year the increased use began. But as Nick and the post clearly state, if this was the only major intervention during 2021, it’s reasonable to assume that at least some of the increase was due to the FEM campaign.
As far as I can see they are inferring casualty (cautiously) because there are other regions that haven’t seen the same drastic uptake in family planning on that year as the region they worked in, and no other major player they know of had a major campaign in that area.
They also said they are engaging with researchers to see how much of the increase in uptake might be due to their work, and said that early results are promising. I struggle to understand what they mean by this and have asked above and hope for a reply.
I would agree that it is dangerous to make any strong claim of casualty from any before and after cross sectional research in such a complex scenario. Because of this nature of the survey I don’t think FEM can claim cause and effect here with high certainty.
But assuming the survey is sound (I had a look and it seems solid) then I would say at least there is a good chance that some proportion of the increase in uptake is likely because of FEMs work—but how much they could claim is very difficult to ascertain.
They might have other data they are using sad well which I can’t see here though, but will wait for a comment from FEM.
To be clear, I’m a gynecologist. I think increasing contraceptive use is extremely important and impactful and that mass media campaigns are likely to be effective. However, I find some of the claims in this narrative very misleading. I encourage the FEM team to engage in more rigorous research and to focus on capacity-building perhaps by partnering with other organizations that have a great deal of experience working in this space over a long period of time.
Hi Paula, (1) “I encourage the FEM team to engage in more rigorous research and to focus on capacity-building...” FWIW, I think giving the organization (who are engaging with this thread) more time to respond on the point you raise about the PMA survey would be significantly preferable than making a conclusion in either direction at least based on my reading of this thread. (2)”...partnering with other organizations that have a great deal of experience working in this space over a long period of time...” Two separate points: a. to what extent do you know they don’t do this? I know them to have extensively engaged with local organizations and had at least information sharing conversations with international organizations (f.e. Development Media International who were founded in 2005) from my very loose following of their work over the years. b. Sadly, and much to my personal heartbreak, from having worked in international women’s human rights myself for a number of years, it is certainly not the case that an organization who has been around a long time is more effective or even effective at all than an evidence-based, well run and connected new organization. I have actually been deeply disappointed by the average long-standing organization in this field but perhaps you have had better luck! :) Thanks!
Fair enough. Was going mostly from what I’ve read in their post and on their website in addition to the RP report. I don’t disagree. Although, some international organizations partnering with local public health departments in LMICs have a decent track record in this regard.
The independent survey was done by PMA Data, here’s a link.
Thanks for the link. So the PMA survey shows a significant increase in modern contraceptive prevalence rate starting in Feb ’21 - a full 7 months BEFORE the start of the FEM radio campaign—without any appreciable change on slope of the increase after the beginning of the FEM campaign (shaded area on the graph). How are you attributing ANY increase directly due to the FEM campaign itself? And any thoughts about what led to the increased rate preceding the start of the campaign?
I think you’re reading the graph wrong. There are two data points, one with data collection “Dec 2020- Feb 2021” (before the intervention) and one with data collection “Dec 2021- Jan 2022″ (after the intervention).[1] There are no in-between measures that would give you a “slope” from Feb ’21 to the start of the intervention.
See page 9 of the report: “Tables: Contraceptive prevalence and unmet need”
Thanks. In reviewing the survey methodology more carefully, I see the data comes from cross-sectional survey over 3 months at yearly time points Therefore impossible to say when during the year the increased use began. But as Nick and the post clearly state, if this was the only major intervention during 2021, it’s reasonable to assume that at least some of the increase was due to the FEM campaign.
“intervention period indicated by shaded area”?
The shaded area is between the two data points, which means that the two data points are one before and one after the intervention period.
Right, my question is how can you infer causality from the data?
Thanks Paula, it’s a good question
As far as I can see they are inferring casualty (cautiously) because there are other regions that haven’t seen the same drastic uptake in family planning on that year as the region they worked in, and no other major player they know of had a major campaign in that area.
They also said they are engaging with researchers to see how much of the increase in uptake might be due to their work, and said that early results are promising. I struggle to understand what they mean by this and have asked above and hope for a reply.
I would agree that it is dangerous to make any strong claim of casualty from any before and after cross sectional research in such a complex scenario. Because of this nature of the survey I don’t think FEM can claim cause and effect here with high certainty.
But assuming the survey is sound (I had a look and it seems solid) then I would say at least there is a good chance that some proportion of the increase in uptake is likely because of FEMs work—but how much they could claim is very difficult to ascertain.
They might have other data they are using sad well which I can’t see here though, but will wait for a comment from FEM.
To be clear, I’m a gynecologist. I think increasing contraceptive use is extremely important and impactful and that mass media campaigns are likely to be effective. However, I find some of the claims in this narrative very misleading. I encourage the FEM team to engage in more rigorous research and to focus on capacity-building perhaps by partnering with other organizations that have a great deal of experience working in this space over a long period of time.
Hi Paula, (1) “I encourage the FEM team to engage in more rigorous research and to focus on capacity-building...” FWIW, I think giving the organization (who are engaging with this thread) more time to respond on the point you raise about the PMA survey would be significantly preferable than making a conclusion in either direction at least based on my reading of this thread. (2)”...partnering with other organizations that have a great deal of experience working in this space over a long period of time...” Two separate points: a. to what extent do you know they don’t do this? I know them to have extensively engaged with local organizations and had at least information sharing conversations with international organizations (f.e. Development Media International who were founded in 2005) from my very loose following of their work over the years. b. Sadly, and much to my personal heartbreak, from having worked in international women’s human rights myself for a number of years, it is certainly not the case that an organization who has been around a long time is more effective or even effective at all than an evidence-based, well run and connected new organization. I have actually been deeply disappointed by the average long-standing organization in this field but perhaps you have had better luck! :) Thanks!
Fair enough. Was going mostly from what I’ve read in their post and on their website in addition to the RP report. I don’t disagree. Although, some international organizations partnering with local public health departments in LMICs have a decent track record in this regard.
Editing my post as I may have come across too critical. Very supportive of this work and just trying to give helpful feedback.