Estimating the Value of Mobile Money
In the US we have many options for sending money around: there’s the traditional cash, check, credit card, bank transfer, etc., and then there’s PayPal, Venmo, Square, Wallet, etc. In poorer countries, however, if you want to send money to someone in another city your best option is often to send cash on a bus. In 2007 Safaricom introduced M-Pesa in Kenya, where it has been very successful. Can we get an estimate of how valuable M-Pesa has been?
In The long-run poverty and gender impacts of mobile money (Suri and Jack 2016) they try to figure out the effect of the M-Pesa rollout on per-capita consumption. They argue that from 2008 to 2010, when M-Pesa was just getting started, new agents were distributed effectively randomly. [1] Then they look at how areas that were close to M-Pesa agents in 2008-2010 compare to ones that weren’t, and argue that this difference represents the effect of having access to M-Pesa.
They find that having access to M-Pesa raised log [2] per-capita consumption by 0.012 (n=1593, standard error=0.005, Sidak-Holm p=0.04). (EDIT: this is an underestimate; see update below.) The reason to care about log consumption is that the more money you have the less valuable additional money is, and this relationship is approximately logarithmic (though it might fall off even more sharply). So by averaging log consumption over all the people in their sample we’re getting a rough approximation of how much better M-Pesa access made people’s lives.
What does a 0.012 increase in log per-capita consumption look like? Here’s a table:
| per-capita consumption | change |
|---|---|
| $50,000 | $603 |
| $25,000 | $302 |
| $10,000 | $121 |
| $5,000 | $60 |
| $2,500 | $30 |
| $1,000 | $12 |
| $500 | $6 |
| $250 | $3 |
| $100 | $1 |
To get a comparision, another way to increase someone’s consumption would be to just give them money. GiveDirectly is currently the best option for doing this, and on average GiveDirectly gives $288 per-person to recipients with an annual consumption of $238, a log-increase of 0.79. This is 66x the 0.012 log-increase for M-Pesa availability. GiveDirectly is about 80% efficient, so getting someone $288 costs about $360. So, very roughly, giving someone access to M-Pesa one year earlier is about as valuable as donating $5.45 to GiveDirectly. [3]
While M-Pesa was very successful in Kenya, its expansion to other markets has been slow. Improving M-Pesa’s expansion, or starting a competitor that can expand faster, could be very valuable. For example, the population of Ethiopia is 94M, so getting mobile money available in Ethiopia a year earlier would be about as good as donating $500M to GiveDirectly. [4]
This is a very rough analysis, and as it is essentially a multiplicative chain errors anywhere along the chain are magnified. For example, two errors in the first version of this post moved the final estimate by 2.3x and 1.5x respectively. This estimate is nowhere near as robust as, for example, GiveWell’s estimates for GiveDirectly or the AMF, and those are still very uncertain. Still, it suggests deploying mobile money is an extremely promising intervention, competitive with the top global poverty interventions.
Update 2017-01-01: Tavneet Suri, corresponding author of the paper I’m drawing on here, responded to my question about the interpretation of cell 1 in table 1. Above I interpret it as the effect of having an M-Pesa agent within 1km, but it’s actually the effect of having one additional M-Pesa agent within 1km. Since the average number of M-Pesa agents within 1km was ~9, naively we’d expect this to mean the effect of M-Pesa is 9x what I estimated above. But since the first M-Pesa agent is probably much more valuable than the tenth, and their sample mostly didn’t include this no-access-to-access transition, the true effect is probably more than 9x. On the other hand, this is kind of big, maybe implausibly so. Instead of a ~1% increase, we’re talking about more like 20%. Could M-Pesa really have made Kenya that much richer?
Cross-posted from jefftk.com.
[1] On the other hand, after 2010 they believe there was a systematic effort to expand in places with higher demand. Overall, this is one of the weakest parts of this analysis: if places that got agents earlier were more actually more economically dynamic then it wouldn’t be at all surprising to see greater consumption growth there.
[2] The paper doesn’t say what base they’re using for the log, but I’m assuming here that it’s e: natural log is standard in economics. Additionally, in Table S1 of the supplement they give both “Daily per Capita Consumption” and “Log Daily Per Capita Consumption”, and the particular numbers they have rule out a base as large as 10. In a draft of this post I had initially assumed log10: thanks to Rob Wiblin for catching this.
[3] My model of this is that you can either wait and get the consumption boost from mobile money eventually, or you can do something to make it happen sooner. If we take a window long enough to include the eventual deployment of mobile money I would expect to see something like:
The solid line represents the path the average log(per-capita consumption) would take, with a jump at the eventual introduction of mobile money, while the dashed line represents moving that jump forward in time. So the benefit is the area between the two curves: how much better mobile money is for people, multiplied by how much you can accelerate it.
[4] Or donating $105M to the AMF, according to GiveWell’s cost-effectiveness estimates.
Because it is helpful to think about exactly what intervention is needed to help mobile money expand (which may differ by country), I’m throwing here a few potential barriers (mostly based on my own experience in Kenya and Myanmar):
Regulatory barriers (India allowed it only recently because of this; in Myanmar it’s still ongoing)
Network effects: in Kenya I heard that an important reason it took off was that Safaricom had a very high market share (maybe near 70%?); in Nigeria I heard that the fragmentation of the telecommunication market is one reason it didn’t take off. I’m not sure if more countries are similar to Kenya or Nigeria, also these are all anecdotal. One interesting thing though is that lack of competition in Kenya may have contributed to the high charges (though there is more competition now including from mobile carriers and banks).
Lack of trust: people may not trust mobile carriers or mobile money agents. Probably less of a problem in a close knit community where agents are shopkeepers. Also, lack of trust in banks is a common problem in developing countries but I have no idea about trust on mobile carriers/agents.
Existing alternatives already good enough: this has been mentioned to me in Myanmar, that the traditional “hundi” system of money transfer works well and is cheap which may dampen adoption of mobile money. If that’s true then mobile money wouldn’t contribute much anyway, but I’m skeptical since mobile money is really much more convenient. (Also it can be used as a savings tool like a checking account, and the poor often face savings constraint too, but I’m not sure how effective that will be; interventions that tackle “self-control” have worked well on this so such elements might need to be bundled in order for mobile money to help with saving)
The intervention I had in mind when writing this post was joining a start-up that has been working on this and has been seeing good results so far: http://www.jefftk.com/p/leaving-google-joining-wave
Wave is really good! (I use it) Another thing one can do is to work for some mobile money company in a developing country to design products that benefit the poor (e.g. saving, credit, that I mention in the other post), like the American guys I met in Myanmar’s Wave Money (but they are still early stage and has many challenges before having an impact). (Not suggesting you should do it though—involves moving to a developing country etc., and could be much less likely to succeed due to regulations etc.). BTW this is the mobile credit scoring company I had in mind: http://tala.co/.
Thanks for this Jeff—a very informative post.
The study doesn’t appear to control for cash transfers received through access to M-Pesa. I was thinking about how much of the 0.012 increase in ln(consumption) was due to GiveDirectly cash transfers.
Back of the envelope:
M-Pesa access raises ln(consumption) by 0.012 for 45% of population (c.20m people).
0.012 * 20m = 234,000 unit increases in ln(consumption)
GiveDirectly gave c.$9.5m in cash transfers between 2012-14 to people with access to M-Pesa. [1]
GiveWell estimate each $ to GiveDirectly raises ln(consumption) by 0.0049
9.5m * 0.0049 = 46,000 unit increases in ln(consumption)
So GiveDirectly accounted for (very roughly) a fifth of the 0.012 increase in ln(consumption) due to M-Pesa.
[1] this is an overestimate as it assumes all transfers went to Kenya and none to Uganda
(Done in haste—may have got my sums / methodology wrong)
Good point! I hadn’t thought about this at all. GiveDirectly’s cash transfers were very large, enough that $9.5m would go to 33k people ($288/person). The population was 43M, so 1 in 1300 people received money from GiveDirectly. Their sample size is just 1593, so you expect 0-2 GiveDirectly recipients. I think they should be pretty visible in the data? Might be worth writing to the authors.
It seems like you’re assuming that the GiveDirectly money would have gone only to the M-Pesa-access side of the (natural) experiment, but they categorized areas based on whether they had M-Pesa access in 2008-2010, not 2012-2014 when access was much higher.
I didn’t notice that GiveWell had an estimate for this, and checking now I still don’t see it. Where’s this estimate from?
(In my post I just took their average amount transferred, figured out what effect that had on the average recipient’s income, and then discounted by .8 for GiveDirectly’s overhead.)
(My method gives an estimate of 0.0022 per dollar to GiveDirectly, so if GiveWell is estimating 0.0049 then my bottom line numbers are roughly 2x too high.)
Ah yes—that kind of invalidates what I was trying to do here.
It came from the old GiveWell cost-effectiveness analysis excel sheet (2015). “Medians—cell V14”. Actually looking at the new one the equivalent figures seems to be 0.26% so you’re right! (Although this is the present value of total increases in current and future consumption).
I spent about two hours looking at this in further depth and made an initial stab at modeling out the impact. I estimate an effectiveness of $200/hr (95% interval: $50/hr to $511/hr), not taking into account the value of donating the salary earned from working at Wave.
Some places where I notice we disagree or I am confused:
1.) I disagree with you here (footnote 1) that there is a 50% chance of failure (or success). I think the chance of failure could be significantly higher. From https://en.wikipedia.org/wiki/M-Pesa: M-Pesa expanded to Kenya (>10M subscribers), Tanzania (5M), South Africa (100K in a year, 1M in five years), India (???), Mozambique (???), and Lesotho (???).
Also, a 2016 Vodaphone press-release suggested M-Pesa seems to have 25M customers worldwide after 10 years of effort.
Based on this, I model that a Kenya-level success (>10M subscribers) thus looks like it would have a less than 1⁄10 chance and a South Africa-level success (1-10M subscribers) looks like it would have a ~3/10 chance. However, I think this success figure could be lower due to diminishing marginal returns since M-Pesa has already plucked low hanging fruit. It’s possible better technology could increase this chance. I’d have to know more specifically about what problems M-Pesa runs into and how these are addressed.
-
2.) I think your estimate that getting M-Pesa a year earlier is only 66x worse than getting a $288 transfer from GiveDirectly is an overestimate because I expect future roll-outs will take place in countries with higher base consumption. However, as you point out, that estimate is also already an underestimate due to misunderstanding the study. I don’t know how to correct for this either way, so I used the 66x number literally in my calculation.
-
3.) I disagree with you here (footnote 1a) that marginal ETG donations are at GiveDirectly levels of cost-effectiveness. I expect them to continue at AMF levels (or greater) for at least a few more years, for reasons OpenPhil mentioned and Carl mentioned. I did an AMF-adjustment in my model for this reason.
-
4.) I really don’t know how many staff years it would take to either complete a roll-out or know that it’s not going to happen and this is an important part of the model. I currently guess 5-10 full-time staff for 2-5 years, or 10-50 total staff years. This does not count field agents or other hired locals. I couldn’t find any information on M-Pesa’s total staff count anywhere. I note that Wave has at least 44 staff (from counting faces on the about page), but I don’t know if they’re all full-time or all focused on expanding cash transfers.
-
5.) I’m confused about why GiveDirectly is stated to be 5x more cost-effective than AMF when from GiveWell’s cost-effectiveness estimate, GiveDirectly has a median of $7702 per life saved, ranging from $2200 to $16000, excluding outliers. AMF has a median of $3282 per life saved, ranging from $2200 to $4800, excluding outliers. Together, this implies a comparison centered around 2.35x but ranging from 1x to 3.3x. Maybe I misread the sheet—I haven’t invested that much time in making sure I fully understand it yet.
Re 1, it’s worth noting that M-Pesa was administered by very different teams in different countries. In Kenya it was allowed to operate mostly as a startup with limited oversight from Safaricom (or anyone else), whereas in other countries (to varying degrees) the people running M-Pesa were constrained by stricter management from the telecom’s country head. This means that there are obvious ways in which M-Pesa was executed less well in other countries. For instance, Wave integrates with M-Pesa in both Kenya and Tanzania, and despite running on exactly the same technology, the Tanzanian system’s uptime is substantially worse. Similarly, the quality of their agents and their customer support staff in Tanzania is noticeably lower.
Since Wave isn’t hamstrung by oversight from a stodgy and risk-averse telecom, I think you should give less weight to examples from countries other than Kenya as a base rate.
Possibly, but they are already starting to operate in the country in question, and my understanding is that’s been going pretty well. My impression is that they’re much more competent than Safaricom. My inside view is much higher than 50%, and getting down to 50% was a discount from there.
I’m confused. I was trying to talk about the counterfactual for a specific very poor country if Wave were not working there. So if future mobile money rollouts by other organizations happen first in countries with higher base consumption then that increases the counterfactual impact of Wave choosing to come into a country with very low consumption.
See http://www.jefftk.com/p/leaving-google-joining-wave#fb-835897806972_835943804792
That, combined with estimating marginal impact, makes this pretty awkward. I figure something like 40 person years?
This comes from cell F31 of the “Results” tab. I haven’t put time into understanding how that’s calculated, but it looked like the relevant bottom line number.
Thanks Jeff!
-
Sounds like you definitely have inside info that I don’t have, so for now I’d have to rely on my outside view, but I can work to acquire that inside info if I look into this more.
-
I don’t know what country Wave is looking at or how they are doing what they do because you have inside info that I don’t have. If it has consumption comparable to Kenya than my point is invalid. I just was concerned that it wouldn’t.
-
Cool. Sounds like this isn’t a disagreement between us then.
-
Agreed that it is pretty awkward to estimate. I modified my model to use some of your inputs—such as a 40% chance of 1-10M subscribers and a 10% chance of >10M subscribers and 40 person years—and it comes out to $383/hr (95%: $145/hr to $834/hr). The new mean is still in my old 95% interval which is about the best I can hope for with this level of uncertainty.
-
Oh, I see that now. I suppose this is a question for GiveWell and not you. I’ll ask them.
If you’re interested in working for Wave, or are advising other people on whether it’s a good idea for them, I could imagine they’d be quite interested in talking to you!
It’s poorer than Kenya.
That sounds pretty awesome, who do you think would be a good person to reach out to when I’m ready?
Ben Kuhn maybe?
Very cool post.
Just saw that the transaction costs for m-pesa are quite high—the company makes ~20% profit… so there might be something that a Wave-like startup could do: https://en.wikipedia.org/wiki/M-Pesa#Cost.2C_transaction_charges.2C_statistics
maybe using crypocurrency—see here:
http://phys.org/news/2016-10-cryptocurrency-bottom-billion.html
The transaction costs listed on the wikipedia page you cite aren’t trivial, but would average well less than 20% unless most transactions are (a) very small and (b) to unregistered users. I’m missing something.
EDIT: could it just be that their profit is 20% of expenses, as opposed to 20% of the money that flows through the M-Pesa network?
That article doesn’t really show that cryptocurrency helps here. Mostly they’re unhappy with transaction fees on international remittances, but you can have low transaction fees just by automating interactions with the money transfer organization, without going to cryptocurrency. And with cryptocurrency generally you pay someone a fee to get your money into the cryptocurrency and then your recipient pays someone else a fee to get it into their local currency.
I think their profit is 20% of their revenue (for a money transfer company, revenue is the total fees brought in, not total money paid into the network).
I just got back from Myanmar and I talked with some people running Wave Money (one of the mobile money companies in Myanmar, and the only licensed one so far; not related to the Wave that Jeff mentioned which sends money to Africa).
Getting people to adopt could be a big challenge, depending on the country. In Kenya, the anecdotal story of why mobile money took off so quickly is 1) the need to send remittances, 2) preexisting methods for this being not very good for various reasons (insecurity is one); some also argue that Safaricom’s unusually high market share in the country played a role in speeding up adoption through bundling of services + network effects (telecommunication markets in other countries seem more fragmented). Mobile money has not taken off in some countries, e.g. Nigeria (this article argues it’s due to regulation https://iea.org.uk/blog/why-mobile-money-transformed-kenya-failed-to-take-in-nigeria). In Myanmar it remains an open question: the traditional hundi system for remittances works well for most purposes (being cheap, reliable, and not too slow), which may hinder adoption of mobile money.
Other potential functions of mobile money: other than through remittances (which is what Suri’s paper is estimating), it can also help poor populations by
Saving: providing a safe place to save. This may be important as many poor people seem “savings constrained” (e.g. see https://www.econ.uzh.ch/dam/jcr:5f6e818a-ad07-466a-8962-fdc77bb1dfc2/casaburi_macchiavello_dairy_20160731.pdf, http://www.simonrquinn.com/research/TwoSidesOfTheSameRupee.pdf) and bank are either scarce or expensive in many rural places—although it might be hard to convince people to adopt mobile money just for saving purposes.
Credit: e.g. the Mpesa-based mobile loan Mshwari. There are some startups (including one in Kenya, whose name I forgot) that creates credit scores for people based on their mobile money transaction history, mobile phone records etc. (Mshwari does a version of this but doesn’t seem very sophisticated; the startups probably use more “big data”) which may help the poor access credit in a way that’s much cheaper and sustainable than the traditional microfinance model. (In Myanmar I know one startup trying to do this and giving out small amounts of loan for shorter periods, basically competing with money lenders in slums—seems hard but could be really good if they succeed; they are quite early stage now.)
Sending government benefits, e.g. India is considering introducing universal basic income, and already have biometric identification for most citizens, but one of the remaining barriers is the scarcity of bank branches in rural places. If each village has a mobile money agent things would be much easier (and this has implications not only for poverty reduction but maybe also curbing corruption and improving local governance etc.).