Liberty in North Korea, quick cost-effectiveness estimate
In 2019, Liberty in North Korea (LINK) claims to have rescued 222 North Koreans, spending $3,660,223 across all programs, including refugee resettlement. In 2018, it was 326 rescues, spending $3,604,423. Combining, that’s 548 rescued for $7,264,646, or $13,257 per rescue. Rescue expenses alone (so excluding employee pay, other program expenses, and everything else) over both years totaled $932,456, which would come out to $1,702 per rescue. For comparison, GiveWell’s recommended life-saving charities are estimated to save a life for $3000-5000 on average.
EDIT (credits to Denis Drescher for pointing this out): What they do: They rescue already escaped North Koreans from China using secret routes, since if they are caught, Chinese authorities would send them back, or the women could be sold into sex trafficking or as brides. LINK doesn’t help North Koreans escape in the first place. LINK claims thousands of North Koreans attempt to escape each year. They also write:
It costs $3,000* to help a North Korean refugee travel from Northern China to safety in South East Asia
*The Changing Costs of A Rescue
The cost of rescues varies on where a North Korean refugee begins their journey. The closer they are to the North Korean border when our partners find them, the greater the cost because of heightened security and increased travel time. A generous foundation funds these high-risk extractions from the border region.
There are likely several ways these cost-effectiveness estimates may be off (including possibly having the wrong sign) due to indirect effects and not accounting properly for the counterfactuals, and a cost-effectiveness model should take such considerations into account. Here are some:
Less cost-effective: How likely is it that these people would have succeeded anyway, on their own or with the help of another group? Their website had someone who made 4 escape attempts (Jo Eun, on this page). How much earlier does LINK move their success if they would have been caught and made further attempts?
Unclear, lean more cost-effective: What are the risks or benefits to the families of rescues, and how likely are they? If someone is caught, their family may be punished, e.g. with labour camps or execution, and by rescuing them, we may prevent this. On the other hand, we may incentivize further escapes, which risk punishment for them and their families. To what extent are the families also aware of and accept these risks? (Credits to Bruce Tsai and Denis Drescher.)
Less cost-effective: They only rescued 15 North Koreans in 2020 due to increased security due to COVID, and maybe we should expect it to be similar going forward.
Unclear: Does this undermine reform in North Korea, by taking those disproportionately likely to push for reform? (Credits to edwardhaigh) On the other hand, LINK says that refugees send money and information back to their families in North Korea, and it’s possible this could undermine the regime. (Credits to Khorton)
Less cost-effective: How would North Korea respond to increasing rescues? Harsher punishments and security? Further encouraging births?
Unclear, but lean less cost-effective: Where does marginal funding actually go? Would they actually rescue more, or just spend more on resettlement and other services? In what proportions? If we funded them, could we get them to spend disproportionately more on rescues? How much could this scale before further rescues became very difficult?
Unclear: How are descendants affected? Escapees who are caught and sent back may be prevented from having children, or have children anyway in North Korea (this is something to check). This may extent to their families, as well. Rescued escapees seem likely to have children outside of North Korea. So, rescuing seems likely to increase the number of descendants born outside North Korea and inside North Korea, and possibly move some descendants from North Korea to outside, in case escapees would have had children in North Korea even if they get caught. How does this interact with effects on the regime, like in 4? (If we’re comparing to GiveWell-recommended charities, we should also include their generational effects. How exactly we include these effects will depend on your views on population ethics.)
There’s also the question of how much welfare they would gain from rescue.
This seems like an intervention worth looking further into.
This post was prompted by an EA I recognized making a public donation to this charity.
This is fantastic!
I wish we had a lot more posts that worked through charities’ cost-effectiveness claims in this way — even charities that are very unlikely to be competitive with standard “top choices”, and even if the estimates include a high degree of uncertainty (as long as this is communicated).
As a community, it seems good for EA to routinely “keep in touch” with a wide range of areas, for a few reasons:
We’re less likely to miss a genuinely strong opportunity from outside our usual focus areas.
We’re able to provide useful information to people interested in specific areas (even a highly uncertain analysis > having nothing but the charity’s claims to work from).
We get lots of practice in building, reading, and discussing models of the world, and are constantly exposed to many different methods of evaluation.
Thanks for this, and thanks Michael for the post.
This made me think we should perhaps have an overall evaluation of the cause area “Helping refugees migrate” from different countries in crisis (e.g., South Sudan, Afghanistan, Haiti, NK etc.) and corresponding projects in receiving countries—such as comparing LINK and GiveDirectly’s cash transfers to refugess in Uganda.
I think that, for some countries, a useful proxy would be avg life expectancy, and maybe HDI differences (though HDI is not a cardinal measure, I think differences could help assess differences in life chances—especially if one can adjust for inequality). I made a personal rough evaluation about helping a specific Haitian family this way. However, I don’t think this would extrapolate well for other countries (where HDI data is unreliable) or for people fleeing from persecution (where the counterfactual is not the average life, but just death); plus, in some cases (like NK and Afghanistan), it would be interesting to take into account political factors (point 4), but I have no idea about how to start to even begin to quantify this.
Saram is an organization in the area that I trust. You can for example get in touch with Nicolai Sprekels (nicolai.sprekels@saram-nk.org), who is an expert in the area.
I’m much less up-to-date here, and I don’t know this particular organization (afaik, there are several that help escapees), but the ones I looked into back in the day weren’t able to help people escape from NK but instead helped them flee from China to South Korea. The northern border is the one that most escapees manage/d to cross, but China tries to catch them and returns them to NK if they can’t escape from China before that happens.
If escapees leave families behind, the families often claim that the person has died. So they don’t suffer consequences at first, but if the person is then returned from China, it becomes clear that they lied, and the escapee and three generations of the family are sent to concentration camps or killed. So helping them to escape from China saves the whole family from that fate, which increases the cost-effectiveness estimate.
So if this is still true and true of this organization (Nicolai may know more), this flips the sign of points 2 and (separately) changes some other points subtly, because the organization doesn’t affect who or how many people escape in the first place.
Point 4 is probably a very small effect because of how few people escape compared to the general population, and how little power they have to organize or even move around in the country. The people who escape are probably mostly from the north because few people who are not from the north can travel there. Conversely, to influence something in the country politically, you’d probably have to be in the top caste and in Pyongyang.
Point 5 is potentially important. They may execute people in particularly public, gruesome ways in response to rescue attempts to deter foreign organizations.
Also, point 7 about other generations/descendants could be very different under the scenario you outline. You wouldn’t mainly be helping descendants be born outside of NK rather than inside, you’d be helping some be born outside NK instead of not born at all (the descendants of escapees), and others be born in NK instead of not born at all (the other descendants of the families left behind by escapees).
Yes, good point.
I’m generally delighted that you’re looking into this! I’ve previously mentioned this as an intervention that is potentially on par with GiveWell’s top charities, so if you’ve discovered it independently, it makes me even more optimistic about it.
Ok, if this is the case, then the number of people escaping from NK in the first place could be a limiting factor we’d run into quickly, if we don’t expect to influence this number much, but if we are increasing this number (people finding out about successful escapes), this may increase the risk of punishments (maybe not overall, though).
They might stop buying this if too many people are escaping and there’s no evidence of the person having died. I’d guess we shouldn’t expect due process from North Korea.
Yeah, I’m thinking about this at the current margin where deaths are probably some three orders of magnitude more than escapes, but those are problems that we’d run into if we were to scale this intervention a lot.
very speculative
Say you’re hit by a car tomorrow and die. An angel comes down, and they don’t quite offer you a second chance at life, they just offer you a day of life, with none of your current memories, as an average middle class person in South Korea.
Do you accept? I probably would, I expect the median South Korean to have a net-positive existence.
But here’s the catch: you also have to spend a day as an average political dissident in North Korea. Would you take that trade? I definitely would not. I think the disutility of the second scenario far outweighs the utility of the first.
So what would the ratio have to be? I.e How many good days in SK would you have to get in return to accept a single day living in NK? It’s hard to say without a better sense of the conditions in each play, but I would genuinely guess something like 10:1. In other words, putting very rough guesses on the utility of each scenario:
Middle class in South Korea: 10
Muzak and potatoes: 0
Political dissident in North Korea: −100
In this view, you’re not just “saving a life”, you’re preventing a huge amount of suffering.
I’m not sure how exactly this compares to GiveWell’s evaluations, or what degree of disutility they expect to prevent with interventions. Dying is bad, getting malaria and then dying is probably really horrible.
I’m not advocating running out and donating to LiNK for all the reasons mentioned by OP, but this is the chain of reasoning I would pursue more rigorously if I wanted to seriously evaluate their efficacy.
I tend to agree that helping NK refugees prevents suffering, and that we should really have some back-of-the-envelope calculation to measure it. (Usually, when I assess the value of helping a refugee, I consider HDI differences between countries as a proxy for the increase in wellbeing; but we can’t do this for NK because we can’t rely on what they publish—and even if we could I don’t think it could still work as a proxy for welfare in a totalitarian state.)
But I don’t know if you considered how this could extrapolate to population ethics. Your conclusion that NK lives are net negative (and that the modulo of their value is 10x greater than that of a SK life) seems to imply that killing (or letting die, if you have deontic objections) NK people is a net good—and that letting 1 NK citizen die produces 10x more welfare than saving a SK life. Or that moving 1 NK citizen to SK produces about .55x the welfare of letting 2 NK citizens die.
I believe NK people would likely disagree with this conclusion, even if they were not being coerced to do so.
I understand your argument is very speculative, but my overall take is that perhaps we should be extra careful when we apply negative cardinal utility measures to people—and that perhaps our own personal utility functions may not extrapolate very well to moral evaluations of the welfare of others.
Unrelated to NK, many people suffer immensely from terminal illnesses, but we still deny them the right to assisted suicide. For very good reasons, we have extremely strong biases against actively killing people, even when their lives are clearly net negative.
So yes, I think it’s plausible that many humans living in extreme poverty or under totalitarian regimes are experiencing extremely negative net utility, and under some ethical systems, that implies that it would be a net good to let them die.
That doesn’t mean we should promote policies that kill North Korean people or stop giving humanitarian food and medical aid.
I tend to agree that there are lives (human or not) not worth living, but my point is that it’s very difficult to consistently identify them by using my only own preference ordering. Saying “I’d rather die than live like that” is distinct from “this is worse than non-existence.” (I’m assuming we’re not taking into account externalities and opportunity costs. An adult male lion’s seems pretty comfortable and positive, but it entails huge costs for other animals) It’s even harder if you have to take into account the perspectives of the interested parties. For instance, in the example we’re discussing, SK people could also complain that your utility function implied that preventing one NK birth is equal to saving 10 SK lives. Even the implication that moving a NK person to SK is better than saving 10 SK lives is sort of implausible—for both NKs and SKs alike.
LINK argues that people who escape often send back money and information that can improve the life of their family and destabilise the regime
This makes the assumption that what comes after the destabilisation of the NK regime will be better outcomes for the citizens. While that might be the case, it’s definitely not certain. There are countless historical examples of regime change that caused more suffering for people. Iraq, Syria, Libya for recent examples.
That’s true, but on the other hand almost every country is currently better for its people than North Korea, so the bar is not high.
$13K per escapee is actually pretty impressive for a charity in the developed world. (If we could rescue all 25 million North Koreans for that low price, it would cost only 20% of one year of South Korea’s GDP! But of course we could not really do this… IRL there would be sharply diminishing marginal returns to attempting the rescue of many more people.)
I would expect that the biggest charitable gains related to North Korea would probably be found among “hits-based giving” strategies devoted to various tail-risk scenarios that could play out:
Working for the peaceful reunification of the peninsula under a benevolent, liberal government.
Preparing for the inevitable influx of refugees and economic shock (a la German unification) that would impact South Korea and China in the aftermath of anything from a best-case reunification to a total collapse of the North Korean government.
Trying to support resistance or reform efforts within North Korea.
Trying to lobby the North Korea policy of relevant powers: SK, USA, China, Russia, Japan, etc.
Preparing to mitigate the humanitarian catastrophe that would ensue if a hot war ever occurred (perhaps advocating that the South Korean government construct more civilian bomb shelters, stockpile more food and medical supplies, etc).
Lots of organizations in South Korea (including the government/military) probably already do work in many of these areas. But there are probably some scenarios that are relatively neglected, considering their probability & impact.
On the other hand, like the Israel/Palestine situation, North Korea is a famously intractable diplomatic puzzle, without any obvious no-brainer solutions. The actors with the most power to dictate the situation (North Korea itself, followed perhaps by China) are pretty opaque in their inner workings, and probably beyond the power of EA to easily influence. That leaves all the more-tractable options in a stance where they are basically watching, waiting, and hoping that they will be well-positioned to help when whatever happens, finally happens.
I think it’s worth exploring those other options further.
Two things to note, though:
The evidence base for such work is likely to be far worse (i.e. there would be little feedback and precedent to go on), and someone sufficiently skeptical might discount them based on priors. I think this is part of what favours GiveWell-recommended charities. That being said, I don’t expect RCTs for rescuing North Koreans, either, so I expect this to fall in-between in terms of strength of evidence.
I think similar arguments also suggest moving on from GiveWell’s recommended charities. This isn’t meant to undermine your comparison, but rather to point out that these kinds of comparisons would have implications for those who support GiveWell recommended charities, too.
Definitely; good point—I am sort of just deploying the generic longtermist argument, with all the advantages and drawbacks that brings.
I wonder if you could actually build up a bit of an evidence base from historical case studies?? There’s a long history of trying to help people escape oppressive regimes: slaves in the American south, Jewish people during WW2, East Berliners and other refugees of communism. It might be interesting to do a historical survey and try to analyze what interventions did the most good for people in those situations. (Spreading information, helping individual people escape, supporting internal resistance or reform, lobbying your own country’s policy towards the problematic regime, fighting the full-scale war.) Obviously the situations are very different from each other, but it might turn out helpful. Although for sheer cost-effectiveness, it might be tough for “helping people escape totalitarian regimes” to compete with “helping refugees from very poor and violent countries immigrate to rich and stable ones”, which seems like it ought to be cheaper to do at scale.
I looked into LINK earlier this year and had the vague impression that they are not funding constrained since newer security rules during COVID have added too many barriers to attempting rescue.
This makes sense. They only rescued 15 in 2020. They discuss COVID on this page:
Note that LINK is on every.org, so for a short time you can get a donation up to $100 matched (and double your cost-effectiveness estimate).
I’m so excited this is being investigated as an intervention. I had lived in South Korea for quite some time and did a lot of research on the welfare of North Korean refugees. The majority of refugees were women, and most women found they could make the most money as prostitutes. Other options were usually abysmally low paying staff in a restaurant. Discrimination against North Koreans in the south is a problem. They’re living a “safer” “more stable” life than in North Korea, with no firing squad or hard labor prisons for dissidents, but with prostitution as the most appealing option to survive is less than ideal to say the least. more assistance to find gainful employment and public education is necessary. There is progress but there are many bumps in the road still. The famous woman now living in Chicago, Yeonmi Park, is very very lucky and a rare success story.
The argument that allowing the most creative to leave is bad is often used in anti immigration arguments. I think it’s clearly wrong there. Those leaving send a strong signal to the regime, they send back money and they improve their own lives. I think the same is true here. I think it sends either no signal or a positive one that people can and want to leave and it’s good for those people.
I’ve updated the considerations since I misunderstood what they did. Denis Descher pointed out that they probably rescue North Koreans who have already escaped into China, and their website confirms this.
This is a great write-up, succinct but with all the relevant info. Thanks!
Less cost-effective: Those who flea North Korea are likely rebellious, anti-authoritarian, and brave. By accommodating these people in fleeing, are we reducing the chance of internal pressure in North Korea that could topple the regime and result in these benefits happening for the whole population?
If we include descendants as ‘more cost-effective’, then we should include them in the ‘less cost-effective’ column when calculating the possibility that someone who leaves, had they remained, could have had an impact on earlier freedom in North Korea.
Good points.
Do you mean that a larger NK population in general makes freedom more likely? It isn’t clear to me we should expect much effect at all here, since many will also support the regime due to indoctrination and incentives, although, I suppose more things in general will happen with a larger population? The descendants of people inclined to flee may be disproportionately likely to contribute to freedom there, too, though, due to the influence of their parents.
Sorry I wasn’t clear, I’m thinking on the spot. Say someone who leaves has an x chance of being the spark that starts a revolution within North Korea. Without this person, the chance of revolution has decreased, so it will take longer for it to potentially happen. Therefore, we should include the entire population of North Korea’s happiness plus their descendants to this calculation based on the increased unlikelihood of their freedom.
These probabilities are made up purely for example but say there’s a 0.01% chance any of these people could have helped start a movement towards freeing the entire population. They’ve helped 222 people escape so that’s (222*0.01)%. The population is 25.78m with an average growth rate of ~1.5%. Assume my made up probability above is correct then we’re reducing the chance of this group of people being free by 2% each year for the sake of 222 lives (plus their descendants). The question, I think, is if the possibility of any of these 222 people being a key figure in a potential revolution is high enough that in a population ratio of 1:116,000 it would be better for them to have remained in their country. It’d take many generations from the 222 people to equal the QALY increase of improving the odds of ~26m peoples freedom for a single year by 2%.
On a side note I don’t actually know if you can use probabilities of QALYs like this, so maybe my thinking is flawed. I’m sort of assuming 26m * p gives a rough calculation but as I write this I’m doubting myself.
I’m uncertain about this—I’ve heard people often send messages back, and I imagine a society where people regularly escape the country would be somewhat destabilising as well.
>I’m uncertain about this
So am I!
Reading more about Liberty in North Korea, here are some questions that I have which may be helpful for assessing the impact of a donation to LiNK:
Does Liberty in North Korea have room for more funding for more trips? If so, to what extent does marginal funding allow more people to travel to Southeast Asia with the help of LiNK?
Without Liberty in North Korea, what is the chance that a North Korean refugee would still be connected with someone to help them travel to Southeast Asia? If the trip costs something (as opposed to how LiNK rescues are free), how does this affect the quality of life for the refugee? Should we think of LiNK is helping people travel to Southeast Asia earlier or for free, rather than at all?
What is the chance of getting caught during the journey and getting sent to prison camp in North Korea? How long will they stay there? How many of their family members will be punished if they get caught?
By staying in China for an additional duration of time, what is the chance of getting caught and returned to North Korea?
What is the quality of life for a North Korean when in North Korea, inside a prison camp in North Korea, in China, in Southeast Asia, or in South Korea?
To what extent does offering free travel from China to Southeast Asia encourage additional people to try escaping from North Korea? What is the welfare impact for someone who attempts to escape North Korea?
I don’t at all understand point #7. Are you saying that people who unsuccessfully try to escape NK get sterilised?
The net effect is: there are fewer North Koreans within the country, but more North Korean diaspora? It seems very likely to me that having escaped North Koreans would do much more to damage the regime than having the average rebelliousness of the population go slightly down, since advocacy is driven by escapees (e.g. Yeonmi Park).
What is the relevance of population ethics? That slightly more future people will exist under this intervention?
I was thinking captured escapees (and possibly their families) would be imprisoned (or executed) and so have fewer children in expectation as a result. By rescuing them, you may allow people living in NK to have more children in NK.
I expect the NK diaspora to increase. It’s not clear what the effect is on the population of NK.
Yes, although it might not be a small effect; it could be more than the number of rescues, considering their descendants.