Focused on impact evaluation, economics, and (lately) animal welfare
geoffrey
Karma: 574
I wish I could strong-upvote this three times over. It’s that good of a piece.
This reads very clearly to me today, and I think younger less-knowledgeable-with-research-world me would follow it too.
It’s a legible example of
how research is increasingly a team effort with many behind-the-scenes people (with Principal Investigators arguably deserving little of the credit in some cases)
what work there is to be done adjacent to research
a theory of change (make it possible → … → make it required)
how movement building and grant making rely more on heuristics than explicit cost-effectiveness
Also cool was the flavor you gave different fields, and the benefits meta-science might have in each. (I broke out laughing reading the anecdote of an economist objecting to the title slide of a presentation.)
Minor typo:
Clinical Decision Support in Health Care
ack in the mid-2010s
This was a good nudge for me to lower the frequency on all my notifications (especially the karma one to weekly, which I’ve been checking more than I’d like lately)
Hi Nick,
Ah I missed only a third of Ugandans cooking with charcoal (I’m guessing a third of Ugandan households since that’s usually how these surveys work). That does suggest we can bump up the estimate by 3x.
I don’t think we can 5x the savings because of family size. Household savings go up (compared to my individual model) but so do household expenses. So the percent income gain from charcoal savings stays the same if both scale the same.
(Technical detail: I’m not following how you got back to 1000-1500 USD from my 0.6 USD per-adopter estimate. That’s about 1500-2500x bigger than what I had!)
Another point of ignorance is what the cooks are doing when not cooking. (Linch raised a related point in a sibling comment.) If someone’s home all day, has a reliable reserve of both beans and fuel, and cooks on a regular basis, then soaking sounds free. I’m sure these are all things you’ll find out while asking around though.
On a final meta note, I’m not sure if you want the benefits here to be as large as possible. You’ve mentioned calculating on the conservative end but it’s not obvious to me that larger benefits are always good
If the charcoal savings benefit is too large, the household would have realized it on their own. To exaggerate this, suppose households could double their income by soaking beans but still weren’t doing it. There’s likely a big obstacle preventing them from soaking beans that we’d have to figure out.
Put abstractly, importance goes up but tractability goes down with larger immediate benefits.
Surprised no one’s done the per-capita income comparison, since extra income from less charcoal usage would be a big selling point in an information campaign.
I did a very rough back-of-the-envelope calculation and estimated only 0.006% extra income via charcoal savings per year per adopter from soaking beans. I suspect that means lower tractability
If 1% of 50 million Ugandans adopt, we have 0.5 million adopters.
If 5-year savings for less charcoal used are 1.5 million USD, then annual savings are 0.3 million USD
So per-adopter savings (annually) is 0.6 USD.
And that seems low. Compare that against per-capita Uganda GDP of 1000 USD and we’re talking 0.006% extra income per year.
(Also glanced quickly at a few other indicators like median daily income, per capita GDP in PPP terms, and they seem ballpark similar)
To put that into a scale my first-world brain can understand, 0.006% over 100,000 USD is 60 USD. It’s definitely something but also feels low return for the habit change. And at that price, could easily see someone reverting back to cooking beans w/o soaking for the convenience.
Like this a lot, especially the plot designs.
Surprised there’s not much in demographic differences even with all the caveats that go with interpreting disparities there. Not sure what to make of that yet but will be chewing on that for a while.
Lastly, got a question. Do you have any sense of what a good baseline for the mental health section might be? The question of “has your mental health increased / decreased/ stayed the same since getting involved with X” is new to me
I think this falls into a broader class of behaviors I’d call aspirational inclusiveness.
I do think shifting the relative weight from welcoming to clear is good. But I’d frame it as a “yes and” kind of shift. The encouragement message should be followed up with a dose of hard numbers.
Something I’ve appreciated from a few applications is the hiring manager’s initial guess for how the process will turn out. Something like “Stage 1 has X people and our very tentative guess is future stages will go like this”.
Scenarios can also substitute in areas where numbers may be misleading or hard to obtain. I’ve gotten this from mentors before, like here’s what could happen if your new job goes great. Here’s what could happen if your new job goes badly. Here’s the stuff you can control and here’s the stuff you can’t control.
Something I’ve tried to practice in my advice is giving some ballpark number and reference class. I tell someone they should consider skilling up in hard area or pursuing competitive field, then I tell them I expect success in <5% of people I give the advice to, and then say you may still want to do it because of certain reasons
Yes, it’s all very noisy. But numbers seem far far better than expecting applicants to read between the lines on what a heartwarming message is supposed to mean, especially early-career folks who would understandably assign a high probability of success with it
Congrats on passing your first year. Some off-the-cuff thoughts from someone still trying to get to where you are and spent a lot of thing on EconTwitter figuring stuff out:
I read online that 2nd econ phd year is usually the best year. Field courses I hear are a treat. If you’re not feeling that year, it’s unlikely the rest of the academic path will sustain your interest.
I think year 3 is where you can possibly have your first test bet, since it’s the first solo year. I also hear it’s the easiest to slack off in. A recurring joke I’ve seen among faculty tweets is “we still don’t know what our students do in their 3rd year, but we know it takes them a year”
I recall a Twitter poll that listed econ PhD year 1 as the highest work hours per week (compared to 2nd year, 3rd year, 4/5th year, job market year, tenure track, tenured). If the 5 days on 2 days off is what you miss, I suspect you’ll swing partly back towards that, which may help.
The wealth of peers with similar interests is a big reason why I’d considering PhDs. I do think you’ll have trouble finding that outside the PhD. That said, you’ll still have those connections and if EconTwitter doesn’t blow up, you can have that as kinda surrogate connection feeling.
I’d separate working on treatment spillovers and growth. Treatment spillovers feels like a measurement and impact evaluation thing, which does seem very academic-research-loaded. But growth is the outcome and working on growth can mean many things (which is one of my frustrations when people say we should work on economic growth. It as specific as saying you want to work on non-global-health systemic change).
In the real world, very few roles work on growth theory or policies for aggregate growth directly. It’s usually some piece, like infrastructure or corruption or business liberalization or so forth. Some might have a growth model as their motivation, but for most people the motivation is as simple as “this is a big problem with big upside, let’s work on it”. If that more growth-adjacent less-evaluation-rigor stuff appeals to you, the PhD is less crucial.
As an aside, to my knowledge, most trade econ doesn’t work on country-level growth directly or stuff relating to that. That seems to result from the history of trade methodology, where it’s very micro focused. There is stuff on firm productivity growth in the “new new trade theory” (look up Melitz) though it’s unclear how that aggregates up into broader welfare gains. I do know there’s a macro-trade / computable trade niche of stuff going on, but I’m less familiar with those, and that community feels like it’s closer to macro than trade.
I’ve found a lot of professional overlap between groups focused on global health and groups focused on LMIC growth (low-and-middle-income country growth). Each group tends to be the biggest audience and best critic of the other approach.
I’m not sure if breaking out the topic would incentivize more attention of LMIC growth, but I do worry we’d lose some interesting discussion.
Try “managing up” with a simple text document during meetings.
I’m the main contributor on a project with a light management layer. The autonomy’ s nice. But it’s given a lot of space for stakeholders to spend check-ins talking about their long term wish list (which is fun for them) while avoiding the prioritization I need them to do.
Recently, I started bringing a text document into check-ins on my understanding on what the priorities, editing it as the meeting goes, and assigning items as (In progress), (todo), or (nice-to-have). It’s Kanban in spirit but without the overhead of actually running Trello / Jira/ Notion.
Here’s some extra (low confidence) info regarding financial aid:
Some programs will have a link on their website where you can talk to a program coordinator or admissions officer. Talking to this person before you apply and forming a good impression may help you secure a much nicer financial aid package when you get accepted. This varies by school and it’s luck-of-the-draw whether you hit it off with someone. Generally, you should be genuine and approach the meeting with curiosity over topics like when certain faculty teach. But that person may have info on how to best negotiate financial aid or they may even be able to champion the financial aid package for you internally.Diversity-based aid is extremely rare and I don’t know anyone who’s personally gotten it. But I think if you have a especially rough hardship story or rare background for your field (the kind that warrants a local newspaper article or university profile) AND you’re a good student, you’re likely to get some money for that.
Finally, if you’re price-sensitive and willing to take some risk, newer Master’s programs and/or lower-tier programs tend to offer generous packages because they’re trying to establish their brands. I wouldn’t recommend this, but if you just want the credential, this does get you through at a lower price. Or if you know and trust one of the program coordinators / faculty personally, maybe you could take a risk at a newer program.
Hi Caspar,
Thanks for the response. On second thought, my objection might be different than what I initially suggested. I do think the test of overlap of scales as you mentioned would be an interesting test to run, but it doesn’t seem to be capturing the overlap I ultimately care about.
Maybe this comment can captures my complaint better. We don’t have any access to what “the most/least satisfied that any human could possibly be”. We don’t even have access to “the most/least satisfied you personally think you could become”.
As a personal example, I would take most of my worst post-therapy days over most of my best pre-therapy days. Younger me has no access to realizing how much satisfied I could be with life, or even how broadly people are in general.
I might be using the language wrong, but I think I’m hinting at differences in the latent scale of well-being or satisfaction… which doesn’t feel like it’s knowable.
I enjoyed this a lot. I’ve been meaning to delve into well-being measurement and this was a nice entry-point into the field.
One thing I’m not clear on is whether vignette anchors (or any of the comparability methods) can correct for non-overlapping well-being scales. You talked about an example like this:
But I’m more interested in examples like this:
Measuring these larger SWB (subjective well-being) differences seems crucial for detecting interpersonal differences across societies and picking up on how intense pain / pleasure can be at the long tails. The non-overlap area seems like it can get extremely big.
Hi Niki, glad to hear it helped. Here’s some more thoughts. Can’t promise they’re any good.
Yes, I agree the consumption smoothing point is critical. I could have worded my answer a bit better. What I meant to say is that rural households are good at trying to smooth consumption given their situation. That can still be a low overall ability given how sporadic income can be. The crux, I suppose, is whether we trust the households to smooth their own consumption or if we should make the decision for them. If we think the households are better able to make the decision, we send the transfer now. If we think they’re worse (perhaps for lack of willpower reasons), then we delay.From the dataset I linked you, it’s tough to decompose the idiosyncratic shock from just family-level income / consumption / wealth data. You’d want to correct for village-level shocks and seasonality and so on. I believe it’s do-able but noisy. And it’s usually cleaner to simply ask the families whether they suffered from X adverse event in the past 12 months (which the India Human Development Survey does ask).
To do this more cleanly, you’d want a dataset that interviews each family multiple times in a single year. Then you could see the variability in income / consumption / wealth over time per family.
I’m not familiar with how GiveDirectly does their operations, but I’d be surprised if the communication was frequent enough that GiveDirectly staff could observe and react to every family illness in real-time.
On systemic shocks, you might be right. I was envisioning a bad natural disaster where the logistics of shipping more food out to villages is temporarily clogged up. But perhaps NGOs are more crafty that I think.
(Some quick thoughts hastily written based off some class papers I wrote a while back.)
One dataset that pops to mind is the India Human Development Survey. This is a rich household-level dataset that includes total household monthly income (disaggregated by source) and if I recall right, also tells you what month it is. These are time-intensive to work with, but I imagine a few others datasets like this exist in the world. And you can estimate “income” per month with them.
My guess is you’ll get obvious insights from this, like income dropping during cold / dry seasons in more agricultural-dependent villages.
That said, my gut feeling with the policy question is that sending cash transfers sooner is better. A few reasons:The book Portfolios of the Poor suggests rural households, especially poor ones, are good at managing their own finances and spreading out their resources over time (or consumption smoothing as economists call it).
Household debt (not captured in income) may accumulate interest, and interest rates can be exorbitantly high.
Idiosyncratic shocks (like a family illness) are hard to surveil and predict from afar. So it’s hard to implement strategic delays
Extreme systemic shocks like very harsh droughts / floods may temporarily constrain food / fuel supply reducing effectiveness of cash transfers at that time.
Households may want to stock up on durable foods like grains or oils ahead of time
Counterintuitively, giving money while households have high-income may push them over a wealth-threshold that lets them make durable investments (roofs, goats, farm equipment). This may be welfare-enhancing in the long run. I think this goes by the “lump-sum” effect in the cash transfers literature.
Hope this helps. Happy to chat about this more.
This is great stuff. I appreciate you posting some initial results quickly, being careful about what claims you can make right now, signposting what you’ll investigate later, and being explicit about what updates you’ve made.
I’ll also echo Lily’s comment about dis-aggregating POC. I’d be interested to see POCs breakdown between countries / regions of the world. For example, being a Chinese-American and being a Chinese national are different things.
Noted! Sorry for the misinterpretation.
One concrete idea could be an article centered on “class migrants”. Perhaps it could be similar to the format of the anonymous interview series, or it could be like the imposter syndrome article where there’s one personal profile and a few mini-profiles attached.
Partly, this is to help people feel less alone. But also, I think the strategy for developing your career differs based on where you’re starting. Even between colleges, there’s variation.
Beyond that, I’m not sure. I get that 80k’s target audience is different from me and messaging is hard. So I’m hesitant to recommend huge strategic changes in content.
I never felt excluded either. And 80k does a lot of things right on this front. The messages of ambition and “here’s some broken stuff why don’t you go fix it” are good and certainly have pushed me to do things I wouldn’t have done otherwise. I genuinely feel people from underprivileged backgrounds need to hear more of it and I try to promote them as much as I can.
>I just had to discount or not read any sections that talked about status, top universities, etc., kind of assumed I’d have to write my own theory of change and have a thick skin about not always being taken seriously.
I think this is where our experiences diverge. I also discounted these sections… by charging straight ahead into applying for my dream jobs and thinking I had the similar chances to Ivy League graduates. But (1) college brand name does matter if comparing two people with no experience, (2) the Ivy League graduate probably had more opportunities, networks and internships than me, and (3) many of my dream jobs were magnitudes more competitive than I initially thought.
Maybe 80k advice worked too well and I flew too close to the sun? I don’t know. Messaging is tricky.
>I’d be more excited about 80k targeting the frustration people older than 25 or 35 have with 80k, that seems more useful than thinking deeply about class.
Agreed. To be clear, I’m not endorsing thinking deeply about class. The “business case” for class diversity doesn’t seem great to me. But I saw “demographic diversity” mentioned in the original post and I was wondering if class was included in that.
Do you have plans for increasing class diversity via 80k career advice / tailoring advice to those with less resources? If so, what are some strategies you have?
I’ve loved 80k career advice and have benefited a ton from it. But one frustration I’ve had (especially earlier in my career) is that it doesn’t offer much advice for people starting with less resources. For example, non-profit jobs can be out-of-reach without relevant / outstanding credentials or money to do a Masters degree if moving into policy.
I also suspect there’s working-class cultural factors some need to un-learn (this was true for me). Manual labor tends to reward putting your head down and doing the hard work. But professional + managerial jobs reward creativity, relationships, and questioning systems.
Adding some thoughts regarding diversity, privilege, and inclusiveness, as someone who was on the fence about applying and going last year and also about interacting with the global priorities community in general.
Like others said, I attended and loved this course last year. I think the value in this course is higher if you’re from an underprivileged background or if you’re a “big fish in a small pond” at a solid but non-elite university.Mainly, it’s because you’ll get to hang out with other strong students across a range of contexts. You attend rigorous lectures, solve problem sets, and socialize with them. Last year, this included talking to PhD students at top universities. Particularly for potential PhD applicants, it’s a very good way to test your fit and seeing what your weaknesses / strengths are relative to other people. And for me, it was a good way to remove some of the idolization I had of other people.
For some, this means realizing their initial career path is arduous but interesting and ultimately doable. For others, this means realizing “Oh wow this is too much and not that useful to me. I need to rethink my career path.” Both are useful updates.
Also, as a general vibe, I found that GPI as a whole talks about diversity, equity and inclusiveness in a very thought out and genuine way. This doesn’t entirely offset the very privileged demographics that econ + philosophy pulls from, but I think it should nudge people towards applying if on-the-fence.
Hi Ozzie, what’s the ask and intended audience here?
The problems here seem interesting and maybe even approachable to a dedicated newbie. So I was wondering if my background was any use. I used to work in software engineering with JavaScript and now work in data cleaning research assistance with R.
But I can’t tell if this is an open call to submit pull requests to an open source library on GitHub, a request for advice, or a request for someone to work part-time / full-time