In DC. Focused on development, economics, trade, and DEI.
geoffrey
Karma: 365
Writing About My Job(s): Research Assistant at World Bank / IMF
Flagging quickly that ProbablyGood seems to have moved into this niche. Unsure exactly how their strategy differs from 80k hours but their career profiles do seem more animals and global health focused
I think they’re funded by similar sources to 80k https://probablygood.org/career-profiles/
This is great stuff. I appreciate you posting some initial results quickly, being careful about what claims you can make right now, signposting what you’ll investigate later, and being explicit about what updates you’ve made.
I’ll also echo Lily’s comment about dis-aggregating POC. I’d be interested to see POCs breakdown between countries / regions of the world. For example, being a Chinese-American and being a Chinese national are different things.
I think this falls into a broader class of behaviors I’d call aspirational inclusiveness.
I do think shifting the relative weight from welcoming to clear is good. But I’d frame it as a “yes and” kind of shift. The encouragement message should be followed up with a dose of hard numbers.
Something I’ve appreciated from a few applications is the hiring manager’s initial guess for how the process will turn out. Something like “Stage 1 has X people and our very tentative guess is future stages will go like this”.
Scenarios can also substitute in areas where numbers may be misleading or hard to obtain. I’ve gotten this from mentors before, like here’s what could happen if your new job goes great. Here’s what could happen if your new job goes badly. Here’s the stuff you can control and here’s the stuff you can’t control.
Something I’ve tried to practice in my advice is giving some ballpark number and reference class. I tell someone they should consider skilling up in hard area or pursuing competitive field, then I tell them I expect success in <5% of people I give the advice to, and then say you may still want to do it because of certain reasons
Yes, it’s all very noisy. But numbers seem far far better than expecting applicants to read between the lines on what a heartwarming message is supposed to mean, especially early-career folks who would understandably assign a high probability of success with it
The candidate pool was much stronger than expected
This one can be sent to every applicant and still provides very useful information. It tells me that my expectations of the hiring bar might have been correct in the past. However, the market has changed and I should adjust my expectations.
For this one, concreteness is essential. One hiring manager phrased it like, “We had to reject many exceptional candidates that would have been instant hires a few years ago. Everyone did well on our take-home test that we thought impossible to complete within the 3 hours.”
Literally any feedback about final stage interviews.
I worry a ton about my final-stage interview performance. Partly this is a me-issue but I think there are structural reasons why final-stage interviews are so nerve-wracking.
They’re the most important to do well in. I can be a marginal candidate in every stage before that. The 20th best resume can still get a HR phone screen. The 10th best HR phone screen can still get a work trial. The 5th best work trial can still get a final interview. But only the top 1-3 candidates in a final interview can realistically expect an offer.
They’re the type of interview I have the least experience with. By definition, final stage interviews are at the end of the funnel so I’m going to have a lot less of them.
They’re oftentimes my first chance to interact with my potential coworkers and managers. And unless I already have contacts in that organization, I won’t know the professional norms or idiosyncratic expectations. This criteria is usually implicit and hard to figure out on my own.
One piece of feedback I really liked went like, “Your interview was very good and I have no doubt you could learn the skills very quickly. We just had someone else who had already done the work.”
Adding some thoughts regarding diversity, privilege, and inclusiveness, as someone who was on the fence about applying and going last year and also about interacting with the global priorities community in general.
Like others said, I attended and loved this course last year. I think the value in this course is higher if you’re from an underprivileged background or if you’re a “big fish in a small pond” at a solid but non-elite university.Mainly, it’s because you’ll get to hang out with other strong students across a range of contexts. You attend rigorous lectures, solve problem sets, and socialize with them. Last year, this included talking to PhD students at top universities. Particularly for potential PhD applicants, it’s a very good way to test your fit and seeing what your weaknesses / strengths are relative to other people. And for me, it was a good way to remove some of the idolization I had of other people.
For some, this means realizing their initial career path is arduous but interesting and ultimately doable. For others, this means realizing “Oh wow this is too much and not that useful to me. I need to rethink my career path.” Both are useful updates.
Also, as a general vibe, I found that GPI as a whole talks about diversity, equity and inclusiveness in a very thought out and genuine way. This doesn’t entirely offset the very privileged demographics that econ + philosophy pulls from, but I think it should nudge people towards applying if on-the-fence.
(Speaking as a Software Engineer with 3 years professional experience, and a casual EA organizer in Washington DC for 1.5 years)
It’s worth thinking about how technical a community-building role is. My impression of tech evangelist roles is that companies want your engineering skills to be top-notch, but are less strict about your community organizing skills. In that case, EA organizing would make sense as a side project but not as much sense as a full-time job.
The other caveat that comes to mind is whether EA offers opportunities for large-scale organizing. I can’t think of that many EA groups with >50 regulars. But I can easily think of few tech meetups, local political chapters, and social groups with >100 regulars and enough engagement to have active Slack channels + multiple ongoing projects.Edit: I no longer endorse this. EA DC got its first paid part-time organizer a few months ago (Q2 2021 if memory serves me right) and we definitely have >50 regulars now. I also suspect we still have considerable room to grow
Has there been a past success story where a drug was developed to mimic the effects of a gene and successfully improved a complicated phenomena (in this case, sleep)?
I’m unfamiliar with drug development, but my limited knowledge of genetics and sleep suggests this would be complicated. A past success story would sway my mind a little bit.
Do you have plans for increasing class diversity via 80k career advice / tailoring advice to those with less resources? If so, what are some strategies you have?
I’ve loved 80k career advice and have benefited a ton from it. But one frustration I’ve had (especially earlier in my career) is that it doesn’t offer much advice for people starting with less resources. For example, non-profit jobs can be out-of-reach without relevant / outstanding credentials or money to do a Masters degree if moving into policy.
I also suspect there’s working-class cultural factors some need to un-learn (this was true for me). Manual labor tends to reward putting your head down and doing the hard work. But professional + managerial jobs reward creativity, relationships, and questioning systems.
Quick thought on the tangent, which I’d also love to hear more thoughts on from other people.
I’m skeptical that corruption is a big obstacle to growth and development. Measurement and historical comparisons are tricky here, but corruption seems to be a pervasive feature across many societies.
Even the United States had its local political machines and share of bribery before the Progressive Movement in the 1920s tried to filter it out. And conventional wisdom credits the Industrial Revolution (of the 19th century before the US reduced its corruption) with our modern wealth.
I suspect if we applied our same concern of corruption to currently-developed countries to their past, we’d find they (1) would fare just as bad and (2) had their development periods before they dealt with the corruption
I live in DC and attended an econ Master’s program that places some of its graduates in the International Monetary Fund. I don’t think your decision matters too much, and I think you should use teaching quality, and your own interest as a tie-breaker. With the exception of Numerical Analysis, none of these classes sound particularly international-focused or standardized so I would guess they all look about the same to an employer.
I’d lean towards dropping Economics of Inequality. It sounds way too general and I’m not even sure what someone would learn from that class just by by seeing the title on a resume. That said, I do want to emphasize that if you should still take it if the curriculum or professor appeals to you.
The skill set of Numerical Analysis may be slightly useful in getting you a job that’s working on macro modeling (since macro models rely on numerical approximation to help simulate multiple sectors). But I think these are mostly for technical roles at macro institutions. I suspect it’s much less useful at think tanks. Even within macro institutions, it’s not most teams working on it and even then, most early-career people can get offers for these roles with just econometrics and econ theory classes (me being one example).
Social Development and Statistical Methods signal interest in international institutions and Excel data analysis respectively. They both seem good, but you’d have to help your resume readers out by specifying Social Development (thru multilaterals) or something like you did in the post above.
In terms of classes you should look out for:
For statistics, I’d rate econometric classes quite highly. Take the highest-level undergrad econometrics class you feel you can do well in. If your school has macro-themed electives like time-series econometrics, it may help to take one. These skills also generalize really well even if you leave research entirely.
For econ theory, I would see if your school has a class that uses the Krugaman, Obstfeld, & Melitz textbook on international economics. Class titles may go by International Finance (for the first half of the textbook), International Trade (for the second half), or International Economics (for a less detailed look at the whole). Personally, I found this textbook dense (especially the finance part) and I’m not sure if it’s something I could pick up on my own. So a class here is definitely useful.
For institutional knowledge, I would highly rate classes if you have professors who worked at think tanks or multilaterals in the past AND they are teaching on a related topic.
I wish I could strong-upvote this three times over. It’s that good of a piece.
This reads very clearly to me today, and I think younger less-knowledgeable-with-research-world me would follow it too.
It’s a legible example of
how research is increasingly a team effort with many behind-the-scenes people (with Principal Investigators arguably deserving little of the credit in some cases)
what work there is to be done adjacent to research
a theory of change (make it possible → … → make it required)
how movement building and grant making rely more on heuristics than explicit cost-effectiveness
Also cool was the flavor you gave different fields, and the benefits meta-science might have in each. (I broke out laughing reading the anecdote of an economist objecting to the title slide of a presentation.)
Minor typo:
Clinical Decision Support in Health Care
ack in the mid-2010s
I liked this a lot. For context, I work as a RA on an impact evaluation project. I have light interests / familiarity with meta-analysis + machine learning, but I did not know what surrogate indices were going into the paper. Some comments below, roughly in order of importance:
Unclear contribution. I feel there’s 3 contributions here: (1) an application of surrogate method to long-term development RCTs, (2) a graduate-level intro to the surrogate method, and (3) a new M-Lasso method which I mostly ignored. I read the paper mostly for the first 2 contributions, so I was surprised to find out that the novel contribution was actually M-Lasso
Missing relevance for “Very Long-Run” Outcomes. Given the mission of Global Priorities Institute, I was thinking throughout how the surrogate method would work when predicting outcomes on a 100-year horizon or 1000-year horizon. Long-run RCTs will get you around the 10-year mark. But presumably, one could apply this technique to some historical econ studies with (I would assume) shaky foundations.
Intuition and layout is good. I followed a lot of this pretty well despite not knowing the fiddly mechanics of many methods. And I had a good idea on what insight I would gain if I dived into the details in each section. It’s also great that the paper led with a graph diagram and progressed from simple kitchen sink regression before going into the black box ML methods.
Estimator properties could use more clarity.
Unsure what “negative bias” is. I don’t know if the “negative bias” in surrogate index is an empirical result arising from this application, or a theoretical result where the estimator is biased in a negative direction. I’m also unsure if this is attenuation (biasing towards 0) or a honest-to-god negative bias. The paper sometimes mentions attenuation and other times negative bias but as far as I can tell, there’s one surrogacy technique used
Is surrogate index biased and inconsistent? Maybe machine learning sees this differently, but I think of estimators as ideally being unbiased and consistent (i.e. consistent meaning more probability mass around the true value as sample size tends to infinity). I get that the surrogate index has a bias of some kind, but I’m unclear on if there’s also the asymptotic property of consistency. And at some point, a limit is mentioned but not what it’s a limit with respect to (larger sample size within each trial is my guess, but I’m not sure)
How would null effects perform? I might be wrong about this but I think normalization of standard errors wouldn’t work if treatment effects are 0...
Got confused on relation between Prentice criterion and regular unconfoundedness. Maybe this is something I just have to sit down and learn one day, but I initially read Prentice criterion as a standard econometric assumption of exogeneity. But then the theory section mentions Prentice criterion (Assumption 3) as distinct from unconfoundedness (Assumption 1). It is good the assumptions are spelt are since that pointed out a bad assumption I was working with but perhaps this can be clarified.
Analogy to Instrumental Variables / mediators could use a bit more emphasis. The econometric section (lit review?) buries this analogy towards the end. I’m glad it’s mentioned since it clarifies the first-stage vibes I was getting through the theory section, but I feel it’s (1) possibly a good hook to lead the the theory section and (2) something worth discussing a bit more
Could expand Table 1 with summary counts on outcomes per treatment. 9 RCTs sounds tiny, until I remember that these have giant sample sizes, multiple outcomes, and multiple possible surrogates. A summary table of sample size, outcomes, and surrogates used might give a bit more heft to what’s forming the estimates.
Other stuff I really liked
The “selection bias” in long-term RCTs is cool. I like the paragraph discussing how these results are biased by what gets a long-term RCT. Perhaps it’s good emphasizing this as a limitation in the intro or perhaps it’s a good follow-on paper. Another idea is how surrogates would perform in dynamic effects that grow over time. Urban investments, for example, might have no effect until agglomeration kicks in.
The surprising result of surrogates being more precise than actual RCTs outcomes. This was a pretty good hook for me but I could have easily passed over in in the intro. I also think the result here captures the core intuition of bias-variance tradeoff + surrogate assumption in the paper quite strongly.
I’ve read conflicting things about how individual contributor skills (writing the code) and people management skills relate to one another in programming.
Hacker News and the cscareerquestions subreddit give me the impression that they’re very separate, with many complaining about how advancement dries up on a non-management track.
But I’ve also read a few blog posts (which I can’t recall) arguing the most successful tech managers / coders switch between the two, so that they keep their technical skills fresh and know how their work fits in a greater whole.
What’s your take in this? Has it changed since starting your new job?
Do you think your area is more talent-constrained or cash-constrained? How about your particular role? Read this in whatever way makes sense
Project-based learning seems to be a underappreciated bottleneck for building career capital in public policy and non-profits. By projects, I mean subjective problems like writing policy briefs, delivering research insights, lobbying for political change, or running community events. These have subtle domain-specific tradeoffs without a clean answer. (See the Project Work section in On-Ramps Into Biosecurity)
Thus the lessons can’t be easily generalized or made legible the way a math problem can be. With projects, even the very first step of identifying a good problem is tough. Without access to a formal network, you can spend weeks on a dead end only realizing your mistakes months or years after the fact.
This constraint seems well-known for professionals in the network, as organizers for research fellowships like SERI Mats describe their program as valuable, highly in-demand, yet constrained in how many people they can train.
I think operations best shows the surprising importance of domain-specific knowledge. The skill set looks similar across fields. So that would imply some exchange-ability between private sector and social sector. But in practice, organizations want you to know their specific mission very well and they’re willing (correctly or incorrectly) to hire a young Research Assistant over, say, someone with 10 years of experience in a Fortune 500 company. That domain knowledge helps you internalize the organization’s trade-offs and prioritize without using too much senior management time.
Emphasizing this supervised project-based learning mechanism of getting domain-specific career capital would clarify a few points.
With school, it would
emphasize that textbook-knowledge is both necessary yet insufficient for contributing to social sector work
show the benefits of STEM electives and liberal arts fields, where the material is easier from a technical standpoint but you work on open-ended problems
illustrate how research-based Master degrees in Europe tend to be better training than purely coursework-based ones in the US (IMHO, true in Economics)
With young professionals, it would
highlight the “Hollywood big break” element of getting a social sector job, where it’s easier to develop your career capital after you get your target job and get feedback on what to work on (and probably not as important before that)
formalize the intuition some people have about “assistant roles in effective organizations” being very valuable even though you’re not developing many hard skills
With discussions on elitism and privilege, it would
give a reason for the two-tier system many social sectors seem to have, where the stable jobs require years of low-paid experience and financially unstable training opportunities require significant sacrifice to even access
perhaps inform some informational interventions like books highlighting the hidden curriculum in executing projects or communicating with stakeholders (Doing Economics: What You Should Have Learned in Grad School But Didn’t or The Unspoken Rules: Secrets to Starting Your Career Off Right)
I always read therapeutic alliance as advice for the patient, where one should try many therapists before finding one that fits. I imagine therapists are already putting a lot of effort on the alliance front
Perhaps an intervention could be an information campaign to tell patients more about this? I feel it’s not well known or to obvious that you can (1) tell your therapist their approach isn’t working and (2) switch around a ton before potentially finding a fit
I haven’t looked much into it though
EA organizations often have to make assumptions about how long a policy intervention matters in calculating cost-effectiveness. Typically people assume that passing a policy is equivalent to having it in place for around five years more or moving the start date of the policy forward by around five years.
I am really really surprised 5 years is the typical assumption. My conservative guess would have been ~30 years persistence on average for a “referendum-sized” policy change.
Related, I’m surprised this paper is a big update for some people. I suppose that attests to the power of empirical work, however uncertain, for illuminating the discussion on big picture questions.
Surprised no one’s done the per-capita income comparison, since extra income from less charcoal usage would be a big selling point in an information campaign.
I did a very rough back-of-the-envelope calculation and estimated only 0.006% extra income via charcoal savings per year per adopter from soaking beans. I suspect that means lower tractability
If 1% of 50 million Ugandans adopt, we have 0.5 million adopters.
If 5-year savings for less charcoal used are 1.5 million USD, then annual savings are 0.3 million USD
So per-adopter savings (annually) is 0.6 USD.
And that seems low. Compare that against per-capita Uganda GDP of 1000 USD and we’re talking 0.006% extra income per year.
(Also glanced quickly at a few other indicators like median daily income, per capita GDP in PPP terms, and they seem ballpark similar)
To put that into a scale my first-world brain can understand, 0.006% over 100,000 USD is 60 USD. It’s definitely something but also feels low return for the habit change. And at that price, could easily see someone reverting back to cooking beans w/o soaking for the convenience.
This post resonated a lot with me. I was actually thinking of the term ‘disillusionment’ to describe my own life a few days before reading this.
One cautionary tale I’d offer to readers is don’t automatically assume your disillusionment is because of EA and consider the possibility that your disillusionment is a personal problem. Helen suggested leaning into feelings of doubt or assuming the movement is making mistakes. That is good if EA is the main cause, but potentially harmful if the person gets disillusioned in general.
I’m a case study for this. For the past decade, I’ve been attracted to demanding circles. First it was social justice groups and their infinitely long list of injustices. Then it was EA and its ongoing moral catastrophes. More recently, it’s been academic econ debates and their ever growing standards for what counts as truth.
In each instance, I found ways to become disillusioned and to blame my disillusionment on an external cause. Sometimes it was virtue signaling. Sometimes it was elitism. Sometimes it was the people. Sometimes it was whether truth was knowable. Sometimes it was another thing entirely. All my reasons felt incredibly compelling at the time, and perhaps they all had significant degrees of truth.
But at the end of day, the common denominator in my disillusionment was me. I felt all the problems in these circles very intensely, but didn’t have much appreciation for the benefits. The problems loomed 10x larger in my head than the benefits did. Instead of appreciating all the important things I got to think about, talk about, or work on, I thought about the demands and all the stress it brought. In my case, leaning into the disillusionment would only perpetuate the negative pattern of thinking I have in my head.
Granted, my case is an extreme one. I have a decade of experiences to look back on and numerous groups I felt affinities with. And I’ve had intense experiences with imposter syndrome and performance anxiety. I can confidently attribute most of these feelings to myself, reverse some of the advice Helen offered,[1] and lean away from disillusionment.
But I suspect I’m not the only one with this problem. EA seems to selects for easily disillusioned personality traits (as evidenced by our love of criticism). And I also suspect that these feelings are common for young idealistic people to go through while navigating what it means to improve the world. Not everyone should be leaning into that.
I’m practicing “maintain and/or build ties outside EA”. It requires intentional effort on my part since making + maintaining adult friends is always hard. However, it has helped me realize my disillusionment still exists outside EA. I’m partly reversing “anticipate and lean into feelings of doubt...” since I like the anticipating part, but not the leaning part. I’m reversing “assume EA is making mistakes and help find them” since I need to see more of the positive and less of the negative. I don’t have any thoughts on “defer cautiously, not wholesale” since this comes naturally to me.