Research Fellow at Open Philanthropy. Former Quantitative Researcher on the Worldview Investigations Team at Rethink Priorities. Completed a PhD at the Paris School of Economics.
David Bernard
Karma: 763
The tag seems focused on how much weight should be assigned to different moral patients. But some people and posts use the phrase moral weight to refer to relative importance of different outcomes, e.g. how much should we care about consumption vs saving a life? Examples include:
Should we include both under this wiki-tag and broaden the definition? Or should we make a new tag and disambiguate between the two?
This paper was a chapter in the book Randomized Control Trials in the Field of Development: A Critical Perspective, a collection of articles on RCTs. Assuming the author of this chapter, Timothy Ogden doesn’t identify as a randomista, the only other author who maybe does is Jonathan Morduch, so it’s a pretty one-sided book (which isn’t necessarily a problem, just something to be aware of).
There was a launch event for the book with talks from Sir Angus Deaton, Agnès Labrousse, Jonathan Morduch, Lant Pritchett and moderated by William Easterly, which you might find interesting if you enjoyed this post.
Thanks for the post, but I don’t think you can conclude from your analysis that your criteria weren’t helpful and the result is not necessarily that surprising.
If you look at professional NBA basketball players, there’s not much of a correlation between how tall a basketball player is and how much they get paid or some other measure of how good they are. Does this mean NBA teams are making a mistake by choosing tall basketball players? Of course not!
The mistake your analysis is making is called ‘selecting on the dependent variable’ or ‘collider bias’. You are looking at the correlation between two variables (interview score and engagement) in a specific subpopulation, the subpopulation that scored highly in interview score. However, that specific subpopulation correlation may not be representative of the correlation between interview score and engagement in the broader relevant population i.e., all students who applied to the fellowship. This is related to David Moss’s comment on range restrictions.
The correlation in the population is the thing you care about, not the correlation in your subpopulation. You want to know whether the scores are helpful for selecting people into or out of the fellowship. For this, you need to know about engagement of people not in the fellowship as well as people in the fellowship.
This sort of thing comes up all the time, like in the basketball case. Another common example with a clear analogy to your case is grad school admissions. For admitted students, GRE scores are (usually) not predictive of success. Does that mean schools shouldn’t select students based on GRE? Only if the relationship between success and GPA for admitted students is representative of the relationship for unadmitted students, which is unlikely to be the case.
The simplest thing you could do to improve this would be to measure engagement for all the people who applied (or who you interviewed if you only have scores for them) and then re-estimate the correlation on the full sample, rather than the selected subsample. This will provide a better answer to your question of whether scores are predictive of engagement. It seems like the things included in your engagement measure are pretty easy to observe so this should be easy to do. However, a lot of them are explicitly linked to participation in the fellowship which biases it towards fellows somewhat, so if you could construct an alternative engagement measure which doesn’t include these, that would likely be better.
Thanks for this Luisa, I found it very interesting and appreciated the level of detail in the different cases. One thought and related questions that came up when reading the toy calculations at the end of each case:
For a fixed number of survivors, there is a trade-off between groups of different sizes. The larger the groups, the more likely each group is to survive, but the fewer groups need to be wiped out in order for humanity to go extinct.
What might this trade-off look like and is there some optimal group size to minimise the risk of extinction?
What are the game theoretic considerations of individuals forming groups of varying sizes and how do these vary depending on the extent to which people care about their own individual survival and human extinction?
What group sizes might we expect in practice and is there anything we could do to influence group sizes in the event of a catastrophe?
Given the low likelihood of extinction you suggest, I think these are relatively low priority questions but could be potentially interesting for someone to look at in more detail.
I’m happy to see an increase in the number of temporary visiting researcher positions at various EA orgs. I found my time visiting GPI during their Early Career Conference Programme very valuable (hint: applications for 2021 are now open, apply!) and would encourage other orgs to run similar sorts of programmes to this and FHI’s (summer) research scholars programme. I’m very excited to see how our internship program develops as I really enjoy mentoring.
I think I was competitive for the RP job because of my T-shaped skills, broad knowledge in lots of EA-related things but also specialised knowledge in a specific useful area, economics in my case. Michael Aird probably has the most to say about developing broad knowledge given how much EA content he has consumed in the last couple of years, but in general reading things on the Forum and actively discussing them with other people (perhaps in a reading group) seems to be the way to develop in this area. Developing specialised skills obviously depends a lot on the skill, but graduate education and relevant internships are the most obvious routes here.
1. Thinking vs. reading.
Another benefit of thinking before reading is that it can help you develop your research skills. Noticing some phenomena and then developing a model to explain it is a super valuable exercise. If it turns out you reproduce something that someone else has already done and published, then great, you’ve gotten experience solving some problem and you’ve shown that you can think through it at least as well as some expert in the field. If it turns out that you have produced something novel then it’s time to see how it compares to existing results in the literature and get feedback on how useful it is.
This said, I think this is more true for theoretical work than applied work, e.g. the value of doing this in philosophy > in theoretical economics > in applied economics. A fair amount of EA-relevant research is summarising and synthesising what the academic literature on some topic finds and it seems pretty difficult to do that by just thinking to yourself!
3. Is there something interesting here?
I mostly try to work out how excited I am by this idea and whether I could see myself still being excited in 6 months, since for me having internal motivation to work on a project is pretty important. I also try to chat about this idea with various other people and see how excited they are by it.
4. Survival vs. exploratory mindset.
I also haven’t heard these terms before, but from your description (which frames a survival mindset pretty negatively), an exploratory mindset comes fairly naturally to me and therefore I haven’t ever actively cultivated it. Lots of research projects fail so extreme risk aversion in particular seems like it would be bad for researchers.
5. Optimal hours of work per day.
I typically aim for 6-7 hours of deep work a day and a couple of dedicated hours for miscellaneous tasks and meetings. Since starting part-time at RP I’ve been doing 6 days a week (2 RP, 4 PhD), but before that I did 5. I find RP deep work less taxing than PhD work. 6 days a week is at the upper limit of manageable for me at the moment, so I plan to experiment with different schedules in the new year.
6. Learning a new field.
I’m a big fan of textbooks and schedule time to read a couple of textbook chapters each week. Lesswrong’s best textbooks on every subject thread is pretty good for finding them. I usually make Anki flashcards to help me remember the key facts, but I’ve recently started experimenting with Roam Research to take notes which I’m also enjoying so my “learning flow” is in flux at the moment.
8. Emotional motivators.
My main trick for dealing with this is to always plan my day the night before. I let System 2 Dave work out what is important and needs to be done and put blocks in the calendar for these things. When System 1 Dave is working the next day, his motivation doesn’t end up mattering so much because he can easily defer to what System 2 Dave said he should do. I don’t read too much into lack of System 1 motivation, it happens and I haven’t noticed that it is particularly correlated with how important the work is, it’s more correlated with things like how scary it is to start some new task and irrelevant things like how much sunlight I’ve been getting.
9. Typing speed.
I struggle to imagine typing speed being a binding constraint on research productivity since I’ve never found typing speed to be a problem for getting into flow, but when I just checked my wpm was 85 so maybe I’d feel different if it was slower. When I’m coding the vast majority of my time is spent thinking about how to solve the problem I’m facing, not typing the code that solves the problem. When I’m writing first drafts, I think typing speed is a bit more helpful for the reasons you mention, but again more time goes into planning the structure of what I want to say and polishing, than the first pass at writing where speed might help.
11. Tiredness, focus, etc.
My favourite thing to do is to stop working! Not all days can be good days and I became a lot happier and more productive when I stopped beating myself up for having bad days and allowed myself to take the rest of the afternoon off.
12. Meta.
The questions I didn’t answer were because I didn’t have much to say about them so I’d be happy to see answers to them!
Thanks for the paper suggestions! Most of my own research is on internal validity in the LaLonde style so I definitely think it is important too. I’ll add a section on replicability to the syllabus.
The first 5 paragraphs are repeated twice. Could someone fix this?
Hey Kaj, I just thought I’d let you know that you’re not alone in Scandinavia! A few of us are starting an EA group in Uppsala, Sweden and Trondheim, Norway launched a couple of weeks ago. I know it’s late notice, but we’re having a Google Hangout this evening, 9pm your time so if you could join, that’d be great!
I want to do good—https://youtu.be/2Y3TTVK8wGQ