ETGP 2022: materials, feedback, and lessons for 2023

[trammell]

Update: feedback from 2023 is similarly summarized here. I am posting it as a link to a doc from this post, instead of as a new post, because I put it together so late (and the prospects of future courses in 2024 or 2025 seem so unlikely) that it seems unlikely to be of general interest to readers of this forum.

From August 20 to September 2, I ran a summer course in Oxford titled “Topics in Economic Theory and Global Prioritization”, or ETGP. It aimed to provide a rigorous introduction to a selection of topics in economic theory that appear especially relevant to the project of doing the most good. It was designed primarily for economics graduate students considering careers in global priorities research.

The purpose of this post is to share

• the course materials as presented this year,

• the feedback, and

• a summary of the lessons learned.

I hope it helps potential attendees get a better sense of whether they would like to attend next year, and potential organizers of similar programs get a better sense of whether and how to go about it.

I’ve erred on the side of thoroughness regarding the feedback and lessons learned. A brief summary is that the course was rated very highly, and that I think this suggests people should consider organizing more structured courses, instead of sticking to the more common “EA formula” of reading groups and summer research fellowships.

The course was sponsored by Forethought and made possible by operations support from the Global Priorities Institute. If you would like to be notified when applications open for next year, email me at [e-mail].
Applications for 2023 are now closed. If you would like to be notified in the event that applications open for 2024, or in the event that an online course or lecture series on this content is made available, please register your interest here.

Course materials and program schedule

The lecture slides and exercises, as presented this year (with corrections), can be found here. Feel free to use them for any purpose.

The program was scheduled as follows:

• Saturday, August 20 - Sunday, August 21: two 1.5h lectures per day on philosophical and mathematical background material, respectively.

• Monday, August 22 - Friday, August 26: two 1.5h lectures per day on various EA-relevant macroeconomic theory topics.

• Saturday, August 27: (optional) punting in the afternoon.

• Monday, August 29 - Friday, September 2: two 1.5h lectures per day on various EA-relevant microeconomic theory topics.

• Attendees were given the option to stay for a (totally unstructured) third week to discuss research ideas with each other, schedule meetings with others in Oxford, and so on.

The lectures, except for those on the first day, came with exercises. Every lecture-day except the first two (August 20-21) opened with a 1-hour session in which I went over previous lecture-day’s exercises. They were not graded.

Lectures and lunches were held at Trajan House, where GPI and Forethought are based. Breakfasts, dinners, and housing were held at Worcester College, Oxford, except for an opening dinner and a closing dinner, which were held at pubs.

Applicant and attendee characteristics

There were 179 applicants. 46 were accepted (26%), and 34 attended at least in part.

Educational backgrounds of the attendees:

• 1 was an assistant professor of economics (3%)

• 16 were enrolled in, or about to begin, doctoral programs in economics (47%)

• 3 were enrolled in or about to begin master’s programs in economics, or recent graduates of master’s programs not doing either of the above (9%)

• 6 were doing pre-doctoral research /​ research assistance in economics (18%)

• 6 were undergraduates studying economics, or recent graduates not doing any of the above (18%)

• 2 had never pursued an economics degree (6%). (One had a graduate degree in a related field, and the other was pursuing one)

Genders of the attendees:

• 29 were male (85%)

• 5 were female (15%)

45 of the applicants (25%) and 9 of the admits (20%) were female. I noticed the relative scarcity of female applicants when reviewing the applications, and I did my best to ensure that they were not rejected unfairly.

Feedback

[Edit January 21, 2023: several attendees have kindly offered testimonials in the comments here. The feedback below was submitted anonymously.]

The feedback survey received 24 responses (71%). Aggregate results are as follows:

Overall evaluation

Much of the open-ended text feedback was clearly not written with a public audience in mind, and I won’t try to edit it down or summarize it. But I have to say, there was a heartwarming volume of intensely positive comments that a data point like “5 out of 5” doesn’t fully capture. I hope it’s all right if I just share one of them, for illustration: “I loved this course. This has been one of the best courses of my life.”

Consistent with the respondents’ unanimous (weak or strong) satisfaction, there were no intensely negative text responses. I’m not sure why two respondents were weakly satisfied but on balance would not recommend it to others with similar skills and interests.

[Edit Sept. 28: One of these two respondents reached out to explain that they felt ETGP was mostly a “survey of results”, without “much exposure to the ‘frontier’ and/​or methods”, so they “would not tell other environmental econ PhD students that [it] would help their research”. It is possible that this impression was strengthened by the fact that they could not stay for the second week, which was moderately more in-depth than the first (albeit had even less environmental economics).]

Accommodation, meals, and activities

From the text responses, the most common source of less-than-full satisfaction with the accommodation was that many of the Worcester dormitory bathrooms only had baths, not showers.

Inclusivity

From the text responses, the respondent who strongly disagreed did not argue that the program stifled debate—just that “[t]he lectures were lectures and in informal settings we discussed philosophy.”

In the text responses, the most common way in which respondents found the course not fully welcoming was that lectures—and informal discussions with the other attendees—could be difficult to follow for those without extensive background knowledge in economics, mathematics, philosophy, and EA thought.

Second most commonly, some respondents found the course’s relative lack of female and/​or non-white attendees off-putting.

Curriculum

As this last question indicates, it seemed to me by the end of the course that I had tried to pack too much in. But the respondents were roughly divided on this. As many respondents recommended keeping it roughly as is as recommended covering less material, lengthening the duration, or both. Also, all 3 people who answered “Other” proposed ways of teaching the existing quantity of material more efficiently in the same period of time. (Still, it may be telling that no one proposed covering more content or shortening the duration!)

Most and least enjoyed aspects of the program

From the text responses, it seems that people’s favorite part of the program was the opportunity to meet and hang out with others with shared interests. To this end, some respondents would have appreciated more organized social activities. The only organized social activity, besides meals, was an hour of punting which only about half of the participants attended.

People’s least favorite part of the program was the problem sets. This seems to have been for three reasons, beyond the fact that problem sets typically aren’t fun in general:

• because the exercises as originally written often had errors which made them unsolvable (hopefully fixed in the exercise sheet in the “course materials” Google doc linked above!);

• because some of them were too time-consuming, even if they were solvable; and

• because, without much incentive to attempt the exercises, and given the two issues above, most people soon stopped grappling with them seriously. This made it hard to unilaterally decide to work on them instead of socializing in the evenings.

Plan changes

18 respondents wrote that attending ETGP may have changed some of their future research plans or directions, of which 6 wrote that it did or almost certainly did so. The most common themes were increased interests in

• growth theory,

• economic theory (as opposed to empirical work) in general,

• doing an economics PhD in general, and

• explicitly longtermism-focused research.

Takeaways and tweaks for 2023

In summary, I’m excited to report that respondents rated ETGP 2022 very highly! As noted above, I believe the feedback supports the idea that a structured course like this can play an important role, likely beyond that of a less formal fellowship or reading group, in helping students and early-career professionals develop their research interests. It can successfully familiarize attendees with complex toolkits and topics that are relevant to EA-aligned research.

Nevertheless, in light of the feedback summarized above, and my own experience, I believe the following changes will improve the program for summer 2023.

Content

Mistakes in the slides and (I think even more harmfully) in the exercises sometimes made it harder to learn the content. I’ve since corrected all the mistakes that were identified on the first run-through, and I’ll try to go over everything again before next year, and/​or ask others to do the same.

I want to encourage attendees to grapple with the exercises, if they would like to, instead of letting them fall into an equilibrium in which little effort is put into the exercises and people typically socialize in the evenings. To make this more likely, in addition to the error-checking noted above, I’ll make the exercises easier. Also, to incentivize effort, I might offer attendees the option to

• have their exercises graded (in which case I’ll try to hire a TA, perhaps from this year’s cohort), and/​or the option to

• join a randomly-assigned study group at the beginning, with each group’s scores (self-graded or TA-graded, depending on whether there is a TA) posted publicly at the end, as a friendly competition.

As a compromise between the “too packed with content” half of respondents and the “keep as is” half, I’ll try to cut an average of 5-10% of the slides from each lecture. I’ll also add an explicit 15m break to the middle of each lecture (making each lecture 1h45), so that the material is less rushed and people have more time to ask questions.

Finally, at least one curriculum revision: The lecture on quadratic funding will be cut. In its place, the lecture on AI and growth will be expanded to two lectures, with the lecture on game theory for AI governance bumped from week 1 to week 2.

Welcomingness

The question on welcomingness got somewhat lower scores than the other questions. As noted above, the two main reasons cited for this were the assumption of extensive background knowledge and the relative lack of female and/​or non-white attendees.

I hope to mitigate the first issue next year mainly by either (a) making the course more accessible to undergraduates or, more likely, (b) marketing it more clearly as a graduate-level economics course. It was originally pitched at late-stage undergrads and early-stage graduate students equally, but when the applications came in and it became clear that the seats would be filled primarily by graduate students, I wrote the lectures to be roughly graduate-level. This was a bit of an unfair bait and switch.

I’ll also encourage attendees to remember that their colleagues have a wide range of prior engagement with philosophy and EA thought, and that they should therefore avoid or explain the relevant jargon in conversation. And of course I’ll try to do the same.

I hope to mitigate the second issue next year by asking Magnify Mentoring, and local EA groups outside North America, Europe, and Australia, to consider encouraging their members to apply.

Scheduling

Several admitted applicants could not attend because the course overlapped with the beginning of a fall term, or with a pre-PhD math camp. Several more could only attend for the first week, or would have wanted to attend the optional third week but couldn’t. By contrast, it seems that relatively few people who could attend would not have been able to attend if it had been scheduled a week or two earlier. So I’ll probably schedule it earlier next year.

Accommodation and social

Worcester got high marks overall, and it’s the natural place for participants to stay, given its proximity to Trajan House. But next year I’ll make sure people are all on halls with showers, or make sure signs are put up that make the showers easy to find. (Lesson for program organizers in general: explore the accommodation facilities in detail before booking, and think hard about what might be missing...!)

To facilitate those highly-valued informal connections, I’ll probably organize a few more weekend social activities, not just punting.

Some participants drank a lot at the closing pub dinner, which struck some other participants as shameful and/​or a waste of money (and perhaps in some cases it was). To avoid this issue, I’ll consider setting a cap—zero or positive—on the number of drinks included with the opening and closing dinners.

Finally, I’ll make sure that some activities, including research idea brainstorming sessions, are organized for the optional third week.

[Kevin Kuruc]

Thanks for working so hard on this! Great stuff.

[trammell]

Thanks! And thanks for presenting!!

[Eleos Arete Citrini]

So many of these topics seem really interesting to me personally that your saying “It was designed primarily for economics graduate students considering careers in global priorities research.” made me wonder: Is there something similar for people with a less robust background in economics? Maybe for economics (or political science) undergraduate students? :)

[trammell]

Glad to hear you find the topics interesting!

First, I should emphasize that it’s not designed exclusively for econ grad students. The opening few days try to introduce enough of the relevant background material that mathematically-minded people of any background can follow the rest of it. As you’ll have seen, many of the attendees were pre-grad-school, and 18% were undergrads. My impression from the feedback forms and from the in-person experience is that some of the undergrads did struggle, unfortunately, but others got a lot out of it. Check out the materials, and if you think you’d be a good fit, you’re more than welcome to apply for next year.

That said, I agree that a more undergrad-focused version of the program would be valuable. I don’t have plans to make one myself in the near future, but I would like to at some point. In the meantime, if anyone reading this wants to do it, please feel free to reach out!