I think this is highly dependent on the tasks that are required of applicants!
I know e.g. Charity Entrepreneurship has thought a lot about how to make the application process, work tests etc. more valuable for applicants. And iirc, I found the CERI Process quite helpful in sharpening my research focus and organizing my thoughts around x-risk.
(An entirely different question is whether long decision timelines negatively affect applicants’ options or result in applicants withdrawing/going with their second best option, but I’m not sure this happened a lot. There was also a post recently about the benefits of opening applications sooner to be able to compete with non-EA summer internships/Fellowships.)
Thanks for your perspective. I think CE’s application process is a great example of how to do tiered filtering with different tasks ranging in time and topic. And I think it’s valuable because you get some feedback. Correct me if I’m wrong but CERI/CHERI/SERI have not provided feedback before?
My low confidence hunch is that for applicants who are self-reflective, like you, there is value in work without feedback too, perhaps in the way that you say but that, at the same time, with a pool of 650 people there’s also productivity loss.
Say there’s probably at least a bottom e.g. 20% of applicants who are less self-reflective or are less familiar with the x-risk or even writing applications, who get no feedback. And that would be about 130 people, call it an hour each, that’s 130 hours of productivity. Had the application been shorter, say 20 minutes, that means about 43 hours spent or 87 hours saved. I’m sure there’s time spent on reviewing applications that’s sinking here too. I know people who have spent upwards of 5 hours on their apps for these fellowships and were then frustrated to not know how they did or what was missing. I’m not sure how representative of an experience this is at all, of course.
The other side that feels underutilized to me is ‘near miss’ candidates. You didn’t get a spot but you were near the top. It feels like a real loss to get no feedback at all?
Agreed on the feedback part, but pretty unsure how to solve this without bringing on one person for a few hundred hours to give individual feedback.
Regarding the hours of productivity lost: I’m not sure about the counterfactual, since CERI applicants are mostly students.
I think my main point however is this: being forced to actually write a proposal for that vague research idea that has been floating around in your head seems pretty valuable to me whether you get feedback or not. I, for example, spent quite a few hours just digging through research agendas in different x-risk fields to make sure I’m not just riding my hobby horse, but that my research actually fills some gap.
So, even if I hadn’t been accepted, I would probably have considered this “time well spent” rather than “lost productivity”.
(Disclaimer that I might be biased here though—motivated reasoning/avoiding cognitive dissonance etc.)
A quick comment on the 2022 application: we did think about the tradeoffs between short vs long applications, and considering our target audience, we decided to create an application which not only helped us assess the applicants, but also helped them explore the research field and organise their thoughts around existential risks. Several applicants commented that they learnt something about existential risks from completing our application, and we think there was atleast some level of nebulous impact there. We are working on our 2023 application, and will have concrete updates to share by late January.
Looking forward to your continued important work in x-risk.
For the fellowship, would you consider doing your application differently this year? I was looking at a separate post recently (https://forum.effectivealtruism.org/posts/yeDRczoJuJZMbJvrq/assessing-seri-cheri-ceri-summer-program-impact-by-surveying) and you got 650 applications for 24 spots. I remember the applications taking a long time! I wonder if you could have a first filter with shorter questions. It seems to me that many productivity hours are lost or underutilized.
I think this is highly dependent on the tasks that are required of applicants! I know e.g. Charity Entrepreneurship has thought a lot about how to make the application process, work tests etc. more valuable for applicants. And iirc, I found the CERI Process quite helpful in sharpening my research focus and organizing my thoughts around x-risk.
(An entirely different question is whether long decision timelines negatively affect applicants’ options or result in applicants withdrawing/going with their second best option, but I’m not sure this happened a lot. There was also a post recently about the benefits of opening applications sooner to be able to compete with non-EA summer internships/Fellowships.)
Thanks for your perspective. I think CE’s application process is a great example of how to do tiered filtering with different tasks ranging in time and topic. And I think it’s valuable because you get some feedback. Correct me if I’m wrong but CERI/CHERI/SERI have not provided feedback before?
My low confidence hunch is that for applicants who are self-reflective, like you, there is value in work without feedback too, perhaps in the way that you say but that, at the same time, with a pool of 650 people there’s also productivity loss.
Say there’s probably at least a bottom e.g. 20% of applicants who are less self-reflective or are less familiar with the x-risk or even writing applications, who get no feedback. And that would be about 130 people, call it an hour each, that’s 130 hours of productivity. Had the application been shorter, say 20 minutes, that means about 43 hours spent or 87 hours saved. I’m sure there’s time spent on reviewing applications that’s sinking here too. I know people who have spent upwards of 5 hours on their apps for these fellowships and were then frustrated to not know how they did or what was missing. I’m not sure how representative of an experience this is at all, of course.
The other side that feels underutilized to me is ‘near miss’ candidates. You didn’t get a spot but you were near the top. It feels like a real loss to get no feedback at all?
Agreed on the feedback part, but pretty unsure how to solve this without bringing on one person for a few hundred hours to give individual feedback.
Regarding the hours of productivity lost: I’m not sure about the counterfactual, since CERI applicants are mostly students.
I think my main point however is this: being forced to actually write a proposal for that vague research idea that has been floating around in your head seems pretty valuable to me whether you get feedback or not. I, for example, spent quite a few hours just digging through research agendas in different x-risk fields to make sure I’m not just riding my hobby horse, but that my research actually fills some gap. So, even if I hadn’t been accepted, I would probably have considered this “time well spent” rather than “lost productivity”.
(Disclaimer that I might be biased here though—motivated reasoning/avoiding cognitive dissonance etc.)
A quick comment on the 2022 application: we did think about the tradeoffs between short vs long applications, and considering our target audience, we decided to create an application which not only helped us assess the applicants, but also helped them explore the research field and organise their thoughts around existential risks. Several applicants commented that they learnt something about existential risks from completing our application, and we think there was atleast some level of nebulous impact there. We are working on our 2023 application, and will have concrete updates to share by late January.