Error
Unrecognized LW server error:
Field "fmCrosspost" of type "CrosspostOutput" must have a selection of subfields. Did you mean "fmCrosspost { ... }"?Unrecognized LW server error:
Field "fmCrosspost" of type "CrosspostOutput" must have a selection of subfields. Did you mean "fmCrosspost { ... }"?
Academic here:
Essentially all of these numbers vary wildly across subfields, across countries, and on other assumptions like how prestigious the labs are that you’re considering. Judging based on numbers from physics, or from US PhDs overall, could leave you off by an order of magnitude or more. They also vary significantly over time.
The populations in PhD programs vary a lot from field to field as well, and how you fit relative to those populations will help tilt the odds. Being intrinsically motivated and a good English writer (the two things I can tell about the OP) could give you a pretty big leg up in your odds of finishing and getting a job relative to the median CS PhD student at a good US research university, at least assuming that you have the technical qualifications to be admitted. In a Philosophy program, by contrast, that’d be baked into the admissions criteria, and wouldn’t tell me much.
FWIW, here are some ballpark 80% confidence intervals based only on my recent experience. These are conditioned on what I know about the OP (AI safety area, good English writer, intrinsically motivated). I’m focusing on the US because that’s what I know, and I’m generally assuming top-50-or-so universities in CS, which is where you can be reasonably confident that you’ll have the resources and public platform to do high-impact research. I work in AI, but I don’t have much yet experience with AI safety. I have been involved in general admissions for two PhD programs with an AI focus, and two others.
P(admission to a good PhD program | serious effort at applying) =
1-15% without substantial prior research experience,
5-60% with limited research experience (at least one serious paper with a recognized collaborator, but nothing presented as a first author at a competitive venue)
50-90% with strong research experience (at least one paper with a recognized collaborator, presented as a first author at a competitive venue).
P(graduate within six years | enroll) = 70-95%
P(assistant professor job at a US top-100 research university directly after PhD | graduate within six years and apply) = 5-50%
P(assistant professor job at a top-100 research university within three years after PhD | graduate within six years and apply) = 30-75%
P(long-term US research job that supports publishing, academic or otherwise, within three years after PhD | graduate within six years and apply) = 85-95%
P(granted permanent tenure within nine years of starting as an assistant professor | make a serious attempt to stay) = 85%-98%
Hey Pablo—thanks for working this up. It’s nice to have some baseline estimates!
As you say, Tregellas et al. shows that the probability of tenure varies a lot with the number of first author publications. It would be interesting to know if tenure can be predicted better with other factors like one’s institution or h-index—I could imagine such a model performing much better than the baseline.
Two other queries:
I feel like we’re talking about tenure, rather than tenure track?
When you say things like “my personal estimate of the baseline probability of getting a permanent (tenured) position in academia should be with 90% probability between 10-30%”, it might be clearer to say you’re 90% sure that 10-30% of students get tenure? Otherwise I don’t know how to interpret this probability of a probability.
You’re right Ryan, I’ll modify the second complicated sentence. I am actually not sure what is the difference between tenure and tenure track, to tell the truth.
However, in one of the documents above I saw that institution is not such a strong predictor (point 4), but h index seemed useful (in point 2 the h-index is discussed).
Interesting. The point 2 article by van Dijk seems decent. Figure 1B says that the impact factor of journals, volume of publications, and cites/h-index are all fairly predictive. University rank gets some independent weighting (among 38 features, as shown in their supplementary Table S1), but not much.
Looks like although the web version has gone offline, the source code of their model is still online!
I strongly agree with Ryan that success is to a relatively large degree predictable, as can be done in the PCA decomposition of point 2 above, figure 1C.
I think it would be very valuable to have such a model, but the current code is only for biology (the impact factor will fail for instance for anything different). If one wanted to fit a model to predict it, it could probably use google scholar and arxiv, but the trickiest part would be to recover the position of those people (the target), which may partially be done using google scholar.
I just posted another article I found on average publication rates in Norway for different positions, ages, fields and gender.
This is helpful, thanks.
The information is probably here somewhere, but is that the probability of getting tenure given you finish your Ph.D.? I.e. Does this account for dropping out?
Somewhat tangential, but I think accounting for the chance of working on AI safety (or something comparably effective) outside of academia will help. I think this is more common in Economics (e.g. World Bank). But I guess OpenAI or similar institutions hire CS PhDs and working there possibly has a similar impact to working in academia.
I would say you basically cannot get tenure if you don’t get a PhD, so dropouts are not taken into account in any of the previous statistics as far as I understood them. All this metrics are of the kind of: x% of PhD alumni got tenure, or similar.
I actually agree that taking into account the private sector could help, but I am much less certain about the freedom they give you to research those topics, beyond the usual suspects. That was why I was focussing on academia.
In the US, about half of people who start PhD programs get the degree. Also, a big factor that I thought I commented about here (I guess they removed comments) is that most tenure track positions at least in the US are teaching intensive, so there is not much time for research.