Identifying Talent without Credentialing In EA
Employers usually prefer to hire employees with credentials. The most common example of this is having a university degree: employees with university degrees receive significantly higher compensation.
Universities are very expensive, and it would be nice to replace them with some cheaper way of identifying talent. For example, maybe Google could just go to high schools, give everyone some two-hour long exam, then offer those who pass the exam six months of training at Google, after which they become Google software engineers.
The fact that Ivy League universities still exist seems like prima facie proof that this isnât possible: corporations could make billions of dollars if they could do an end run around universities and hire people right from high school, so the fact that they donât do this must indicate that itâs a hard problem.
Even if itâs hard in the general case, though, there might be specific subsets of employees or jobs for which we can avoid universities. Coding boot camps, for example, seem to demonstrate a credentialing system for software engineering jobs which is just as effective as existing ones but massively cheaper.
In this article, I explain the concerns with existing credentialing systems and try to speculate on which idiosyncrasies of EA would enable us to avoid them.
Why Credentialing is (Sometimes) Bad
One simple solution to identifying talent is expert deferral: if Iâm hiring a software engineer, I could just say something like âif Google offers you a job then I will also offer you a job,â thereby deferring to Google about hiring standards.
This is not a terrible strategy, but it has the downside that we are, by definition, competing with Google for every employee. Similarly, if we recruit only from Ivy League colleges, we are competing against a bunch of other employers who recruit from Ivy League colleges.
We might prefer to find someone with good open-source contributions who didnât finish college (as I almost did), or graduated from a state school (as I did). These people might be better fits (in the specific case of programmers, open source contributions are arguably more predictive than educational pedigree), but even if they are less skilled than the average Ivy League graduate, we have much less competition for recruiting them.
Secondly, credentials donât exist for many things that EAâs consider important. Someone researching existential risks may come across infohazards, and having a PhD in no way certifies âthis person will avoid publishing infohazards even if publication would advance their career.â Therefore, we need an alternative certification process.
Finally, the EA community currently contains a lot of people who are early in their careers and have not had the time to build a track record of success. There are legitimate concerns with putting people into positions of authority without credentials, which means that these early-career people have a lot of difficulty contributing.
What are some ways EA could take its unusual requirements into account, in order to hire more effectively?
Hypothesis 1: Hits-Based Hiring
Established companies are very risk-averse. Boards will pay a large premium to hire a provenly mediocre CEO versus taking a chance on an untested CEO who might be really good but also might be really awful.
Certain types of EA work are possibly more hits-based, such that the organizations doing that work might prefer the âuntested, high varianceâ employee to the âtested and mediocreâ employee.
As an example, some types of research might fall into this category: if the research project is done poorly, it will just go into the dustbin as an un-cited paper, but if it succeeds, it could define a new field.
For these positions, it seems like we can rely less on credentialing in favor of a âspray and prayâ approach where we hire a lot of people with the expectation that most of them will be bad fits but the few who are successful will cover their costs.
Rocket Internetâs model of starting vast numbers of companies and then shutting down the ones which arenât successful is one for-profit example of this strategy. (Itâs worth noting that Rocket has a horrible reputation as a result â it turns out that repeatedly laying off massive numbers of people within six months of hiring them doesnât make you popular.)
Hypothesis 2: Altruistic Internships
Suppose a corporation feels that someone with no relevant background could be brought up to speed after completing a two-year training program. The corporation might think of some strategy like: train someone for two years (during which the company loses money on them), but then have them employed at below-market salaries for three years (during which the company makes money on them). This would seem to benefit both the corporation and the employee.
In practice, this doesnât usually happen because thereâs nothing to prevent the employee from leaving after they complete the training program and going to a competitor which pays market rates. Corporations sometimes try to put a âpoison pillâ in their contracts, forcing employees who leave before a certain date to pay money back to the corporation. But itâs hard to legally enforce these contracts.
Altruistic employers might not have this problem: if I train someone to become awesome at corporate outreach campaigns, and then they go do corporate outreach campaigns somewhere else, thatâs kind of okay?
More specifically: altruistic funders would be okay with this. If The Humane League has unusually high HR costs because their skilled employees leave to go work at Mercy For Animals, donors to THL might be happy to subsidize that cost because the donor is almost indifferent between work at THL versus MFA. Or a funder might support a grant for someone to work on a problem apart from either organization.
Hypothesis 3: Avoiding Negative Salaries
The US has a cultural and legal norm that employees must be paid if they provide value to their employer. University students, for example, do not need to be paid for their studies (and indeed usually pay for the privilege of studying) because their studies do not benefit the University.
This prevents a lot of mutually beneficial apprenticeships: young apprentices might be willing to work for free or even pay to help more skilled workers, and the skilled workers might be happy to have free labor. But this is culturally and often legally prohibited, e.g. by minimum wage laws. For example, shortages of surgeons are sometimes blamed on the fact that hospitals are required to pay surgical residents, even though the market equilibrium would be for the residents to pay the hospitals.
Nonprofits are not as shielded from these requirements as you might expect â there are laws preventing volunteers from doing the work that employees would otherwise do. But, as in the previous section, nonprofits can coordinate with funders in ways for-profits canât. For example, a donor could subsidize someoneâs employment through a donation to their employer, letting the nonprofit avoid taking a risk on their new employee while still complying with minimum wage laws. The donor is thus effectively investing in the apprenticeâs employment, with the hope that the apprentice will provide positive value later on.
Another approach is entrepreneurship: the self-employed are generally exempt from minimum wage laws and often lose money on their companies â if EA has valuable projects which can be worked on independently, people could do that instead of more expensive credentials.
Conclusion
It seems like there is some momentum in EA behind these approaches. BERIâs Individual Level-Up Grants is one relevant program; some EA Grants recipients have been funded to gain important skills through independent study. Eliezer Yudkowsky, the founder of MIRI, is an extreme example of success without credentials, having attended neither high school or college.
Given that EA has a lot of people with raw capabilities but relatively few credentials, I would be excited for more people to explore this.
I would like to thank Aaron Gertler for reviewing a draft of this. Any remaining mistakes are mine. Views expressed are my own, and do not represent my employerâs. I have not been involved in CEAâs hiring, nor in any rounds of EA or Community Building Grants. This post is related to several recent ones such as EA is Vetting Constrained but was written several months ago and not influenced by them (it just took me a while to get around to publishing).
Having a reliable work-sample test and interview process allows me to hire fairly confidently without much regard to credentials.
At basically every company in the world, someone who comes in with a 10-year-long track record of success is going to be way more likely to be hired than someone fresh out of college, even if they are equally skilled. It would be pretty surprising to me if you are able to outperform the hiring practices of all these companiesâdo you have a sense of why you think you can? Many orgs use work-sample tests, so that seems too simple of an explanation.
I canât reply on behalf of Peter, but I would imagine the following:
Individuals at companies choose to hire for reasons other than expected performance (e.g. itâs a publicly defensible decision to hire someone with recognized credentials and track record, whereas itâs not publicly defensible to hire someone who lacks those but who otherwise seems like theyâd perform really well). See general discussion of the signalling value of education.
Individuals at companies are bad at hiring for expected performance: e.g. relying on things which the evidence suggests donât predict job performance well (such as subjective impressions in an unstructured interview) and (possibly) credentials.
Many companies in the world can in theory hire people with 10 year track records doing similar roles in similar companies. People hiring for EA researcher roles typically canât find anyone with a 10 year track record in similar work- and even if you relax the assumptions somewhat, can still find far fewer people with any kind of track record in similar work.
The competencies Peter is hiring for may be more test-taskable than many that companies are hiring for. e.g. creating a cost-effectiveness model may be a better predictor of performance at creating cost-effectiveness models than the best available test tasks for âbe an executiveâ or âmanage HR.â
I can reply on behalf of Peter and I approve this message.
Fair â an implicit assumption of my post is that markets are efficient. If you donât think so, then what I had to say is probably not very relevant.
I assume youâre not arguing for the strong EMH here (markets are maximally efficient), so the difference to me seems to be a difference of degree than kind (you think hiring markets are more efficient than Peter does, Peter thinks hiring markets are less efficient than you do.)
If you are arguing for the strong version of EMH here Iâd be curious as to your reasoning, as I canât think of any credible economists who think that real world markets donât have any inefficiencies.
If youâre arguing for a weaker version, I think itâs worth digging in to cruxes⊠Why do you think that the hiring market is more efficient than Peter does?
I very much like this approach to finding ways to deal with credentialism, however Iâm also unsure how much of an impact credentials are having on current EA hiring. That is, my impression is that current EA orgs are hiring folks more based on work experience rather than credentials, and in fact EA orgs are unusually willing to consider candidates without traditional credentials (EA-orgs within universities being an exception due to their hiring processes being tied to those of the host institution). This suggests your premise may not apply (EA orgs not hiring folks because they lack credentials), but regardless I think your solutions apply anyway because they also address the issue where candidates lack experience rather than credentials.
Thanks for the feedback! I meant âcredentialsâ to include things like work experience, and perhaps should have used more examples like that to be clear.