There’s two opposing arguments: 1) You get more information about your friends than you get about strangers, and 2) you are more likely to be biased in favour of your friends.
Personally, I think it would be very hard to vet potential funding prospects over just having a few talks, and the fact that I’ve “vetted” my friends over several years is a wealth of information that I would be foolish to ignore.
Our intuitions on this may diverge based on how likely we think it is that we’ve acquired exceptional friends. If you’re imagining childhood friends or college buddies, then I see why you would be skeptical. If on the other hand you’re imagining the friends you’ve acquired from activities that you think only exceptional people would engage in, then that changes things.
I think for funding a project, most of the important and relevant information about a person who might run the project can be obtained from a detailed CV.
I thin most of the information that a funder could obtain about a friend which they couldn’t also get from the friend’s CV is their impression of difficult-to-accurately-evaluate things like personality traits. I place very little value on a funder’s evaluation of these things because these things are inherently difficult to evaluate anyway and I expect their evaluation to be too heavily biased by their liking for their friend.
Perhaps we disagree on the difficulty of evaluating personality traits, but I think we probably disagree on the extent to which liking someone as a friend is likely to bias your views on them.
My view has long been that the bias is likely to be so large that funding applications should include CVs but not the names of people. I think many EAs feel like systems like these overvalue credentials, but that could easily be gotten round by excluding university names and focusing CVs more on ‘track record of running cool projects’.
I think for funding a project, most of the important and relevant information about a person who might run the project can be obtained from a detailed CV.
Wait what? Predictive validity of CVs is minimal for most jobs, one might naively guess that they ought to be even less predicative for funding entrepreneurial projects than for jobs.
Why do you think companies rely on referrals more than on CVs?
There are lots of ways to accurately predict a job applicant’s future success. See the meta-analysis linked below, which finds general mental ability tests, work trials, and structured interviews all to be more predictive of future overall job performance than unstructured interviews, peer ratings, or reference checks.
I’m not a grantmaker and there are certainly benefits to informal networking-based grants, but on the whole I wish EA grantmaking relied less on social connections to grantmakers and more on these kinds of objective evaluations.
Do you mean like people referring potential employees to them or the references of job applicants? I wasn’t aware of what companies rely on more in recruitment.
I meant people who work in a company referring future potential employees.
I was very surprised to see Peer Ratings being so useful for predicting performance
Very belated edit to add: if your workplace has a bunch of real work, someone needs to get things done on your team. If your teammates are slackers or generally incompetent, this means you need to pick up the slack. This could end up being pretty rough, so there’s a strong incentive to give honest referrals to future colleagues about their work performance (and not just charisma).
This is probably a very important distinction for those reading the above comments for the first time. “Referral” might be a better word so as to distinguish from “reference” letters written by past supervisors.
There’s two opposing arguments: 1) You get more information about your friends than you get about strangers, and 2) you are more likely to be biased in favour of your friends.
Personally, I think it would be very hard to vet potential funding prospects over just having a few talks, and the fact that I’ve “vetted” my friends over several years is a wealth of information that I would be foolish to ignore.
Our intuitions on this may diverge based on how likely we think it is that we’ve acquired exceptional friends. If you’re imagining childhood friends or college buddies, then I see why you would be skeptical. If on the other hand you’re imagining the friends you’ve acquired from activities that you think only exceptional people would engage in, then that changes things.
I think for funding a project, most of the important and relevant information about a person who might run the project can be obtained from a detailed CV.
I thin most of the information that a funder could obtain about a friend which they couldn’t also get from the friend’s CV is their impression of difficult-to-accurately-evaluate things like personality traits. I place very little value on a funder’s evaluation of these things because these things are inherently difficult to evaluate anyway and I expect their evaluation to be too heavily biased by their liking for their friend.
Perhaps we disagree on the difficulty of evaluating personality traits, but I think we probably disagree on the extent to which liking someone as a friend is likely to bias your views on them.
My view has long been that the bias is likely to be so large that funding applications should include CVs but not the names of people. I think many EAs feel like systems like these overvalue credentials, but that could easily be gotten round by excluding university names and focusing CVs more on ‘track record of running cool projects’.
Wait what? Predictive validity of CVs is minimal for most jobs, one might naively guess that they ought to be even less predicative for funding entrepreneurial projects than for jobs.
Why do you think companies rely on referrals more than on CVs?
There are lots of ways to accurately predict a job applicant’s future success. See the meta-analysis linked below, which finds general mental ability tests, work trials, and structured interviews all to be more predictive of future overall job performance than unstructured interviews, peer ratings, or reference checks.
I’m not a grantmaker and there are certainly benefits to informal networking-based grants, but on the whole I wish EA grantmaking relied less on social connections to grantmakers and more on these kinds of objective evaluations.
Meta-analysis (>6000 citations): https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.172.1733&rep=rep1&type=pdf
Do you mean like people referring potential employees to them or the references of job applicants? I wasn’t aware of what companies rely on more in recruitment.
But yeah my model of recruitment was very evidence free and mostly based on my limited experiences in recruiting people for things, turns out my model of what’s most useful was basically the opposite of what some of the evidence says (https://www.talentlens.com/content/dam/school/global/Global-Talentlens/uk/AboutUs/Whitepapers/White-Paper-The-Science-Behind-Predicting-Job-Performance-at-Recruitment.pdf), I was very surprised to see Peer Ratings being so useful for predicting performance.
I meant people who work in a company referring future potential employees.
Very belated edit to add: if your workplace has a bunch of real work, someone needs to get things done on your team. If your teammates are slackers or generally incompetent, this means you need to pick up the slack. This could end up being pretty rough, so there’s a strong incentive to give honest referrals to future colleagues about their work performance (and not just charisma).
This is probably a very important distinction for those reading the above comments for the first time. “Referral” might be a better word so as to distinguish from “reference” letters written by past supervisors.