There are definitely a lot of selection effects prior to us making our selection. I think what we are trying to say is that our selections based on interview scores were not very helpful. Perhaps, they would be helpful if our system worked very differently (for instance, if just interviewed anyone who put down their email). But it seems like with the selection effects we had (have to make an effort to fill out the application, do a small amount of background reading, schedule and show up to an interview) we arrived at a place where our interview scoring system didn’t do a good job further narrowing down the applicants.
We definitely do not mean to say that other groups definitively shouldn’t be selective, or even shouldn’t be selective using our criteria. We just don’t have the evidence to suggest that our criteria were particularly helpful in our case, so we can’t really recommend it for others.
There are definitely a lot of selection effects prior to us making our selection. I think what we are trying to say is that our selections based on interview scores were not very helpful. Perhaps, they would be helpful if our system worked very differently (for instance, if just interviewed anyone who put down their email). But it seems like with the selection effects we had (have to make an effort to fill out the application, do a small amount of background reading, schedule and show up to an interview) we arrived at a place where our interview scoring system didn’t do a good job further narrowing down the applicants.
We definitely do not mean to say that other groups definitively shouldn’t be selective, or even shouldn’t be selective using our criteria. We just don’t have the evidence to suggest that our criteria were particularly helpful in our case, so we can’t really recommend it for others.