some capable applicants will not have very good examples of “past situation X”
I agree with you completely. I view the main problem of evaluating someone based on the last time they had to do X, is that perhaps this person has never had to do X (even though they are capable of doing X well).
I was always told that an applicant is less likely to make up an answer if we ask about past behavior than if we ask about how he or she would act in a future hypothetical scenario, but I actually don’t have any sources for that. As a result of you asking this question I poked around a few PDFs that I have on my to-read list and I’ve updated my beliefs a bit: I used to strongly recommend behavioral interview questions, but know I’m going to lessen the strength of that recommendation and only moderately recommend. I’m really glad that you brought this up, because now my view of reality is ever-so-slightly more accurate. Thank you. I’ll add a sentence in the body of the post to clarify.
The reviewed research suggests that SQs have slightly lower criterion- validity than PBQs, especially for high-complexity jobs. Yet, this finding does not necessarily mean that only PBQs should be used, as both SQs and PBQs have good validity for most jobs. Instead, researchers commonly recommend the use of both question types, in part because SQs and PBQs tend to measure different constructs and may complement each other (e.g., Campion et al., 1997; Day& Carroll, 2003; Huffcutt,Weekley et al., 2001; Krajewski et al., 2006) to increase overall criterion-related validity and also because they allow some variety in question format.
I will provide a caveat by saying that there seem to be different ups and downs according to different research. Regarding biases in interviews, according to one journal article when applicants of different racial groups apply, behavioral interviews seem to had lower group differences than situational interviews: the average of the differences between the two groups was .10 with behavioral interviews and .20 with situational interviews (this is from a paper called Racial group differences in employment interview evaluations). Although there are many ways to interpret this, my rough interpretation would be that behavioral questions allow less bias to creep in. However, this is only a single journal article, and I have to admit that I am not as well-read on behavioral interviews as I would like.
I agree with you completely. I view the main problem of evaluating someone based on the last time they had to do X, is that perhaps this person has never had to do X (even though they are capable of doing X well).
I was always told that an applicant is less likely to make up an answer if we ask about past behavior than if we ask about how he or she would act in a future hypothetical scenario, but I actually don’t have any sources for that. As a result of you asking this question I poked around a few PDFs that I have on my to-read list and I’ve updated my beliefs a bit: I used to strongly recommend behavioral interview questions, but know I’m going to lessen the strength of that recommendation and only moderately recommend. I’m really glad that you brought this up, because now my view of reality is ever-so-slightly more accurate. Thank you. I’ll add a sentence in the body of the post to clarify.
According to The Structured Employment Interview: Narrative and Quantitative Review of the Research Literature, it turns out that:
I will provide a caveat by saying that there seem to be different ups and downs according to different research. Regarding biases in interviews, according to one journal article when applicants of different racial groups apply, behavioral interviews seem to had lower group differences than situational interviews: the average of the differences between the two groups was .10 with behavioral interviews and .20 with situational interviews (this is from a paper called Racial group differences in employment interview evaluations). Although there are many ways to interpret this, my rough interpretation would be that behavioral questions allow less bias to creep in. However, this is only a single journal article, and I have to admit that I am not as well-read on behavioral interviews as I would like.
Thank you for a very thoughtful reply! (and for the sharing those papers :))