Can’t Live With Them, Can’t Live Without Them
With few exceptions, talent assessments present candidates with a series of closed-ended, multiple choice questions. We’ve looked at the value of pre-employment testing before: it takes around 70 multiple choice questions to establish a correlation with job performance. The shortest assessment of this type takes a little less than 30 minutes to complete. A lengthy multiple-choice test might not be the best experience for the candidate, but it makes identifying top performers much easier for a busy recruiting staff. Since assessment scores are predictive of performance, candidates who score the highest are more likely to be high performing. That’s the theory, anyway. But if you’re looking to create a fairer hiring process, a multiple choice test might not be the way to go.Bad News for the Talent Assessment: America, Europe, and Asia all have “Testing Cultures”
Standardized tests are deep-seated in the American education system. Every state has its own elementary school testing protocol, and nearly every American college requires applicants to report an SAT or ACT score. America is not unique in this regard: standardized tests are even further ingrained in European and Asian nations. Around the world, standardized tests have one thing in common: they’re mostly multiple choice. It would be impossible to grade every child’s test in a timely manner otherwise. Like most multiple choice tests, there are right answers and there are wrong answers. Now consider how most talent assessments are presented. Most will tell candidates that there are “no right or wrong answers.” Based on their years of taking standardized tests, in school or otherwise, citizens of testing cultures know this is a lie. Consider this common talent assessment question: My former employer would say that I get along well with others.- Strongly Agree
- Agree
- Disagree
- Strongly Disagree
The Dark Side of Traditional Assessments
Most assessment providers will tell you that “pre-employment assessment prep kits” won’t actually prepare candidates for their assessment (a quick Google search will return dozens of these assessment preparation sites). They argue that these sites are a scam, and that they never leak the “right” answers. They could be right. Then again, they might not. Game of Thrones gets leaked every year by one of the most profitable producers of original content, so it’s not unthinkable that the same could happen to an assessment provider. Either way, it’s a subpar situation for those that rely on the assessment to make their screening decisions. Let’s consider these two possible scenarios:Scenario #1: Test Prep Sites Sell Faulty Study Materials
Even if the study materials sold by test prep sites are flawed, the problem still remains: your candidates are not answering honestly. Maybe their flawed study materials actually made them perform worse on the assessment. Great – some of your best candidates (those who perform third party research and put the effort into studying) selected themselves into the worst scoring bracket.Scenario #2: Test Prep Sites Sell Accurate Study Materials
If the study materials sold by test prep sites are accurate, you’ve got an entirely different problem on your hands: you’re giving preference to candidates who study, rather than those who answer honestly. Which, if the testimonials on assessment prep sites are to believed, might be happening more often than you think:“Not only did I get the position I wanted, the testing coordinator said I achieved the best score on the personality test she had ever seen. The JobTestPrep guide was clear and well explained. It included several different approaches to cover all the personality test type variations and formats, and allows you to practice the strategy laid out in the guide.” – A Satisfied Assessment Prep Customer “I wanted to work at a certain software company, so I got the [assessment name removed] pack. Did all 5 tests and went through all the tips. I took the [assessment name removed] calm, prepared, and confident. I was hired.” – Another Satisfied Assessment Prep Customer
Don’t Blame the Candidate
People around the world have been trained from a young age to study for multiple choice, closed-ended tests. You can’t blame them for doing what they’ve always done to stay competitive. Candidates who study should not be penalized for making the most of the materials that are out there. But candidates who don’t should also be given a fair shot. When it comes to assessing talent, it’s critical that everyone is on the same playing field so they can let their true abilities shine.Don’t Blame the Assessment
You shouldn’t blame the assessment either. Pre-hire assessments have a decades-long history, and are backed by solid science – so long as the test takers are answering authentically. Unfortunately, the internet has made it easy to build a business around leaked exam materials. Don’t blame the candidate, and don’t blame the assessment: blame the delivery mechanism. The closed-ended, multiple choice questionnaire (that, to its credit, has worked for decades) looks like a odd, stagnant artifact in our rapidly evolving technological landscape. We need to bring this tried-and-true screening tool into the modern era.Open-Ended, Candidate-Friendly Assessments: Courtesy of Artificial Intelligence
Artificial intelligence (AI) is powerful because it can draw powerful inferences (like identifying cancer) based on terabytes of historical data. It’s also given us the unprecedented ability to make sense out of open-ended, unstructured information. With the ability to make sense of unstructured data (like keystrokes in games or responses in recorded video interviews), asking candidates a series of closed-ended questions is no longer necessary to gather the data you need to predict job performance. Instead, you can ask them to record open-ended responses to questions you (or a team of I-O Psychologists) have identified as predictive. These most often include:- Situational judgement questions: candidates are asked what actions they would take when confronted with a hypothetical situation,
- Scenario-based simulations: candidates simulate their actions in a hypothetical scenario.
- Past behavior questions: candidates are asked to relate past experiences and on-the-job behaviors.
