So, You Don’t Test? A Closer Look at Why You Should Consider It

None of the conversation or resources shared on this podcast should be construed to be legal advice. As a best practice, if you have concerns over these issues or opinions expressed, you should explore them with an attorney or other expert. And with that, let's begin.

With me again today is Mike Callen, the President of TestGenius.

Today our topic is You Say You Don't Test.". And let me give a little context of that. Our team attends about 25 conferences and trade shows a year. And sometimes people will come up to our trade show booth and they'll be looking at our backdrop and our sign and they'll look at and they go, oh, testing, you do testing. Oh, we don't do that. And sometimes it leads into a conversation. So we thought it would be beneficial today to talk about that So you say you don't test?

And, but how can we make sure that the job applicants have the requisite amount of success in that skillability area on day one of the job as is required by the job. So that's what we're trying to do. And now after we started off in that route, we've subsequently added a lot of soft skills tests and even some personality tests in different markets. And so we're doing more than hard skills testing, but also soft skills testing as well.

So what we're gonna talk about today is, why is it that an organization should absolutely consider to employ these skill and ability testing and personality, soft skills testing products ours and other people's as well. Particularly if they have an attitude where they're against doing so. So that's it.

Really this was a hundred percent inspired by real life conversations, that we have at conferences where, HR managers, recruiters, people that are right in the space that, the folks that we want to talk to and the folks who should be really interested in what we do sometimes walk up to us and say, oh, we either don't do this or we've been told we can't do this. Or even sometimes people say it's illegal to do this. And, of course that's not the case.

But that's what we want to do is break it down. And so I have five main points that I want to talk about with you today, Jenny. And within the first one, there's about 10 subpoints that are in there. And yeah, let's just do that. Let's just go ahead and kick it off.

So point number one whether you're testing or you're not testing, it's super important to know or remember that nearly everything that occurs in a recruitment is already a test. Even if you're not doing hard skills and ability testing or personality testing or behavioral testing, it's very likely that what you are doing some of what you're doing is literally testing. I think the next slide has a point on here.

What we want to talk about here when we talk about testing people tend to gravitate towards a test, a typing test, or a multitasking test or a personality test, and to think of that as tests.

But back in the 1960s there was a landmark case called Griggs versus Duke Power. And I want to say right up front I've learned all this from our CEO Dan Biddle I was not educated in this arena. And working around him and learning from him, I was able to understand the genesis of this particular topic, When the Civil Rights Act of 1964 came into to play there was basically it was explained that as an unlawful to discriminate against race or to not hire minorities intentionally.

And so to us it's duh. That's the most obvious thing in the world. But there was literally a time and a place and our not too distant past where this was a regular occurrence. And so the Civil Rights Act of 64 came out. It was ratified but it didn't come into effect until 1965 and the Duke Power Company was hiring for a particular position.

And so what they did is they decided to employ a test that they thought would screen out minorities because they had to now have some sort of decision making process in place. And they waited until the very, very end of 1964. as it turns out, that test, as they had intended, had really bad adverse impact. But furthermore, it wasn't valid or consistent with business necessity for this particular job. It was testing for things that went way above and beyond the requirements of this particular job.

And so there was a plaintiff named Griggs who, hired an attorney, and sued Duke Power because of this unfair testing practice. And during the course of this case it was determined or what was defined was something called PPTs, which we refer to a lot within, our internal Biddle Consulting Group space PPTs, as the slide shows stands for practices, procedures, or tests. And so when we talk about a test there's two layers of definition.

If you zoom right in on it, there's the testing that TestGenius does. If you need to type on a job, we're gonna give you a typing test that's gonna measure speed and accuracy within a certain amount of time with content that might be similar to the content that would be typed on the job.

That's a test and it's really obvious, but there are tons of other aspects of the recruitment process that are practices, procedures, and tests that result in some portion of an applicant population not advancing in a recruitment. And so therefore, any of these steps that occur within the recruitment process should be considered to be tests or PPTs practices, procedures, and tests.

Certain PPTs that are in place that maybe wouldn't directly screen a group of people out, but maybe because of how they are constructed or where they are posted or located, they actually end up precluding a portion of the population from even being included in the process from the very start. Most organizations use a software program called an applicant tracking system.

the main purpose for the applicant tracking system is to collect demographics at each step of recruitment and compare it to the census data on availability of the population within your recruitment area who would be available for this particular job. comparing the census data on availability and the actual results. It can see if there's a statistical disparity between the available population and the actual population that end up moving through each and every one of the steps that occur.

And unless you have any questions or comments, We can go ahead and move. Forward to the first one.

And there's some sort of sorting of what are the fundamental duties that require knowledge, skills, and abilities that are needed on day one of the job before there's on the job training or brief orientations So the job posting is going to have within it text that would include things like, here's the position of the job. It might be the pay that's offered. It might be the days and hours of the job. It might be minimal qualifications that are necessary to be successful on the job.

And then also desirable qualifications. We call those MQs and Dqs that are helpful and can maybe position one applicant ahead of another in terms of the job. And so when you look at the job posting, two really important aspects of it are one, where is that job posting located or posted to?

So if there is a venue, a newspaper, or a bulletin board or whatever that isn't inherently accessible by the same population that is the available population for the job, then you can literally have adverse impact right outta the gate. Let's say that there's some sort of newspaper that is predominantly read by white people, for instance.

And so if you have this newspaper and you're advertising your job predominantly to white people and their availability for people of color for this particular job, is significant but they're not seeing the job posting, then right out of the gate your demographic of your job applicants could be lacking as compared to the availability. The second thing that's really important about it is the content.

And so when you have a job posting, It's vital that before an organization posts something regarding available job to their website or newspapers or wherever they post them, that they stop and they do a review. Let's make sure that the things that we're saying about this job are correct. Say for instance, you have a job that you're posting for that requires people to pick up 50 pound bags of concrete and load them and stack them onto a pallet to get ready to have them shrink wrapped.

So if you go through and you post that but then later you find out that those bags are no longer 50 pound bags. The safety people did a review and they found out there was too many people getting hurt and they realized if they bumped 'em down to 25 pound bags, that not as many people would get hurt on the job, and therefore they made that change.

If you tell people you have to lift 50 pound bags all day long instead of 25 pound bags, and that's wrong, you have actually created adverse impact that you otherwise could have avoided. Understanding where a job posting is located and making sure that the content in that job posting is correct, is really vital to avoiding adverse impact right out of the gate that otherwise would not have occurred if you had done things correctly.

And again, to go back to the a ATS, the applicant tracking system, they're able to go through and know what is the demographic of our job applicants versus the availability. And to be able to look at those populations and to determine if there's any statistic differences between the two groups. So does that make sense?

Their body types are different, and they are more well suited to doing work like lifting concrete bags. if you gave a physical ability test that ask people to pick up these 25 pound bags or 50 pound bags, and they were to stack them on there, they may find that the rate at which women fail that test could be nine or 10 times the rate at which men fail that test. So that in itself is adverse impact. However, the test is a simulation of the job.

And if you cannot complete the test, then it's very likely to assume that you cannot complete the job. And so that's where that issue of validation comes into play. So on one hand or one one side of the shop, we want to make sure that we avoid adverse impact wherever we can avoid adverse impact.

But on the other side of the shop, the testing side of the shop, we go into that knowing that sometimes there are tests that are valid and they're consistent with business necessity that will either over the life of the test, during one recruitment, during a year of recruiting, Inevitably, you may find that there is some adverse impact that occurs, but if your test is valid and consistent with business necessity, then you are

accept it is an acceptable process to go through and to do that testing. And that's really a whole nother side of this particular issue we're talking about today. Which probably, I think we've explored it in a prior podcast where we talked more about validation, but we might want to dig into it again. Great question. Thank you.

So the second point is the job application. And people often don't even think about this as being a test. But job applications some of them require very minimal information, but some require more information. And sometimes the information that is collected or the way that the information is being requested can result in adverse impact.

One of the things that can happen, just for an example, is it may be an issue where somebody is asking on a job application if you have college degree or if you have a high school diploma, and those questions for a particular job may result in adverse impact against a protected group. And so the question becomes do, why are you asking that question? If you, we go back to the job where you know the person is gonna be stacking concrete bags.

What about a high school diploma Makes that an appropriate question? And then if you ask that question and you end up in a disparity of job applicants versus availability, now all of a sudden, you've caused adverse impact. So you want to be really cognizant of and analytical about the content that is on the job application, and you want to make sure that you're not asking things that are unnecessary. Very often people have asked questions regarding how far are you located from work?

The reason being that they may decide that the people who live closer to the office are less likely to be late on the job or have issues getting into work. And so what starts off as, maybe a well-meaning reason depending on where the work is located, it could end up causing adverse impact if the location is located way out in the suburbs where you may have less people of color versus the inner city where you may have more people of color.

And this is again, just an example, by asking that question and trying to not call people in to become applicants moving further on into the process, again, you may cause adverse impact. And so think about the questions that are being asked. And even if you have good well-meaning reasons the fact that somebody has to commute 30 miles to get into a job is not a good reason to not allow them to continue to become an applicant and then to move on to becoming a candidate for hire.

And I would reference just the Bay area of California, how many people there commute, two hours a day from 60 or more miles, and it's hundreds of thousands of people that do that probably.

And so to that end, on the latter part of this meeting today, we'll talk more about why it is that they need to consider using tests because there are really good reasons, compelling reasons why that's the case.

Maybe you do background checks for everybody because, there's security or merchandise or whatever. It's public safety. There can be really good reasons why background checks are used, but you want to make sure that you are applying those universally. Don't test people for this position for background checks if you're not gonna test these people over in this other position if they are in the same realm. And I want to differentiate there 'cause there could be a little confusion.

Let's say you're hiring the cash carriers that drive the armored cars and carry, money into, the banks. You may need to do background checks on all those people. It doesn't necessarily mean that you need to do a background check for admin assistance who work in the office who aren't around the cash at all.

However, if you have for instance, the entry level worker that would be the armed guard and the supervisor who would be the armed guard, those are two different positions, you should use the background checks on both of those sets of people so you're applying it equally.

That's something that happens and that is a test.

And I would imagine that nearly a hundred percent of the people who aren't using the traditional tests in point C are doing in interviews. They're not just hiring people outright from a job application, and so they are testing people whether, they'd like to think so or not.

And so even if you're not using traditional tests, you're gonna be using some of these items somewhere along the way, and you are testing because these are all, PPTs. These are all practices, procedures, or tests that are determining whether or not a human being is moving on in the recruitment process.

And it's probably important to to talk about adverse impact and disparate impact. Adverse impact is a aspect of the statistical disparity that is non not intentional. And that's usually what happens when there's adverse impact. Somebody will inadvertently post a job where it says you gotta lift a bag that's heavier than the bag that you have to actually lift on the job. And that would be adverse impact and it's unintentional adverse impact.

But one of the main reasons why you want to go through and do this is that you want to make sure that there isn't disparate impact that's in play. When you have a person who is in human resources, such as a recruiter who's making decisions, there's a lot of authority within that one human being.

if you have somebody who is not of a proper mindset who's making decisions, it could be that they are making decisions that are based upon invalid selection criteria that could be adversely impacting women or people of color or other protected groups in a way that's really darn insidious. And again, that's where we have these applicant tracking systems is to make sure.

No organization or leaders of an organization want to wake up someday and read about themselves in a newspaper because they've got some rogue HR person who's out there throwing away all the job applications from people with Hispanic surnames, for instance. It's just, that's just plain evil and wrong. despite what's happening in the news right now that's wrong and it's always wrong and, that's something that should never happen.

People should never be denied an opportunity where they otherwise should be given the opportunity to step through this process and become, go from an applicant to a candidate, potentially to an employee if you fail for a reason that's valid and consistent with business necessity along the way. That's all fine and dandy. So just important to understand that difference between adverse impact and disparate impact.

And now we're going to move on to the next main point that is subjective versus objective measures.

So if you're not using those kinds of concrete measures then what are you using to make your selections? And that's where you go to the other items that you pointed out. You've got your job posting, you got your job application, and then it comes down to the in-person interview. Now, your typical applicant isn't gonna be thinking about the job posting or the application, maybe unless they encounter something along the way that makes 'em go, wow, I don't really feel good about this.

Then they might ask a question. But if you are applying for a job and you have passed through the first two steps, the posting in the application, and you get invited in for an interview and you know that you have the requisite skills, abilities, maybe experiences that are necessary for this particular job, and you sit down and you do a job interview with someone and you don't get called back for something and you feel like that interview went well then that causes an undue amount of focus on

that subjective measure, that interview. And so really this has to do with just how we're situating a person in terms of how they feel about the process. And so if you back it up one step and you say, prior to the interview, I go and I take skill and ability tests, and those skill and ability tests are face valid to me. That is they feel like the kind of thing that I would be doing on this job that I've applied for.

And I feel like I don't do well on those tests and I don't get advanced to the next step, then I probably am gonna feel pretty well about that. If I am skilled enable, and nobody gives me a test. And I sit down and I talk to somebody and they interview me, and they're asking me questions that don't really make me feel like they're job related. They're just sorting out whether or not they like me.

Then it's really important for organizations recruiters, HR staff in particular, to own that and to say, is that a really, good process that we have in place?

It comes down to the point where you want to make sure that your job applicants feel as though the process was fair. And if they should have had a good chance to move forward, you know that the odds are that they will. And if they didn't have a good opportunity to move forward, it's for a good reason. And so that's where having objective measures in place is really valuable.

I don't think it's a stretch to say that an interview is a subjective measure. Anybody who's gone in for an interview knows that because we always prepare. And right. We make sure our hair looks good, we'd make sure we present well. We want people to feel good in our presence. Is there is there a place for subjective measures in the hiring process?

And so one of the reasons why we want to test is that the recruitment process does require almost always that there are subjective measures, like the interview that you're talking about where people have to go through and make a determination that's based upon, some set of human components. And it might come down to, I feel that they won't do a good job, or I feel that they won't fit in. Our test can't feel.

And so if that is a valid and valuable part of the process, that's something that a human can provide and probably a human alone. And so that's fine. It's absolutely fine, but you want to do it in a positive proportion so that there's a good feeling about that.

So you've got optics.

And If you are not offered a job or you are not advanced through a particular process with an organization, the HR should be thinking about what are the optics of that. And again, it goes back to the prior point when you're using objective measures. And if I go through and I'm asked to, take a Word test, an Excel test, and a PowerPoint test, and I struggle with those tests, I'm going to feel as though I probably didn't deserve to be moved on to the next step.

And if I don't get moved on to the next step, I'm not gonna feel really bad about it. I'm gonna, I might feel bad because I was looking forward to the job, but I'm not going to blame the employer or the process for that. Now, you're always going to have somebody who has a problem and who's gonna complain, but that's also a part of social media. But when you have 99 good comments and one bad comment, think about Yelp. You're gonna go, you're gonna look for a restaurant.

Somebody might have four stars. The last review might be a terrible review about service or something. But if there's 95 reviews behind it, you're willing to forego that one and then take in the opinions of the others. And I think that's a really important part of the recruitment process that often gets forgotten in terms of looking at these PPTs.

So now we're at work sample tests.

The first one is that, again, if I go and I take these tests and I don't do well on the tests, I feel like it's okay that I lost the opportunity to move forward in the process. I feel good about that. But the other aspect of this is that they can serve these kinds of work sample tests can serve as something called a realistic job preview. One of the best examples that we have for that is in our CritiCall testing.

We have this whole brand of tests for emergency services dispatcher, police, fire, 9 1 1 EMS, utility, ambulance, all those environments where people have to get phone calls from the public or from other people that they work with. Where issues are reported, it's a very high stress situation. They have to be multitasking and they have to be able to go through and work through that job.

And so if you have a good work sample test that puts someone who does not possess the skills and abilities to, for instance, multitask, the test itself is going to make you feel uncomfortable. Now, I have taken this test a hundred times since 1999 when we were first developing it, and I am not a multitasker, and I know exactly what's gonna happen in the test, and yet it makes me feel very bad about that job making me realize that I'm not a multitasker, I really don't want that job to begin with.

And so what I will do is I will self deselect from the recruit recruitment process. I will say, this job really isn't for me. And, maybe I wouldn't have been moved on, but I'm not gonna pursue this job any longer. But again, this is the most important part of this point. And this is the thing that people who don't do testing don't understand is that this self deselection process is so good for everybody, okay? Obviously it's good for the organization.

You didn't hire, train onboard and have fail somebody who should never have gotten the job to begin with. That's a really tragic issue. Not only, for that particular person, but for all the people that they work with. They've been waiting for this person to get hired. They're excited, they get to know this person. They come in and they fail, and then they get dismissed. That's a big problem. But again, to go back to this optics situation, it's really good for the person.

Juxtapose the person who took a work sample test failed and realized the job wasn't for them as compared to the person who got hired, onboarded, trained and then went on to fail on the job and get dismissed. How does that person feel in scenario B versus the person in scenario A?

And that's a hundred percent avoidable by going through and making sure that you have a good testing procedure that's in place wherever possible, exposing these job applicants to a realistic job review, giving them an out in the process, giving you an out that they also would accept. So it's a really super important point for HR departments to keep in mind.

There are certain maxims that are in place regarding human resources that determine what kind of tests should be used on the job. If you're just using Excel at a basic level and you give an advanced Excel test, you have tested beyond the job. You can't do that. If you have to lift 25 pounds of bags of concrete. You can't say you're only gonna be testing for five minutes, so I'm gonna have you lift 50 pound bags of concrete. You cannot do that. It's illegal to do that.

And the Dial case is a case where the Dial corporation tested on a line job at a level beyond the level of the job itself. And they ended up having to pay a lot of money out to plaintiffs because they did that. And so it's just one of those areas that's clearly wrong. So it's each organization's responsibility to make sure that their tests are valid and consistent with business necessity. When you look at our tests, we provide a tool that allow you to do that.

It'll literally generate a 30 plus page validation report that's for this particular position at your organization, complete with, a table that tells you of these seven tests, here's the ones that are valid, and here's some suggested cutoff scores that would show competence in this area. So we're literally walking you down the primrose path allowing you to do this in a safe way. Now, look at all those other PPTs, you're making decisions at those levels.

What do you have in place that allows you to make sure that your interview happens in a valid and business consistent way? What makes sure that your job application is done that way, or any of the other PPTs that we listed, or any other ones that we didn't list. You are already doing testing and you should be concerned about your validity.

And by the way, if you use concrete testing or even soft skills or behavioral testing, we give you the tools that are necessary to show that this is valid and consistent with business necessity, either in establishing it locally for that position or in transporting the criterion validity through a transportability report.

Again, we do a lot of work with dispatch centers and we're at a lot of conferences in that space, and I have over the years, many times, had the same conversation with different center directors and they will tell me that, I could sit down with a person and talk to 'em for five or 10 minutes and know whether or not they're gonna be successful on the job. I know absolutely positively that they're telling the truth. They have been around the game long enough. They know how people operate.

They know enough about the human nature as it pertains to this particular position, and they could have a brief conversation with somebody and determine whether or not somebody was gonna be successful on the job. However. Even though they would be successful in that regard. The problem there goes back to the optics aspect is I sit down, I have an interview with somebody and somebody decides through a short conversation with me that I can't multitask.

People talk to me all the time while I'm texting, therefore I can multitask. That's not the same thing, but in their mind it is the same thing. And so if I have a problem with that recruitment process, the best solution for that is having some sort of paper trail that documents the process. And so what the same people who talk to us at conferences and they say, I can sort out whether or not somebody's gonna be successful or not.

We'll finish that conversation by saying, but I choose to not to do that. First of all, I don't want to spend my valuable time talking to people who don't have the skill and ability necessary for success on the job. But more importantly, they understand this paper trail aspect and they understand that if somebody doesn't feel good about that part of the recruitment, it's probably the most highly subjective thing that we've talked about this entire conversation here.

But if somebody goes through that particular process and they're weeded out to some subjectivity in five or ten minutes, there isn't any paper trail that allows people to show that the process was valid and to have the degree of defensibility that you would want to have.

And so if you go back and you say, this center director is able to sort out whether or not people have the skills or abilities, then you can take that one step back prior, and you can say why don't we just give tests for those skills and abilities and personal characteristics, and let's make sure in a concrete way that they possess them. So the center director isn't put into that situation.

Having a testing process in place is a really important aspect because there is documentation all along the way as to what decisions were made and in most cases why those decisions were made. And if the test is gonna screen out the greatest number of applicants you want that to be the most concrete and most documented step of the whole entire process. And so that's the reason why we want to talk about the paper trail and make sure that there's a good paper trail in place.

If you have a job application, and every year as a process, HR is reviewing their application to make sure that their application is asking the things that should be asking and not asking the things that doesn't, then that's a part of the paper trail. Hey, your application is bad. We review it annually. Okay. That's documentation in your favor. If you go through and you have tests, are they valid and consistent with business necessity?

Our tests can validated by a group of subject matter experts in case of content validation in virtually the same amount of time as it takes one applicant to go through and take those tests. There's a couple of survey items. It takes a few more minutes, but essentially they go through that process. You choose a select hand. Chosen pool of subject matter experts that are good performers on the job. You ask them to go through and take these tests in validation mode.

They take the test, they answer the survey, and the validation report is delivered. And that's a really great bit of paper for the paper trail. Somebody says I don't think that test was valid. Email 'em a copy of the validation report. Here's our validation report. You don't think it was valid? Here's 35 pages of documentation that shows why. And in the instances where we've been asked for something like that by or a client will ask us, Hey, this is being challenged. What should we do?

And I say, send off the plaintiff's attorney a copy of the validation report and they send it off to 'em and we don't ever hear back from 'em again. It just evaporates because you can say, I don't think it's valid, but here's 35 pages that say otherwise.

So having this paper trail in place and especially not having some sort of bottleneck that somebody experiences that doesn't have a paper trail to justify it, wherein x percentage of the people are screened out and y percentage of the people continue forward is a valuable situation to have. What allows you to support your defensibility of the hiring process.

And so when we're talking to somebody who could be using this process to their benefit, and they say to us, I just don't have time to look at this right now because I'm busy recruiting people that aren't gonna stick because I don't know whether they're skilled and able, it's very frustrating for us. We're always trying to help people understand that the process is automating things that human beings would be doing otherwise.

And so there is a valid concern to say adding testing, may slow down the recruitment process just enough that we'll lose somebody who otherwise would've been a good hire. And it's important for us to hire people. But the flip side of that is if you're hiring the wrong people and you're turning them over, you're hiring the wrong people and you're having to recruit people using a bad, broken process to replace the person that you didn't get right to begin with.

And again, being able to test people in their bunny slippers is a really important aspect of a recruitment because you get an applicant in, as soon as somebody says this is a qualified applicant, you send them a test link probably before you could have scheduled them for a phone interview. You can have them test and you can determine whether or not they would be subject to or should be subject to a telephone interview. What was the second aspect of that you asked about?

It's gonna take, what, two, three weeks, a month before somebody realizes, okay, we've got 'em all trained up and they can't do the job. The effect upon the chemistry and the engagement of your existing employees is palpable. And you want to avoid that at all costs. These people are the people who are there for you. They're the people who are working there for you, and you want to hire people to help them out.

It's not a concrete number of dollars, but what one of our reps, Nick Brown will say, at the conferences, when somebody comes up and asks, what is it? have you ever received a resume from somebody who says they have experience using Excel, and then when they sit down to use Excel, they're putting numbers in the cells and pulling out a calculator to sum them because they don't know that Excel has functions that add those up for you? Everybody laughs but then they all say, yes, I know that person.

I've run into that person before. And so you want to encounter that situation ahead of time. You don't want to encounter that situation at the end of the process. So very important.

To the people who come up to me at conferences and say, we don't do this because of A, B, C, or D reason. I just highly recommend that you reconsider that. I think in light of the things that we've discussed today, it would be really hard for someone to maintain that position as being valid. Because, I don't think it really is. That's my opinion. You said a disclaimer right at the front. But I think there's a lot of good concrete reasons as to why that's the case.