What is predictive validity and why do recruiters need it?

Predictive validity is the evidence that a hiring assessment actually forecasts later job performance. Score candidates before they are hired, wait until there is real performance data, then correlate the two. The number that falls out, the validity coefficient, is the entire claim.

Here is where hiring teams get burned. A vendor says its assessment has strong predictive validity. The recruiter reaches for the rule of thumb they half-remember from statistics class, that a correlation above 0.7 is strong, and decides the vendor is overselling. Except nothing in the market clears 0.7. No selection method does.

The federal testing guide for employers puts "very beneficial" at anything above .35, and calls .21 to .35 typical for a single test. That gap between the imagined bar and the real one is expensive. It makes good assessments look weak, bad ones look unfalsifiable, and vendor claims impossible to compare.

TL;DR

• Predictive validity is one of the two criterion-related designs: test first, measure performance later, correlate. Concurrent validity collects both at the same time.
• Above .35 counts as very beneficial in federal guidance written for employers. Above .21 is likely to be useful. The 0.7 bar borrowed from general statistics would disqualify every selection method there is.
• Two federal sources put the practical ceiling for a single assessment between .40 and .50.
• The headline coefficients everyone quotes are genuinely contested right now. A 2022 re-analysis revised many of them downward and the correction is still being argued in print.
• Under the Uniform Guidelines the employer, not the vendor, carries the burden of showing a selection procedure is job related.
• So buy on the quality of the evidence, not on a decimal: the validation sample, the criterion, the fairness evidence, and whether the study transports to your jobs.

What is predictive validity?

Predictive validity is the degree to which scores on an assessment predict a future outcome, usually job performance, sometimes retention or ramp time. It is one of two criterion-related validity designs, and the only thing separating them is timing. If the criterion is collected at the same time as the test, that is concurrent validity. If it comes later, that is predictive validity.

The parent category matters when you read vendor documentation, because the words get used loosely. Predictive validity sits inside criterion-related validity, alongside its same-day sibling, concurrent validity. A third relative, discriminant validity, asks the opposite question: does this assessment stay clear of traits it is not supposed to measure?

One term to keep straight: the criterion. That is the outcome you are trying to predict, and it has to be a real work behavior you can observe and score. Get the criterion wrong and the coefficient is meaningless no matter how high it goes.

How do you measure predictive validity?

An assessment only proves its value if its scores tell you something useful about how candidates will perform on the job. Predictive validity measures that relationship by comparing assessment scores with real-world performance after people are hired, giving you evidence of whether the assessment can distinguish stronger future performers from weaker ones.

1. Define the performance criterion

Start with the outcome that matters for the role. Depending on the job, that could be quality ratings, sales output, productivity, retention after one year, or time to full productivity.

Define how that outcome will be measured before candidates take the assessment. This prevents the success criteria from changing after you see the results.

2. Assess candidates and preserve the full score range

Give candidates the assessment and record their scores independently of the hiring decision. Ideally, retain scores for everyone who participates in the validation study.

If you only study candidates who passed the assessment and were hired, you restrict the range of scores you can observe. That range restriction can weaken the correlation and make the assessment appear less predictive than it actually is.

3. Collect meaningful performance data

Give new hires enough time to demonstrate meaningful performance. A 30-day check-in may tell you more about onboarding than sustained performance.

For many roles, you need to wait through a complete performance or review cycle so the criterion reflects how employees actually perform once they are fully ramped.

4. Correlate assessment scores with performance

Once you have both sets of data, compare assessment scores with job performance outcomes. The resulting validity coefficient shows how strongly assessment performance is associated with subsequent job performance.

A stronger positive correlation suggests that higher assessment scores tend to be associated with better performance. It does not, however, prove that the assessment itself caused the difference.

5. Interpret the coefficient in context

The correlation is important, but it should not be treated as the entire decision. Consider your applicant volume, selection ratio, the cost of a bad hire, and the practical value of improving your hiring decisions.

You should also examine whether the assessment creates adverse impact or disproportionately screens out candidates from a protected group.

6. Revalidate when the job changes

A validity study describes the relationship between an assessment and the job as it existed when the study was conducted. If the role, technology, responsibilities, or performance standards change substantially, the existing evidence may no longer apply.

Treat predictive validity as an ongoing validation process rather than a number you calculate once and never revisit.

What is a good predictive validity score?

For hiring, above .35 is very beneficial and .21 to .35 is likely to be useful. That is the federal employer guide's own interpretation table, and it is far below the 0.7 figure that circulates in vendor marketing and general statistics explainers. Anything above .50 for a single assessment should prompt questions about the study, not applause.

Where is the ceiling? Two federal sources answer slightly differently, and the spread is worth knowing. The Labor Department guide states that validity coefficients, unlike reliability coefficients, rarely exceed r = .40.

The U.S. Office of Personnel Management's Assessment Decision Guide puts it at .50 for a single assessment, and treats .30 or higher as generally useful for most circumstances. So the honest answer to "what is good" is a range with a hard roof: somewhere above .21 to be worth using, .35 and up to be strong, and a practical limit between .40 and .50 for any one assessment.

The same guide lists what to weigh alongside it: the level of adverse impact, the selection ratio of applicants to openings, the cost of a hiring error, the cost of the tool, and the odds of hiring a qualified applicant by chance alone.

Predictive vs concurrent validity: what changes?

Timing, and what the timing costs you. Predictive designs test candidates and wait, so they mirror how you actually hire. Concurrent designs test current employees and use performance data you already have, so they are faster and cheaper, and they describe people who already survived your hiring bar.

For a hiring screen, the predictive design is the one that answers your question. The catch is that almost nobody has the patience or the volume for it, so most published evidence is concurrent, corrected after the fact with statistics. Which brings us to the argument currently running through the research.

Why the published coefficients are contested

Treat any quoted validity coefficient as an estimate with a method behind it, and ask which method. In 2022, a re-analysis in the Journal of Applied Psychology argued that the range-restriction corrections used in decades of selection meta-analyses were often applied where they did not belong, including to concurrent studies, and that this inflated the published numbers.

The revised figures reorder the familiar hierarchy. Writing in the Society for Industrial and Organizational Psychology's own publication, two HumRRO researchers summarize the revised validity estimates: structured interviews come out with the highest mean operational validity at .42, while cognitive ability lands at .31, fifth on the list, after decades of being treated as the focal predictor.

What survives every re-analysis is the shape rather than any single number: structured beats unstructured, the older coefficients were probably too high, and years of education or general years of experience sit near the bottom as predictors.

For a hiring team, the practical read is simple. Do not buy an assessment because a decimal on a slide is bigger than a decimal on another slide.

Two numbers produced under different correction methods are not comparable, and the people who built them are still arguing about it in print. If you want the predictor that holds up best across every version of this argument, it is a structured interview run the same way for every candidate.

Who is responsible for a test's validity?

You are. Not the vendor. If a selection procedure screens out a protected group at a different rate, the employer has to show the procedure is job-related and consistent with business necessity, and vendor paperwork does not transfer that duty. The Equal Employment Opportunity Commission states it plainly in its guidance on employment tests and selection procedures.

While a test vendor's documentation supporting the validity of a test may be helpful, the employer is still responsible for ensuring that its tests are valid under UGESP.

UGESP is the Uniform Guidelines on Employee Selection Procedures, adopted by the EEOC in 1978 and still the operative federal rule. It sets out three routes to showing a selection procedure is job-related, and criterion-related validation, which family predictive validity belongs to, is one of them.

The other two are content validation, where the assessment content represents important job behaviors, and construct validation, where the assessment measures a characteristic shown to matter for the job.

The guidance also sets an expectation that outlives any single study: if a procedure screens out a protected group, look for an equally effective alternative with less adverse impact. A high coefficient is not a defense against never having looked.

Can you rely on a vendor's validity study?

Sometimes, and there are conditions. Borrowing outside validity evidence is legitimate and often the only realistic option, because running your own study needs more people in one job than most employers have. What you cannot do is treat a vendor's study as automatically yours. The federal guide is explicit that evidence of transportability is required, and names what has to line up.

• The validation procedures used in the original study have to be consistent with accepted standards.
• Job similarity has to be verified with a job analysis showing your job and the original job are substantially similar in ability requirements and work behavior.
• Fairness evidence has to be considered for each protected group in your labor market, and where it does not exist, an internal fairness study should be run if that is feasible.
• Other variables have to match well enough: the performance measures used, the essential work activities, and how similar your candidates are to the original reference sample.

Read that list against the last assessment you bought. A technical manual reporting a coefficient from a different job, in a different labor market, with no fairness data for the groups you hire from, has not actually told you what you need.

What to ask a vendor before you buy

Four questions, and they are all about the study rather than the number it produced. A vendor who can answer these is selling evidence. A vendor who redirects to the coefficient is selling a decimal.

• How large was the validation sample, and who was in it? The federal guide tells employers to check the size and characteristics of the validation samples, and whether the statistical procedures were appropriate.
• What was the criterion? Supervisor ratings from one manager are not the same evidence as output data across sites.
• Was it predictive or concurrent? And if the figure was corrected, corrected how. See the argument above.
• What fairness evidence exists, by group? If the answer is none, that is your study to run.

Pro Tip: ask for the sample and the criterion before you ask for the coefficient. A .45 from 38 incumbents rated by a single manager is weaker evidence than a .28 from 400 hires across three sites, and the first number is the one that will be on the slide.

This is the loop the Testlify Quality-of-Hire Learning Model describes: connect pre-hire evidence to post-hire outcomes, find out which signals actually predicted success in your business, then adjust the assessments, weights and benchmarks accordingly. It is a methodology rather than an automated prediction engine, and it only closes when you feed real outcome data back in. Nobody can hand you validity for your jobs off the shelf.

Hire on evidence you can check

Running your own validation study starts with having the predictor side in a usable form. Testlify reports item-level psychometrics to recruiters, including a difficulty index plus percentile benchmarking and score logs, CSV and PDF exports.

See how Testlify can help you turn candidate assessment data into clearer hiring decisions. Book a demo to explore the assessments, reporting, and insights you can use to identify candidates with the skills your role actually requires.

Frequently Asked Questions (FAQs)