Discriminant validity: meaning, examples and how to test it

Your candidates sit through a cognitive test, a leadership test and a communication test. What if two of them rank people in almost the same order?

Discriminant validity is the evidence that a test does not overlap with tests of different skills or traits. A low correlation with an unrelated test is the result you want.

This guide shows what discriminant validity means, how it looks in hiring and how to test it on your own scores. You will also get the cutoffs to trust and the steps to take when two tests overlap.

TL;DR:

• Discriminant validity is evidence that a test does not overlap with tests of different skills. A low correlation with an unrelated test is the result you want.
• Two overlapping tests give one signal counted twice. They cost candidates extra time and can make one trait count double in a weighted scorecard.
• Test it by correlating both scores for at least 100 candidates, correcting for measurement error, and comparing against a cutoff set in advance.
• There is no universal cutoff. Raw correlations below 0.30 are often treated as acceptable, while 0.85 and 0.90 are HTMT thresholds.
• When two tests overlap, drop one, merge them, re-weight them, or keep both on purpose without adding the scores together.
• Under EEOC guidance, the employer, not the vendor, stays responsible for making sure its tests are valid.

What is discriminant validity?

Discriminant validity is the degree to which a test stays separate from tests that measure something else. A coding test should not track a personality test closely, and a typing test should not track a reasoning test.

When scores on two supposedly different tests move together, one test is quietly measuring the other.

Discriminant validity is one half of construct validity, the wider question of whether a test measures what its name says. The other half is convergent validity, covered below.

Discriminant validity is also a matter of degree rather than a pass or fail stamp: the evidence can be weak, moderate or strong.

Three points to keep in mind:

• It compares tests, not ideas: You need scores from two measures taken by the same people.
• Low is good: A weak correlation with an unrelated test is the evidence you want.
• Context sets the bar: Two related skills, such as reading and writing, will always correlate a little. The question is how much is too much.

Why does discriminant validity matter in hiring?

Discriminant validity matters in hiring because every extra test costs candidate time, and an overlapping test adds that cost without adding information. Two tests that measure the same thing do not give you two signals.

They give you one signal counted twice, and the candidate sat through both.

Across the teams we work with, overlap tends to show up in three places:

1. Long assessment stacks: A role gets a cognitive test, a role-specific test and a situational judgment test, and two of them end up ranking people the same way.
2. Weighted scorecards: When two overlapping scores both carry weight, that one trait counts double in the final ranking, and the team rarely notices.
3. Fairness reviews: If a "leadership" test is really a verbal fluency test, it can screen out strong candidates who write in a second language. That becomes a compliance question, not just a quality one.

Key takeaway: a hiring process is only as informative as the number of genuinely different things it measures. Five stages that measure two things are two stages with extra drop-off.

What are examples of discriminant validity?

Examples of discriminant validity compare a test against something it should not track. A low correlation supports the test, and a high one is a warning.

The cases below use made-up figures to show what each result looks like.

Two of these deserve a closer look.

The leadership test that is really a reading test: Say a company's leadership test correlates at 0.86 with its verbal reasoning test. Leadership and communication are related at work, so some correlation is expected.

But 0.86 means the two rank candidates in nearly the same order. Promote off that leadership score and you are promoting the fastest readers.

The coding test that is really a reasoning test: A coding test that correlates strongly with a general reasoning test looks like proof that it works. On closer inspection, it means the coding test rewards the same general ability your cognitive ability tests already measure, so it adds little about whether someone can ship working code.

The pattern usually surfaces when you are reading technical assessment results and two score charts look like copies of each other.

What is the difference between convergent and discriminant validity?

Convergent and discriminant validity answer opposite questions. Convergent validity asks whether two tests of the same skill agree.

Discriminant validity asks whether tests of different skills stay apart. You want a high correlation for the first and a low one for the second.

You need both before you can say a test measures one specific thing.

Convergent and discriminant validity are checked together because each one alone can mislead. A test that agrees with everything has great convergent evidence and no discriminant evidence.

A test that agrees with nothing has great discriminant evidence and may measure nothing at all. Confirm convergent validity first, then check discriminant validity against a meaningful comparison test.

Is divergent validity the same as discriminant validity?

Divergent validity is another name for discriminant validity. The property, the evidence and the tests are the same.

Some writers prefer "divergent" because it pairs neatly with "convergent", and both terms appear in the same fields with no difference in meaning. If someone asks for divergent validity evidence, give them your discriminant validity results.

One related term does cause confusion. "Discriminative validity" is usually a third synonym, but in health and education testing it sometimes means whether a test can tell apart two groups already known to differ, such as experts and beginners.

That is closer to concurrent validity than to the overlap question on this page, so check which meaning a report is using.

How do you test discriminant validity?

Discriminant validity is tested by correlating two tests taken by the same people, then correcting for measurement error. Compare the result with a benchmark you set in advance, using scores your platform already exports.

1. Name the two skills and write down what you expect: Two tests you believe are distinct should correlate weakly. Two related skills will correlate moderately, so decide before you look how much is too much.
2. Collect both scores for the same candidates: Candidates who took only one of the two tests are no use here. Aim for at least 100 people with both scores.
3. Run the correlation: A spreadsheet CORREL function on the two score columns is enough to start.
4. Correct for measurement error: No test is perfectly reliable, and unreliable scores pull the correlation down. Divide the raw correlation by the square root of the two reliability values multiplied together.
5. Compare against your benchmark and record which number and cutoff you used.

A worked example: A team exports scores for 140 candidates who took both a leadership test and a communication test. The raw correlation is 0.62.

The two tests report reliability of 0.80 and 0.75. Multiply those (0.60), take the square root (about 0.77), and divide: 0.62 divided by 0.77 is roughly 0.80.

A raw 0.62 looked fine. A corrected 0.80 says the two tests overlap heavily, and the team should look at the fixes further down.

Pro tip: Step 4 is the one most teams skip, and it changes the answer most often. If the vendor does not publish a reliability figure for each test, ask for it before you trust any correlation between two of their tests.

For larger datasets, factor analysis adds an item-level view: each question should load on its own skill and not on the other one. A question that loads on both is a sign the two tests share content.

What are the Fornell-Larcker and HTMT tests?

The Fornell-Larcker and HTMT tests are the two named methods most research software uses to check discriminant validity. Both work on survey or test data where each skill is measured by several questions.

Hiring teams rarely run them by hand, but vendor reports and technical manuals often quote them, so it helps to know what each one means.

The Fornell-Larcker test (often called the Fornell-Larcker criterion) compares two numbers for each skill. First, take the average variance extracted (AVE), which is how much of the variation in a skill's own questions that skill explains.

Then take its square root and compare it with the skill's correlation with every other skill. The square root should be larger.

In plain terms, a skill should explain its own questions better than it explains anything else.

The HTMT ratio (heterotrait-monotrait ratio) compares how strongly questions from different skills correlate with how strongly questions from the same skill correlate. A value near 1 means the two skills cannot be told apart.

SmartPLS documentation now recommends HTMT over Fornell-Larcker and describes both Fornell-Larcker and cross-loadings as outdated, because they often miss real overlap.

The practical upshot: a Fornell-Larcker pass on its own is weak evidence, HTMT is stronger, and a report that quotes only the test that passed deserves a follow-up question.

What value shows discriminant validity?

Discriminant validity has no single universal cutoff. The numbers people quote each belong to a specific statistic, and mixing them up is the most common error on this topic.

The 0.85 and 0.90 figures are HTMT thresholds, not pass marks for a plain correlation.

Simply Psychology notes that raw correlations below 0.3 are often treated as acceptable, while stressing that the bar varies by field. SmartPLS sets 0.90 as its HTMT threshold.

Whatever you use, state the statistic, the cutoff and the reason. A number quoted without its method is not evidence.

What should you do when two tests overlap?

Two overlapping tests leave you with four options: drop one, merge them, re-weight them, or keep both on purpose. The right choice depends on which test is cheaper for candidates and which one ties more closely to the job.

• Drop one: If both tests measure the same skill, keep the shorter or more job-relevant one. Fewer stages usually means fewer candidates lost to drop-off.
• Merge them: If each test covers part of the skill, combine the strongest questions into one assessment.
• Re-weight them: If you must keep both, cut their combined weight so one trait does not count twice in the final ranking.
• Keep both on purpose: Sometimes two tests overlap but serve different stages, such as a quick screen and a deeper final round. That is fine if the team knows it and does not add the scores together.

We have seen teams cut a stage this way and lose nothing in quality of hire, because the stage they removed was repeating another one. Keep separate evidence on how well each remaining stage predicts job performance, since a test can be perfectly distinct and still be irrelevant to the role.

What should you ask an assessment vendor?

Assessment buyers should ask vendors for evidence, because the legal duty does not move to the vendor with the invoice. The EEOC's guidance on employment tests says a vendor's documentation may help, but "the employer is still responsible for ensuring that its tests are valid."

Under 29 CFR 1607.14, a test built around a trait should be shown to measure that trait, and the trait should be linked to important work behavior.

Automated tools add another layer. In New York City, Local Law 144 bars employers from using an automated employment decision tool unless it has had a bias audit within one year, with a public summary and notice to candidates 10 business days before use.

Five questions to ask, in order:

1. What evidence shows this test measures the skill it is named after?
2. How does it correlate with the other tests we plan to run alongside it, and was that number corrected for measurement error?
3. Which method did you use for discriminant validity, and what was the actual value?
4. What does this test not measure? A vendor who cannot answer has not checked discriminant validity.
5. Can we export candidate scores so our own team can repeat the check?

How does Testlify help you check discriminant validity?

Testlify helps by giving you candidate scores you can check, not just a pass or fail. Use them to run the correlation steps above on your own candidates.

The reporting and analytics page shows what each report contains.

The test library has 3,500+ validated tests across role-specific skills, coding, psychometric tests, cognitive ability, situational judgment and language. Pick one test per skill you care about instead of stacking several that overlap.

Want to see whether your current stages measure different things? Book a demo with our team, and we will help you identify gaps, remove overlap, and make every stage more purposeful.

Requently asked questions (FAQs)