What should you include in your technical interview questions to ensure comprehensive candidate evaluation?

Technical interviews usually fail in a specific way. The candidate talks for 45 minutes, the panel nods along, and nobody can say afterwards what was actually proven. The questions were fine one at a time. As a set, they tested the same two things four times over and left three load-bearing skills completely untouched.

A technical interview question set should cover eight areas: tool and language fundamentals, applied problem-solving, reading and critiquing someone else's code, debugging under pressure, design at the scale the role really works at, collaboration and technical disagreement, explaining the work to non-technical colleagues, and the judgment to verify machine-generated output instead of trusting it. Each area gets its own question, asked of every candidate, scored against written anchors.

TL;DR

• Cover eight areas, not eight questions on the same area. Fundamentals, applied problem-solving, code review, debugging, design, collaboration, communication, and verification judgment each need a slot of their own.
• Ask the same questions in the same order of every candidate, and score with a written 1-to-4 scale. Structured interviews carry the highest measured validity of any single selection tool at r = .42, ahead of cognitive ability tests at .31.
• A 60-minute slot realistically holds one warm-up, two deep questions, one code-reading or debugging exercise, and 10 minutes of candidate questions. Anything more and you get shallow answers in eight areas instead of real ones in four.
• With 84% of developers using or planning to use AI tools, recall questions no longer measure what panels think. Shift the question from producing an answer to judging one.
• Your question set is a selection procedure under US federal hiring rules, and the rules name casual interviews specifically. An ad-libbed question set is the hardest kind to defend.
• Run the skills test before the interview, not after. The interview is for the things a test cannot see.

What should technical interview questions include?

Start from the role, not from a question list. The Testlify Competency-to-Evidence Matrix maps every role to the competencies that matter, then connects each competency to measurable evidence through assessments, simulations, interviews, references, and structured feedback. Applied to a technical interview, it means one plain discipline: write down what the job needs before you write a single question, then check that every competency has exactly one question pointed at it.

The order matters more than it sounds. A panel that starts from a question list ends up with whatever that list happened to cover, and the gaps are invisible because nothing on the page is wrong. A panel that starts from the role notices the blank row.

Here is what that looks like for a backend engineering role. Eight coverage areas, eight distinct pieces of evidence.

The table is the useful part of this article. Print it, cross off the areas your current question set already covers, and look hard at what is left blank. Most panels find they have four questions on fundamentals and nothing at all on verification or outward communication.

The applied problem-solving row is the one worth over-investing in, because it is the row that generalizes. If you want worked examples for that row specifically, there is a longer set of questions that test problem-solving with the signal to listen for on each.

Why does question structure change the outcome?

Structure is not bureaucracy. It is the whole reason the interview predicts anything.

The US Office of Personnel Management defines a structured interview with two conditions: all candidates are asked the same predetermined questions in the same order, and all responses are evaluated using the same rating scale and the same standards for acceptable answers. That is it. No exotic methodology, just consistency applied twice, once to the asking and once to the scoring.

The payoff is measurable. In the most recent major re-analysis of selection research, structured interviews came out with the highest mean operational validity of the common hiring tools, r = .42, above cognitive ability tests at .31. That reversed decades of received wisdom that put cognitive ability first, and it has a blunt practical reading: the interview you already run is probably your strongest predictor, and it is probably the part of your process with the least discipline applied to it.

Here is the catch, and it is worth sitting with. Those validity numbers belong to structured interviews. An interview where each panelist improvises from the resume in front of them is a different instrument wearing the same name, and it does not inherit the score. If your panel cannot produce the question list in advance, you do not have a structured interview. You have a conversation with a hiring decision stapled to it.

How many questions fit a 60-minute interview?

Fewer than you want. A realistic 60-minute technical interview holds one warm-up question, two questions explored properly with follow-ups, one hands-on exercise in code reading or debugging, and about 10 minutes at the end for the candidate to ask things. That is four scored items, not twelve.

Panels consistently overload the slot, and the failure is subtle: you still get an answer to every question, so nothing feels wrong. What you lose is the follow-up. The first answer to a technical question is rehearsed. The second and third follow-ups are where the real depth sits, and a 12-question agenda has no room for them.

If eight coverage areas will not fit in 60 minutes, that is correct, and it is an argument for splitting the work rather than cramming it. Move fundamentals to a skills test before the interview. Keep the interview for the areas a test genuinely cannot reach: how someone reasons out loud, how they take disagreement, how they explain a tradeoff to someone who does not share their vocabulary.

Pro tip: write the follow-ups into the question set, not just the questions. For every main question, draft two probes ("what would change your answer?", "what did that cost you?"). Panels that script follow-ups ask them. Panels that improvise them run out of time and move on.

How do you score technical interview answers?

With written anchors, decided before the first candidate. A rating scale without anchors is just gut feel with numbers attached, and it drifts over the course of a hiring round: the fourth candidate gets graded against the third rather than against the job.

A 1-to-4 scale is enough. Four levels force a decision because there is no safe middle.

Two rules keep a scale honest. Score each area immediately after its question, not at the end of the hour, because recency flattens everything into a single impression. And have each panelist score alone before the group talks, because the first number spoken out loud becomes everybody's number.

On what to actually listen for: the signal is the reasoning, not the polish. A candidate who says "I would check the query plan before adding an index, because I have guessed wrong about that before" has told you more than one who recites index types fluently. Confidence is not evidence. Specificity is.

What changes when candidates use AI tools?

Everything about the fundamentals half of your question set.

In the 2025 Stack Overflow Developer Survey, 84% of developers reported using or planning to use AI tools in their development process, up from 76% the year before. The same survey found 46% actively distrust the accuracy of what those tools produce, against 33% who trust it. Read those two numbers together and the hiring implication is specific: near-universal use, widespread doubt about correctness. The scarce skill is no longer producing code. It is knowing when the produced code is wrong.

So the question type that has lost the most value is the one most panels still lead with: explain this concept, name this pattern, write this well-known function. A model answers all three fluently, which means the question now measures preparation rather than capability.

Three replacements that hold up:

1. Hand them a wrong answer. Give the candidate 30 lines of plausible, machine-generated code with a real defect in it, and ask what they would change before shipping. Reviewing a confident wrong answer is the exact job now.
2. Ask for the verification step. Not "how would you solve this" but "how would you know your solution is correct". Testing habits, reading the docs, checking the edge case, reproducing the bug first. These are hard to fake and highly predictive.
3. Let them use the tools, and watch how. An open-tools exercise tells you more than a closed-book one. Testlify supports this directly with vibe coding, where candidates direct AI tools to reach a working solution instead of writing syntax by hand, and the evaluation sits on the result and the direction given rather than on recalled syntax.

The honest tradeoff: open-tools exercises are harder to score consistently, because two candidates can arrive at working solutions by very different routes. That is a real cost, and it is why the written anchors above matter more in an AI-assisted interview than they ever did in a closed-book one.

Is your question set legally defensible?

This is the part nearly every article on technical interviewing skips, and it is the part that turns into a problem at exactly the wrong moment.

Under the US Uniform Guidelines on Employee Selection Procedures, a selection procedure is not just a test. The text states that selection procedures include the full range of assessment techniques, from traditional paper and pencil tests and performance tests through informal or casual interviews and unscored application forms. Casual interviews are named. The same rules set the four-fifths test: a selection rate for any race, sex, or ethnic group that is less than four-fifths (80%) of the rate for the highest-scoring group is generally treated by federal enforcement agencies as evidence of adverse impact.

Put those together and the practical consequence lands on your question set. If your technical interview screens out candidates at materially different rates by group, the questions are the thing that has to be justified as job-related. And an improvised question set is close to impossible to justify after the fact, because there is no record of what was asked, no standard for what counted as a good answer, and no way to show the questions related to the work.

Three records make the difference, and all three are free if you build them in from the start: the written question list, the written scoring anchors, and the per-candidate scores with notes. Keep them and you can show the process. Skip them and you are reconstructing a defense from memory.

None of this is legal advice, and the specifics vary by jurisdiction. The design lesson travels anyway: write the questions down, score them the same way for everyone, and keep the paper.

How do you interview outside your own stack?

It happens constantly at small companies. A founder hires the first backend engineer. A frontend lead interviews a data engineer. Nobody on the panel can judge whether the answer about partition keys was good.

You cannot fake domain depth, so do not try. Stop grading correctness and grade the shape of the reasoning instead, which is visible without expertise:

• Did they ask what the system is for before designing it?
• Did they name a tradeoff and choose, or list options and stop?
• Did they say what they do not know, or paper over it?
• Did the explanation get clearer when you asked for it simpler, or just louder?
• Did they describe a failure in their own work, with what they changed afterwards?

Those five questions are answerable by anyone who has worked with engineers, and they map onto the collaboration and communication rows of the coverage table. Then cover the correctness half with evidence you do not have to judge yourself: a scored skills assessment, a work sample reviewed by a contractor or an advisor with the domain knowledge, or a short paid trial task. Combining several independent signals rather than one interviewer's impression is the point of the Testlify Multi-Signal Talent Evaluation Model, and it matters most precisely when the interviewer is the weakest signal in the room.

Borrowing a vetted question list for the role is also a legitimate shortcut when nobody on the panel owns the domain, as long as you still ask the same questions of everyone and score them the same way. A ready-made set for hiring a full-stack developer or for a technical support engineer role gives you a defensible starting point to cut down, which is a much better position than an empty page 20 minutes before the call.

Where an assessment belongs in the process

Before the interview. Not after it, and not instead of it.

Running a skills test first does two things at once. It filters on fundamentals without spending panel hours, and it hands the interviewer something concrete to dig into: a candidate who scored well on SQL but poorly on data modeling gives you an obvious, specific first question. A test after the interview mostly confirms a decision already made, which is why candidates read it as a hoop rather than a step.

What a test can carry: language and framework fundamentals, coding under realistic conditions (a project with multiple files, up to 20 test cases, visible or hidden), role-specific knowledge, and cognitive ability. What it cannot carry: how someone handles being contradicted, and whether a non-engineer can act on their explanation. Those stay in the interview, which is the real reason the interview still earns its hour.

The sequencing also fixes the overload problem from earlier. Move fundamentals out and the eight coverage areas fit comfortably, because the interview is only carrying four of them.

Mistakes that quietly wreck a question set

These are the ones that keep showing up in question sets that look reasonable on paper.

• Trivia dressed as rigour. If the answer is a fact a developer would look up in four seconds, the question measures memory, not skill.
• Puzzle questions with one trick. They sort candidates by whether they have seen the trick before, which is a proxy for interview practice, not for the job.
• Four questions on the same competency. The commonest fault and the hardest to see from inside, because every question looks individually fine.
• Letting each panelist pick their own. Three improvised sets mean three different jobs were interviewed for, and the scores cannot be compared.
• Grading communication style as technical skill. A quiet candidate who reasons well is not a weaker engineer. Score the two rows separately, exactly as the coverage table does.
• Asking for unpaid work. A take-home longer than about two hours selects for free time, which is correlated with things you are not allowed to hire on.
• Never calibrating the panel. Two interviewers using the same 1-to-4 scale can sit a full point apart on the same answer. Score three past candidates together once a quarter and argue about the gaps, and the scale starts meaning the same thing to everyone.

One more worth naming, because the market makes it tempting. Demand for engineers is not collapsing: the US Bureau of Labor Statistics projects employment of software developers, quality assurance analysts, and testers to grow 10% from 2025 to 2035, much faster than the average across occupations, with about 106,100 openings a year and 2025 median pay of $134,040. In a market like that, a sloppy interview does not just risk a bad hire. It loses good candidates to the company that ran a clear, fast, well-designed process, and strong engineers read a disorganized interview as a preview of the job.

Hire technical talent on proven evidence

Map your eight coverage areas, decide what the test proves and what the interview proves, and score both against written anchors. If you want the assessment half handled with coding tests, work simulations, and structured scoring that your panel can actually compare, book a Testlify demo and bring your current question list to the call.

FAQs