Interviewing and Assessment
Interviewing is the stage where hiring decisions are actually made — and it's shockingly easy to do badly. Unstructured interviews driven by gut feel and "culture fit" turn out to be poor predictors of job performance and reliable amplifiers of bias, yet they remain the default almost everywhere. The research points clearly toward a better way: structured, consistent assessment focused on the actual skills the job requires. For engineers who interview candidates, learning to assess well — not just chat and go with your gut — is a genuine, learnable skill.
Interviewing and assessment — evaluating candidates to decide who to hire — is the core of hiring, and it’s often done poorly. This post covers what to assess (job-relevant skills), the case for structured interviews (consistent, more predictive, less biased), how to avoid bias, and practical assessment. It builds on the hiring process (interviewing is the key evaluation stage). Assessing well is a learnable skill that most people (and companies) do worse than they think.
The problem with typical interviewing
Typical interviewing — unstructured, gut-driven, “culture fit”-focused — is poor at actually predicting job performance, and it’s the widespread default:
- Unstructured interviews are weak predictors. The common unstructured interview (a free-form chat, different questions per candidate, gut-feel judgment) is a surprisingly poor predictor of actual job performance — research consistently finds unstructured interviews weakly predictive. Yet they’re the default almost everywhere. Unstructured interviews predict poorly. Gut-feel chats don’t work well.
- Gut feel is unreliable and biased. Judging candidates on gut feel / general impression is unreliable (feels confident, isn’t accurate) and biased (gut reactions reflect biases — favoring people like us, superficial impressions). Gut-based hiring amplifies bias and misjudges. Gut feel is unreliable and biased. Confidence isn’t accuracy.
- “Culture fit” often masks bias. Judging vague “culture fit” often masks bias — “fit” frequently means “like us/me,” which reduces diversity and screens on similarity rather than the ability to do the job. Vague culture-fit judgments are a common vehicle for bias. “Culture fit” often means bias. Beware fuzzy fit.
- Most people overrate their interviewing. A humbling truth: most interviewers overrate their ability to judge candidates (feeling their gut is accurate when it isn’t). Recognizing that typical interviewing is weak (and that you’re probably not the exception) is the first step to doing it better. Most overrate their interviewing. You’re probably not the exception.
Typical interviewing (unstructured, gut-feel, “culture fit”-focused) is a poor, biased predictor of job performance — yet it’s the widespread default, and most interviewers overrate their ability at it. Recognizing this weakness is the first step. The better approach starts with assessing the right things — job-relevant skills.
Assess job-relevant skills
Good assessment focuses on the skills and abilities the job actually requires — not vague impressions or irrelevant factors:
- Assess what the job needs. Effective interviewing evaluates the specific skills, abilities, and qualities the role requires — defined by what the job actually needs (from the hiring criteria — the process post). Assess job-relevant capabilities (can they do this job well?), not general impressions or unrelated traits. Assess job-relevant skills. Evaluate what the role needs.
- Evaluate actual ability, not proxies. Where possible, assess actual ability to do the work — through work-sample-type evaluation (having candidates do representative tasks) or well-designed questions probing real skill — rather than proxies (credentials, pedigree, how well they interview-chat). Demonstrated ability predicts better than proxies. Assess real ability, not proxies. Can they actually do it?
- Focus on relevant, not irrelevant, factors. Assess factors relevant to job success (skills, relevant experience, problem-solving, how they work) and avoid irrelevant ones (superficial impressions, similarity to you, unrelated traits) that don’t predict performance but do introduce bias. Focus on what predicts performance. Relevant factors only. Judge on job-relevance.
- Define what “good” looks like beforehand. Decide in advance what you’re assessing and what a good answer/performance looks like (clear criteria) — so you evaluate consistently against defined standards, not shifting gut impressions. Pre-defined criteria enable fair, consistent assessment. Define “good” in advance. Set the bar before, not after.
Good assessment focuses on the job-relevant skills and abilities the role actually requires (evaluating real ability, ideally via work-sample-type tasks, not proxies or superficial impressions), with clear criteria defined in advance. Assessing the right things is the foundation — and doing it consistently is the case for structured interviews.
The case for structured interviews
Structured interviews — consistent, predefined questions and evaluation for all candidates — are substantially better than unstructured ones, and this is one of the clearest findings in hiring:
- Structured = same questions, consistent evaluation. A structured interview asks all candidates the same predefined, job-relevant questions and evaluates them against consistent criteria — versus unstructured (different questions, gut evaluation per candidate). Structure means consistency and comparability. Structured = consistent, comparable assessment. Same questions, same standards.
- Structured interviews predict better. Research consistently shows structured interviews are substantially more predictive of job performance than unstructured ones — the consistency and job-relevance make them better at identifying who can do the job. Structure improves prediction. More accurate hiring. It’s one of the clearest hiring findings.
- Structure reduces bias. Consistency (same questions, same criteria for everyone) reduces bias — everyone’s evaluated on the same job-relevant basis, rather than varying gut impressions (which carry bias). Structure makes assessment fairer and more comparable (you can compare candidates on the same questions). Structure reduces bias and improves fairness. Consistency curbs bias.
- It’s a clear, underused improvement. Structured interviewing is a clear, evidence-backed improvement over the typical unstructured approach — yet it’s underused (unstructured remains the default). Adopting structure (consistent job-relevant questions and criteria) is one of the highest-leverage hiring improvements. Structure is a clear, underused win. Adopt it.
Structured interviews (same predefined job-relevant questions and consistent evaluation criteria for all candidates) are substantially more predictive of job performance and less biased than unstructured ones — a clear, evidence-backed, yet underused improvement. Adopting structure is a high-leverage hiring upgrade. Beyond structure, actively avoiding bias matters.
Avoiding bias
Bias in hiring — favoring or disfavoring candidates for irrelevant reasons — is pervasive, harmful, and worth actively countering:
- Bias is pervasive and harmful. Human judgment carries biases — favoring people like us, superficial impressions, and many others — that distort hiring (misjudging candidates on irrelevant factors). Bias hurts hiring quality (misses good candidates, picks worse ones) and fairness/diversity (systematically disadvantaging some groups). Bias is pervasive and damaging. It hurts quality and fairness.
- Structure and job-relevance reduce bias. The best defenses (above): structured interviews (consistent, comparable) and job-relevant assessment (evaluating actual ability, not similarity/impressions) — which anchor judgment to relevant factors, reducing the room for bias. Structure and relevance are the main bias defenses. Anchor to job-relevance.
- Awareness and process help. Beyond structure: awareness of common biases (knowing they exist helps you check them), diverse interviewers (multiple perspectives), evaluating on evidence against criteria (not gut), and consistent process all reduce bias. Deliberate anti-bias practices (not just good intentions) matter. Awareness and process reduce bias. Deliberate practices, not just intentions.
- It improves quality and diversity together. Reducing bias improves hiring quality (judging on real ability, not irrelevant factors, finds better candidates) and diversity (not screening out good candidates who are “different”) — these go together (bias reduction serves both). Reducing bias is both fairer and more effective — a rare win-win. Less bias = better and fairer hiring. Quality and diversity together.
Avoiding bias (favoring/disfavoring candidates for irrelevant reasons — pervasive and harmful to both quality and fairness) is served by structured interviews, job-relevant assessment, awareness of biases, diverse interviewers, and evidence-based evaluation — which improve hiring quality and diversity together. Interviewing and assessment done well — job-relevant skills, structured and consistent, bias-aware — is a learnable skill that most do worse than they think. Next: onboarding — the neglected step after hiring.
Key takeaways
- Typical interviewing (unstructured free-form chats, gut-feel judgment, vague “culture fit”) is a surprisingly poor and biased predictor of job performance — yet it’s the widespread default, and most interviewers overrate their ability at it (culture fit often masks bias by favoring people “like us,” reducing diversity).
- Good assessment focuses on the job-relevant skills the role actually requires — evaluating actual ability (ideally via work-sample-type tasks) rather than proxies (credentials, interview-chat charm) or irrelevant factors — with clear criteria for what “good” looks like defined in advance.
- Structured interviews (the same predefined job-relevant questions and consistent evaluation criteria for all candidates) are substantially more predictive of job performance and less biased than unstructured ones — one of the clearest findings in hiring, yet underused, making it a high-leverage improvement.
- Bias (favoring/disfavoring candidates for irrelevant reasons — favoring people like us, superficial impressions) is pervasive and harmful to both hiring quality (misjudging candidates) and fairness/diversity — countered by structure, job-relevant assessment, awareness of biases, diverse interviewers, and evidence-based evaluation.
- Reducing bias improves hiring quality and diversity together (judging on real ability finds better candidates and doesn’t screen out good “different” ones) — a rare win-win — so interviewing done well (job-relevant, structured, bias-aware) is a learnable skill that most people and companies do worse than they think.