Hiring & Interviews
Hiring engineers on evidence: define the bar first, structured interviews with rubrics, work samples over puzzles, and independent feedback before debriefs.
Every practice in this playbook is carried out by people, and the most consequential decision a team makes about them is who joins. An interview loop is a measurement instrument, and most are bad ones: interviewers ask different questions, judge against private standards, then talk each other into a decision in a debrief where the most confident voice wins. The result predicts job performance only a little better than chance and is full of the interviewers' biases. Everything below exists to make the measurement better.
Define the Bar Before You Meet Anyone
Before the first interview, write down the work the person will actually do in their first year, the skills it requires, and what "meets the bar" looks like for each. If the hiring team can't agree on that list, no amount of interviewing will fix it; you'll hire whoever the loudest interviewer liked. The list also decides what you don't test: a backend role that never touches algorithms doesn't need a dynamic programming question, and every slot spent on something the job doesn't need filters out people who'd have been good at the actual work.
Structured Interviews
A structured interview asks every candidate for the role the same planned questions and scores the answers against a written rubric. It's the single biggest improvement most loops can make, and the evidence for it is unusually strong.
The classic meta-analysis by Schmidt and Hunter (1998) put the validity of structured interviews at .51 and unstructured ones at .38. A 2022 re-analysis by Sackett and colleagues found both estimates had been inflated by overcorrection, revising them to .42 and .19, and with that correction structured interviews came out as the best single predictor of job performance of the methods it covered. The gap between them roughly doubled: an unstructured conversation predicts less than half as well as a structured one.
Why it works isn't mysterious. You can't compare candidates who were asked different things; a rubric written in advance makes "good" mean the same to every interviewer; and interviewers who practice on a question see the range of real answers and drift less. Structure doesn't mean a script read aloud: follow-ups are where the signal is, and they just probe the same competency in the same direction for everyone.
Work Samples Over Puzzles
The closest thing to the job is the best test of it: reviewing a pull request with real problems in it, debugging a failing service, extending a small codebase, designing a system the team actually built. Brain teasers mostly measure whether someone has seen the trick, selecting for interview practice over engineering skill. Make the expectations explicit: what you're looking for, how long it should take, and that asking questions is part of the exercise, as on the job.
A Loop With Distinct Signals
Each interview owns specific competencies from the bar, and no two measure the same thing. Four interviews that each probe "can they code?" give one signal measured four times, plus four chances for a bad day to sink a candidate. Coding, system design, collaboration on a real problem and the role's domain give four signals.
Independent Feedback, Then the Debrief
Every interviewer writes their feedback and their decision before seeing anyone else's or talking about the candidate. Once a senior person says "strong no" in a debrief, the room anchors on it, and the independent signals you paid for collapse into one opinion.
The debrief then reviews evidence against the rubric: what did the candidate say or do, and which level of the rubric does that match. "Not a culture fit" and "I just didn't get a good feeling" aren't evidence; ask what was observed, and if nothing can be named, the concern doesn't count.
Reducing Bias
Structure is itself the most effective bias reduction, because it replaces private standards with shared ones. Beyond that, score before discussing, build diverse panels that catch each other's blind spots, distrust the "like me" signal (shared schools, employers or hobbies feel like evidence and aren't), and review outcomes: if pass rates differ sharply by group at one interview, look at that interview.
Candidate Experience
Every candidate is evaluating you as hard as you're evaluating them, and the ones you reject talk to the ones you'll want later. Tell people the process up front, how long it takes and what each step measures. Respond within days, not weeks. Give rejected candidates something useful when you can.
Take-home exercises need care. They shift unpaid hours onto candidates, and they quietly favor people without children or second jobs. If you use one, time-box it honestly (two to three hours, and design it so that's true), consider paying for longer work, and offer a live alternative. Never use a take-home on top of a full loop that tests the same thing.
Closing and References
Close the way you interviewed: on evidence. Tell the candidate specifically why you want them, what the first months would look like, and who they'd work with. References are most useful when they're specific ("tell me about a time they disagreed with a technical decision") and least useful as a formality checked after the decision is made.
References
In practice
- SIOP: Is Cognitive Ability the Best Predictor of Job Performance?, Sackett and colleagues' revised validities, with structured interviews highest at .42
- Master International: New Study Providing Updated Validity Estimates, Schmidt and Hunter's 1998 estimates (.51 and .38) beside the revised ones (.42 and .19)