Interview Questions for Software Engineers: The Epic Bug Hunt
This post was driven by Rands Writing Prompts, a weekly feed of topics for folks looking for inspiration or an excuse to practice.
My favorite interview question is a pretty simple prompt:
What is the worst bug you ever had to track down?
I would often use this one as a bit of an intro question. An invitation for them to talk about themselves — an opportunity to trade fish stories with a fellow engineer. For most engineers, that would help them be more comfortable in the interview while providing me some insight into the candidate.
The Signals
“What do you consider ‘worst’?” was a common follow up question. I would shrug and point out that part of the question was their definition of worst. This was the first signal I wanted from the question. Some folks would focus on strict technical difficulty (good). Some would pick a minor bug that allowed them to tangent into selling themselves (okay). Some wouldn’t pick a bug per se, but would describe some customer emergency where they flew in to be a savior (not good). The only really bad interpretation was the “I can’t think of one” or “I never really had a bug I couldn’t handle”. That sort of stuff just makes me wonder if the candidate has written code at all.
The second hiring signal that I was looking for was bug difficulty. The worst bug an engineer has ever tracked down correlates pretty well with the edge of their abilities. Engineers who have a lot of skill and experience tend to produce nastier bugs. Asking this question early allowed me to calibrate some of my later questions since the skill of a candidate doesn’t always match what’s on their resume or the expectations of the role. If you’ve been programming for twenty years and the worst bug you ever encountered was using string rather than StringBuilder, then that is concerning. Likewise, if a more junior candidate has been fighting distributed systems problems or had a team-wide issue that they realized was a leadership problem rather than a technical problem, then that shows potential.
The last hiring signal is simple communication skills that I look for in every question. Can the candidate describe the bug in a way I can understand? Do they use too much detail? Too little? Do they check with me to confirm understanding, or is it just a monologue? Are they candid about the actual problem (and their part in it)? Basic stuff, but important none the less.
The Noise
Beyond getting some hiring signal, what sets this question apart from others is that I get some great stories out of it. Candidates would have terrible, outlandish, hilarious examples from various industries. Power outages. Suddenly dangerous sex toys. Self-driving car fails. Million-dollar global catastrophes caused by a single missing character. In the best case scenario, it was a great conversation starter and stress release opportunity for all involved.
And that’s why I rarely ask it any more. Even in the best case scenario, it is still a behavioral interview question with all of the problems those entail. There is no consistent rubric for a question like this. A skilled lie is more effective than the truth. A great storyteller will succeed more than a great engineer. As much as I love to hear programmers’ epic bug hunt stories, the bias it introduces undermines any signal it provides.
Bonus Content
Some candidates would return the favor during the interview and ask me what the worst bug was that I ever had to track down. It might be a poor interview question, but it still produces entertaining stories. I have a few that I use for different audiences which I will be writing up over the coming weeks:
- The Debugger Bug — a more lighthearted bug, when I need to be a little self-effacing or help the candidate feel at ease.
- The Hamburger Bug — the hardest bug to actually track down, when I think that technical depth is most important to my audience.
- The Compiler Bug — the most impressive bugfix, when I need to drop names or otherwise show off to less technical folks.
