AI Broke Coding Interviews
While we’re debating how AI is transforming software engineering in general, there is a field within it that got completely flipped on its head: the technical interview.
The classic coding problem used in interviews, one that is self contained, tests some logic, data structures and algorithms, and can be reasonably solved in 30-45 minutes, is the exact thing that coding assistants are perfect at.
In other words, this is exactly what no software engineer ever needs to do again “on the job”.
Even before AI, “leetcode” style questions were getting less popular, with most companies moving toward “real world” type questions, focusing less on “gotcha” algorithms and more on practical “good engineering” practices.
But now, we end up in a funny situation:
In my experience, most companies will ask the candidate to turn off any coding assistant they have in their IDE, explicitly telling them to do the opposite of what they’d do on the job.
A minority will encourage candidates to use any tools “as you would in real life”. This sounds like the more sane alternative to me.
What’s the Interview For?
Back to basics: what we want out of the technical interview is to approximate the work the engineer does in real life, but in a very time constrained manner. Sure, this is flawed, but it’s the best we can do.
So how should we approach this in the age of coding assistants?
The typical interview process consists of some coding tasks and some design / architecture tasks.
The design / architecture ones are safe. We should probably focus on those even more because that’s going to be a larger part of the job (I wrote more on this).
As for the coding questions:
We should transform the coding questions even more into end-to-end, “real-world” styled projects.
We should allow coding assistants like we allow web search.
We should now present larger scoped problems, because some of the implementation details will now be handled by the coding assistant.
I would even err on the side of something that cannot be reasonably accomplished within the allotted time (see below).
We should focus on watching the candidate work alongside the coding assistant and pay attention to:
- How they drive the process
- How the design the end-to-end solution before handing off to the coding assistant
- How they break things down and prioritize tasks to get to a working solution, even if it doesn’t have 100% of the features asked for (this is a real skill they’ll need on the job)
- How they guard against the coding assistant producing garbage: do they critically inspect and modify the product before putting it into their solution?