The Layers of a Good Technical Question
Use layers of complexity to drive conversations that leave the candidate feeling stretched but successful.
New interviewers I’ve trained often ask me for help choosing a technical question for their interviews. There are many attributes important to making a good question including the experience level of the candidate, the specific role you’re targeting, company culture, and whether you’re looking for answers that pertain to a specific technology. One thing the best questions have in common is they have multiple layers that probe different skills, knowledge, or understanding. The layer allows candidates to take different amounts of time at each part of the question and leave your interview feeling accomplished. In this post, we’ll look at an example problem prompt and peel back the layers one at a time.
The prompt
You have access to an external API for fetching stock quotes based on their ticker symbol. Once retrieved, a quote is valid for 15 minutes before a new quote is needed. This API is expensive to call (in time or money). We’re using the following abstraction to handle the API call.
double GetQuoteFromApi(string ticker)
Write a function that optimizes the retrieval of quotes using the information provided.
Layers of a successful conversation
The candidate:
Understands the problem
Chooses the right data structure
Codes the solution
Extends on the solution
Optimizes for new information
Provides feedback to you
At each layer, be prepared to offer additional hints or guidance to help the candidate get to the next layer. Many candidates will get through the first few layers without any help but need some assistance later on in the process. Seeing how far candidates are able to go without help is one signal. Seeing what they can do with a little guidance is another. Creating software is a collaborative exercise. They won’t be doing it in a vacuum when they are working, so providing feedback and guidance to help them go further will show them what it’s like to work with you. Once the candidate gets stuck (even with hints) or you run short on time, help the candidate along towards the next layer. They’ll leave having learned something and with a more positive impression of the interview than if they felt like they failed to solve anything.
Put yourself in the candidate’s shoes, and look where you’d start having some challenges. This can be a good point to start transitioning from the candidate solving the problem alone to the two of you discussing it and working towards the next phase of the solution together. If you get to a spot where the candidate can’t add much more to move the question forward, switch gears one more time to teach them something new.
As the candidate completes each layer, check back on your agenda and figure out if you have enough time to go further. The deeper layers and follow-ups to your problem might not need to be completed in order. Adjust your plan on the fly to look for strengths with the time you have left.
Layer 1 - Understands the problem
Your first task is aligning with the candidate on expectations for the problem.
Is the candidate able to recognize what this problem is asking them to do?
If not, do they ask clarifying questions to figure it out?
Does the candidate suggest well known general solutions to this kind of problem?
The candidate should recognize that this problem is asking them to create a simple caching mechanism with a 15 minute validity window. The most common problem I’ve seen at this layer is an attempt to jump towards using an external cache provider like Redis, ElasticSearch, output caching, or even a database. I redirect towards an in memory “keep it simple” approach when this happens.
Layer 2 - Chooses the right data structure
Once you’ve established a shared understanding of the problem, get the candidate to talk through their strategy for solving it. At this point, they should only be jotting down notes. No code yet!
Does the candidate jump to a HashTable / Dictionary based solution?
Can they explain why they chose that structure?
If another structure is chosen, is there something specific they’re trying to optimize for other than retrieval time?
Do they suggest creating a Quote struct or class to contain the value and a timestamp?
If not, how are they keeping track of the expiration? Some candidates less comfortable with OOP will use an Array or Tuple here.
Layer 3 - Writes the basic code
Next, ask the candidate to write the code for the solution you’ve discussed. If you’ve had a smooth conversation so far, this should be straight forward. If he or she struggled to communicate their choices in Layer 2, they have a chance to make up for it with stronger written code.
The emphasis on this layer will vary from writing syntactically correct code when interviewing virtually with a good editor to getting the idea and logic out quickly on a whiteboard in person. In our example, the solution itself is pretty simple. The only tricky part below is the syntax on the date difference. I let candidates hand wave over this bit or define an IsExpired boolean value in their Quote object. There are lots of different ways to structure the if statements here. I don’t get too particular about it. A working C# solution for this stage is available here.
Are there bugs in the candidate’s code? Are they using UTC? If not they’ll need to handle daylight saving time. Yikes!
Are they capturing the basic requirements of populating the cache and checking for expiration?
public double GetQuote(string ticker) {
Quote result;
if(!_cache.ContainsKey(ticker) || (DateTime.UtcNow - _cache[ticker].TimeStamp).TotalMinutes >= 15) {
result = new Quote() {
Value = GetQuoteFromApi(ticker),
TimeStamp = DateTime.UtcNow
};
_cache[ticker] = result;
}
else {
result = _cache[ticker];
}
return result.Value;
}
}
public struct Quote {
public double Value { get; set; }
public DateTime TimeStamp { get; set; }
}
Layer 4 - Extends the solution for completeness and advanced use cases
Once the candidate has developed a working (or at least logically correct) solution, you can extend in a number of directions.
How would you write unit tests for your solution? Would this change anything about your design?
If you’re running this system on a web server with many users, how would you enable multithreading? What are the critical sections in your code?
In a long interview, these follow-ups can be powerful performance indicators for many technical questions. If time is short, try to probe for these skills in other parts of your discussion.
In our example, the candidate would want to mock out the external API call and make the expiration time a configuration parameter, rather than hard code it, to support unit testing. In the multithreading case, use a concurrent dictionary to get fine grained locking or evaluate the effectiveness of a simple lock around all of the writes. This layer can help you separate candidates between those who can write and understand their code and those who can think through the implications of their solutions in a deeper way.
Layer 5 - Optimizes for new information or constraints to drive further discussion
At this point in the discussion, I like to make things a little more complicated by adding a new constraint to the prompt. I add these after the initial code is written, rather than at the outset, to avoid premature optimizations. I love spending time discussing tradeoffs between solutions with strong candidates. If you have time to make a lot of progress in this layer, you’ll have good signals about the candidate’s skills.
In the example, we add a memory constraint to the problem as follows. The premise is somewhat contrived, so even good candidates can get hung up here.
We’re really cheap, so let’s discuss how to extend our solution if we only have memory capacity for a fraction of the universe of quotes.
Do you need to reiterate the goals of the original problem? Those are still in place.
Can the candidate identify this step as a cache eviction (or page replacement) policy?
Are they able to name and describe a few policies like Least Recently Used, Least Frequently Used, Random, or Clock?
Can the candidate connect their chosen strategy back to a run time complexity?
How does their strategy handle domain specific events like IPOs?
Can we optimize for memory usage? How can we use a single bit to optimize our eviction policy? Check out the Clock algorithm.
When the candidate gets stuck in these steps, how does it impact them? Do they demonstrate curiosity to learn something new from the question?
Layer 6 - Provides low level details and feedback
The best candidates will also help make you better. If he or she has successfully completed the earlier layers, they may be able to provide additional insights, such as new solutions or suggestions about your problem.
Here are some changes I hope candidates might propose to the solution presented above:
The decimal type is probably better for storing money values, but has a larger memory footprint than a double.
Depending on the language and compiled architecture, the referenced bit for Clock will still need a word in memory. You can manipulate the least significant bit on the value or timestamp to ignore this.
Your question probably won’t follow the exact steps I’ve laid out here. However, if you structure the layers from broad and basic to narrow, challenging, and esoteric, you’ll ensure that all candidates will feel like interviewing with you was time well spent. With practice proctoring your problem, you’ll smooth out candidate questions and learn to give the right hints without giving away the answer. You’ll also get a feel for the progression of the work from “You do this,” to “We’ll do this together,” and then to “I’ll show you how to do this.” They’ll have a chance to show you what they know, get collaborative, and learn something new from you. Maybe the candidate will even teach you a new way to look at your problem.
Thanks for reading. Leave a comment below to let me know if this post changed how you’re going to ask your technical question in your next interview. Up next time, we’ll dive into how to make candidates you’re not going to hire have a good experience so they’ll tell their friends about your company. Speaking of telling friends — if you share this post with your friends and subscribe to the newsletter, you’ll get more candidate experience tips in your inbox weekly.
Layer 5: If you put a placeholder value into the cache when you kick off the API call, and subsequent (concurrent) calls that see that placeholder and wait instead of making their own API call, then you can avoid making two calls to the API at the same time even when two users request the same quote simultaneously. :-)