Interview Coding Challenges: Hire the Best Developers

How do you use interview coding challenges?

There’s a lot of discussion back and forth about coding challenges in technical interviews. But with years of experience, we can safely say that they are a must for successful hiring processes.

That said, you DO need to use the right type of coding challenge to keep candidates happy and identify the right developers.

Here’s what you need to know.

What are coding challenges?

First, let’s define coding challenges.

Coding challenges are short tasks that candidates complete either in-person or remotely. Their purpose is to help hiring managers choose the right candidates based on their real-world skills.

The problem they solve is simple:

It’s really hard to hire developers. Based on just interviews, assessing someone’s coding ability is difficult. In fact, 74% of employers say they’ve hired the wrong candidate in the past.

Worse, interviews are prone to bias. People look for people like themselves when interviewing them for jobs.

Skill assessment tests, on the other hand, are effective for measuring a candidate's skills, and they make the hiring process more objective.

Skill assessments:

• Mirror real-world work that a developer would complete at the job
• Are standardized, so everyone gets the same challenge
• Give data and a grade instead of a “yes/no” result

76% of organizations with more than 100 employees rely on some form of assessment test. And as a study published in the Journal of Applied Psychology shows, practice tests can increase the quality of applicants, reduce the cost of testing unqualified applicants, increase the chance of a successful hire, and increase human capital.

According to Aberdeen Group, organizations that use pre-hire assessments are also 24% more likely to hire employees who exceed their performance goals.

But developers don’t tend to be overly excited about coding challenges. Here’s one example of a disgruntled candidate and here’s another.

The reason? Most coding challenges don’t offer a great experience.

Even worse, they don’t help the hiring process because they don’t measure candidates’ real-world skills.

So, are they even worth it?

That’s what we’ll look at next.

Do coding challenges work?

There are different types of coding challenges.

With years of experience in technical recruitment (and having talked and worked with hundreds of technical teams), we’ve seen that there’s a few types of coding challenges that works.

Note that the specific coding challenge you choose for your company depends on the seniority level of your candidates. More experienced developers don't want beginner-level questions. Good coding challenges for beginners, on the other hand, are often challenges where the developer gets to prove their skills without the test being too hard.

Here are your alternatives:  

Take-home coding challenges

Take-homes are coding challenges that developers complete on their own time. The reason they’re such a great way to assess developers is that they measure real-world skills because candidates complete the same tasks in the same way as if they were part of your team.

The best part? Take-homes are objectively assessed. You understand candidates’ coding skills, instead of basing your hiring decision on arbitrary “first impressions” or subconsciously hiring people who look like you or have the same background.

You can (and should) hold a feedback session after your assessment to talk through the challenge with candidates. This is an excellent opportunity to discuss their thought process and ways of working.

That said, take-homes aren’t the only alternative. We realize that there are circumstances and situations that make companies want to choose a different type of coding challenge.

Pair-programming

Another good alternative is pair-programming.

If you want to work through the project with your candidates, this might be the right type of interview for you. You see, pair-programming typically involves a coding challenge that you and the candidate solve together.

The reason pair-programming is a second-best alternative is that it can be stressful like whiteboarding tasks. That’s why our #1 recommendation is take-home coding challenges.

A pair-programming interview is best kept for senior candidates who won’t be as intimidated by the live-coding experience. Pair-programming interviews can also be a great way to discuss a candidate's take home challenge in a follow up interview.

And why aren’t whiteboarding tasks a valid option? Let’s take a look.

Whiteboarding

Whiteboarding is an in-person coding challenge, where candidates solve a task on a whiteboard in front of you/the hiring team. Obviously, this is a stressful situation. And as studies show, whiteboarding tasks are a better measure of how well someone deals with stressful situations than their actual coding skills.

Even FAANG companies are shying away from the whiteboard interview more and more. We’ve seen that a bad whiteboard interview can leave candidates with a really negative impression of a company and also, that the company itself may lose out on otherwise awesome candidates who got nervous - what a waste!

Screening questions

Finally, there are screening questions. These are quiz-like questions; they’re often pretty short so they’re not exactly full-blown coding challenges.

Either way, we don’t recommend screening questions. They don’t measure candidates’ skills and only serve to distract you and candidates from the real goal -- to identify the best applicant.

The only situation where screening questions are worth it is if you have too many applicants (many enterprises and big tech companies have this problem) and need to filter out underperformers quickly.

How are coding challenges assessed?

To make the right hiring decision, you need the right types of tests. And to learn how other companies are doing this, we took a look at content by Matasano, corrux, and Atlassian. Here’s what you need to know about creating and assessing your test.

(Note that Atlassian uses a smaller test before their take-home challenge. For a company like Atlassian, that’s reasonable. But smaller companies with fewer applicants should likely focus on just one challenge.)

Use the same test for everyone

You might err on the side of thinking that because someone has a great GitHub profile or resume, they must be really good. That’s not necessarily the case. Use the same tests for all candidates with the same assessment criteria.

Try different tests

Use different tests until you find a winner that gives you as much data as possible. For example, Matasano used a test that asked candidates to write a fuzz testing tool, which seemed like a great test at first because they got specific code to read. But after a while, they realized that they didn’t learn anything from the tests.

So even if a challenge seems like a good one at first, keep testing.

Avoid stressful situations

You want objective data on coding skills, not stress management skills. Avoid any “gotcha” moments or tasks that ask candidates to be clever, rather than straightforward tasks based on real-life work. And prep candidates so that they have the information and support they need.

Share your scoring criteria beforehand

Coding challenges are no competition. You just want to identify the best person for your team, that’s it. That’s why you should share the scoring criteria with your candidates.

Plus, candidates will understand your culture and values (which helps you weed out candidates who don’t see eye-to-eye on them). For example, do you look at how complex their solution is, even if they solved the challenge successfully? Do candidates get points for including tests and documentation?

What’s more, with objective scoring, you’ll also save your reviewers’ time and giving feedback will become a lot faster and easier. Plus, you’re more likely to hire the right candidate because you reduce the risk of bias in the decision-making process.

Interested to learn what this can look like in practice?

You can see corrux’s scoring criteria here.

Atlassian, on the other hand, looks at things like:

• Do candidates leave good documentation?
• How do they approach problems?
• Do they run tests and do they do enough testing?
• Do they follow the requirements?
• What technologies do they use?

Use a time limit and clear task descriptions

You don’t want to waste your or the candidate’s time. Use a clearly defined task and a time limit that indicates how long a task should take. 3-4 hours is a good rule of thumb for most companies.

Use real-world tasks

Your task should mirror tasks that someone would work on at the job. That means: no inverted binary trees or other “brain teasers” but actual problems they’d solve if they were part of your team. This is especially important for companies that don’t have a flood of candidates (so, most companies out there).

Give feedback

One of the most important parts of the process is to give feedback. Candidates have invested their time in your hiring process, so it’s only fair that you share feedback and show them how they compare to the average candidate.

Plus, you never know if someone will be a good fit in the future. Make your hiring process an enjoyable learning process for them.

Communicate well

Last, using coding challenges means you need to communicate with your candidates. As most candidates value clear communication, be timely and specific in your communication. Give candidates the feedback when you have it or on a date you’ve shared as the deadline.

Interview coding challenge examples

Want some examples of what coding tests can look like? Here are a few samples of some of the most popular languages and frameworks out there (note, however, that your challenges should simulate the work someone would do as part of your team). If you want more, sign up for our demo assignments.

And if you want to see a real-life example, take a look at corrux’s coding challenge.

JavaScript interview coding challenges

Here’s one of our own assignments:

“Using JavaScript and React, your assignment is to implement a responsive signup form and get it looking as close to the design as possible.”

Users also get a brief with specific instructions and a list of the tasks that are to be completed:

“Your task is to build out the project to the designs inside the /design folder. You will find both a mobile and a desktop version of the design to work with. You can use any tools you like to help you complete the challenge.”

Python interview coding challenges

A Python coding challenge can look something like this:

“Using Python, your assignment is to write a program that returns a list that contains only the elements that are common between two lists.

The lists are:

a = [1, 1, 2, 3, 5, 9, 13, 14, 34, 55, 89]

b = [1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 15, 22, 54]

Make sure to create a list without duplicates and that your program works on both lists.”

Java interview coding challenges

How do you create a Java coding challenge? Here’s the “rock, paper, scissors” challenge:

“Using Java, create a feature that simulates “rock, paper, scissors.” The functionality should return the result as:

• Player 1 wins
• Player 2 wins
• Tie

The feature takes the first parameter from the first player and the second from the second player.

The rules of rock, paper, scissors are:

• Two players have to say either “rock,” “paper,” or “scissors” at the same time
• Rock beats scissors, paper beats rock, and scissors beat paper”

Front-end interview coding challenges

What do front-end coding challenges look like? Here’s one example:

“Your assignment is to create a credit card form that includes number formatting, validation, and automatic card-type detection.

Use your language and framework of choice. Make sure to create a responsive form.”

Over to you!
There you have it! Now you know how to use interview coding challenges to find the best candidates.

Want to use them in your own technical hiring processes?

Get started today! You can try CodeSubmit for free (no credit card required).