Tracy Phillips
CodeSubmit Team
How to Use Coding Tests for Interviews: Hire Developers with Ease
How do you use online coding tests for interviews to hire the right developers?
Coding skills tests are a crucial part of your technical hiring process. And today, you’ll learn how interview coding challenges will help you assess programming skills so that you can identify the best talent for your team.
Want to learn more? Read on.
How do you test a programmer for an interview?
Looking to hire a developer?
Then you know just how hard it can be. According to the Society for Human Resource Management, 75% of HR professionals who had difficulty recruiting in the past 12 months reported that there are skills gaps in job candidates.
This is especially true for technical roles. According to Dice, software developer and engineering jobs are the hardest to fill in major tech hubs in the US. It takes, on average, 39 months to fill those positions.
And nearly three in four employers have been affected by a bad hire, according to research by CareerBuilder.
All of this data just goes to show how hard it is to hire technical talent.
That said, there are ways to improve your hiring processes, from attracting top talent to improving your employer brand and identifying the right candidates.
One of the most effective solutions? Skills tests.
Research by Aberdeen Group shows that employers who use pre-hire employment screenings are 24% more likely to hire employees who exceed their performance goals.
What’s more, they have a 39% lower turnover rate and hiring managers report 36% more satisfaction with their final decision than organizations that don’t use these tests.
Other research backs this up. Well-designed work assignments correlate with job performance.
Now, “pre-employment screening” is a wider concept than skills assessments, but coding skills tests are an essential part of screening procedures for technical hires.
The point is:
The right coding tests for interviews will help you…
- Objectively assess candidates
- Create an engaging candidate experience
- Lower employee turnover
- Improve candidate selection
- Understand exactly how candidates perform on the job
But these benefits require that you use the right programming tests. That’s what we’ll look at next.
Synchronous versus asynchronous interviews
Before we dive into the coding challenges, let's take a look at the two different types of interviews.
We have synchronous interviews, which are scheduled, real-time interactions between the candidate and hiring manager. These are typically pair programming interviews below.
Then, we have asynchoronous interviews, which don't require you to schedule a set time, but the candidate completes the task at their own pace. These are take-home coding challenges below.
That said, all types of interview coding challenges are controversial.
Why? Let's find out.
What are interview coding challenges?
Coding skills tests are controversial. Why? Because software engineers tend to dislike them.
(Just google something like “developers dislike coding tests” and you’ll find hundreds of hits like this discussion thread on YCombinator.)
Some common arguments against them are that they’re free work and that they don’t really show a developer’s skills.
However, this depends on the types of tests that you use. The right skills test can improve the candidate experience.
The different types of programming tests are:
Brainteasers and algorithmic quizzes
Brain teasers and quizzes have little to do with how candidates actually perform on the job. In fact, companies like Google decided not to continue using them as they don’t add anything to the hiring process.
In other words, skip questions like this:
“How many golf balls would fit into a Boeing 747?”
Ultimately, they’re useless.
Screening tests
Screening tests are, just like brain teasers, a less accurate way to assess someone’s real coding skills. At CodeSubmit, we offer short and concise coding problems that require candidates to write code against predefined tests as an alternative to traditional “screening tests”.
That said, there is a time and place for screening tests. When companies simply have too many applicants to be able to sift through them, screening tests can be a simple way to filter candidates. However, this mostly applies to companies like Google and Amazon -- in other words, big enterprises and corporations with a strong, existing employer brand.
While we primarily recommend take-home coding assignments (see below), we also offer screening tests as part of CodeSubmit, CodeSubmit Bytes.
Curious to see how they work? You can try a demo Byte here.
Whiteboarding tests
Whiteboarding tests are coding challenges that are performed on a whiteboard in front of hiring managers. And all we can say is: Don’t use these programming tests!
Research shows that whiteboarding assignments are terrible for making hiring decisions. Unfortunately, they only work to increase anxiety levels and they create a terrible candidate experience.
Pair programming
Pair programming, e.g. pairing someone from your team with a candidate to go through a challenge, is a pretty good option. But there are a few things that could quickly make this assessment option a terrible experience. Let me explain: If the person on your team isn’t great with candidates (and let’s face it… most of us are not amazing in interviewing situations!), pair programming can make a really bad candidate experience. It can also create the same undo stress as a whiteboarding test.
We’re working to launch our version of candidate-friendly pair programming at CodeSubmit soon!
Take-home coding challenges
Take-home coding challenges are real-world coding assignments that candidates “take home” to complete at their own pace. This option is the best way to identify top developers. Why? Because candidates work on the same type of tasks they’d work on if they were part of your team. The process is also far less stressful than if they had to perform in front of recruiters.
Plus, take-homes are more objective than other coding tests just because of that reason -- there is less room for bias. The best part? Ultimately, these tests can be exciting for candidates because they are based on real work, giving them insight on what it’s like to work with your team!
Take-home challenges are typically assigned after one or two rounds of interviews.
Here’s how Daivat Dholakia, Director of Operations at Force by Mojio, describes the company’s approach to take-home tests:
“Take-home coding challenges are a critical part of our hiring process. Force by Mojio depends on having knowledgeable developers on call to update and fix bugs in our app and website.
We have two untimed coding challenges. One involves pulling data from an API, manipulating it, and pushing it to another API. The other is more UI-centric. Depending on the position being hired, we’ll give the candidate one or both of these tests.
We use take-home coding tests because they give candidates the opportunity to showcase their skills in a way that a multiple-choice or short-answer test can’t. They also help us weed out candidates who aren’t invested in the company. A take-home test requires some level of dedication and discipline, and people willing to go above and beyond with it are always worth our attention.”
Anthony Wilson, Vice President of Delivery at Digital Artisans, also uses coding challenges. He describes their approach to using them:
“We use two phases of testing.
The first is in the recruiting platform that presents a simple test when the candidate submits the application and the second is a deeper test of skill during the interview process.
The first test is simply screening. Keep it very simple and quick; it’s not used to assess skill. The majority of people will simply skip it which disqualifies them. The test weeds out the people who submit dozens of resumes indiscriminately, agencies and contractors applying for full-time jobs.
The second test is a true test of skills that is presented after the first round of interviews. This second test is only given to validate skill after both parties think there is a match. By this point, a candidate appreciates the test because most good employees want to work with other strong employees.”
Interview coding challenge examples
Curious to see what coding tests can look like?
Here’s a demo take-home challenge you can try out.
And here’s one of our coding interview questions for JavaScript and React:
As you can see, it includes a clear objective, brief, and list of tasks. You have access to tasks like this one in CodeSubmit’s challenge library.
Here’s another task, a WordPress coding challenge:
How to create coding tests for interviews
Now you know what coding test you should choose and why.
But how do you actually create a coding assignment for your interviews?
Here’s how.
What coding language is used for interviews?
First:
What language should you use for your interviews?
Simple: Use the language that candidates would use on the job. CodeSubmit supports 60+ languages and frameworks. You can either use one of our library assignments or upload your own.
Some of the languages and frameworks we support include:
...And many more!
But how do you create your tests? That’s what we’ll look at next.
How do you create online coding tests for interviews?
There are a few things you can do to ensure that your coding tasks fill their purpose. Your coding challenge should:
- Be simple and clear
- Not be input/output of a single method
- Welcome creative interpretation
Also, your challenge should mimic real-world tasks, be difficult but achievable, and be reviewed by people on your team. That’s what we’ll look at next.
Use real-world tasks
First, use real-world tasks, e.g. the same type of task that one of your team members would work on. Bonus points if the brief feels specific to your company’s industry or product!
This way, you’ll see how candidates will perform on the job. Plus, it makes much more sense for candidates to complete a “real” task. It’s a win-win -- they get a better understanding of the job, you see how they’ll fit with the team.
Also, this means that you should mirror the real work environment. Here’s how Pinterest does it:
“We allow candidates to use online resources like Google or Stack Overflow during the interview process to mimic as closely as possible what they’d see in a real work environment. We also provide a machine with the most common IDEs, like VSCode and Sublime, so candidates feel at home with the code completion, syntax highlighting, and shortcuts they’re used to.”
By the way, CodeSubmit allows candidates to use their own IDE. This is a unique feature that most other coding assessment tool providers don’t offer.
Make your coding challenges difficult (but not too difficult)
According to a Glassdoor study, difficult job interviews are linked to higher employee satisfaction. This can be due to many reasons -- for example, candidates might feel they’ve had to work harder for their job, which makes them feel more motivated. Or, the fact that they completed a challenging interview process might be a sign that they were motivated from the start.
Either way, you should create an actual challenge for your candidates. But don’t make it too challenging -- after all, your applicants might have full-time jobs, families to spend time with, and other commitments that make it hard to dedicate time to your task.
Our recommendation is that you limit your take-home challenge to 3-4 hours.
Provide feedback
Remember what we said about take-home challenges, that developers tend to like them more than other coding skills tests? Well, we’d like to include a small caveat. For a truly great candidate experience, you need to offer feedback.
Think about it: Applicants have put hours on your task. Take a few minutes to go through the task and show them where they can improve. Yes, this takes extra time and effort. But considering that your candidates will be awed by your hiring process, it’s well worth it.
How to assess developers
Once you’ve put together a coding challenge, it’s time to assess applicants.
To do so, you do need evaluation criteria. Your assessment criteria depends on your priorities and company culture.
For example, this is what Slack looks at to assess developers:
“The exercise is graded against a rigorous set of over 30 predetermined criteria. We’re looking for code that is clean, readable, performant, and maintainable. We put significant effort into developing these criteria to ensure that, regardless of who grades the exercise, the score is an accurate reflection of the quality of the work.”
If you’re not sure what to look for in your candidate’s code, use these pointers to assess quality code:
- The code works (features like CodePlay make this part easy to assess!)
- It’s well-documented (as much as it needs to be)
- And it’s clear with well-chosen variables, methods, and class names
- It’s easy to maintain
- There are no security breaches
- It’s fast
Milosz Krasinski, Managing Director at web consulting company ChilliFruit.com evaluates developers based on the following criteria:
“You need to be clear about what it is that you want to evaluate. Some pointers for this can be:
- Can they write code and, at what level?
- How do they approach problems?
- Are they cautious, methodical or experimental?
- Are they proficient in documenting their work?
- Do they run QA tests?
You also need to make sure that the results are measurable/quantifiable as you’ll be comparing candidates in order to make a final decision. The candidate will be evaluating your company, so you should try to make the test applicable and relevant to your business.
Finally, don’t forget those soft skills including communication, problem solving and presentation. While you don’t want to put unnecessary pressure on candidates, it's a good idea to include a deadline to give you an idea of his or her timekeeping skills.”
What to look for in a coding challenge tool
Now you know what it looks like to set up and create coding tests for interviews.
Just one more step to go:
Decide on the coding assessment tool you want to use for your challenges.
Your tool should...
- Be easy to use
- Offer a great candidate experience
- Support the languages and frameworks you need
- Offer an option for you to upload your own challenges
- Allow for the number of applicants you need
- Offer the number of seats you need for your team
- Have great customer support
- Be flexible and customizable
- Make it easy to detect plagiarism
CodeSubmit offers these features (and many more). And if you want to take a look around, you can test our platform for free here.
Aimee, Engineering Mentoring Program Lead at Babbel, describes why Babbel chose to work with CodeSubmit:
“The feedback that we received from the candidates about their experience was overwhelmingly positive - specifically regarding the ease of the process, as well as the personal and engaging experience the process offered.”
Other things users have said about our platform include:
“The whole assessment process and hiring experience is just so much better for our hiring managers and candidates."
“A great way to test candidates. Very hands on approach and it is quite easy to see how relevant it is towards the job.”
Get started today!
There you have it. Now you know how coding tests for interviews will help you find the right developer for your team.
Want to use this for your own technical hiring processes?