Tracy Phillips
CodeSubmit Team
How to Test Programming Skills Before Hiring [Updated for 2023]
How do you test programming skills before you make a hiring decision?
Great question. Today, you’ll learn the exact process to test developers’ skills so that you can identify and hire the best technical talent.
Want to learn more? Read on!
How do you test programming skills?
Hiring a developer is an investment, as technical talent tends to be the most expensive payroll cost. According to a study by the Society for Human Resource Management, hiring an employee costs, on average, $4,129. And it takes 42 days to fill a position.
Overall, making a new hire tends to cost 20-25% of that person’s base salary. And the Bureau of Labor Statistics reports that the median annual wage for software developers was $110,140 in 2020. If we take employer costs into account (benefits and other similar costs), the average annual salary would be around $142,741/year. Costs for hiring a new developer would amount to $28,548 to $35,685.
We made our own example calculation of what these developer hiring costs can look like. And we landed on $41,049 if you use an external recruiter and $13,514 if you don’t use one.
In other words, to make your investment count, you’ll want to be sure that you’re making the right hiring decision. And a great way to ensure that your top candidate’s technical skills are what you expect them to be is to test them.
But what are programming tests?
There are various ways to test programming skills; they should all be used together to make the best assessment. The different ways to assess programmer skills are:
1. Evaluate candidates’ portfolios, projects, and GitHub/Stack Overflow activity
The first step is to look at your candidates’ portfolio, projects, and GitHub and Stack Overflow pages.
What types of projects have they worked on? These aren’t necessarily work projects, but pet projects, examples of code, blogs, and open-source projects.
As this is the first clue to understanding the quality of work a developer puts out there, you can use portfolios to start weeding out candidates and decide who gets to the first round of interviews.
2. Hold interviews
The main objective of your interviews isn’t to assess technical skills. Instead, interviews are a great way to understand how candidates communicate, solve problems, and work together with others.
That said, you can use interviews to get an overview of your candidates’ technical skills, for instance by asking about projects they recently worked on.
Here are some of the interview questions you can ask, depending on the programming languages or frameworks you’re hiring for:Python interview questions
3. Use programming tests for interviews
The last part of your recruitment process is coding challenges. If used in the right way, coding tests can be a great way to objectively assess your candidates’ skills.
Plus, if you know how to set up the right type of programming test, you’ll improve your candidate experience and make the whole process more enjoyable for candidates.
Note: Not all programming tests work for every seniority level. There are coding tests for beginners and other tests work better for more senior developers. So consider the overall user experience to decide on which coding test to use at your company.
Next, we’ll look at how you use them in your hiring process.
What are coding tests?
Programming tests for interviews are tests that your candidates complete on their own or together with your hiring team.
Typically, you send out coding tests when you’ve already held a few interviews. The reason is simple: coding tests require more resources, so you likely won’t test all of those who apply.
The process can look something like this:
In other words, you send the test when candidates have gone through an initial round of interviews (or after two rounds of interviews).
There are different types of programming tests to test programming skills. The one you choose to use depends on your organization and goals. The different tests are:
1. Take-home coding challenges
Take-home coding challenges are coding tests that your candidates complete remotely on their own. Typically, you’ll ask candidates to complete a real-world task (albeit a small one; think 3-4 hours rather than 10-20 hours).
With some platforms, such as CodeSubmit, your candidates use their own IDEs, so that you get a thorough understanding of your candidates’ skills in a real-world setting.
Take-homes are effective for figuring out how your candidates would perform if they were part of your team. They’re flexible and work well for people with less adaptable schedules, such as parents and people with full-time jobs.
The drawback is that your hiring team needs to evaluate every take-home challenge and provide feedback. That’s why you’ll likely send take-homes later in the process after a few rounds of interviews when you have fewer candidates to test.
We offer a flexible way to create take-home coding challenges. You can either use our library of proven take-home tests or upload your own.
2. Pair programming
Pair programming tasks are coding tests that someone on your hiring team completes together with the candidate. Rather than just watching how candidates complete the task, the hiring manager participates and discusses the task with them. This way, you get a great overview of how the candidate works, solves problems, and communicates.
Pair programming can be done in person or remotely. That said, a remote coding test enables more people to participate and apply for your open positions.
You can use CodeSubmit’s pair programming tests for your own interviews.
3. Screening tests
Screening tests are short quiz-like tests with the sole purpose to weed out candidates. They don’t really test programming skills and shouldn’t be used to assess someone’s knowledge.
In fact, screening tests should only be used in one situation: when enterprises and organizations that get hundreds of applications for every open position are hiring and want to filter out candidates in the early stages of their hiring process.
You can try one of our screening tests, Bytes.
However, if your organization is smaller and doesn’t get hundreds of applicants, use take-homes or pair programming tests.
4. Whiteboarding tests
Whiteboarding tests are programming assignments that candidates complete on whiteboards in front of your hiring team. Whiteboarding isn’t a great way to assess developers; they complete tasks physically on whiteboards and not in their normal developer environment. In other words, you assess their presentation skills rather than their coding skills.
Plus, a physical test means that fewer candidates will apply for your position, which limits the talent pool you’re able to reach.
Coding test examples
But what does a coding test look like?
This depends on the role you’re hiring for. The task you choose to use should resemble the work you’re hiring for as much as possible. You can choose a small task the developer you end up hiring would work on in a real-world setting.
Here’s what a task looks like in our CodeSubmit library:
Either way, use a task with clear instructions. Your task shouldn’t be too complicated or require a lot of background research. The best tasks are small tasks developers would work on as part of the job.
How do you assess programming tests?
What should you look for in a developer?
Use your entire recruitment process - portfolio assessments, interviews, and programming tests - to understand how your candidates score on these assessment factors:
Clean maintainable code
Great developers write code that doesn’t have to be rewritten for a long time. This is an incredibly important skill, as developers spend a lot of their time maintaining code.
Databases
A developer who is skilled in database management systems can backup, secure, and manage their database.
Algorithms
Algorithms can be used to solve problems faster. A developer who understands algorithms is, in other words, a great asset for your team.
Source control
Source control is necessary when many developers work on the same set of files. It helps developers track different file versions and the source code.
System design
Someone who understands system design understands information about the system and its elements and how changes can be made to them. An important skill for developers!
Cloud services
Developers who have cloud computing skills master cloud platforms like Google Cloud and Amazon Web Services. They understand how to use these platforms to support your company’s cloud computing strategy.
Data privacy and data security
Data privacy and security are more important than ever. If the developer you hire masters these areas, they can analyze risks and work to mitigate them.
Integrated Development Environment (IDE)
IDEs help developers write, compile, run, and debug their code. A developer who masters their IDE is a great asset for your team.
Problem-solving skills
Knowing how to code is essential. But a great developer knows how to solve problems. If a developer can spot bugs, they can fix problems in their code much faster.
Soft skills
Other skills that take developers from good to great are non-technical skills. These are teamwork or how well your developer works together with the rest of your team. A 10X developer doesn’t just have amazing skills; they also understand that they need the rest of the team for the best results.
What’s more, developers need to know how to communicate well. Not only will collaboration improve within the technical team, but developers with good communication skills are also able to help non-technical team members understand technical topics.
Plus, great developers are creative thinkers and they adapt well to new situations. Work changes fast these days and flexibility is key to overcoming new and unexpected challenges.
Related: The 9 Soft Skills Developers Need to Succeed
Over to you!
There you have it! Now you know how to test programming skills before you make a hiring decision.
What it comes down to is that you use all three parts of the recruitment process to assess coding skills; portfolio assessments, interviews, and coding tests. With real-world coding assignment tasks, you get a thorough understanding of your candidate’s skills and how they would perform on the job.
Want to try coding tests in your own organization?
With CodeSubmit, you can try our developer skill tests (no credit card required).