In Search of the Elite Few – Finding and Hiring the Best Software Developers in the Industry

The Challenge

Hiring software development talent – real talent – is a multi-faceted skill that lies at the crossroads of social networking, technical acumen, process management, and intuition. Anyone who has ever had hiring responsibility understands all-too-well the extent and depth of the hiring challenge.

Particularly challenging is the fact that the hallmark qualities of exceptional software engineering candidates are extremely difficult to evaluate. How do you explore a candidate’s ability to innovate and think creatively? How do you determine if he’s a team player? How do you diagnose her ability to take constructive feedback? How do you investigate someone’s moral fiber?

While evaluating these “soft” criteria is no easy task, doing so is essential in one’s quest to hire the very best. Yet all too often, attempts at evaluating these crucial, yet elusive, attributes consist of questions or challenges that are sufficiently transparent for the “correct” answers to be obvious to most candidates. Thus, nothing of substance is achieved by asking them.

A common resulting pitfall is to instead focus too heavily on technical minutia, rather than evaluating the candidate’s ability to problem solve, to think creatively, and to work compatibly with the rest of the team.

Admittedly, effective interviewing and hiring of advanced software developers is as much of an art as it is a science. Nonetheless, approaches and methodologies do exist for evaluating the more subtle dimensions of a software engineer’s skills and abilities.

When used collectively, these recruiting techniques yield a highly effective screening process with a proven track record of success for hiring salaried or freelance programming talent. That process is what this post is all about.

Getting Started: Filling the Pipeline with the Best

The process of finding and hiring the elite few begins long before the interview itself. In fact, a systematic approach to properly identifying potentially qualified candidates can streamline and increase the efficiency of your entire recruiting process significantly.

Without a doubt, the single best source of qualified candidates is personal networking, as quality people tend to associate with quality people. Personal referrals dominate as the most productive source for successful new hires (source).

Other valuable sources of top-notch technical talent include:

• Blogs and online technical postings. Developer blogs can be a great source of strong candidates. Technical posts often provide valuable insights into a developer’s technical acumen, approach to problem solving, and writing skills, all of which are important criteria for identifying cream-of-the-crop candidates.
• Open source code contributors. Perusing open source projects on sites such as GitHub and SourceForge can yield highly qualified candidates. An added benefit of this approach is that it allows you to see and evaluate a candidate’s code samples before even deciding to make contact.
• Conferences. Conference speakers and co-attendees at relevant technical conferences often provide a valuable resource for qualified candidates. If you are unable to attend, review the online program and consider contacting the presenters. You may just happen to contact them “at the right time”, or they may be able to refer you to someone else who is highly qualified.

The beauty of these resources is that they supply engineering candidates who you have vetted, in one way or another, before even interviewing them. Either someone in your company, or someone you know, knows them and recommends them highly. Or you have reviewed their open source code and are duly impressed by it. Or their blog posts suggest a mature level of technical expertise and understanding (and ideally even a sense of humor :-) ). The key here is that, by the time you make contact, the candidate is already significantly more than “just another resume”.

Additionally, consider using job description templates (like these for SEO, front-end web development, and back-end web development) that set a high expectation for the types of candidates you desire.

Evaluating Software Developers’ Technical Acumen

An effective determination of technical proficiency goes far beyond the nuances of a specific programming language or technology. While these technical details should certainly not be ignored, they typically are not the most important element of the evaluation process.

The best developers don’t waste time committing to memory that which can easily be found in a language specification or API document. Moreover, learning a new language, technology, or design pattern is a relatively trivial exercise for a strong developer (and is actually an essential skill given the accelerated rate at which new technologies are being introduced and gaining market share). It is therefore the candidate’s fundamental technical strength and understanding, creative thinking, and problem solving that are critical to evaluate.

A proven method for accomplishing this is to pose a programming challenge unconstrained by language. Present the candidate with a problem and ask him or her to code a solution in the language of their choosing. The only requirements should be that the solution be thorough and correct, including addressing any edge conditions or potential errors. Remember, the goal here is to evaluate the candidate’s (a) ability to problem solve, (b) knowledge of computer science, and (c) coding style. You’re not at this point evaluating aptitude in a specific programming language. (An added benefit of this approach, incidentally, is that it avoids requiring the interviewer to be an expert in the language the candidate is coding in.)

Numerous online resources exist from which you can select such programming challenges to incorporate into your interview process. Those that focus on algorithms (rather than knowledge of a specific programming language) facilitate evaluation of a candidate’s computer science foundation and problem solving capabilities. Some noteworthy examples include hackerrank.com, projecteuler.net, and beatmycode.com. Bear in mind, though, that you don’t need to use these sites to administer the tests; rather, you can simply use them as resources from which to select challenges and then have the candidate code a solution during the in person interview (on a white board, etc.).

That said, online testing can be a valuable additional component of your interview process. When using online testing services, though, it is important to avoid the risk of focusing too heavily on a candidate’s site-generated scores rather than the specifics of their solutions. For one thing, you may not agree with the evaluation metrics employed by the testing service. Moreover, there may be aspects of a solution that you consider particularly clever or elegant, to which the automated evaluation process may be entirely oblivious.

There is also the potential for “false positives” or “false negatives” with ratings provided by online testing services. On the one hand, false positives can result in wasted time and resources interviewing unqualified candidates, while false negatives can disqualify candidates who were highly worthy of consideration. In the long-term, the latter has more serious ramifications and is therefore something that one must carefully monitor for.

In general, tests administered in person are inherently more reliable than those administered online, since they reduce the potential for deception. Moreover, in-person testing provides the opportunity to observe how a software engineer candidate operates under pressure. Nonetheless, use of online testing services can be an effective component of the more comprehensive type of screening process advocated herein, which incorporates other techniques to help weed out either fraudulent or under-qualified candidates.

Online development tests can also serve as a valuable preliminary filter to eliminate the staggeringly high percentage of candidates whose technical skills are far below par. Interestingly, many such candidates abandon the process without even trying to take the tests, due to a lack of confidence. That said, confidence is not always the best barometer of attitude, as attested to by the Dunning-Kruger effect.

Another extremely valuable technique for evaluating technical acumen is to ask the candidate to provide a list of open source projects that they have authored, or at least contributed to, on sites such as GitHub and SourceForge (if they were only a contributor, ask them to indicate to you which portions of the code they personally authored). You can then come to the interview having reviewed their code, prepared to ask them about specific design decisions, coding styles, patterns employed, and so on. Such a discussion can be substantially more valuable than simply quizzing the candidate on the technical minutia of a language.

Speaking of language…

Members of the elite few are not just technologically savvy, they are consummate software development professionals who communicate clearly, effectively, and succinctly, both verbally and in writing.

Verbal communication skills are easy to evaluate in the course of an interview, whether in-person or over the phone. While verbal skills are admittedly less critical in certain “back room” technical roles that don’t involve user contact, they are nonetheless still important for maximizing effective team interactions and interchange of ideas.

Written communication skills can best be evaluated by asking the candidate to provide writing samples. Technical writing samples are strongly preferred, since they demonstrate not only writing skills, but also technical clarity. Requesting these samples in advance of the interview can be particularly useful. For one thing, this provides the opportunity to review and analyze their writing ahead of time so that you can come to the interview prepared to discuss and critique their writing with them. In addition, some technical candidates will be “put off” by the request for writing samples which in itself can serve as a valuable filter.

The Moral Dilemma

There is an old joke about a father who wants to teach his son about ethics. The father tells his son that he and his partner have a dry cleaning business. One day, says the father, a customer comes into our store to pick up her dry cleaning. As the customer turns around to leave, I realize that she’s overpaid by 10 dollars. OK, so here comes the ethics… do I or don’t I tell my partner?!

In contrast, real ethical dilemmas – moral dilemmas – can often be murky and complex and they most certainly present themselves in business. The strongest software developer can wreak untold havoc on a project if he or she lacks a properly calibrated moral compass. This dimension of a candidate is therefore critical to evaluate.

A highly effective technique for accomplishing this is to present the candidate with a hypothetical moral dilemma and ask what they would do in the situation. For this to be effective, it is essential that the dilemma not have an obvious answer. Ideally, any possible answer should be imperfect and somewhat problematic. The answer the candidate provides to such a question can provide enormous insight into how they think, their ethical standards, and their priorities.

Here’s an example:

The company you work for has been hired to work on a project by another company. The project requires use of a particular subcontractor. You previously worked for that subcontractor and you therefore know first-hand that they often pad their hours and overcharge their customers for their time. If they do this on this project, though, it won’t hurt your company, since those costs will be passed on to your company’s customer. You’ve mentioned this to your boss and he has advised you not to say anything to the customer, especially since the subcontractor’s portion of the project only represents a small fraction of the overall budget. You have a very close friend who works for the customer though. One day over lunch, your friend asks you what you know about this subcontractor, since he knows you used to work there. How do you respond and what do you tell him?

The beauty here is that there’s no clearly “right” answer and the situation mirrors the moral complexity of real life and the business world. If asked properly, the answer to such a “moral dilemma” question can be one of the most telling and valuable parts of the interview process for hiring software developers.

Getting to Know You (Who are you really?)

One of the inherent flaws of the interview process is that there is really no way to get to know what it will be like to work with a person until you actually work with them. Getting a realistic feel for a candidate’s personality and temperament is nonetheless crucial to a successful hire.

The technical elite are not just technically superior, they are also highly professional. The search for them is therefore not limited to a strictly technical evaluation.

With that in mind, here are some interview techniques that can be beneficial for evaluating a candidate’s personality and professionalism…

For one, consider intentionally beginning the interview process with a prescreen by a non-technical staff member (such as an administrative assistant). You can then ask this staff member how the candidate treated them once they realized that they were not technical. Were they demeaning, impatient, and condescending, or were they pleasant, patient, and respectful? What the staff member relays to you can be very telling indeed. You want to know if the candidate treats others with respect, regardless of who they are.

Throughout the interview, look for opportunities to provide the candidate with constructive feedback. Observe carefully the reaction that this elicits. Is the feedback appreciated or does the candidate appear to become at all defensive?

Engage in non-technical “chit chat” for the first 5 or 10 minutes of an interview as a means of obtaining insight into the candidate’s personality. The candidate may think this is simply a prelude to the actual interview and will therefore be more likely to “let their hair down”. You may thereby be able to discern a surprising amount about their personality during this time. And of course, if they exhibit a sense of humor, that’s always a good sign. :-)

In addition to these general techniques for hiring engineers, here are some sample questions to help assess a candidate’s personality, confidence, honesty, and professionalism:

• As you look back on your career, what it is a specific difficult situation you’ve been faced with that stands out in your mind? Describe the challenge and how you dealt with it in detail. What do you feel you handled particularly well? What might you do differently if you had the opportunity to do it over again?
• What is the biggest mistake you think you ever made in your career?
• What do you enjoy the most about software development? What do you enjoy the least?
• If you could paint the ideal position for yourself, what would it be? What would it look like in terms of your role and responsibilities, size and type of company, the work environment and so on?
• What are your greatest strengths? What would we gain by hiring you?
• None of us are perfect. What are some aspects of your personality that you’re working on improving?
• If I called your current boss and asked him about you, what would he or she tell me?
• What should we be concerned about in hiring you?

Collectively, these questions and techniques have been proven to foster a much stronger and deeper understanding of who the candidate really is, which is a vital to making a successful hiring decision.

Is the Feeling Mutual?

So, you’ve found an A+ candidate. Highly skilled. Positive, upbeat personality. Excellent cultural fit. That’s great! But…

There’s still one pivotal question that remains to be answered: To what extent is the candidate specifically interested in your available position and in working for your company? The answer to this is of paramount importance in ensuring a successful hire. One-sided relationships work no better in business than they do in any other aspect of our lives.

Once again, simply asking the question directly is of minimal value since, in the context of the interview process, one must be suspicious of a candidate’s candor in answering such a question. Rather, this is something that one must attempt to ascertain through other, less direct, means.

For starters, the way the candidate approaches you from the onset is a fairly reliable gauge of interest. Does the candidate offer a cogent, compelling explanation of why he or she is particularly interested in your company and the available opportunity? If so, this not only displays real interest, but also a level of professional maturity and sophistication. Such a candidate has clearly done at least some level of prior research into your company, and the opportunity, before contacting you (rather than simply “shotgunning their resume around town”).

Candidates who enter the developer hiring process having done their homework on the company, and its products or services, provide a useful indicator of both their interest and their savvy. Candidates who have gone as far as doing advance research on the interviewer (e.g., reading his or her bio, LinkedIn profile, blog posts, and so on) are particularly noteworthy. More sophisticated candidates will make a point of revealing this information in the course of the interview. For others, simple open-ended questions such as “What do you know about our company?” and “What in particular interests you about our company and this position?” can elicit very telling responses indeed.

Finally, a candidate who is genuinely interested will likely be asking questions throughout the interview process. That notwithstanding, it’s always a good idea to close the interview by asking if the candidate has any questions. A lack of any questions at all can be a red flag, even with someone who would otherwise have been considered an A+ candidate.

Wrap Up

The techniques described herein can serve as a valuable core process for finding and hiring software developers that are the best in the industry. Yet it is important to remember that effective hiring is not a destination, it’s a journey. One must continually re-evaluate and tweak their process, as a static process is destined to become a dead process.

Every company’s journey will be different, based on its own culture and priorities. Find the path that’s right for your organization and execute on it. You will have successes and you will have failures, but as long as you learn from the latter, you can be assured that you’re on the right path.