Hire screening bucketology

improve your screening by defining and using no-go buckets

I’ve found hiring is a very inexact science – using good screening techniques is essential to both optimize the chance of a good hire and minimize time invested.

Below are some of the more effective approaches I’ve used and have seen used as a job-seeker.

Tech Screen

In my experience, it works best for a Tech Screen to be done first for both developers and testers (and for technology managers too) , and it’s far more efficient to do these by phone than FTF for both the interviewers (can cut it off sooner) and the seeker (one less day to wear a suit). Using Skype or other videoconferencing can provide additional early insight into their communication effectiveness.

I’ve found it’s much more important to focus on aptitude than specific skills, because things change fast in the technology biz, and you’re always going to need your people to be pushing the envelope and learning if you want to innovate and consistently deliver value over time. I like to focus on their approach to how they delivered one example “Most Interesting Project” (MIP) (of their choice), including overcoming obstacles, rather than only on the technical aspects of how they used language X or pattern Y – this usually shows whether they had to learn new skills, troubleshoot or think outside the box.

Many recruiters and ads have a tendency to screen based on a specific skill, which I refer to as Buzzword Buzzkill. For example, a Developer role “must have 5+ years C#, using ASP.NET MVC jQuery”, or a Tester role “must have Selenium Webdriver experience”. Examples of more enlightened criteria I’ve used or seen are “5+ years of C# or Java, full-stack web development, using at least one Javascript framework” and “must have experience in developing test automation using one or more compiled or interpreted languages”.

I also usually try to make a preliminary determination during a tech screen of how well the current interests (not just abilities) of the candidate align with the needs of the offered job.

Toward the end of a tech screen I’ve seen CoderPad used, which is a great way to see if they “think in code” to a sufficient degree they can type simple constructs without an IDE or intellisense, and to hear them describe their thought process.

Toward this end, I have found the following “buckets” useful to quickly weed out candidates who are not a good fit. Note my criteria are weighted toward hiring for product-oriented start-ups / SMBs, or small high-performing teams within larger organizations:

  • Corporate Cog: Has worked primarily for larger companies / on larger teams often has to make an adjustment to be effective on a smaller team. Assess whether the person is going to need lots of hand-holding, wants to do only one thing, is not comfortable with perceived risk, doesn’t want to leave their comfort zone of what / how they are accustomed to doing – in general, needs a lot of structure. Example red flags: “I thought you were bigger than this”, asking “do you offer tuition reimbursement or sabbaticals” early in the interview process, or if their past work was mostly driven from an assigned task list vs. working more collaboratively and independently.
  • Lone Ranger: Has worked alone much more than collaboratively with a team. Can occur even on project teams. Example: Someone who has done only narrow contract engagements which didn’t require much interaction or collaboration, or has worked in an isolated manner on teams not following agile principles of frequent review and feedback.
  • New New Thing Groupie: Emphasizes interest-in / experience-with latest / bleeding edge technologies or buzzwords to the exclusion of describing in-depth experience delivering projects of any depth. Example: Someone who repeatedly says they have most liked learning new things / are interested in your job because you’re using cool technology, to the exclusion of talking about what they’ve delivered or interest in what your product does.
  • Tire Kicker: S/he is just not that into you. They are often seeking the highest-paying / cushiest-benefits job they can get, to the exclusion of being interested in your product, how they could contribute to it or how they could be challenged. Example red flags: “I’ve got 2 other offers and expect to get 2 more within the next week”.
  • Hyperbolator: Exaggeration of experience on either resume or during tech screen, e.g., listing experience with some technology when it was used only by someone else on a project, or so shallow as to be irrelevant. Examples: “I listed InstallShield but never personally used it, it was used on a project I was on”, “I’ve never written a web service, only consumed them”.
  • Blabbergaster: Similar to Hyperbolator, but differing in that this individual might actually have the relevant experience, but have focus issues and / or underdeveloped listening skills. For example, if they spend more time explaining why they didn’t do certain things on a takehome test than just having attempted them, or if you picture this person leading a meeting where they leave everything open-ended with no ability to drive to closure. Examples: “While working on the takehome test I wasn’t sure what you were looking for, so I stopped”, or “blah blah blah… what was your question again?”
  • Disgruntled Knowledge Worker: A fair amount of work experience who seems to have never been satisfied at any job (vs. making the most of each opportunity), complains of being “misunderstood” or “underutilized”, might be too prone to blame their not having found that perfect job on outside factors like the economic downturn.
  • Egghead: Has very strong theoretical understanding, but not enough balance with real biz apps and / or app delivery. Example: Candidate who had worked for years at NCSA, had very strong math skills, but working on primarily research projects of 6 months to 2 year duration, no concept of frequent delivery via an SDLC.
  • Syntax Jockey: Understands syntax of a programming language fairly well, but does not demonstrate a solid approach to clean design. Most often seen with fresh grads but sometimes with experienced candidates who have shown more ability in memorizing syntax rules than analysis, design and application of patterns.
  • No There There: Doesn’t have much depth (at least in areas of interest to the company / job), despite a lot of experience as measured in years or projects – they were in roles on projects where they didn’t do much technically challenging work. Example: Someone with 5 years .NET experience, who had never used a collection construct; a tester who had years of work experience but solely manual testing, no automation or use of a programming language.
  • Take me Mold me: Too inexperienced for the position, who would either require an inordinate amount of training, or does not show enough initiative to come up to speed quickly. When hiring interns or junior level, the latter is the key criterion.

Soft-skills Screen

Having a different person, such as the hiring manager or an HR recruiter, assess “soft skills” and cultural fit, delve more into what we’re looking for in the position and to what degree it meshes with what they’re looking for – like the Tech Screen, I find this is best done by phone or videoconference. The order of the Tech Screen and Soft-skills Screen may be flipped, I’ve done it both ways.

FTF Interview

During the FTF interview I like to use two interviewers at a time for at least one of the time slots, which allows the interviewers to trade off between talking and thinking (it’s hard to do both well simultaneously, at least for me!) – this drives out how well the candidate deals with multiple streams of interaction as often happens in a work setting, and compresses the interview schedule. I encourage technical interviewers to prepare a short programming exercise for the candidate to do in-person, particularly if no live code evaluation was done during the phone Tech Screen. Two of the key “soft skills” I like to focus on are time management and ability to summarize and analyze information. I also like planning a group lunch with at least some other team members, which helps determine how they would mesh with the team.

Other Resources

  • Laczlo Bock, Google: In Head-Hunting, Big Data May Not Be Such a Big Deal
    • “brainteasers are a complete waste of time” – I’ve found the same, they both artificially inflate and deflate impressions of candidates
    • “Behavioral interviewing also works — where you’re not giving someone a hypothetical, but you’re starting with a question like, “Give me an example of a time when you solved an analytically difficult problem.”” – as mentioned above in Tech Screen, I find focusing on a project someone did, rather than solely leading with specific skills questions, drives out their analysis and coping skills.

7 thoughts on “Hire screening bucketology”

  1. Thanks for your comments – clarified article to state I usually use two interviewers for only part of the FTF session. I recently was on the interviewee end of a phone interview with two on the other side, which I thought was pretty effective, it cut to the chase more quickly

    Like

  2. So I went and looked at “CoderPad”. Unless I am missing something, I can not see how it would be useful in a job interview. The “demo” that CoderPad created was a simple 5-10 line C program. In my mind, that does little to evaluate real engineering talent.

    When local graduates used to call me for a job interview, my first question was usually: What is the largest program that you have written? The typical answer was less than 100 lines. I would then ask, how would you manage a 100,000 line program. Typically there was silence at the other end of the line. (The main app that I sell now is closer to 400,000 lines, but if they can do what 100,000, I figure that they can manage 400,000.)

    I thought that we were beyond just trying to determine whether people new syntax? A beginning programmer that was under my supervision some 35 years ago came to me after he had finally gotten his 8080 program to assemble without errors. He had not even tried to run it! He was ready for his next assignment…

    This seems to be little more than a gimmick to me.

    Like

    1. i agree something like coderpad is not alone sufficient to evaluate someone’s deeper ability – where I’ve seen it used, and intend to use it myself, is as an early screening tool before too much time is invested in the interviewing process. I think it’s effective as an early screening device, particularly in this era of exaggerated, often ghost-written resumes.

      I like your approach of asking someone the largest program they’ve written, I’ve sometimes done a variation on that such as “roughly how many classes did you create in the app you wrote, and for what purpose” – but I’ve learned the hard way that some people can talk a good game but don’t always back it up. Watching them do something live via a tool like CoderPad gives me more confidence to proceed further with the interview process.

      Like

      1. I like your question of how many classes. I now am curious, how many classes are in my 400,000 lines of code. I know of no tool that would easily calculate that for me. Any ideas?

        Like

    2. I’m the CoderPad guy. You’re obviously correct in the sense that CoderPad cannot holistically inform you of a candidate’s maximum skill level. Then again, nothing but working with someone for a while can.

      What CoderPad CAN tell you is if a candidate is capable of doing the basic code, debug, and iterate loop. This is often a huge signal when hiring. Top notch candidates will pass most of your tests with flying colors – your challenge as an interviewer is to be able to efficiently interview enough candidates without alienating the good ones.

      Like

  3. offhand i don’t know of such a tool, but searching for “static analysis” tools in your language of choice should lead to something of that nature. There are some other good static analysis metrics for an active project, like Churn vs. Complexity, provided by RubyCritic and others, which we’ve found useful to monitor on the teams i’ve been a part of.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s