I’ve done a lot of interviewing over the years of both developers and testers.
Hiring is a very inexact science, but you can maximize your success rate by using effective screening techniques.
Below are some of the more effective approaches I’ve used, and seen used as a job-seeker.
In my experience, it works best for a Tech Screen to be done first for developers and testers (and for technology managers too) , and it’s far more efficient to do these by phone than FTF for both the interviewers (can cut it off sooner) and the seeker (one less day to wear a suit). Using Skype or other videoconferencing can provide additional early insight into their communication effectiveness.
I’ve found it’s much more important to focus on aptitude than specific skills, because things change fast in the technology business, and you’re always going to need your people to be pushing the envelope and learning if you want to innovate and consistently deliver value over time. I like to focus on their approach to how they delivered one example project (of their choice), including overcoming obstacles, rather than only on the technical aspects of how they used language X or pattern Y – this usually shows whether they had to learn new skills or think outside the box.
I also usually try to make a preliminary determination during a tech screen of how well the current interests (not just abilities) of the candidate and needs of the offered job align.
Toward the end of a tech screen I’ve seen CoderPad used, which is a great way to see if they “think in code” to a sufficient degree they can type simple constructs without an IDE or intellisense, and also how fast they type which you cannot take for granted.
Toward this end, I have found the following “buckets” useful to quickly weed out candidates who are not a good fit. Note my criteria are weighted toward hiring for product-oriented start-ups / SMBs, or small high-performing teams within larger organizations:
- Corporate Cog: Someone who has worked primarily for larger companies / on larger teams often has to make an adjustment to be effective on a smaller team. Assess whether the person is going to need lots of hand-holding, wants to do only one thing, is not comfortable with perceived risk, doesn’t want to leave their comfort zone of what / how they are accustomed to doing. Example red flags: “I thought you were bigger than this” or asking “do you offer tuition reimbursement or sabbaticals” early in the interview process.
- Lone Ranger: Has worked alone much more than collaboratively with a team. Can occur even on project teams. Example: Someone who has done only narrow contract engagements which didn’t require much interaction or collaboration, or has worked in an isolated manner on teams not following agile principles of frequent review and feedback.
- New New Thing Groupie: Emphasizes interest-in / experience-with latest / bleeding edge technologies or buzzwords to the exclusion of describing in-depth experience delivering projects of any depth. Example: Someone who repeatedly says they have most liked learning new things / are interested in your job because you’re using cool technology, to the exclusion of talking about what they’ve delivered or your product.
- Tire Kicker: S/he is just not that into you. They are often seeking the highest-paying / cushiest-benefits job they can get, to the exclusion of being interested in your product, how they could contribute to it or how they could be challenged. Example red flags: “I’ve got 2 other offers and expect to get 2 more within the next week”.
- Hyperbolator: Exaggeration of experience on either resume or during tech screen, e.g., listing experience with some technology when it was used only by someone else on a project, or so shallow as to be irrelevant. Examples: “I listed InstallShield but never personally used it, it was used on a project I was on”, “I’ve never written a web service, only consumed them”.
- Blabbergaster: Similar to Hyperbolator, but differing in that this individual might actually have the relevant experience, but have focus issues and / or underdeveloped listening skills. For example, if they spend more time explaining why they didn’t do certain things on a takehome test than just having attempted them, or if you picture this person leading a meeting where they leave everything open-ended with no ability to drive to closure. Examples: “While working on the takehome test I wasn’t sure what you were looking for, so I stopped”, or “blah blah blah… what was your question again?”
- Disgruntled Knowledge Worker: Someone with a fair amount of work experience who seems to have never been satisfied at any job (vs. making the most of each opportunity), complains of being “misunderstood” or “underutilized”, might be too prone to blame their not having found that perfect job on outside factors like the economic downturn.
- Egghead: Someone who has very strong theoretical understanding, but not enough balance with real biz apps and / or app delivery. Example: Candidate who had worked for years at NCSA, had very strong math skills, but working on primarily research projects of 6 months to 2 year duration, no concept of frequent delivery via an SDLC.
- No There There: Someone who doesn’t have much depth (at least in areas of interest to the company / job), despite a lot of experience as measured in years or projects – they were in roles on projects where they didn’t do much technically challenging work. Example: Someone with 5 years .NET experience, who had never used a collection construct; a tester who had years of work experience but solely manual testing, no automation or use of a programming language.
- Take me Mold me: Someone too inexperienced for the position, who would either require an inordinate amount of training, or does not demonstrate enough initiative to come up to speed quickly. When hiring interns or junior level, the latter is the key criterion.
I like to use a takehome test as the next step after a successful Tech Screen, for both developers and testers, usually with a deadline of 3-4 days or a weekend, taking into account any schedule constraints they have. This is very useful to see how they approach a simple problem, document their approach in simple terms, make and list their assumptions and invest enough time to demonstrate interest. A key is how well they organize and summarize information – for testers I use a test that requires some minimal statistical analysis, since that is crucial to avoid the data-proliferation-without-summarization syndrome.
Optional second-round Screen
If the Tech Screen and Takehome Test are a slam dunk, I usually skip this – otherwise having a different person, such as the hiring manager or an HR recruiter, assess “soft skills” and cultural fit, delve more into what we’re looking for in the position and to what degree it meshes with what they’re looking for – like the Tech Screen, I find this is best done by phone or videoconference.
During the FTF interview I like to use two interviewers at a time for at least one time slot, which allows the interviewers to trade off between talking and thinking (it’s hard to do both well simultaneously, at least for me!), drives out how well the candidate deals with multiple streams of interaction as often happens in a work setting, and compresses the interview schedule. Two of the key “soft skills” I focus on are time management and ability to summarize and analyze information. I also like planning a group lunch with at least some other team members, which helps determine how they would mesh with the team.
- Start-up Spotlite’s insightful essays on how they do engineering interviews: “Interview Goals: Engineering Skills” and “What to Expect at a Spotlite Interview”
- Interviewing at TrialPay 101: Nice brief outline of what this highly selective employer (extends offer to 1 in 50 interviewed, 1 in 500 who apply), focuses on in the interview process for engineers. They generally do a phone screen -> live coding round -> onsite FTF.
- Laczlo Bock, Google: In Head-Hunting, Big Data May Not Be Such a Big Deal
- “brainteasers are a complete waste of time” – I’ve found the same, they both artificially inflate and deflate impressions of candidates
- “Behavioral interviewing also works — where you’re not giving someone a hypothetical, but you’re starting with a question like, “Give me an example of a time when you solved an analytically difficult problem.”” - as mentioned above in Tech Screen, I find focusing on a project someone did, rather than solely leading with specific skills questions, drives out their analysis and coping skills.
- Johanna Rothman: Hiring Technical People Blog, including Cultural Fit, Hiring Testers articles
- David K. Williams: The Case For ‘Under Qualified’ Employees: The 5 Best Reasons To Hire For Aptitude, Not Skills
- Adam Kooperman, CultureFit Technology Staffing: Hiring Contract and Permanent Technical Employees with an emphasis on Culture Fit
- Adam Bryant: New York Times Corner Office twice-weekly interviews with executives on the topics of hiring, culture and leadership
- Good site to mine for ideas, mainly for tech screening…
- But it is hardly exhaustive…
- I’ve observed a fair number of companies seem to use this or similar sites, as there are “greatest hits” questions that get asked a lot like:
- OO: What is the difference between an abstract class and an interface?
- SQL: What is the difference between inner and outer join?
- SQL: What is a clustered index?
- CoderPad: Online tool to facilitate coding evaluation during a phone or video conference.