How I interview: An exploratory approach

Recently Michael Bolton posted some Interview Questions as a thought experiment. This post prompted a bit of a discussion on twitter about how people interview. Like most I’ve interviewed and been interviewed countless times so I’ve experienced a wide array of methods, techniques, and approaches. Beyond that I minored in Human Resources so I’ve studied interviewing in-depth. As one might expect some interviewing methods are better than others and often what is best is based on the specific needs of the role and company. With that said I thought I would share my approach which has served me well over the last 8 years, your mileage may vary.

I use a framework that I adapted (stole) from testing, I call it Session-Based Exploratory Interviewing (SBEI). It’s based on Session-Based Exploratory Testing (SBET) created by Jon and James Bach. The Session-based portion of the name refers to the amount of time spent. A session is a limited amount of time, known as a time box. By design interviews are time boxed, usually between one to four hours depending on the role and the number of people involved. The exploratory portion refers to how I conduct the interview. Instead of a list of canned questions that I that I ask each candidate I instead consider topics I would like to cover within the time box. During the interview I use these topics and I keep notes about what was discussed so I can refer back to them later. After the interview is complete I meet with everyone involved and do a debrief.

So when I’m hiring someone I come up with a charter or end-state goal for the interview (could be written down if you like). For a tester I want to understand their background, how they think, their skill-set, and if they would be a good fit for the team. I then consider more specific areas (topics) I would like to cover during the interview. For example interest in the job, career path, motivations, preferred type of work (structured vs unstructured), chaos management, view of software testing, and experience (if any). Beyond that I like to cover “day in the life,” company culture, team dynamics, and how we view software testing. After the “traditional” interview then I wrap up with a short testing exercise (the dice game which James Bach taught me).

No two interviews are alike and this is important because no two people are alike. To ask the same set of canned questions for every interview contorts the discussion and risks not discovering the one thing that might make them a good fit. During the interview you want a natural flow so you can learn about the person but you also need to balance that with your charter and coverage (I don’t always cover everything, sometimes more interesting things come up which trump my charter or coverage. These are called opportunities). Ultimately interviews are meant to be conversations not a set of prescribed questions with numerical ratings of responses. For me I prefer an exploratory approach.

Dice Game for Interviews:

The reason I do the dice game with candidates is because some people are so nervous during an interview that it disguises their intellect and testing ability. Doing an exercise allows me a glimpse into how they might approach testing problems without worrying about saying the right thing.

How I play

First you need to experience the dice game first hand (I suggest attending CAST or hunting down someone in the Context-Driven community). I time box the game to 15-20 minutes and preface it by saying that I don’t expect them to solve it in this short amount of time but I would like them to work through it. After the time box has elapsed I bring in someone who acts as a Product owner or Project Manager and the interviewee then gives a report about their testing. The Product Owner or PM are allowed to ask questions of the tester to better understand what took place.

Explore posts in the same categories: Software Testing

6 Comments on “How I interview: An exploratory approach”

  1. Chris K Says:

    Excellent breakdown Ben. I’ve been in interviews that are time-boxed but I don’t think I’ve heard of anyone classifying them as session-based. I like using a time-boxed dice game to watch how the interviewee responds. Why use someone else as the project manager?

  2. dynamoben Says:

    Why use someone else as the project manager?

    I want someone who hasn’t observed the game, isn’t a tester, and doesn’t know how the game is played. This then allows the reporting skills to be put to the test.

  3. […] How I interview: An exploratory approach Written by: Benjamin Yaroch […]

  4. Mike Lyles Says:

    Good blog and good notes. I have been furiously interviewing for over 16 mid to senior level testers for our team and I agree that everyone is different and you have to find a way to bring out their skills with the questions.

    I have used a stapler recently – in which I have carried it in with me to the interview, and not given any attention to it, but as the interview progresses, I put the stapler in front of them and ask them to write test cases for “this”. I want to see how they start, what they think, what questions they ask right away, whether they assume it’s a stapler or if they ask “what is this”, if they go through all the scenarios they can think of, more than just “put in staples”, and “staple paper together”. I want to hear them asking me how i would WANT them to use it – would it be used as a stapler, and if so, would it be used as a traditional stapler or would there be things we would want it to do that are not usually done with staplers.

    Another thing I noticed was your note on making the questions different for people – unfortunately large organizations (such as mine) do require us to ask the same situational questions and prepare a 3-4 question set for the technical questions that we choose. We try to keep the premise the same for everyone so that we can at least compare how person X answered the question verses person Y. However, I would agree that based on how they begin to respond, the interaction between our interview team and the candidate could branch out into dozens of new questions and ideas that will help personalize the candidate and either give them an advantage or disadvantage when comparing to others who have applied.

  5. dynamoben Says:

    …unfortunately large organizations (such as mine) do require us to ask the same situational questions and prepare a 3-4 question set for the technical questions that we choose.

    I’ve worked for large organizations too and I’m familiar with this technique. It’s not a bad idea to have leading questions like that, as long is its paired with other things and you are willing to branch from there. Actually they can work well at times, I have four “canned” behavioral questions at the bottom of the outline I use for interviewing testers. I will use one of them if it seems appropriate or the topic hasn’t come up another way.

    I didn’t cover this in the post but much of what I do is behavioral, “have you been in this situation and how did you handle it” sort of thing. The rational is the actions of the past are indicators of the future. With that said I don’t entirely buy into that I think context matters, just because someone was frustrated with a short deadline in the past doesn’t mean they will be in the future…then again they might. So you have to be careful about trying to predict the future from the past.

  6. […] questions for technical positions; good responses, bad responses, and rational. I’ve written about interviewing before so I was honored to have the opportunity to chat with her about interviewing. Check it out and […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: