Foundational user research
Leading initial user research and synthesis at bebop
Background
I joined bebop in June 2015 to help reboot their design team. At the time I joined, the larger team was demotivated and there was a significant tension between the engineering team and the design/product team. Their design director had recently left, and the previous version of the product had been scrapped. I was the second design hire on the new team and we were tasked with starting over from scratch to design an Applicant Tracking System (ATS).
The first thing we needed to do as a new team was to establish a shared understanding of the recruiting space and its key users. This was critical for our own knowledge, but also because creating a shared foundation of knowledge with the team would help us build trust and bring the team back together. Ultimately, our goal was to use this research to help the team create a shared product vision and product strategy, grounded in solving the most pressing problems we heard about from our users.
To do this, we launched a foundational user research project to understand and document the goals, pain points, and processes of the key user types involved in the hiring process.
The research was hugely successful not only in helping us form the foundation of our product strategy, but also by bringing our team together with a deep shared understanding of the recruiting space and creating a user centered culture.
Bebop was acquired by Google and was converted into Google Hire about four months after this research concluded.
Approach
1. Create a research plan
Establish research goals, write out questions, and plan logistics around recruiting, interviewing, observing, and note taking.
2. Conduct interviews
Interview your participants and take notes
3. Synthesize results
Analyze your interview notes to extract meaningful, actionable insights.
4. Apply the research insights
Research is only helpful if people are aware of it and actually use it to make decisions. Create artifacts that help summarize your research and make it easy to digest.
Creating a research plan
With my design partners, I created a research plan aimed at establishing foundational, shared knowledge from which we could begin building the product. The plan included questions to help us understand goals, pain points, and processes.
Interview guides typically start broad with open ended questions, then slower taper in toward specific questions. We do this in order to ensure we don’t bias interviewees with specific topics before they have a chance to say more broadly whats important to them.
Everyone in the company, regardless of role, was encouraged to come observe the sessions so we could all have a shared foundation of knowledge.
This kind of shared first hand knowledge enables engineers participate more deeply in the product definition process, reducing need for documentation and minimizing risks of unexpected technical constraints later on.
My colleague mapping out the workflow of a Sourcer during an interview
Conducting interviews
We conducted dozens of interviews with recruiters, sourcers, hiring managers, interviewers, coordinators, applicants and more to get a holistic view of the hiring process.
We limited the number of observers to 1-2 per session to avoid overwhelming the participant, and asked each observer to take notes so that the interviewer could fully focus on conducting the interview.
Since we were conducting a large amount of interviews, we asked everyone to type their notes so we could more easily organize them to uncover meaningful insights later on.
After each interview, we’d have a 15 minute debrief where we’d write down our top takeaways.
Uncovering meaningful insights
After the interviews, I manually triaged all the notes for each role (Recruiter, Hiring manager, etc…) and grouped the key pieces across interviews to uncover common themes and patterns. I also color coded the notes by which interview they came from so we could keep track of who said what. Names are covered to maintain privacy of participants.
Through this process, we uncovered several common themes that ended up forming the basis for our product strategy.
Current tools aren’t optimized for collaboration
Hiring is a highly collaborative process, but existing tools were build almost exclusively for recruiters, making it hard for hiring managers and interviewers to seamlessly collaborate. In addition to creating extra overhead, this tangibly slowed the process down and resulted in more errors and bad hires.
“The recruiter I work with usually just shares a Google Spreadsheet with me to track current and rejected candidates for my jobs. I grade each candidate here, because it’s the easiest way to get on the same page. Unfortunately, none of this is saved in our ATS.”
- Hiring Manager
Fragmented processes lead to a waste of time
If we learned anything, it’s that speed and accuracy are critical in hiring. Despite this, we heard over and over again how fragmented processes across tools created high manual labor for recruiters that slowed them down and increased silly errors.
“I use a broad set of tools to address different needs I have throughout the process, but the manual process of keeping everything in sync between them all is hugely cumbersome”
- Recruiter
Tools aren’t optimized for mobile
Recruiting is a job that never really stops. In interviews I conducted, I heard stories of recruiters needing to pull over on the side of the road and tether their laptops to their phone to take care of a simple task because they couldn’t do it on their mobile device. This later led to the formation of our mobile team, which I led design for for 1.5 years. You can read more about that here.
“I always check my schedule the night before to make sure all interviewers have confirmed. Somehow, I’m still putting out fires on my way to work the next day - like when people call in sick. It’s hard to get all this done efficiently without easy access to my ATS when I’m on the go.”
- Hiring Manager
Applying the research insights
Research can be enormously informative, but it can be hard to keep conversations grounded in what was uncovered when decisions are being made. To help us, we created several artifacts that we put up around the office to make the main research takeaways easier to digest, keep the team grounded in the research we’d conducted when making product decisions, and help bring new hires up to speed faster.
I created personas of each user group we interviewed. These don’t represent specific people, but rather are an aggregate of the most important and representative things we heard these users say.
In addition to personas, I also created a user journey map that tied the personas together and illustrated the broader hiring process and its major pain points. The journey map emphasizes the collaborative nature of hiring by focusing on interaction points between the recruiter and hiring manager (the green and blue lines) while pointing out common collaboration points with other key personas along the way.
This research formed the foundation of our product strategy, marketing strategy, and set our team up to have a user centered culture
The insights we gained from this research formed the foundation of our product strategy, marketing messaging, and helped build a strong user centered culture and shared understanding that lasted throughout the time I was working on Hire. This project helped establish a culture where our engineering teams were highly invested in the product definition and design process from early stages, making it infinitely easier to build a high quality, user centered product.
Below is a snippet from the Google Hire marketing website where you can see the insights we uncovered in the research such as collaboration, seamless integration, and speed.
Please reach out if you have any questions!