The diagram depicts the specific areas where the HireVue platform was affected by this project's implementation.
HireVue:
Live Video Interviewing
Enhancements and new feature development 2020-2022
The HireVue Platform
The HireVue platform enables users to create, schedule, and assess asynchronous and in-person job interviews. The primary interview format is on-demand video interviewing. Interviewing panel members can record their questions, and candidates can record their answers for later evaluation. On average, candidates complete 18,000 interviews a day. Candidates have the flexibility to participate in these interviews at their convenience.
Project Context
When I joined the Interview Experience team at HireVue, I was immediately immersed in an exciting project centered around developing a video conferencing system purpose-built for live in-person interviews. There was an older platform a majority of our users were utilizing, but the team was in the process of rebuilding the entirely new live interviewing platform from the ground up. We named this system "Live 3."
Beta testing was underway, and we had a select group of customers who were avid users of our older live interviewing platform, conducting interviews at a high volume with Live 3.
.
In this project, my primary focus was on designing for three main user personas: Hiring Managers, Interview Panelists, and Job Candidates. Each of these personas encountered distinct challenges during virtual interviews while prioritizing adherence to their company's hiring practices.
Candidates aimed to present themselves effectively to the hiring panel by providing confident answers to questions. Interview panelists required an intuitive interface with easy-to-use controls, enabling them to concentrate on asking questions, taking notes, providing recommendations, and efficiently managing the progression of candidates to upcoming interviews.
Team Members
Alison Hartle - Product Management
Jake Allen - Product Management
Bradley Trinnaman - Product Design
Bryce Johnson - Engineering Lead
Chris Thatcher - Engineering
Lani Tripoli - Engineering
Design Responsibilities
Partner with a product manager to assist in discovery efforts
Usability research and data analysis
Contribute to release strategy
Assist in competitive analysis
Handoff prototypes and static mockups for development inspection
Review the development builds before launching to production
When I joined the team, the Live 3 Beta was in a state where interview participants were only viewed in a grid gallery, humorously referred to as the "Brady Bunch" view. However, this view presented challenges, particularly in 1:1 interviews, as participants found it difficult to establish proper eye contact with one another.
Problem Space
During the development of Live 3, we encountered several challenges. While some customers liked certain aspects of the new interview experience, Live 3 lacked feature parity with the older version that was going to be discontinued. Beta clients were hesitant to renew their subscriptions until they were confident that Live 3 could seamlessly integrate with their existing workflows. Additionally, I joined the team during the pandemic when users had become accustomed to platforms like Zoom and Microsoft Teams, raising their expectations for performance and convenience. Despite having a small team, we worked hard to overcome these constraints and deliver a competitive product.
Release Strategy
I collaborated with my partners to develop a prioritized list of features essential for achieving parity with the older system. Our top priorities included incorporating the capability to conduct multiple interviews in one session, introducing a coding whiteboard for collaborative coding challenges, enabling screen sharing, and allowing users to join interviews using their mobile devices. We planned to implement these changes incrementally, releasing them to the beta pool of users and gathering feedback.
Throughout the process, I also made continuous interface improvements. In addition to our prioritized list, we took breaks to work on small feature enhancements to refine the overall user experience.
Who I am designing for:
Candidates
They are ready to showcase their education and qualities to potential employers, despite gaps in experience or education. They also aim to evaluate if the company and role align with their career aspirations and personal values.
Goals and Behaviors:
They look for jobs that meet their requirements in terms of compensation, schedule flexibility, commute time, career growth, and skill development.
Pain Points:
They feel overwhelmed by the extensive forms, assessments, and additional materials required for each application.
Interview Panelist
Interview panelists assist hiring managers in conducting candidate interviews, and evaluating them according to criteria and job requirements. Their responsibilities include conducting structured interviews, posing relevant questions, and actively listening to candidates' responses.
Goals and Behaviors:
Evaluate job applicants thoroughly to find qualified individuals aligned with the company's values and requirements.
Pain Points:
Training on potential biases and mitigating them through structured interview techniques and objective evaluation criteria.
Hiring Manager
Responsible for overseeing the recruitment and selection process for various positions within the organization. They align candidates with company values and goals. Hiring manager responsibilities are additional to regular duties.
Goals and Behaviors:
Identify and hire candidates with the required skills and qualifications.
Pain Points:
Struggle with learning and adapting to new technologies. Responsibilities are additional to regular duties.
Discovery and Research Overview
Our discovery efforts primarily focused on contacting users within the beta pool to assess how well the releases from our prioritized parity list aligned with their expectations.
During the pandemic, users expressed the need for certain improvements to enhance their interview experience while at home. Two key requests were the ability to blur backgrounds or add background images for privacy and professional reasons, and ensuring that interview panelists didn't inadvertently share sensitive information when sharing their screens on single-screen devices.
To address privacy concerns, we decided to implement background blurring. However, after conducting interviews with Hiring Managers and IO Psychologists, we chose not to pursue the option of allowing users to upload background images. This decision was based on the potential introduction of unwanted bias during interviews.
The coding whiteboard in Live 2
Live 2: Observational Study
By recording live interviews, we have the opportunity to analyze the footage and gain valuable insights into the experiences of the interview participants. During the review of several interviews conducted with both the older system (Live 2) and the Live Coding Whiteboard, a recurring issue became evident: the text edit panel was overlapping the video streams of the participants, making it challenging to view the code they were writing. This issue particularly affected technical candidates who felt uncomfortable with their images placed underneath the whiteboard while coding.
As a result, some users chose to switch to alternative conferencing platforms like Zoom, leading them to leave the HireVue system altogether. This issue was a significant contributing factor to the migration of users to other platforms.
Background Blur: A/B First Click Study
As part of the interface update, we implemented new controls and affordances. To evaluate the effectiveness of these changes, I conducted a series of A/B 1-Click tests using Optimal Workshop. In Cell A, the buttons were separated, while in Cell B, all the buttons were centered in the control. Both cells featured a new "Background Blur" button, and I was particularly interested in understanding how well the designed icon conveyed its purpose.
We gathered feedback from eleven participants, and 80% of them expressed a preference for Cell A over B. Cell A resonated with them because they were already familiar with the layout used in popular video conferencing software like Zoom, MS Teams, and Skype.
.
The Click Maps analysis indicated that placing the Background Blur button in the middle of Cell A was a success, taking participants an average of 8 seconds to discover it when prompted.
Furthermore, we asked participants why they believed someone would want to blur their video background while using video conferencing software. The most common theme in their responses was that background blurring enhanced professionalism and minimized distractions during video calls.
In addition to the test results, we gathered feedback from our beta pool users. After carefully considering and formalizing the necessary requirements, we proceeded with the development of the release, incorporating the preferred layout from Cell A.
A/B test results. Cell B has a high success rate, but a longer time to complete
Design Solutions:
Redlines of the new session schedule with spacing details
Multi-session Live Interviewing
The metaphor for multi-session interviewing resembles the experience when job candidates interview in person at an office. They are guided to various rooms to meet with different interviewers or panels. Similarly, multi-session interviews take place in a virtual space, where candidates start with an interview panelist, and upon completion, they are moved to the next session with a different set of panelists and questions.
The virtual nature of multi-session interviews can pose challenges, as there is no physical location to anchor the process. To address this, it was crucial for both candidates and interview panelists to have a visible schedule displaying all the interview sessions. This helped candidates know when each session was happening, join them on time, and seamlessly transition to the next session.
Multi-session interviewing is one of the key differentiating features that the HireVue live interviewing platform offers. However, it presented unique challenges since it differs from the conventional workflow users might be familiar with in platforms like Zoom.
User-flow demonstrating how Interview Panelists escort candidates in/out of the "Candidate Lobby" during multi-session interviews.
Active Speaker View
The idea for the Speaker View feature arose from feedback gathered during interviews with our beta pool users. Initially, the interview participants were displayed in a gallery grid layout, with everyone having the same window size. However, hiring managers and recruiters expressed the need to maintain a continuous view of the candidate throughout the interview process. Additionally, they highlighted that during 1:1 interviews, panelists faced challenges making natural eye contact due to the split windows.
.
To address these concerns, we introduced a new view that prominently displayed the active speaker in the largest stream panel, while ensuring the candidate's stream remained visible in the top left corner. This arrangement allowed for better eye contact and enhanced the overall interview experience. The other interview participants were thoughtfully positioned below, maintaining an efficient and user-friendly interface.
Active Speaker became the default, users could switch to Gallery view
Active Speaker view activates in multi-person interviews (when >2 participants)
When introducing the Active Speaker view, we also added pinning panels
Background Blurring
The idea of incorporating background blur into our live interviewing platform emerged from real-life experiences during the pandemic, where remote conferencing tools like Zoom and Microsoft Teams became prevalent. Many users encountered distractions in their virtual meetings, such as children in the background or barking dogs. When it came to job interviews, it became crucial to avoid such interruptions. This insight led me to consider the implementation of background blur as a feature.
.
Simultaneously, I took the opportunity to redesign the control panel component of the live interviewing system. By reorganizing the buttons and aligning them with the layouts of other services like Zoom or Google Meet, we achieved a more intuitive interface. These changes significantly improved the usability of the system, as interviewers could now access necessary controls effortlessly. The streamlined interface encouraged interviewers to focus more on the interview itself rather than searching for control options.
Participants join the interview with Blur enabled
"Quality of Life" feature: visible code writer label visible to all interviewers using the whiteboard.
Live Coding
The live coding whiteboard was a crucial feature we aimed to achieve for feature parity. One particular customer insisted on having this functionality in the live interview interface, and their subscription renewal hinged on its implementation. Fortunately, my prior experience with our Codevue product, specifically in designing the IDE for the candidate experience, proved valuable. Drawing on my domain knowledge of CodeVue, I seamlessly integrated a panel that allowed interview participants to write code within the same text editing space. To ensure consistency across the interviewing experience, I reused components from the On Demand IDE, resulting in a more cohesive and streamlined process.
It is common practice for technical candidates to complete an asynchronous On Demand interview, which includes a coding challenge. The next step often involves an in-person live interview where they can review their answers from the coding challenge alongside a hiring manager. With the live coding whiteboard and the On Demand IDE components harmonized, the overall experience became more coherent and efficient for both candidates and interviewers.
The mockup illustrates the launch of the coding whiteboard. Live coding with a Hiring Manager is a standard second step in the interviewing process. For candidates who have previously taken an On-Demand coding challenge, the controls will be familiar and intuitive.
Project Outcomes
The feedback received from usability tests and subsequent interviews with the beta pool participants was positive. Additionally, a customer who exclusively utilizes the tablet version of the apps provided feedback stating that it significantly enhanced their team's productivity during candidate reviews.