In 2017, users had access to three dashboards under the "Reports" section. However, feedback from users indicated that the charts and graphs on these dashboards didn't meet their expectations. Among these dashboards, the Positions Funnel stood out as the most valuable to users, as it allowed them to track the movement of candidates' statuses throughout the hiring process.
The HireVue Platform
The HireVue platform enables users to create, schedule, and assess asynchronous and in-person job interviews. The primary interview format is on-demand video interviewing. Interviewing panel members can record their questions, and candidates can record their answers for later evaluation. On average, candidates complete 18,000 interviews a day. Candidates have the flexibility to participate in these interviews at their convenience.
Project Context
Upon joining the Success Visibility Product Team at HireVue, a key objective was to focus on customer retention. The primary responsibility of this team revolved around developing products that would improve data collection and reporting for the users. However, there was a notable issue with the platform's report section being underused, with some users being unaware of its existence.
.
During this project, our target user persona was the Talent Acquisition Team Leaders and Analysts. These professionals are tasked with optimizing hiring processes and recruiting top talent while adhering to compliance standards. They possess strong analytical skills, which enable them to effectively gather and interpret data and generate insightful reports for leadership.
Team Members
Adam Hennings - Product Managment
Bradley Trinnaman - Product Design
Dave Haslam - Engineering Lead
Matt Burr - Engineering
Tom Stroman - Engineering
Paul Robertson - Front End Engineering
Design Responsibilities
Partner with a product manager to assist in discovery efforts
Interview both internal teams and external users
Data analysis
Sketch and usability test solutions
Handoff prototypes and static mockups for development inspection
Review the development builds before launching to production
Problem Space
In pursuit of the retention objective, we hypothesized that enhancing the platform's reporting section would boost retention rates. We believed that providing users with access to previously unavailable information would increase their inclination to stay. However, the existing reporting pages displayed data from multiple warehouses and suffered from reliability and consistency issues when compared to applicant tracking systems' data.
.
User feedback indicated that the charts and graphs didn't effectively showcase data relevant to a user’s key metrics and workflows. To address these concerns and improve the user experience, we focused on refining the reporting functionality to ensure data accuracy, and align the displayed information with users' specific needs. We strove to strengthen the platform's value proposition and, in turn, enhance customer retention.
Who I am designing for:
Talent Acquisition Leader
A Talent Acquisition Leader is responsible for optimizing processes and hiring top talent in a compliant manner. They possess strong analytical skills to gather and interpret data effectively, generating meaningful reports and insights for recruiting leadership.
Goals and Behaviors:
Minimize time-to-fill positions through recruitment strategy optimization, technology leverage, and coordination with hiring.
Pain Points:
Manual and repetitive tasks are time-consuming and redundant, such as entering candidate information into multiple systems or clicking through various screens.
Talent Acquisition Specialist
A Talent Acquisition Specialist is responsible for managing the end-to-end recruitment process within an organization. Their role is to attract, source, and hire top talent to meet the company's staffing needs.
Titles:
Recruiter, Senior Recruiter, Recruiting Manager
Goals and Behaviors:
Track candidate progress throughout the recruitment funnel, staying updated on applicants, qualifications, and movement through stages.
Pain Points:
Dealing with unresponsive interviewers or candidates, requiring effort to bridge communication gaps and avoid delays.
Research and Discovery Overview
An example of a slide from a Quarterly Business Review from 2019
During the discovery phase, we conducted interviews with Talent Acquisition Leaders, Data Analysts, and internal Customer Success Directors to gather valuable insights. To maintain consistency across the interviews, I developed an interview guide for our team to follow. Additionally, we actively participated in quarterly business reviews between the TA Leaders and HireVue Customer Success Directors.
These reviews featured custom reports demonstrating the impact of integrating the HireVue platform into their hiring process and its return on investment. By closely observing these reviews, we began identifying key performance indicators and metrics that users desired to see on our enhanced dashboards. This approach allowed us to gain a comprehensive understanding of the users' needs and expectations, laying a solid foundation for our dashboard improvement planning.
Presentation slides about usability issues with the colors used in the charts
Following the initial interviews, we identified several accessibility and visualization issues that users were experiencing with the current reporting dashboards. Notably, charts were problematic due to misleading color palettes, numbers being displayed without commas, and a lack of clear chart descriptions. Moreover, users expressed difficulty in filtering the dashboards, as the component was too large and would push the charts below the visible area of the screen.
Additionally, user feedback provided valuable insights into areas for improvement in the reporting experience. One significant frustration users faced was the resetting of filters when navigating between dashboards, requiring them to re-filter the data repeatedly.
Specific improvements were implemented based on user feedback. Firstly, we ensured that the chart color palette is accessible and viewable by individuals with color blindness. Secondly, we matched the color palettes of the charts with the data they represent, making it easier for users to interpret the visuals accurately. Lastly, we redesigned some charts to better indicate the data they represent, enhancing overall clarity and usability.
Color blindness emulation in sequential palette design: Blue and Purple shades offer optimal contrast for Achromatopsia sighted users.
The final sequential color palette combines purple and blue, resulting in periwinkle.
Usability Testing
After addressing the issues identified during our interviews, the first step involved creating prototypes for new analytical dashboards to replace the existing ones. This process began with redesigning the filter components and optimizing the layout of the pages. To ensure the success of our redesign, we reached out to the participants from our previous interviews and conducted iterative moderated usability studies.
.
The feedback from these studies was positive, as users expressed satisfaction with the proposed changes, and they found the information on the dashboard to be more aligned with their expectations.
The primary dashboard of the new system was explored through wireframes and mockups, presenting various prototype concepts (Conversions and Velocity). Subsequent feedback from these concepts played a crucial role in shaping the design of the Interview Funnel Dashboard.
An iterative release timeline displaying the successive launches of the new Analytics system.
Release Strategy
With this validation, we proceeded to develop a comprehensive release strategy. This strategy involved testing ideas from the prototypes, implementing iterative changes to the production pages, and actively seeking feedback from users during each phase of the iteration process.
.
By adopting this approach, we aimed to continuously improve the dashboards based on user input, ensuring that the final product would be intuitive, user-friendly, and specifically tailored to meet their needs.
Design Solutions:
Before my redesign, the filters posed issues. They pushed dashboard charts out of view and allowed only one filter open at a time, causing users to frequently reopen the component.
Filter Component
In our usability studies, we specifically inquired about users' experiences with challenging tasks related to the limitations of the reporting dashboards. Notably, we observed that many users opted to use Tableau for creating reports, mainly because of its convention of placing filters in the upper right of the interface. Recognizing the significance of enhancing usability, we deliberately positioned the new filter component in a similar location, making it easily accessible without being hidden behind a button.
To further refine the interaction design, we took inspiration from Excel's custom filters, a familiar tool for users. As a result, we developed a multi-select filtering dropdown component that provided a comfortable and recognizable experience. By incorporating these user-friendly design elements into the new dashboards, we aimed to streamline the user journey and ensure an intuitive interface that simplifies data analysis and reporting tasks.
The new filter component allows users to apply multiple selections at once
Interview Funnel
The Analytics section includes the Interview Funnel tab, providing a visual representation of candidate progress in the interview process. Users can refine results using filters in four categories: Interview, Team, Position, and Custom Fields, accessible by clicking the down arrow next to each category. Data can be exported by clicking the Export button.
The Dashboard Metrics: This section within the Interview Funnel tab shows key performance indicators. The Interview Rate indicates the percentage of candidates who completed interviews after being invited. Time to Interview displays the average time taken by candidates to complete an interview after receiving an invitation.
The Evaluation Rate shows how many interviews were evaluated, while Time to Evaluate displays the average evaluator completion time.
Funnel Trends: In the Funnel Trends section, distributions of incomplete and completed interviews and evaluations are displayed. Users can update the Funnel Drop-out graph dynamically by hovering over data points and clicking 'Update Funnel.'
.
Funnel Drop-out:The Funnel Drop-out graph reveals candidate progression through the interview process. Starting with Interview Created, it shows the total number of interviews. Then, Invitation Not Sent displays interviews created but not sent. Invitation Not Clicked represents candidates who didn't click the interview link.
Interview Not Started shows candidates who accessed the Welcome Page but didn't start the interview. Interview Not Complete indicates candidates who started but didn't finish their interviews. Interview Complete shows the number of candidates who completed their interviews.
For evaluations, the Funnel Drop-out graph continues with Evaluation Not Assigned, denoting completed interviews without an assigned evaluator. Evaluation Not Started shows interviews with an assigned evaluator but not yet started. Evaluation Not Completed represents evaluations started but not completed. Finally, Evaluation Completed shows interviews with completed evaluations
Candidate Experience Dashboard
The Candidate Experience tab on the Analytics page provides visual representations and downloadable reports of candidate data, aiming to offer valuable insights into their experience with HireVue.
Interviews Taken by Time of Day: One of the standout features of this dashboard is the "Interviews Taken by Time of Day" section, which utilizes a dynamic heat map to showcase the distribution of candidate interviews throughout the week and at different times of the day. This visual representation allows users to identify patterns and trends in candidate engagement, helping them optimize interview scheduling for maximum candidate participation and responsiveness.
.
Application Type: The Application Type card shows how many total applications were received, as well as how many of those applications were received through the iOS app, Android app, iOS browser, Android browser, internet browser, or other.
Candidate Origination: The Candidate Origination card shows how many candidates started their interview by an email invitation or an interview self-sign-up page.
Evaluations Dashboard
The Evaluations Dashboard provides users with a comprehensive understanding of candidates' progress in the evaluation process. TA leaders utilize this data to support interviewing panels and hiring managers in improving their evaluation efficiency.
Average Rating Given: This metric displays the average rating given by evaluators during the specified period.
Started Completion Rate: The Started Completion Rate indicates the percentage of evaluators who initiated and completed the evaluation process.
.
Average Time to Evaluate: The Average Time to Evaluate represents the average duration taken by evaluators to complete their assessment after receiving the interview notification. Only completed evaluations contribute to this figure. The Total tab provides the overall average time to complete all evaluations, while the Per Candidate tab offers the average time for a single interview to receive its first completed evaluation.
Evaluations Completed: This section shows the total count of completed and pending evaluations. The Total tab displays the number of all assigned evaluations that have been completed, while the Per Candidate tab shows the count of individual interviews that have been evaluated.
The newly designed dashboards adapt to portrait tablet screens
Project Outcomes
During the phased release of features, we proactively collected user feedback and made continuous improvements. The introduction of enhanced filtering capabilities led to an instant surge in usage. The dashboards now offer more relevant data and charts, which users frequently export as images for reporting. Moreover, the implementation of self-service workflows significantly decreased the requirement for custom reporting requests. These enhancements in the platform contributed to increased retentions. However, the work is ongoing, as users have started providing feedback on areas where they desire more detailed information displayed in the charts.