Get In Touch
Dialpad QA scorecards
Overview
Creating an unified call quality control assessment for contact center managers.
I designed the end-to-end experience of creating, assigning and grading the scorecards.
Team
Contact Center Team (Dialpad)
Role
As the product designer, I worked closely with a lead designer in creating a qa scorecard on the Dialpad web app.
Responsibility
Product design, IXD, user research, product strategy, and overseeing development
Duration
Aug 2020 - Nov 2020 | Released '21
About
Dialpad
Dialpad is an all-in-one voice-over-IP (VoIP) platform that provides voice calling, messaging, and meetings in a single app.
With cloud-based communications powered by AI, Dialpad is used by small/medium sized businesses and contact centers across 7 countries (USA, Canada, England and Japan ect).
Problem & Pain points
Managers cannot assess and conduct call quality control in Dialpad
To ensure the optimal customer satisfaction, businesses invest hefty on building a team of high-quality customer support agents. Contact Center managers want to be able to evaluate calls based on criteria that they care about. Thus allowing the managers to provide agents with quantitative feedback on their calls so they can learn and improve.
Currently our customers with do this informally or outsource it to another (paid) product. Even managers at Dialpad use another 3rd-party tool to perform call quality control.
As turnover rate for agents is high, companies may not be able train or prepare the employees fully. Managers often would want to provide coaching to those agents who are new or struggling.
Goals
Creating an quality control assessment and feedback cycle for Contact Centers
Help Users
- • Allow Managers to grade agent calls all in Dialpad
• Allow Managers to assess and view agent performance
Help Dialpad
- • Create an unified and single platform experience
• Increase customer stickiness and loyalty to Dialpad
• Increase in customer satisfaction
Help Contact Center team
- • Optimize and enhance features for Contact Centers
• Strengthenproduct's end to end coaching story
Solutions
Grade and view agent performance with custom scorecards all in Dialpad
Provide recommended templated questions in the scorecards
Managers can create new scorecards with prebuilt templated questions, or create new templates to build new evaluation criteria that cater to the managers' needs.
Customizable grading criteria in questions
Customize question with different types and rate how important a question is.
View graded calls, find calls to grade, easily notify others to grade call
At a glance, view graded calls or find calls to grade.
Additionally, share calls to invite others to grade.
Grade calls with transcript or recordings for reference
Use existing voice intelligence to conduct semantic analysis and provide suggestions to a question in the scorecard.
View agent performance and find agents who need coaching support
View agent performance at a glance or drill down individually to view their performance in detail.
(Not my design, but adding to complete the whole project)
Result & Next Steps
Most sought after feature in Contact Center License
Unfortunately the a new PM joined the project when it was closed to project launch, so desired metrics were not collected properly. However, through chatting with internal sales and CSAT, Ai Scorecard is one of the most sought after feature in the Contact Center product.
View QA Scorecards in Demo (2023)
The project was first released in 2021. The company has been through a rebrand since then with an emphasis on labelling Ai. The visual may look different however the general design has stayed the same.
Research
How should the entire call quality assurance cycle look like?
Crazy 8s Design Workshops
We conducted 2 sessions with 15 internal stakeholders, ranging from C-suites, engineers and support agents. We asked them to brainstorm and come up with sketches that would helpful for the users.
User Interviews
- 5 user research interviews were conducted to the managers to understand better about their goals and workflow in assessing quality control.
Creating Findings Synthesis and User Flows
We synthesized the findings from the workshops, user interviews and competitor analysis. We presented the general user flow and came up with fundamental design feature requirements. This process allow PM and engineering lead to understand the project overview to prepare scoping and resources.
The scale of the project was much bigger than what the team has anticipated, so we decided to split up the flows to work in parallel to meet the desired deadline. I was tasked to come up with the solutions for 1.) Creating a rubric, 2) Find and assign calls and 3) Score calls. The rest of the flows were designed by the lead designer.
Understanding Different License Types
As Dialpad is a SaaS company, different subscription and licenses are offered. Employees same roles within the company might have different feature access based on the license they are on. We came up with charts to further differentiate the type of access and permission they have on individual page.
Ideation & Validation
How should scorecards be created?
Initial Iteration
White space with limited information was purposely intended, as I want the manager to solely focus on creating questions one at a time.
Final Design
We discovered that managers are so used to seeing many graphs and details in one page, so opted for a more compact view when creating a score card.
How can we create the best grading experience for the supervisors?
Many options were explored on how we can create the best grading experience for the managers. I originally went with the existing call summary pattern to figure out how can grading be incorporated into the page.
Low-fidelity explorations
Proposed Solutions
Hoping to create a more modern and refreshed look with bigger real estate to grade call, I deviated from existing pattern and explored two column layout. It was well received from the stakeholders.
Final Design
However despite the well-received exploration from internal stakeholders, the desire of revamping the transcript/ call summary page received a strong pushback from the engineering team. By doing so, the project will be in risk of not being able to meet the deadline.
Additionally some users are already used to the three-column layout, and changing to a two column layout may interrupt user's mental load and experience with Dialpad.
.
North Star
Automate call grading with recommended semantic- analysis suggestions
Edited on Jan 2024
The two exploratory north star feature has been rolled out. Even though I am no longer to over see the project but I am happy to know that my exploratory design was still used and implemented.
This was one of the most anticipated features, however due to time constrained it was not included in the engineering sprint. However I still came up with exploratory design to present to the the stakeholders to get buy-ins.
Use sentimental analysis (Vi) to help answer the questions: allow admins to come back to finish the work as the process may be tedious.
Connect call summary to custom moments: allow admins to double check on their work before creating class creation, plan eligibility mapping that might impact the entire benefits package
Learning
Don’t deviate from familiar patterns
Use familiar interaction patterns that matches users’ mental models
