2023 - 2024 • Artificial Intelligence / B2B SaaS / GrowthDriving revenue with Premium Conversation Intelligence
AI boosts customer revenue by 19% and cuts call review time by 60%
Context
CallRail is an industry-leading call tracking and analytics platform that helps businesses monitor marketing campaign performance across phone numbers. To understand the call content assess lead quality, customers relied on manually reviewing calls by either listening to the audio or reading the transcript.
Solution
I streamlined our customer’s call review process by leveraging artificial intelligence to analyze customer conversations at scale. I redesigned the call log and call detail views to clearly surface a call summary and actionable insights about a call or caller.
I designed aggregate insights reports, further reducing reliance on manual call review by summarizing a week of calls and providing insights into trends and outliers over time. Scheduled emails met the uer wherever they were, extending the value of the product beyond the application.
Key CallRail outcomes
31%
44%
38%
Increase in MRR for Conversation Intelligence
Increase in Premium CI trial conversions
Reduction in churn for Premium CI subscribers
Key customer outcomes
60%
50%
19%
Reduction in time spent reviewing transcripts
Reduction in time spent qualifying leads
Increase in MRR for Premium Conversation Intelligence subscribers
Problem
As customers scaled their marketing efforts, the increasing call volume overwhelmed their ability to identify quality leads and created a growth ceiling.
Business Impact
By automating much of the call review workflow, our customers were able to spend less time reviewing calls, missed fewer leads, and increased their lead-to-conversion rate.
Accurate call summarization meant customers could quickly understand what a call was about without reading an entire transcript or listening to a call.
Sentiment analysis provided a clear signal regrading the quality of a call and/or lead.
Aggregate insights reporting surfaced qualified leads and agent performance.
ProblemHow might we reduce customer reliance on manual call review and surface crucial caller and conversation insights?
Research & Discovery
Short opening quickly highlighting research process.
Market context and opportunity
The conversation analytics market was projected to reach $1.8B by 2023, with most solutions targeting enterprise call centers. Our research identified a significant gap for SMBs who needed enterprise-level insights without enterprise complexity or cost. This aligned perfectly with CallRail's strategy to increase ARPU through premium features while addressing competitive pressure from emerging conversational intelligence tools.
Figure: Competitive analysis table
120 mins transcribed
Customers needed to have at least two hours of calls transcribed per day.
Fewer than 25 employees
Small businesses with fewer than 25 full-time employees were prioritized.
Segment demographics • 24 total interviews
6
5
4
3
6
Home services
Real estate
Legal
Medical
Marketing agencies
Figure: FullStory graph of total active page time
Who are we designing for?
Small Business Owner
Sole proprietor for whom marketing is a tool secondary to running their business.
Motivations: Cares about growing their customer base, as well as being responsive to current customers. Reluctant to hire additional staff due to increased operational overhead.
Behavior: Engages with CallRail frequently, using the Call Log to check campaign health, review calls, and prioritize leads for follow-up.
Pain Points: Lacks time to listen to each call individually. Worried that they may miss calls from potential new leads.
Design considerations: Needs easy access to operational insights to streamline process and develop targeted training for employees.
Marketer
Freelancer or a part of an agency. Marketing is their business.
Motivations: Cares about campaign impact. Works primarily with spreadsheets to generate reports for clients.
Behavior: Rarely interacts with the CallRail application, preferring instead email summaries, reporting, and raw data exported as a CSV file or via API.
Pain Points: Frustrated by lack of clear signals indicating campaign or channel effectiveness, limiting their ability to drive lead generation for current clients. Wants aggregate insights to better identify trends and outliers in campaign performance.
Design considerations: Needs analysis of changes in call volume or sentiment before and after marketing campaigns.
Customer Service/Sales Agent
Employed by a small business, where customer satisfaction and word of mouth are critical.
Motivations: Cares about customer retention and providing a high quality support experience.
Behavior: Engages with CallRail daily. Appreciates clear messaging on the outcome of a call.
Pain Points: Frustrated by lack of clear signals indicating caller sentiment. Wants to quickly understand call outcomes and if there is further follow-up required.
Design considerations: Needs information on common issues and customer satisfaction levels.
Research methodology
Quantitative analysis: Analyzed 150+ user interactions with call data using FullStory
Customer interviews: Conducted in-depth sessions with 24 small businesses of with fewer than 25 FTE employees
Competitive analysis: Evaluated 5 leading conversation analytics platforms
Hypothesis“Can AI deliver the insights needed to automate call review and drive customer revenue?”
Pain points
Call volume was too high to manually review calls for high-quality leads or customer service monitoring.
Too few signals indicating lead quality.
No aggregate insights across conversations.
Due to staff and resource constraints, business owners need to remain focus on core business operations.
Opportunities
Summarize individual calls and extract/surface relevant and actionable insights.
Surface sentiment insights about customer interactions and their likelihood to book an appointment, convert, or churn.
Summarize multiple calls, surface trends and outliers across data.
Meet the customer where they are and extend value beyond the product with scheduled email summaries.
Explorations
Short opening quickly highlighting ideation and wireframing process.
Prompt crafting
Finally - it was time to put our prompt-writing skills to the test! Using Postman and Assembly’s API documentation (as well as a healthy amount of assistance from our engineering partners), the content strategist and I were able to quickly test and iterate prompts for each of the insights we wanted to surface. At the end of the process, we found that the best prompts were relatively short and clearly articulated both the call context and the expected output format. This minimized hallucinations and delivered the consistent results needed to serve as the foundation of our new product.
Figure: Call Summary auto-chapter prompt return, and how we translated that via interaction design
Figure: Call Summary single summary prompt return, and how we translated that via interaction design
Figure: Sentiment Analysis prompt return, and how we translated that via interaction design
Further Design explorations
Through iteration with the LLM and wireframes, we explored multiple options for representing the call summary and sentiment analysis.
Figure: Call Summary iterations, featuring auto-chapters.
Atomic design principles
We audited the current workflow to better understand where our opportunities lay for integrating AI insights.
Figure: Prompt crafting workflow:
Draft prompt <--> LLM response -> Wireframe mockups (and flows back into draft prompting, as visualizing the results helped us refine our prompt)
Figure: Sentiment analysis iterations, featuring the spectrum design.
Figure: More wireframes (arranged nicely, of course)
MVP Definition
We took a phased approach to designing and rolling out features.
Phase 1
Call summaries, basic sentiment analysis visualization
Phase 2
Sentiment analysis timeline and highlighting, customer/agent sentiment breakdown, aggregate insights
Phase 3
Questions asked, appointment booked, sale made, call coaching
Figure: Prioritization framework
-
Technical
The engineering team identified processing limitations that would impact real-time sentiment analysis. Together, we decided to focus on post-call analysis first, with questions asked and real-time features being explored for a future iteration.
Design
Additionally, the integral nature of these views to our customers Jobs To Be Done limited how drastic the design changes could be. Redesigns of the interface or experience were off the table.
-
This research phase involved close collaboration with:
Product Management: Defining feature scope and pricing strategy.
Engineering & ML: Engineering helped evaluate processing requirements for potential real-time sentiment analysis. AssemblyAI refined sentiment detection algorithms based on our visualization needs.
Sales & Customer Support: Validating value proposition with existing customers.
Marketing: Aligning with go-to-market messaging strategy via Pendo and other channels.
Product Launch
Short opening quickly describing the overall theme of this section.
Iterating on customer feedback
Product Evolution
Short opening quickly describing the overall theme of this section.
Iterating on customer feedback
04 . Summary
So…can AI actually drive revenue?
If the great promise of AI is to be a productivity enhancing tool, then the AI-enabled features of Premium Conversation Intelligence delivered. Small businesses with limited staff were able to more effectively address a greater volume of calls, following-up with more high quality leads and converting more customers.
50%
Decrease in time spent qualifying leads.
As calls come in, leads must be sorted by quality. Those with the greatest likelihood of conversion are prioritized as “Qualified Leads”.
31%
Increase in MRR for Conversation Intelligence.
Reduction in churn and greater trial conversions.
60%
Decrease in time spent reviewing calls.
Increased marketing spend results in greater call volume. Manually reviewing calls is time-consuming and often not feasible.
19%
Increase in MRR for Premium Conversation Intelligence subscribers.
Surface high-value leads, trigger follow-up based on keywords that matter to the business.
Milestone 03 • Key results
It’s not just about spending less time reviewing calls. It’s about helping our customers get more from their marketing efforts so that they can do more business. CallRail customers were able to increase revenue without hiring additional staff because they could rely on Premium Conversation Intelligence to surface high quality leads and insights.
AI can be a key driver of growth and revenue with a clear and realistic use case. The tech is still relatively young and requires an understanding of its limitations, as well as its capabilities. AI won’t solve every problem, but it certainly helped our customers focus on their core business and enabled them to market with confidence.
Design for AI-enabled humans. AI doesn’t add value alone. It’s how we design for AI-enabled features and the humans that use them that makes the difference. Understanding how user mental models change - and the resulting evolution of UX - patterns will determine if customers actually realize value.
What did I learn? 🧐