AI CS Part 1: Understanding Resolution Rates

The single biggest transformation we’re seeing in Customer Service is the transition from Support Leader to AI Support Leader. There is a lot going on in this, it’s not as easy as updating your LinkedIn title and asking for a payraise (though be my guest), there are new roles in your org (e.g. who designs AI workflows, who builds your Fin Tasks etc.) , and new priorities (your help docs are suddenly mission critical infrastructure).

As AI Happens™️ to every industry, we see the same story. It starts with augmenting your team and Copilots while everyone tells themselves nothing will really change (aka the Cope-pilot stage).

It quickly upgrades to Agents that actually do the work, and once that starts people have lots of questions, questions like…

1. How do I measure if the work is getting done (that’s what this post is about)
2. How quickly will this happen? (this post on timelines)
3. Shouldn’t we build our own Agent? (sure thing, you’ll be back though)
4. If >50% of the work is now Agents, what are my new metrics?, (this post on metrics)
5. What’s my org chart look like in a world where 1 Agent is doing half the work? (this post is to follow)
6. What are the new roles & responsibilities here? (to follow)
7. and a whole heap of other questions yet to emerge (let us know what you want answered)

We have so much to say about this transformation, the new org charts, the new roles, and we’ll get to it all

For now though there’s two things everyone needs to understand, specifically about evaluating different agents based on AI Resolutions, so let’s start there:

Note: Resolution Rate means "Of the questions the AI Agent is involved in, what percentage does it resolve". The points I'm making in this post are about comparing agents against each other given the same setup.¹

1 – Every percentage in resolution really matters

If one agent costs 99 cent a resolution and claims to do 72% of your volume (might sound familiar) and another can do 35% but is absolutely ✨free✨, you might auto conclude “we’ll go with the free one“. And who’d blame you? No need to talk to your CFO, no need to load up your procurement tool, just pull the trigger right?

The maths you’re intuitively doing is something like this

My Conversation Volume × 72% × $0.99c = $$$
versus
My Conversation Volume × 35% × 0.00c = $0

It’s easy to pick the winner, but you’re playing the wrong game.

The right game is to talk about Total Cost Of Running Support. You need to ask yourself “What’s happening to the queries that the AI doesn’t handle“

The answer is humans, humans are happening to them. This means you’re asking your support team to shoulder the burden of these queries when in reality there’s loads more valuable work for them to do, and you’re still growing your team’s size as your business (and thus support queries) scales.

The fully loaded cost of a human handling a support ticket is something approximating the % of their fully loaded salary + all associated costs for the person. So for easy maths let’s say your support person is $4k a month, and handles 25 tickets a day, with 20 working days in a month that’s $8 per ticket. Obviously this ignores a load more complexity, e.g. have you heard of Payroll taxes, health insurance, benefits, equity, laptops, software licences per seat (Slack, Zoom, Zendesk, G-Suite), management overhead, and that’s before we get into office spaces etc. Let’s be optimistic and say the $8/ticket turns out to be more like $10/ticket.

Okay, so at $10 per human support resolution, and a choice between 99c and 0c for AI resolutions, how does the maths play out if you have, say, 100,000 conversations per year? The answer is like this…

? The job of an AI Support Leader is to minimise human handover, not to minimise dollars spent on AI resolutions.

2 – The hardest percentages matter the most

Most queries to your team are not the basic “how do i reset my password” ones. They’re messy ones that require information from multiple sources to answer. As we discussed in Good Bot / Bad Bot, it’s important your agent can do the hard stuff.

But your AI Agent also needs to move past informational queries (e.g. ones answered through text alone) into personalized queries (unique to the user) and into action-based queries (e.g perform an action in another system).

These are often a smaller percentage of the total volume, but they’re a larger percentage of the total time spent by your team. This is why we built Fin Tasks – we know that to really deliver on the promise of AI Support, we have to complete the messy harder queries end to end, to leave no crumbs as the kids say. (I know, I can’t believe I wrote that either)

The right way to think about these less frequent but more painful queries is frequency × handling time. Which looks more like this…

? The job of an AI Support Leader is to minimise the time wasted on repetitive actions, not just automate extremely frequent questions.

Ultimately the first step when moving to AI CS is picking the best agent for your business and it putting it live. The second step is ensuring that you’re minimising handovers and minimising repetitive schlep work. If you get those two done, you’re ahead of the majority of your competitors, but you’re not done, not by a long shot…. we have so much more to show you. Stay tuned.

1. When you’re optimising your agent, once you’ve picked one, your actual goal is just “total resolutions” (in the same way that when you’re evaluating a website design you care about conversion rate, but once it’s live, you care more about total conversions) ↩︎