Everything you need to deploy, scale, and transform CX with AI Agents, from articles and videos to proprietary research and deep-dive manuals.
Featured content
Articles
Topic:
Videos
Shorts
Transcript: Transforming CX with AI - Live from Paris
- Okay.
- Welcome everybody to our Blueprint live event.
- Really glad to have some new faces in the room and some other folks who are actually with us all day today for various workshops.
- Nice to see some familiar faces here as well.
- Before we dive in, I just want to acknowledge one thing.
- If you're in this room, you're obviously somewhat bought into the idea of using AI for customer service.
- We'll talk a little bit about where you are on the maturity curve in that space right now.
- But if you've actually shown up today, you know, there's some sort of interest or somebody told you you have to go and show some interest in using AI for customer service.
- So in terms of the maturity curve, we've broken this down in a few different levels at Intercom.
- We've, found this out from working with various different customers or prospects that we're dealing with every day.
- And it goes from level zero of traditionalist, like not really into the idea of AI quite yet, all the way up to AI pioneer where your whole system and organization is rearchitected completely around the use of AI.
- And what we found is that most folks that we speak to are somewhere between level one and level two at the moment.
- They've gotten going with AI.
- They have gotten some good content sources going with AI for customer service, and they're starting to explore further automation.
- So have a think about, as you're looking at the slide, like, where you think you're sitting on this today.
- I like to think that our team are somewhere between, like, level three and probably up into level four now.
- But I work for Intercom who build an AI agent, so I really should be in that space.
- Right?
- And I'd love to just chat with you all a little bit more about what we've learned as we've moved into that level that we're in at the moment.
- So over the past few years, we're going back about three ish years now when we launched Finn first.
- So about six months prior to that, ChatGPT burst onto the scenes late twenty twenty two.
- And the team that I work on, the customer support organization at Intercom, I lead the AI support team there now, but I was leading the frontline folks who work with our customers directly.
- So at the time, when Intercom decided to build an AI agent, which we call Fin, we were the first team to use that, and nobody else had done this before.
- So as the first team to actually start using an AI agent in this kind of generative AI world, there was a lot of new stuff to try and figure out, you know, who's gonna be responsible for this, what are our success metrics, what's this gonna mean for our customer experience across the board.
- And every single thing that we were doing was brand new and there was no documentation, there were no workshops or webinars.
- We were figuring all this stuff out from scratch.
- But then over the course of those past few years, we've worked with a lot of customers on implementing AI agents, implementing Fin, and getting some really excellent success.
- So we realized that for folks who are a bit newer on this journey, it would be incredibly helpful to actually have some sort of guidance or what we call a blueprint.
- So we built the blueprint, and it's actually a book that you can pick up there at the moment.
- There's a landing page of a website for us, but it covers a lot of concepts around how to scale like launch AI, first of all, for your customer service organization, but then also scale it and start getting some really, really amazing results from us.
- So in terms of launching it, within the blueprint book that you can have a look at, what that covers in the launching it phase is, you know, how to build a business case to actually go and, you know, quantify the return on investment and ensure that you're actually getting investment from your company to go ahead and do this.
- Or potentially, if you're being told you have to do this, how you can start having the conversations about how to make it successful.
- There's also a section on evaluating the AI agent.
- How do you know which one to go for?
- Do you buy one?
- Do you build one yourself?
- And then deploying it.
- But what we've been focused on all week in Paris has actually been the scale outside of things.
- Once you've gotten going, you've launched it already, you're starting to see some, like, okay to good results.
- What does it mean to actually start scaling and really transforming your business around the success that you can have with this AI agent?
- So within the scale it section of the blueprint, we break it down into three steps or pillars.
- So customer experience, organization, and system design, and economics.
- So before I dive into those three pillars, I just want to note, like, what's the difference between a scaled AI organization and one that's a bit more stalled?
- So scaled AI looks like deep deployment, organizational change around deploying AI and scaling it across the board.
- Your customer experience improves because you're using AI rather than, you know, maybe you just it on and start deflecting things doesn't necessarily mean that it's going very well.
- And it's also strategic in its nature, so you have a proper road map and plan for what you're doing and the kind of improvements that you want to make.
- It's not AI for the sake of AI.
- It's AI to actually do something helpful for you, your team, your business, your customers.
- But then stalled AI looks more like shallow deployment.
- So the kind of bolt on where you're just deflecting some conversations, you're not even sure if they're truly resolved or if your customers are happy.
- There's been no organizational change, so potentially nobody's really responsible for this thing.
- Like, we somebody said it live and now it's just there.
- Customer experience might actually get worse or disjointed, and the return on investment is unclear, and it's just not a strategic deployment at all.
- So let's talk a little bit about pillar one when you are scaling your AI operation in terms of customer experience.
- So the next part then of organizational design, and system design, and then economics.
- So I'm gonna dig into the customer experience side of things.
- And the point that we're trying to get across with events like this is that you have the opportunity in front of you now and the technology available and the blueprint there for you to actually reinvent the way your business shows up for your customers.
- So you can make this transition from potentially a very reactive space that the support world was in previously into a space where you're being more proactive with your customers, you're helping them become more successful.
- And overall, it's an AI powered customer experience design journey.
- So before you think about customer experience or before you've, like, made that one of your pillars for your AI scaling, what does this look like?
- So support is reactive, handoffs are jarring, so the AI to human handoff is potentially awful.
- The customers need to repeat themselves again.
- Maybe the bot sent them in a bot loop of doom, as I call it, or bot jail.
- Your brand tone is inconsistent, like the way the AI agent is speaking does not sound like you're humans.
- And your support is very much seen as a cost center and is a cost center to the business.
- Whereas after scaling AI and and really honing in on the customer experience side of things, AI is the first responder and often the last.
- It resolves the queries end to end for the majority of your interactions with your customers, and the handoff is actually seamless.
- So if AI can't answer, the way it sends the conversation or the interaction to humans is actually a very nice and not friction filled process for your customers.
- AI is speaking like your brand, so it feels like whether you're interacting with the AI agent or your humans on the team that everybody's on the same page about how you speak as a business.
- And support doesn't just resolve issues, it actually works on things like retention, activation, expansion, and that leads to actually driving value and money into the business.
- Then in terms of organizational and system design, this is another whole section within our blueprint.
- This is where most teams actually hit the wall.
- So they've launched the agent, they're getting some good results, and then, like I said earlier, potentially, no one is really responsible or, like, solely responsible for the success of the AI agent's performance.
- And something that doesn't get enough attention, and I'm really excited to talk to our panelists about today, is actually the change management side of working with the human team to align them with these organization and system changes.
- So the point we're making in the blueprint around organizational redesign is that you need to rewire your teams and your systems.
- And to realize the true value of AI, the way your team works really does need to change.
- What you've been doing all along isn't actually gonna cut it anymore in this new world.
- And it doesn't mean starting from scratch.
- You can work with the people that you have, and you can change their roles and responsibilities.
- You just need to rethink how things are actually done and how things are structured.
- So this is a quote from Nick Mehta, who was once the CEO of Gainsight, calling AI the biggest change management need in human history.
- I've certainly felt this in in living through this the last few years.
- It's actually working with the humans on the change management side of things that's been some of the most challenging, but then also rewarding side of things in this space.
- Then Nick Clark from BCG spoke about that just because the tech works so I think we all know it's very powerful, but just because it works doesn't mean that your organization is ready for it to allow it to actually be capable of doing some of the best things for both your team and your customers.
- So where to start with organizational changes?
- We go into this in a lot more detail in the Blueprint itself, but the idea is that you start thinking about roles that unlock performance for AI.
- So these are just some of the types of roles that there's more detail on in the Blueprint book, But I'm gonna show you what this looks like for my own team.
- So I work in the customer support organization at Intercom.
- That means that I help our customers get the best out of Intercom and Fin, but I'm also a customer myself.
- So you'll see that we've changed up our org structure.
- This was like mostly a classic looking org structure previously.
- But now we've split out into a whole part of the organization for human support specifically.
- We have support operations and optimization, which originally was classic support ops roles like workforce management, analytics, enablement, some QA.
- But all of the roles and functions within that organization now have become very much AI first in how they go about what they're doing.
- And over on the fireside where you see head of AI support, that's actually me.
- This is a new role, new function, and the roles that sit under my team, conversation designer, knowledge manager, and systems analyst, they're new positions as well that have only been around in our organization for the last couple of years.
- And we realized quite quickly after we started deploying Finn and I was still in the human support side of the organization trying to lead a frontline team that I needed to move away from the reactive day to day because that was always pulling me to the side and I was never able to spend proactive time working on making our AI strategy successful.
- So that's why we split out my role over to that side of the business and hired a team around us.
- So in terms of, you know, these new roles are starting to exist.
- What does that mean for the existing folks on the team?
- So they need to start thinking about the types of skills that they need to hone in on.
- So they need to ensure that their subject matter experts and their technical capability is really strong.
- I think by now everybody knows this kind of concept in the industry that AI agents will do, like, a lot of the easier work.
- Now it's doing it more and more complex work.
- So folks in the customer support support sphere need to ensure that they're getting more and more technical expertise, more and more subject matter expertise to ensure that they're able to stay ahead of the curve and that essentially their roles are not taken by AI being able to do a lot of that easier work.
- We need systems thinkers.
- We need consultative skills, so folks who are able to go beyond looking at the issue that's right in front of them, but actually look at how can I make this customer more successful across the board?
- AI literacy is a really big need at the moment in this world, in this space, and continuous improvement mindset.
- So ensuring that they're always trying to make things better and better for the customers and better and better in the AI space.
- As always, we still need humans in the customer support worlds to have empathy and customer obsession, but then for them to be really driven by data and insights in the improvements that we expect them to make.
- At Intercom, this is something we're continuously speaking about with our customer support organization that the first time they answer a question should be the last time.
- So how when you pick up an issue and answer it, do you ensure that next time around, the AI agent, so Fin, will be able to answer it?
- Or better again, that you're giving the feedback to your product team so they sort out the issue and it's never a problem again in the future.
- So these are the systems that we believe sustain great performance.
- So clear ownership, lightweight governance, so somebody making sure they're keeping an eye on everything while everyone else suggests the improvements.
- Systems that learn themselves, so using AI to improve AI, and then content as infrastructure.
- So in an AI agent world, content is absolutely king.
- If you put garbage into an AI agent, you're gonna get garbage out of an AI agent in terms of the answers that it gives to your customers.
- So you need to ensure that, like, no matter how fancy the tech is and all the abilities that are there, you need to actually focus on your content and your information that you're giving to your AI agent first above all else.
- So just quickly to speak about the economics before I move over to speaking to our lovely panelists here.
- Economics are the final pillar that we're gonna speak about from our blueprint.
- So rethinking the metrics that we use within the customer support world.
- So if you just have a look at the first graph here, as your business grew in the old world so let's say your business grew by twenty percent, you'd need twenty percent more people to try and keep up with the demand of inbound questions.
- Right?
- However, with AI, that curve starts to flatten.
- And with that flattening, it means that you don't have to have the same linear growth that's very expensive working alongside the growth of your business.
- So this means you actually need to think about different metrics.
- So a few years ago, we weren't thinking about things like AI agent resolution rate and automation rate, involvement rate, but they're all metrics that we need to actually have as part of our day to day now in the customer support world.
- And in terms of customer experience, CSAT has been the North Star for such a long time, but Intercom has launched a piece of functionality called customer experience score, which is an AI generated score of a hundred percent of the inbound conversations that you get from your customers, and it gives you a much more holistic view about what's really happening.
- Also, things like resolution status.
- Was the issue actually resolved both by the AI agent or by your humans?
- Customer sentiment, AI is fantastic at analyzing this at scale.
- Service quality and repeat contact rates.
- So really digging in in spaces that potentially, if you were a super busy support org before, you couldn't really get into it because you were just so busy dealing with your backlog.
- So this comes back to the fact that simply deflecting contacts, it's not good enough anymore.
- So previously, deflection was a big word or term in the customer support space.
- However, now we're really really focused on actual resolution.
- Because if you're deflecting a customer, it doesn't necessarily mean that you've helped them.
- They might have gone away because they couldn't get help from you.
- They might have given up.
- And ultimately, that can actually end up leading to dissatisfaction and churn.
- So you really need to make sure that whatever you're doing, either with your humans or your AI implementation, that you're actually trying to resolve customer queries, and then ideally going beyond that and trying to be more proactive with them and help them be successful longer term.
- So ultimately, AI performance plus customer experience, when they both come together, they're gonna drive real business outcomes.
- And what do real business outcomes unlock?
- They unlock cost efficiency, so support costs flatten, capacity is freed up for the team that you have to do more strategic work, and it's balancing a lot of that spend as well.
- So you don't need to keep again that linear growth in hiring hiring either more contractors or full time staff.
- You can do more with the people that you already have.
- It also unlocks revenue influence, so faster time to value for customers.
- If your customers are able to get an answer or their issue resolved faster, they can move faster in your product and ideally spend more money with you, and not again like end up going away or potentially moving to a competitor who was more willing and able to help them.
- And then the third unlock, compounding, reinvestment.
- So the savings that you get from not needing to grow your team massively, or the the space that you're freeing up for them to do other work, you can actually reinvest that in different parts of the business.
- So maybe, for example, expansion programs, retention programs.
- So essentially, scaling is the next frontier.
- Most folks, would imagine, in this room have either launched or are about to launch their AI agent, but scaling is really where you're gonna start unlocking true value for your business.
- And what you're building towards is that your AI agent resolves at least eighty percent of your queries.
- That's where we're at with Intercom today.
- Our team is at eighty three percent automation rate, so of everything that comes into us about our pretty technical product, our AI agent, Fin, is actually resolving that.
- That leaves space for human teams to focus on system rethinking and redesign, continuous improvement, and then the organization is changing, we have new roles that people who were once working directly with customers full time are moving into more strategic positions.
- And then customer experience is actually becoming something we're productizing.
- So if you imagine your product teams, you know, there's product teams building roadmaps, success metrics, strategic plans, customer support needs to become that now as well so that you can keep driving more value for your your company and your business.
- So for support leaders, this is a chance to pioneer new technology, new strategies.
- What we're finding with a lot of folks who are doing really well in this space is that they're actually the first within their companies to really pioneer with AI and drive some amazing results.
- And they have people from the rest of the business asking them to come and help them with their road maps for their own AI deployment.
- So it's a really like a career defining opportunity right now for those who are ready for it.
- And support, in this sense is no longer a cost center.
- It's actually an actual value add money into the business.
- So to help me chat a little bit more about that, our panelists are going to join in just a moment.
- And if you want to learn more about the blueprint, you like I said, you can either grab some of the books that are blue, that are on the tables right now or you can check it out on our intercom website.
- There's fin dot a I forward slash blueprint.
- And like I said, that's broken out into launching it and scaling it.
- And just know that obviously the books that you have now, they're static because they're printed copies.
- The online version is gonna continue to be updated.
- We're still figuring out new ways of scaling, new ways of driving more value for the business and anything that we learn ourselves as a team or that we learn from our customers, we package that into the blueprint to keep sharing it with you all.
- Okay.
- I'm gonna be joined by our great panelists here.
- So this thank you.
- Okay.
- So we have two of our wonderful customers here, Inika and Claudia.
- Inika, would you be able to introduce yourself, like what you do, what your company is, and then over to Claudia after.
- Sure.
- So I'm Inika Oates, and I'm the head of support at Agora Pulse.
- That's a social media management tool that helps agencies kind of manage all their social media in the one place.
- And like everybody else here, we're passionate about support and how we do it and was reluctant for AIs at first, but all of a sudden, it's kinda our cornerstone now.
- Nice.
- Yeah.
- I'm Claudia Byers.
- I'm senior operations strategy manager at Redo.
- At Redo, we are a technology subscription service.
- So rather than buying your technology outright, you would pay for it monthly and at the end upgrade and return driving a more circular economy.
- Nice.
- So, Inika, can you tell us a bit about how support operates in your business?
- And what made you start even looking into launching Finn?
- Yeah.
- Absolutely.
- So we were customer support just humans, simple as.
- And as you said earlier on, every time the the industry grew, we added more people to it.
- And that is hard because it's the training of them, number one, takes months and months and months to become product experts.
- So for them to be useful in the team, it took an awful long time.
- And we were trying to move more mid market and trying to look at those higher paying customers and give them that dedicated time that was so important.
- The deep troubleshooting, the kind of really meaty stuff that most support team members really love and not have the repetitive stuff.
- So our CEO came and just goes, get AI, sort it out, here's your plan.
- And yeah, we started to talk to Intercom about it.
- And because the way our support team was structured, it was kinda hard at at first because we have three different languages.
- We have our level one support, which is just everything the customer can ask, and then the technical side.
- And we weren't a big team.
- I think we were around thirty when we first started this.
- We've contracted, but that's through natural attrition anyway.
- But we wanted to make sure that the frequently asked questions were just bypassed.
- We didn't want to know, like, how do I add a social profile or how do I find my invoice?
- And we wanted Finn to be able to do that.
- So, yeah, we've been talking about it for eighteen months.
- We were petrified.
- And once we started, we were very maybe we were unrealistic because we just didn't know what was going to do, but we had said, let's aim for ten percent resolution rate.
- Let's see what happens within the first year.
- And we started in the January, so it was around the fourteenth of January, we had a new, contract and Finn comes in.
- And within the first week, we got forty percent resolution rate without very little work being done on our help center.
- Now lots of work done by our ops team.
- So we had workflows already, but we really needed to build out on them better.
- And, yeah, it kind of shocked us, to be honest.
- And it's grown, like, really has.
- And it's just made life an awful lot easier for the team because everything we do now is completely different.
- Yeah.
- I'm thinking back to when we launched Finn ourselves, and I know I work for Intercom, but at the same time, I was still, you know, a support leader like any other support leader where I was essentially being told we had to use this because we built it ourselves, and I was worried about it.
- Because I didn't really have a lot of experience with this.
- Mean, I had tried ChatGPT a couple of times, and I was worried about what their customer experience was gonna be like, so I totally understand the apprehension, and then I was very pleasantly surprised when we got going.
- How about at Reylo?
- What was the question?
- What made you get going?
- I I guess, whatever Intercom launched in, we were straight away wanting to try that.
- I think, as I mentioned, we're a subscription platform, so we do get a lot of contact from our customers over the life cycle.
- So they could be asking us about their monthly payments, upgrading insurance claims.
- There's lots of things going on.
- So we do have high volume of customers coming in, daily to speak to us.
- So I guess, obviously, like any business, we we wanted to be able to scale without having to scale the team in the same way.
- And AI, straight away, as soon as Finn was launched, that was something that we wanted to get on.
- I mean, I was nervous as well.
- I think everyone, yeah, like, experience is a huge focus for us.
- And I think, yeah, being aware of that and AI in the really early days, I think that was the year Chatuchipity kind of kicked off in the January, and it was a few months later that we launched Finn.
- Initially, it was a couple of weeks, and we realized we weren't actually ready for it.
- We didn't have the content ready, so we we dialed back a little bit.
- And then a couple months later, whenever we were, you know, feel more prepared, yeah, we went straight back into it.
- And over the last year, I guess that was three years ago, we maybe didn't invest in it as much in the beginning.
- But over the last year, we have really invested in that, and the results have have been pretty amazing.
- Yeah.
- I'm just thinking through, like, that piece around not having the content ready to go, and that's actually something that we find when we work with customers or, like, folks who are trying it, and they're like, it's not given the right answers.
- You're like, well, where are they pulling the information from?
- You know?
- And I I think it is a bit of an moment for for people that they need to kinda get their house in order a little bit before they they get going.
- Like you said, Edica, like, if you have some good content, it can start answering well right away, but if the content is really not in order at all, for sure some work needs to go into that first.
- So in terms of scaling at Inika, you know, you went live, you were surprised by the results.
- Like, what was the journey from what you went live with first, the types of queries, and then you started scaling it out to maybe more different types of customers or different types of issues?
- So we started awful small.
- Again, we were, you know, worried.
- So we went with our lowest paying clients in English first.
- We had more English team members that we're going to be able to handle if it bounced back at us.
- And not not that we disrespect our customers, but there was less less chance that they were going to be really annoyed if they got the wrong answer.
- And we could jump quickly on it.
- And they were used to getting a six hour, first response time.
- So all of a sudden, they were getting an instant response time.
- Even if it wasn't perfect, it was still better than what they'd been getting.
- So when we had the results and they came back so so good, like it really did shock us.
- And we'd initially said we're never putting it into our VIP customers ever ever.
- That's never gonna happen.
- Of course, they're the ones that love it the most, believe it or not.
- But it was so easy for us to move it then into the languages and the lower paying clients first and then into the next and the next and the next.
- And we found the languages went pretty easy.
- Interestingly enough, we thought the French would go against it most because they just want a human.
- But actually, they've adapted quicker than we thought, you know.
- So it it it was as smooth as you could possibly think it could be when you were so skeptical.
- And the team got on with it.
- K.
- Great.
- It's amazing when you're pleasantly surprised by something and obviously not not the the flip of it.
- But, you know, you were speaking, Claudia, about, like, the content wasn't ready.
- Like, what did you do then in that time where you were saying, right, we have to kinda take a bit of a step back and do some work here first.
- Like, what did you do?
- So a few things.
- I mean, the content was one thing.
- We also had a lot of Resolution Answers, if anyone remembers Resolution Bot.
- So I guess when Finn was introduced, some of our older bots were it was kind of colliding.
- So that was one thing, kind of rethinking everything else that we had built and then getting the content in order so that AI could read it and read it well and relay the answers back to the customer in a better way.
- We also realized we were missing content.
- And, yeah, I think it kind of opened our eyes to what the content should look like.
- We have completely rebuilt our help center for AI readability, but also realizing that actually if AI can read it better, the customers are probably reading better as well.
- So, yeah, both those things, there's kind of the what we had previously built behind the scenes for the bots that we had in place and then also the content that was in our help center.
- Yeah.
- We had a similar situation.
- We did have a lot of content out there, but it wasn't necessarily completely up to date.
- And we shipped so many products and features that, you know, content got stale pretty quickly.
- And our team were actually quite good at if they answered something, you know, thinking through like someone shouldn't have to find out the answer to this again.
- But they would often write the content and put it into an internal knowledge base that customers couldn't see.
- But often, it was stuff that customers should know about, and when we launched Finn, we had to do a big job on the internal content side of things and pull out all this content that could have been fined to be out in our public health center so Finn could have it, and then do a lot of work around, like, not having duplicate content in two places, making sure the internal knowledge was only for, like, internal processes and things like that.
- So definitely, it was, like, quite a bit of work, but it was worth it when the results really started to to speak for themselves.
- So were there any internal processes that you needed to adopt, Inika, in terms of, like, continuous improvement or, like, getting the scale really moving?
- Yeah.
- There was.
- And and like, like Claudia said, we did have to revamp our help center, but what was there was quite okay.
- Wasn't great, but it was okay.
- But we wanted to make sure that what was missing, if there was stuff missing, and and we did did have internal stuff.
- It's that tribal knowledge that you keep and you shouldn't.
- But we got the team to start, submitting optimization tickets.
- So every time they saw something that was it was nearly right, but it wasn't exactly right or was missing a piece of information that would have made a huge difference, it was submitted.
- And then that went to the ops team, and they made a decision whether it's into the help center or whether it's a snippet or a guidance.
- It took a while, and it still takes a while, like, we're a year in and we still have to remind them, like, I now have a Geekbop question every week that how many did you submit?
- So they have to tell me an exact number so that it's top of mind, you know.
- So it does take a while for them to kinda get in with us and understand us, but they're very passionate and they are they they're very well meaning that we tag pretty much everything one way or another between the workflows or if we have to do with a human when they're submitting.
- And so those type of processes changed.
- And one of the things that was important to get buy in was from the product team as well because they're designing these new features and we want it to look right and we want them to think about letting us know first so that we can be prepared and have the help center details and if there's going to be screenshots changed and so on.
- So now we have a great system where they submit to us that we want this change happen on this help center article.
- The deadline is this.
- The release date is that.
- So that has really, really made it smooth.
- And even having another brand that we we acquired last year that we're now working on with Finn coming in the next weeks, their team is now already doing that.
- So there's no delay anymore because it's a constant habit now.
- Yeah.
- And it sounds like you've built in that knowledge with the product team that they understand the consequence of if they were to ship something and not tell you about it.
- It's happened.
- Or yeah.
- It can still happen.
- Yeah.
- It does.
- It's it's not meant to happen.
- Yeah.
- And they don't mean to because they're so excited about their their their feature that they forget, oh, that's actually a big change.
- And the customer needs to know that.
- Customer support needs to know them, but customers need to see that.
- And they and the minute they see it, they go, oh, that's how I can find that or do this.
- Yeah.
- We've built out a new product introduction process.
- Kinda sounds somewhat similar where product managers in advance of the release need to fill out.
- Not a huge document.
- They can actually just throw something together because now we can use AI functionality to expand on it and build out things like macros or Articles, so that's fantastic.
- But there's still that side of things in the customer support world where potentially someone if my product team were here, I'm sorry, but sometimes product might ship something seemingly small.
- Yeah.
- So it's like, it was just a UI change.
- It was just a tiny UI change.
- But the consequence of that, so often, like, customers can actually have quite a reaction to, like, just a UI change, and there could be, like, two hundred articles that need their screenshots updated and suddenly or, like, the steps changed, and Finn is now giving the wrong information or the team don't know how to troubleshoot something.
- So it's definitely been, like, some work to make sure that other teams understand the consequences.
- And the good thing, I guess, like, we're working for Intercom is that our product team cares so much about our Fin success that they're really bought into making sure that Fin has the information that it that it needs.
- But, Claudia, how about continuous improvement from your side?
- Yeah.
- I would say kind of relate to both of both of you and what you said.
- I think something kind of off the back of what you said around how, I guess, that connection between product and product releasing something and make sure that Finn knows, I think it's also helped us connect with our customer support team when something changes.
- I think, as you said, I don't know, maybe there's more of a view that it's more exciting to get Finn to be able to tell the customer when something has changed, but I think that actually has helped the connection when a new product release comes out and helping our customer support team be up to date as well.
- I think, yeah, similar.
- We we we don't necessarily have a formal ticket process for, you know, optimizations, but all of our support team do it all the time.
- They send Slack messages to, you know, myself or my team member who works on Finn all day every day for, yeah, things that they've noticed that Finn could maybe say slightly better or maybe something that we've missed.
- So, it's not formal at the moment, but it is something I mean, speaking with you and hearing what you guys do, it's truly interesting, and it's something we'll probably look to develop more.
- Yeah.
- We have something similar with the back office ticket process.
- You know, if the team spot that something is off or Fin should have had some other information, they'll open a back office ticket off the back of that conversation and it goes into a queue for the folks on our support, our AI support team to review.
- Because, you know, initially that process of, like, the Slack, whoever, was fine, but, like, the more the team become passionate about continuous improvement, the bigger the queue of continuous improvement things becomes in all these different spaces and channels.
- So starting to formalize the process is helpful.
- And we've actually brought in a KPI for our team now as well around continuous improvement.
- We have like actual number expectations, and we've just gotten going with, like, putting a number on it, and it's an interesting journey because we wanna make sure that they're high quality and that people aren't, like, just suggesting something for the sake of it.
- So we're going back and forth a little bit about, like, the number versus, like, just really trying to get that intrinsic motivation around continuous improvement.
- And part of how we've tackled that is, like, we've reduced their productivity KPIs because they're dealing with far more complex queries now.
- Their inbound workload is not peppered with all of those quick wins that Finn is absolutely dealing with.
- And because we have all these expectations on them about continuous improvement, we have to be realistic about tougher conversations and also more expectations around work off the queues and not working with customers directly.
- So we've been trying to find a good balance there, but I'll have to report back to you all about how the the kind of metric side of things goes with continuous improvement because quality over quantity is important as well.
- Okay.
- So let's talk a little bit about measuring impact.
- So, Claudia, what business impact have you seen since you launched and then scaled Finn?
- Yeah.
- I I guess a key metric for our business is our operational efficiency ratio.
- So we measure that by the number of subscriptions that we can manage by our headcount.
- And I guess underneath that, we have our automation rate which feeds into it.
- So over the last two years, we have increased our operational efficiency ratio times two.
- We made it two times better, I guess, largely driven by the automation rate from Fin.
- So in the last year, we've increased our AI automation rate from seventeen percent to forty percent.
- But the other side of that, which I think is really interesting is actually how we've improved the customer experience when they contact our support team.
- So CSAP, FIN CSAP has really gone through the roof in the last six months, I guess, since we've really put a focus on FIN and introducing personalized information into those contacts, allowing customers to receive support twenty four seven.
- They can come speak to us anytime that suits them, get personalized answers.
- And I think that's really driven that positive customer experience as well, which obviously ultimately supports the business overall.
- Yeah.
- And are you finding that your customers are getting more used to dealing with an AI agent?
- Because, you know, before, it was like chatbot doom, you know, old school chatbot, and I think most people kinda hated interacting with the old versions of chatbots.
- And I don't think anyone loves picking up the phone and waiting on hold for ages either.
- So it's like, it's a different world now, but I I know for for myself initially, if I told people, like, I worked with an AI agent, they were like, oh, chatbots.
- And I was like, no, it's good now.
- It's different.
- It's very different.
- But I think, like, the world is getting a bit more used to that now, and the expectations are are there for you to have a better implementation.
- But are you finding that customers are just happier to interact that way now as well?
- Yeah.
- Definitely.
- I think having the conversation with AI is completely different to, you know, the older versions of workflows and those chatbot decision trees and, yeah, you can see customers getting more frustrated.
- But as I say, our CSAT or CX score for for Finn has significantly improved and obviously what we're doing, we're we're optimizing it, trying to make it better but I do also see that customers seem much more open to it and having those conversations with AI especially when it is it's quite human like and it's instant, so yeah.
- Mean, why not?
- Yeah.
- It's that immediate side of things, isn't it?
- Especially if it like if it just resolves the issue immediately, you know, before again, like chatbot loop of doom or wait on hold for like, sometimes like an hour or more, it depends on what business provider you were working with, but, you know, this is now giving you the same answer as you would have had to wait ages for previously, so I'm kinda like, what's not to love about it?
- But then as support leaders, we have a responsibility to actually make it good So that people do, like, trust us and want to interact with it more.
- So, Yunica, how about your team then in terms of, like, business impact and how you measure that?
- Well, CSAT is, of course, the the metric that we look at for everything.
- I mean, we we know it's important.
- It it can be weird because sometimes they're gonna give you a negative CSAT because we don't have a feature they want or the API doesn't provide us.
- Because we look after, like, we say a lot of stuff with Meta.
- So if they don't do it, we can't have it.
- And it might be native, but it's not in their API, so we can't do it.
- So we get a lot of that.
- So we're trying to balance that.
- And we didn't know how people would raise a a bot.
- And a lot of them, you know, they're they're they're happy enough to get the answer and they move on.
- They don't always give us an CSAT, so we're we're not really sure if they're happy.
- So we're looking at CX because I think if we can get everything rather than kind of eight to ten percent, it's going to be much better to look at.
- But the the feedback we're getting, and and not everybody leaves a comment when they're leaving a CSAT, but when we got last week, and I kept it and I screenshot it and I put it in every Slack channel I could put on because it was so important to us because it was a very high paying client.
- They got their instant response, so less than their thirty minutes, so instantly, the tickets got closed the conversation got closed within three minutes.
- And the guy turned around and he goes, this is the first chatbot that gave me a correct answer, we're doomed.
- But it was the best we're doomed I could have had because when I can show that to our our VPs, they're kind of looking at, oh, like even for our highest paying clients, this is they're happy.
- They're three minutes done.
- You can't beat that.
- So while we don't look at all the metrics all of the time, they are the type of things that I can kind of shine a light on, and that makes things different.
- Yeah.
- And I just think, know, obviously, again, I work in this space, so I have fairly high expectations of what can be done with this, and I see great customers like yourselves doing it really well.
- So when I do end up, say, a website or calling, you know, whatever it is, like a service provider that's just still in the old world, I'm like, there is no excuse anymore.
- Yeah.
- This technology is at your fingertips, you know.
- But let's start talking about the change management side of things.
- You know, I had a slide up there a while ago about this being the biggest change management or people change in in history in terms of how we need to work.
- How did your teams react?
- Like, Claudia, how did the the wider support organization react when you were launching this?
- Was there apprehension or There wasn't as much as I would have thought there would have been.
- I think what you find I mean, I think you talked about it earlier, but I guess support people sometimes it can be a long day for answering all those simple things over and over and over, and maybe not, like, using your problem solving skills a bit more and, you know, getting into the detail and trying to solve customer problems.
- So actually, we find more so that our support team were actually quite excited by Finn being able to take away those simple things and they didn't have to deal with them.
- And we find they're very engaged, as I said, constantly optimizing or suggesting optimizations.
- And, yeah, I I don't think it was maybe as negative as it it might be in some other places.
- Okay.
- How about Inigo?
- We've been kinda harping on about it for about twelve to eighteen months.
- So it was constant chat about it, like AI this, AI that, talking about how bad AI could be because we'd had several, like, oh, my god.
- I was on this AI chatbot.
- Look at the the responses I got.
- But once we started to really, really dig into that, yeah, we're gonna pay for this, so it's gonna be coming.
- And the yeah.
- There was trepidation.
- There's no no two ways about it.
- Not always said out loud, but you could tell and they'd going, what can I do different?
- How can I look at this?
- You know?
- So that was great because it was you know, they cared enough about their jobs.
- But when we got it in first, obviously, there was a core group going to looking after us, particularly the ops, and we had some people doing small little jobs, but we made everybody do all the fin training.
- So we put it out there, they had a timeline to do it, they had to give feedback on it, they were talking to their manager, so it became routine even if they weren't doing anything themselves.
- So it wasn't in their inbox, it wasn't happening to them just yet.
- So when they did get us, it was kinda going, oh, okay.
- Yeah.
- We we can kinda see what it's doing and and how we can help.
- So we have changed job descriptions, so we have our knowledge manager now.
- And his work is is changing everything when we need to have it changed all of the time.
- We have who's in the APAC time zone, and she's doing all the workflows.
- And and, like, that's a complete change for her.
- She was a support agent.
- Now she's rarely touching a ticket.
- So it is different, but they've bought into us.
- And I think because we were positive about it all of the time and and saying, like, it's not taking your job.
- We actually have more work for you than you're you're probably going to be able to cope with because of all the things.
- And I think visiting you guys and seeing what you were doing and how you were doing, it made a difference for me.
- It made me feel more comfortable that you had all these different things that everybody was doing because now they didn't have the volume of tickets, because Finn was taking those, but here's all the things to make Finn work.
- So you have to have that human behind Finn, and that I think settles them a little bit.
- Yeah.
- We had a bit of a mix.
- So there's always some folks who are like really excited by new technology, and we worked with those kind of as our champions.
- You know, we got them to start training the rest of the team on how to work with AI, even just on the the back end.
- You know, this is going back now at the kind of start of ChatGPT, so, know, like, summarize or make my message sound more professional, you know, and really getting going with that stuff to show how it can make your life better.
- And we were so underwater as a support team before we had Finn.
- We had terrible first response times.
- We like, the ones that we had set at our as our targets were actually not that ambitious and we really weren't even meeting those.
- And the team were dealing with, like, pretty frustrated customers regularly because customers were waiting ages for us to get back to them.
- So when we launched Finn and we were pleasantly surprised at the beginning about how much it was resolving quickly, like suddenly the ratio of super annoyed, been waiting ages, customers reduced, and obviously that made our teams lives Better overall.
- And then after that point, you know, when we started to scale it more and figure out responsibilities and who should be doing different parts of the role, like, their their roles are changing now.
- You know, we were speaking about this earlier today, that we've had to up level the roles because the tier one role doesn't exist on our team anymore.
- We were able to up level those folks.
- Finn does all the tier one work, but that also meant that we had to change the job descriptions.
- We're hiring more experienced people.
- We actually have to pay them more, which is always good.
- Right?
- That's better for everybody.
- We do have higher expectations of them as well.
- So it's been quite the journey and, you know, kind of organizational change when it comes to the the frontline folks as well.
- So what's your top piece of advice then in terms of buy in for like, if you're rolling out an AI agent and the team is skeptical?
- It sounds like yours weren't too bad, Claudia.
- So, Inika, like, yours were mixed.
- What's your advice then in terms of getting them on board?
- I think you just have to persevere and keep pushing through it.
- You know, it's been the positive because it is a positive interaction.
- And and I think when they realize that they didn't have to answer the same old same old, the same question over and over again because that is tedious, you know, and if you're like, we wouldn't have a huge number of of conversations compared to some companies.
- But if you're answering fifteen, twenty conversations a day and two of them are very technical and you're having to your headspace there and then there's somebody over here asking for something so basic that actually if they looked in the help center, they'd have got it themselves.
- They they realize that they don't want to be doing that, you know.
- So it it I think just keep the positive side of it.
- Explain to them that it's not taking your job.
- AI is here to stay, so you need to get on that wagon and jump on with it and stay at it because it is good and the team now get it.
- Yeah.
- And, Claudia, even if there wasn't too skepticism, what works really well that you would try and package to say to other support leaders about getting going and having the team well bought in?
- I mean, I think I related a lot to what you said around as well, like, you know, feeling like the team's underwater.
- I would say we went through a lot of that over the last few years as well.
- It it's it's starting to feel like that's not the case, and I think the team feel that.
- And then, I guess, they're more motivated rather than feeling like, I guess, yeah, they're underwater every day.
- And I think getting the team involved is really important.
- Like, I think much like the rest of us, everyone is very excited by AI and excited about the technology and want to be involved.
- And I think getting, I guess, the people who know how to deal with customers involved in getting the AI To deal with the customers.
- I think that's been really important, and I would definitely advise that everyone, you know, at least does a bit of that.
- Yeah.
- We have folks in full time roles for, you know, AI implementation.
- We call it, like, conversation design, also knowledge management.
- We have full time positions for that, but we do still have folks on the frontline team who are working with customers every day contributing to that work, so they get some hours of their week every week to do some of that type of work because the I think you probably all know this.
- Anyone who's worked directly with customers before, as soon as you stop dealing with them with their questions every day, your product knowledge starts to fade away a bit.
- And even how customers ask questions or understand things, like, there's nobody better than the people who talk with them day in, day out who can actually help you make sure that things like your content or your journey design are in order.
- So again, even though I have some full time folks within our organization, we have to have the team continue to contribute because we actually wouldn't know half of the issues that are arising or the way we need to put things if we didn't have those those folks on the team.
- So looking ahead, let's talk about the future.
- We've been talking about all the implementation stuff of the past.
- But into the future, Yunica, how do you see your AI strategy evolving, of course, across this So we have ambitious plans.
- And like one of the first things is implement and fit into a second brand.
- And it's it's a very complex tool, so that's going to need a lot of extra help.
- The team are only learning that tool at the moment, so that's going to, I'd say, ramp up fast because they're they're used to it now.
- But procedures is something that we're really interested in because we want to kind of really look at how we can answer the questions that he's not answering right now.
- So, like, we have to connect to our our payment system so that he can answer the basic, at least the basic to start with because there's obviously some fear around, oh god, that's my payment system.
- Can can they get everything out of it, or can they see my credit card or anything like that?
- So that's really, really important to us.
- And I think monitor.
- I'm like a child on Christmas day.
- It's not you wouldn't even believe it.
- So excited for that because I think it's really hard to to to QA anything anyway anyway.
- And if we can automate a lot of that so that it's getting better all of the time and we're spotting what's not working and fix it without having to trawl through every single Finn conversation because we were doing that at the beginning.
- Everything he said, we checked.
- And then we had to scale that back because that's not something we can do, and we're now doing five percent.
- And I think using monitor will be the best way for that.
- And then, of course, when it comes to the humans, I won't have to spend hours doing all of that myself because I like, I'm I review the managers reviews of their team, and that can be quite tedious.
- That's how I keep my product knowledge up, by the way.
- So those things are really exciting, and that's kind of where we're going to.
- And obviously, there's trends there.
- There's like all the new little pieces that we haven't yet touched is all part of it.
- But that's Hillary's job, and I just kinda go, tell me all about it.
- Nice.
- And for anyone who's not familiar with monitors, it's a new feature that we launched on Tuesday.
- It's QA and observability within the Intercom platform, so you can do your fin QA, and over time, you'll be able to do your human QA all in that one space.
- I love how excited you are about It's fantastic.
- Unreal.
- Yeah.
- My team have been using it for probably the last month or so, and they're getting, like, really, like, hands on and finding all these new ways of getting the kind of observability or kind of immediate information that they need about our queues.
- So it's very cool along with the whole QA process as well.
- But for the rest of this year, so, Claudia, what's the plan?
- I guess more procedures is one part of it.
- Yeah.
- I mentioned earlier, we're around forty percent automation rate.
- We obviously want we we're aiming for what you guys are doing.
- So was it eighty two percent?
- Eighty three.
- Eighty three.
- Yeah.
- But, yeah, I guess, to help us get there, launching more procedures.
- We've been doing that over the last couple of months.
- We want to get Finn taking more actions.
- We've been mostly using personalized data in our procedures.
- We have one action procedure, and, yeah, that has been amazing to see the results, and we we want to see more of that.
- We've also just launched FinVoice.
- So, again, there'll be more of that as well.
- That'll definitely be a learning.
- It'll be interesting to see how our customers interact with AI over voice as well.
- We've obviously been doing it now with digital digital for quite a while, but opening AI up to that channel really, again, supports our operational efficiency that I discussed that is kind of our, you know, our top metric that we're we're targeting towards in operations.
- So, yeah, lots lots more of all of that and monitor.
- So I'm so excited about that too.
- Nice.
- For us, we are at eighty three percent automation rate.
- In case anybody hasn't heard, I brag about it all the time.
- Anyone who's here all day probably heard this maybe ten times today.
- But it that took a lot of time and effort, you know, and I'm really happy for our team now to have the opportunity to work with customers and try and help them get to the level that we're at.
- So that's something on the cards for us across the course of the next year is like how can we do things like run more workshops like we had today?
- How can we expand this blueprint so that that book?
- Like, what else can we add to it as we learn more from customers like yourselves to help more and more folks be really successful with Finn?
- But I my team have also been voluntold that eighty three percent is not enough and that we need to get to at least ninety five percent over the course of the next few quarters, and we're getting there.
- Like, we're every month, we're moving up about one percent.
- But when you're when you have such a high automation rate, every one percent actually gets more and more difficult.
- So there's a lot that goes into that around the procedures that we need to build, the amount of work that we need to put into resolving smaller volumes, but when you add all those together, it starts to tick up the percentages for automation rate.
- It's definitely becoming more challenging to move the needle the higher we go with our automation rate.
- But again, as we continue to learn how to do that, the plan is to try and share that as much as possible with our customers or anyone who attends these these events so they can keep getting that kind of success.
- And then the other piece for us over the course of the coming quarters, rolling out Fin for other jobs.
- So we started using Fin Sales Agent over the last while.
- That's coming soon for all folks who are using Fin.
- Fin is doing the job of a salesperson now.
- So the way we're using it is that if a prospect or a lead comes to our website, we're trying to qualify them, you know, like what kind of business are they, what pricing plan might be suitable for them, and then Finn is making a determination whether we pass them kind of to maybe like a lower cost pricing solution, or if they're gonna be like maybe an enterprise price business that we're getting them a call scheduled with an account executive.
- But Finn is doing all of that now.
- Wow.
- So we have Finn working with the, you know, our service team and having huge impact there, but now we're starting to stretch it into the sales space as well.
- And it's really exciting to see all that coming down the line, and of course, then start trying to help teach our customers how to use that too and build all that into the blueprint.
- So that's all all coming for our team over the course of of the next while.
- So we have a couple of minutes left before we need to wrap up.
- Wanna open the room to any questions.
- There's a microphone walking around the place if anybody has a question for our team.
- Thank you so much for the insights.
- It's really cool to hear from other people who are a bit further along the journey than we are.
- Actually, if we look at support ten years ago, support was quite you know, doing its own thing away from the business, just taking care of those issues.
- Now, the last ten years, it has become a very powerful insight machine for companies, for SaaS businesses, especially that's where we are at, where you can really use the knowledge from the support team, the knowledge from everyday conversations to really make your product way better.
- Now we're moving into a new era where it's agentic.
- Right?
- And Paul mentioned that on Tuesday that it's all about agent orchestration now.
- And you already mentioned that, you know, you wanna aim for ninety five percent automation rate, so the human support team is gonna do less support.
- There are more roles, knowledge management roles, guidance roles, new roles opening up, and at the end, we will have maybe no one actually touching the tickets anymore.
- So I would love to hear your opinions on how you think having those insights suddenly not with the humans anymore, the people who are bringing in their own judgment, their own feeling, but having the insights rather only aggregated from LLMs, how that potentially changes positively and impacts or negatively the whole product innovation process because of that change from human to agentic.
- I think there's still gonna be a need for some support people speaking with customers for quite some time yet, even if we get to our ninety five percent.
- Because there's always gonna be something we didn't know about or a customer who is having maybe a super emotional issue and they really need to chat with a human.
- And, like, there's a big piece for us around relationship building as well.
- So say there's, like, customers that we want to build an ongoing relationship to help them be more successful.
- So that that's the kind of stuff I foresee humans taking care of, especially that, like, edge case stuff that eventually we do want to automate over time, but something new just kinda generally always comes up.
- But your point about ten years ago, support being so different, I used to work for a BPO and we worked, you know, in a different country, offshore, off-site, kind of out of mind.
- And when customers would give us feedback about the product, we would just tag it, but it definitely went into, like, a black black hole.
- Nobody I know nobody ever looked at it.
- Right?
- So now the fact that support, you know, support questions are being seen as this mine for being able to actually improve your product overall.
- Like, I'm so glad that we're moving further into that space now.
- Like, the product teams really see the goals that is in customer interactions and use them to make improvements to the product overall.
- But, yeah, I mean, over time, like, these roles, these human roles are gonna completely completely change.
- I would say the majority of roles within the support organization are gonna be positions like implementing AI, optimizing AI, you know, setting it up for maybe new products and things like that.
- But I still think there's gonna be, like, a fraction of kind of frontline type work that's gonna be gonna be happening.
- Any other thoughts from you two?
- I think you're right.
- I think you can't get rid of the human interaction because depends on your business as well.
- Like, some of them, it's just easy answers, you can send them out, and that's fine.
- But certainly, in our world, some of the stuff is so specific to that customer and how they've set up their Facebook page and their their roles within their Facebook page and and why they can't suddenly do something that that that the AI can't get into that yet.
- Maybe in time, it will.
- Well, even we can't.
- We have to ask them.
- So you need a human to then troubleshoot that or if they're looking at something the wrong way and they're they're not understanding, you just have to get on a call with them or, you know, we try not to do calls when we when we don't have to, but we will always jump on a call.
- And then it's a great selling point as well to have a human there that, you know, you've you've got these customers that come in.
- They want to pay it the world and the sun, moon, and stars, but their only thing is I want a human at the end of it to do support.
- But if they're gonna pay us enough, you're getting a human.
- It's as simple as that.
- So that's why it Fin will take a lot and is taking a lot for us.
- It's we have a sixty percent resolution rate and we're just a year at it.
- So like what's gonna happen in twenty twenty six, it has to improve and we have metrics that we have to get to, but there's there's always going to be a small percentage of our humans that'll be there.
- So with attrition, naturally, we're looking at, well, will we rehire?
- And in most cases, we won't unless it's very specific to a language or or a region that we have no choice.
- Yeah.
- For the roles on our AI support team, the conversation designers, the knowledge managers, all the ones that have been hired onto that team in the last two years have come from the frontline organization, and we've transferred them over and converted their role rather than me getting new headcount on the team that report to me.
- That's been a way of, like, managing, you know, not downsizing the team necessarily, but also not asking for new headcount for these new resources.
- It's like transferring, and, like, hopefully, most businesses will be able to do that with their their organizations, and that the humans just move into these kinda different types of positions, the higher the automation rates go.
- Now, I mean, obviously, right, like, we all know this.
- You know, if if there's a massive support team and suddenly, you know, the business only cares about cost, like, probably there are businesses, and I mean, I see it in the news.
- There are some businesses who use this technology to massively cut costs and reduce headcount, but my hope is that anyone who shows up to events like this today are kind of more in the camp of like, how can I with what I have, how can I do more and kind of transfer roles, change responsibilities rather than just only focus on that kind of cost cutting side of things because there's so much more that you can do with what you have when you're empowered by AI?
- Is there anything else you wanna add, Claudia?
- Yeah.
- I mean, I think I I do agree.
- I'd be surprised if humans are not interacting with the customers in this, I guess, in maybe in, the next year or two.
- I think from our point of view, we deal mostly with consumers, and there are certain circumstances where that human touch is required at this stage.
- I'm quite intrigued to see how it goes over the next few years, but, you know, we might have customers who are vulnerable or they're experiencing financial difficulties.
- And at this point in time, it's not necessarily something that we want AI to handle.
- But I'm not completely against it.
- But at the same time, there's obviously certain things you need to take into consideration.
- Yeah.
- And how how it does that, I think.
- So, yeah.
- It might go like that, but I don't think I can see it just yet either.
- But I think the exciting thing is allowing like our our support teams to kind of expand into those other skills and explore something that maybe they didn't even think they would have ever done.
- I mean, I started in support.
- I I didn't see me working on AI agents at all.
- But, yeah, it's obviously a very exciting time to kind of take that shift into what sport means.
- Yeah.
- I even think about, like, what do people want from businesses also.
- So there's some businesses that you're happy to interact with, like, completely AI, that's all I need, I want my answer fast, I don't care about building a relationship with this business.
- But then there's, like, that need in some situations as humans that we have for, like, relationship building.
- So in the SaaS world, like, that's, that you have a customer success manager who knows your business inside out, and I do think support is gonna become a lot more success over time.
- We've seen that with our team at the moment.
- But I just think about times where, as a consumer myself, I choose not technology.
- So I go to my local coffee shop because I love that the barista makes the the coffee in front of me, they know me, they give my dog a bowl of water in the cafe.
- I mean, I could choose to go in and press a button in like, the supermarket and just get something faster and and cheaper, but I'm actually choosing to pay more for the experience that I'm gonna have.
- And I think that that kind of mentality that customers have depending on the business that they're working with, Like, that means that I don't think humans are gonna be taken over by all the different technologies over time.
- So that's all we have time for, folks.
- Thank you so much for the question.
- I'd say we could talk about this absolutely all night for actually, for for days on end, but we do need to wrap up.
- Thank you to everybody who's joined us.
- There is, there are drinks and food down the back if you wanna hang around for a little bit afterwards and network and connect or ask any more questions that you might have.
- But, yeah, thank you very much, and thanks to our two panelists.
- Thank you.
- Thank you.
Transcript: The case for 100% AI involvement - Numan x Glean x Fin
- Hello, everybody.
- Big clap again for Paul and that product announcement and Liz in the live demo.
- Some really cool and exciting features that were announced today, and I'm gonna talk to our lovely customers here about how they're using them at the moment.
- My name is Ruth O'Brien.
- I lead our AI support team at Intercom, so the folks who implement Fin for our own customer support organization.
- So while we work for Intercom, supporting our customers in their own use of Fin, we're actually a customer ourselves as well.
- And I'm joined today to speak to some other lovely customers, Rideon and Kat.
- Rideon, could you introduce yourself and your role and company?
- Yes.
- Evening everybody.
- My name is Ridian Boubia and I'm a Head of Customer Operations at Newman, which is a D2C digital healthcare company.
- We're based in the UK.
- Hi, Kat Crichton.
- I manage a technical support team at Glean, and Glean is an AI company, a work AI company, with deep roots in enterprise search.
- So you both made a deliberate decision to go all in on Fin and adopt an AI first strategy in quite different industries, right?
- So, Radian, would you be able to tell us a little bit about how that went for you?
- Yes.
- So as I said, we operate within the healthcare industry, and I would say it wasn't necessarily like a bet on AI first support.
- It's more around patient first support, and what we found was that AI was something that could support that.
- It could free up agents.
- And if you look at our lots of our use cases, so a lot of the queries that come through to customer care, they're more on the administrative side.
- But occasionally, although there is the option for patients to speak to clinicians, occasionally, do get some clinical queries that come through.
- And what AI is able to do there is able to triage those queries straight to the clinical team.
- I think naturally in healthcare, is quite a lot of skepticism, or there was at the time, towards AI because when we launched it, we're talking about a couple of years ago now, but I think landscape has shifted.
- I think for us, a lot of it was around just satisfying the regulatory requirements and demonstrating that we could operate AI safely within the healthcare industry.
- Nice.
- And Kas, obviously you're at an AI company, so the buy in immediately probably looked a bit different.
- Absolutely.
- That other end of the spectrum that Paul mentioned, we wanted an AI first solution, and we were we are still in hyper growth, so we needed something that could scale with us.
- And, having already been Intercom customers, then it was a very exciting looking, feature.
- Nice.
- And I know that you went from forty one percent involvement rate to a hundred percent pretty quickly.
- Can you tell us a bit about that?
- Yeah.
- So what I ended up doing is Glean is really good at these internal queries, so my support team is very AI enabled internally, but we didn't have anything external for our customers, so I was able to put Fin on kind of our external facing surfaces, our help center, our community site, and also a little bit in product for our admins, and, was able to get really quick involvement rate from on those surfaces.
- Yeah.
- And in terms of resolution rate, you didn't see a dip when you put up involvement rate.
- Right?
- That's right.
- It stayed steadily around eighty plus percent.
- We were similar when we rolled out Fin.
- We went pretty all in on involvement rate early.
- We say we wanna try to have Fin answer as many queries as possible upfront to give it the best chance, and then the plan, obviously, is to hand it off to a human if it can't answer as seamlessly as possible.
- Yeah, exactly.
- And we are admin gated support in our support portal, so we actually didn't have a solution for end users as well.
- So this is a nice solution to have in front of our end users.
- Nice.
- Radean, you spoke a bit about the slightly more restrictive nature in the healthcare sector.
- How did you build confidence?
- I think we just had to operate with the principle.
- As I sort of mentioned previously, it was around safety first, so demonstrating that we could operate Fin safely within the healthcare industry.
- And I think what we found was sort of satisfy what we agreed initially was that we would review one hundred percent of FIN interactions manually.
- And over time, we just developed more confidence.
- We could demonstrate that it was operating safely.
- And now actually, we've reached a point where we still manually review around five percent of interactions every month.
- And then, yes, we have sort of regular clinical governance meetings, so we can sort of demonstrate within those five percent of interactions that we are operating safely.
- So again, yes, it's just a lot of demonstration, but again, we've proved that we can do it.
- What was the turning point so where you moved from that one hundred percent to going to five percent?
- I think it was sort of a mixture of both.
- There was quality.
- So actually, we do look at things like FIN CSAN.
- Actually, we're seeing that the quality was very high, so it was demonstrating that patients were valuing the output that FIN was pulling out.
- And then I think it's just in terms of those edge cases where sometimes FIN might have been providing some sort of clinical guidance and just showing that we were able to sort of capture it very early, review those.
- And I think, yeah, it took a couple of months, but I think after that, the sort of yeah, the landscape shifted.
- Okay, nice.
- So you both got a sneak peek at monitors.
- I know, Kat, you have some interesting ideas about how you want to use it.
- Yeah.
- So I'm thinking a lot about, all the industries we support.
- We support health care industry.
- We also have public sector and financial services, so really making sure that we're prescriptive about, the answers that the AI is giving and making sure we're checking on if those answers are correct or not.
- And then, like Paul mentioned, we're an AI company, so our product life cycle is very fast, very on hyperscale, and so monitoring product launches and that product feedback, I think my product team would really like to see that in some of those monitors.
- Yeah.
- We've been using it ourselves.
- We get to test everything early.
- It's great.
- And we're using it for this kind of new product introduction, and somebody in the audience was asking about an example of using monitors.
- We're using monitors to track conversations about monitors.
- That might sound a little bit meta, but now that we're rolling monitors out for all of you to use, we need to be able to see what kind of questions we're getting in about it, obviously, like how Fin is doing in terms of resolving those queries, and then what feedback do we need to gather for our product team as well.
- Any sort of new release, same as yourself, we release a lot of stuff, as you can see, a lot of the time, we wanna be able to have really quick observability of what's going on in our queues.
- Yeah, makes perfect sense.
- My job was always trying to make that feedback loop smaller and smaller, and I think this is really gonna help with that.
- Yeah.
- We're also using monitors for new procedures that we roll out.
- If we set a procedure live, we wanted to obviously take a certain action and we want to get a good idea of how it's performing, like what's the CX score like, or whatever the procedure is for, is it actually doing what it's meant to be doing, right?
- And Rhydian, you've actually played around with it a bit?
- Yeah.
- And I think just to follow-up from my point earlier, so as I said, we're manually reviewing around five percent of interactions.
- And the beauty of what I've seen from Monitor so far is that's going to enable us to review one hundred percent of interactions.
- So, it's going to allow us to scale at much greater speed, while also still sort of demonstrating that safety requirement.
- And I think just on that as well, like I loved the bit Paul was showing earlier around with the alerts as well.
- So I think when we do have safeguarding concerns coming through or things that need to be escalated, the fact they can capture that in the moment.
- So it's not the agent would have probably escalated it already, but it means that somebody is aware.
- So actually we can speak to the clinical team and we're aware of this, there's a case happening.
- So, yeah, no, very excited to be using it more.
- Yeah, and we were speaking earlier, the three of us, about old school QA, like back in the day, or maybe not even back in the day for still some companies, right, like spreadsheets, manual checking boxes or change something from green to red, sell in a spreadsheet, those times are behind us now, thankfully.
- But what are you doing really in terms of weekly continuous improvement at the moment before monitors came along?
- We actually have sort of two sessions that we run about Fin.
- One of them, I have to give a plug to a team member of mine, it's called Finnovation.
- So they use the session, use sort of recommendations, we're looking at Fin's answers, the content, help center, where it could have been better using that time there.
- And we've seen on the back of that sort of a steady increase within the resolution rate.
- And I think from a compliance point of view, as I mentioned, that's a much more of manual process at the moment.
- Reviewing, like we said, we do a sample of five percent, but that's still like a very manual process.
- It takes a lot of time.
- Yes, I'm very excited with Monitors as it's rolled out, it's going to actually free up a lot of the team's time to work on other projects.
- Yes, I think about like in a previous life, I was a QA manager in the spreadsheet world, and it was something like we'd do ten tickets you know, of deep analysis of QA for those and then listen to one call that was over twenty minutes long or something, you know, and it was such a, like, small snapshot and also super manual, to be honest, kind of boring checkbox y work, so to be able to get that level of QA observability over a hundred percent of everything that's happening, a whole new world that we're living in.
- And similar to CXCR, right, we were all speaking about this earlier too around CSAT, like, you need to do it, right?
- You do need to ask your customers how they feel about the service, you need to do something about the negative ones, but it's such a tiny fraction of all the work that your team does.
- There's like, what about the rest of the customers who, like, maybe they weren't angry enough to do a negative survey or they weren't quite happy enough to leave a positive survey?
- There's this kind of middle that was lost before.
- So can you tell us a bit, Kat, around how you're using CXCR?
- Yeah.
- It's it's really great to have that observability because in Glean as a product, we we don't we don't have that.
- We can't see into other people's instances for good reason.
- So having the ability to see the conversations our customers are are asking about our product, Really, really helpful.
- CX score is super impactful.
- I have these conversations with my leadership team to show them, like, how our documentation, how important it is that we keep it up to date, especially with our product velocity launches, and how that impacts the customer experience with our product overall.
- You I mean, Paul said eight percent.
- I did the math.
- We're at one percent of actual, like, CSAT, so having a much better holistic view, not just the self selected, is super helpful in me understanding our our customers' experience with the product.
- Yeah.
- There was an interesting question at one of the workshops that happened in Paris this morning.
- One of our presenters asked the audience to be really honest and say, like, even for the CSAT that you get in, do you have a few automations set up where you never send a survey in certain situations and a lot of like hands kind of went up like this?
- So it's really not indicative of the broader experience across the board.
- How about you, Rhydian?
- Yes.
- I think for us, as sort of mentioned earlier, I think with CSAT, sometimes it can be quite binary.
- And I think a great example is sometimes we're looking at the performance of an agent and you'll see actually we've seen a dip week on week and we'll actually like deep dive into some of the interactions.
- Once you're actually listening to the call or if you're reviewing the chat, you actually see the agent hasn't necessarily done anything wrong.
- They've been very polite.
- They've been very empathetic.
- They've diagnosed the issue.
- And unfortunately, we just haven't been able to find a resolution with the customer.
- And then that's led to them obviously then getting a low CSAT, which is not entirely fair on the agent.
- And I think where CX score is a lot more quality based, I think where CX score has really helped us, if you see actually an agent has had a week where their CSAT has gone down, you can actually look at their CX score and it's actually gone up, and you can deep dive into some of their other interactions.
- You can see actually, you know what, they actually performed really well last week.
- So I think it just another extra layer to the customer experience.
- Yeah, and it can help highlight spaces that you didn't even know were an issue before.
- You know when you go into Analyze and you're taking a look at CX score and the different topics that you're applying to it, and suddenly, like, one area could have absolutely tanked and you had no idea anything was going on because maybe you didn't get that many surveys from customers about it?
- So it's like it just gives again, it's part of that observability side of things that we never really had before, which is super cool.
- Definitely.
- So in terms of that trust side of things, we spoke a bit about or Paul spoke a bit about deploying AI as one thing, but then really building the trust to go all in.
- Kat, can you tell us a bit about the conversations that you had with your leadership team about, you know, even like that, like, go from lower involvement rates up high and then how they reacted to CX score because it's a new concept?
- Yeah.
- So I think for for me, when I was deploying, it wasn't not trusting the AI per se, but it was not trusting my documentation to be in a place that the AI can actually use it in front of our customers.
- So, you could have a great test feature.
- Fin has a a really easy way to run some queries, test how it's responding, and and give it a grade.
- And running through that test, I had zero, like, poor.
- It was all fair or good, which made me very confident in in launching it, and I was able to show that to my leadership team.
- And then through launching on the public surfaces, we even got more data for more questions.
- I also had my technical engineers hammer away with other customer queries to make sure, like, we were handling real situations, and, and it performed really well.
- So, I I definitely had confidence in the AI itself.
- And then with the CX score, was able to even show, okay.
- Look.
- It's actually doing a great job.
- We're over seventy plus percent on CX score alone, so we we we are confident that it is answering the queries and then the way we want it to be.
- Yeah.
- Something we it was new to us when we rolled out CX score first was that it was lower than CSAT.
- So we were super proud of our really high CSAT scores that we had.
- And then to see that CX score was actually lower initially was a little bit of a shock, but it was a shock about the truth, you know?
- So again, CSAT was kind of masking where there were some issues that we really needed to dig into.
- So again, like, because I work at Intercom, there was a lot of buy in for CX score initially that I'm not sure every team and every company has from the beginning because NPS and CSAT are like, you know, the holy grails from, like, many years.
- But how about at Newman?
- Like, how is CXCR received?
- Yes.
- So I think we're still on that journey.
- As you said, CSAT, NPS have just been around forever, and I think it's very much ingrained and imprinted in people's minds.
- So we sort of launched it slowly, and I think we were talking about it earlier, we actually track CSAT and CX score right next against each other.
- And we're still on that journey.
- We still need to close that gap.
- But what we see quite often is if CSAT goes up, CX goes up, and then likewise.
- So I think they do sort of track each other.
- They do track each other quite well.
- And I think actually just touching on the sort of CSAT piece, what we saw actually with I was mentioning a lot around safety and compliance earlier for FIN.
- But what we actually realized was from an experiential point of view was last year, so our FIN CSAT went from sixty percent to ninety percent.
- And at the same time, we've actually seen our agent CSAT increase and our agent CX score because FIN is able to handle more queries, which means basically agents are freeing up more, which means wait times are going down.
- So it actually just shows how Fin and agents actually work hand in hand.
- Oh, nice.
- Yes.
- Like I mean, it's that classic piece when you roll out Fin first.
- Your team has more space and time to deal with customers and give them, like, a better level of service.
- And then when you see that being reflected back in the numbers, it's really cool.
- Something that we've experienced, though, and I speak to a lot of customers who are getting going on this journey about this, that in terms of KPIs for your team, another learning that we had when we were rolling out Fin first was that when Fin does take so many of those, like, quote unquote easier conversations initially through, like, FAQ type questions.
- And over time, you start to get Fin answering more and more complex queries.
- What's left going to the team is really hard stuff, and it actually means that, like, while they can spend more time with customers, the issues are more difficult.
- And we actually changed our KPIs for our frontline team where, like, we have an expectation for them to actually do less conversations with customers each week or month.
- Have you found something similar that it's the really complex stuff that's there and it's actually taken humans, like, that bit longer?
- The next conversation I need to have with my leadership team is RTTR, which is the metric, RVP wants to focus on is definitely gonna go up or is going up because of that very problem.
- Fin's handling the easy answers, the the one offs, or the the one touch tickets.
- And my engineers, my support engineers are handling these deep complex issues that do take a lot more time.
- And so when my one metric right now is TTR and it's going up, I have to tell that conversation and that story with my leadership team about what we're actually solving for.
- Yeah, how about at Newman?
- Yeah, we have something very similar.
- We haven't sort of reached there yet, but one of our productivity metrics is resolutions per hour.
- And we were talking about this when we were setting targets at the beginning of the year.
- We knew, given that we're just about to roll out Fin procedures, Fin is going to be taking more and more of the sort of one touch or simpler queries.
- So we recognize once we reach a confident stage with FIN procedures, we will have to lower the expectation for resolutions per hour.
- And it's the same with sort of handle time on calls because it's going to be the same thing.
- What we're expecting is as Fin takes more of those procedures, we're probably going to have more patients calling us with more complex queries.
- And it's just taking that into consideration.
- Because they are these calls, they're like you said, they are more complex.
- It does require a sort of high level of empathy.
- So, it's definitely something we're going have to change going forward.
- Yeah, and you were speaking with me a bit earlier around like your queries because it's in the healthcare space, they can be more emotional as well and you don't necessarily always want Fin dealing with those?
- Yeah.
- So we've got I think there's a couple of examples to take from that.
- So I think we were talking about attributes earlier.
- So what we see is we've created attributes.
- So if patients reach out to us and they sound like they're in distress, something to do with financial problems, anything like that, we just bypass Fin because we can the natural language that it takes it senses, actually, you know what, I think this patient needs to speak needs human support.
- And I think we haven't actually quite got to a point yet with FinVoice.
- We were a beta customer.
- And we've been talking to a lot of our patients.
- And traditionally, they've sort of said, well, if I'm calling you, I want to speak to a human because I need I've got a very complex problem.
- I need empathy.
- But now, I think it is sort of shifting a little bit.
- What a lot of them have said is or starting to come around to it, they're sort of saying, I'm happy to speak to a bot as long as it gets resolved.
- However, if it's not able to resolve it, I just want to make sure that I get through to a human.
- Yes.
- And I think now with Fin Voice, it's so much more natural.
- It's like natural language sounds like, you know, a natural voice, and it's not you know, we spoke about earlier that, like, you spoke about botch oil.
- I was calling it bot loop of doom.
- That applies to both chat and phone across the board.
- Like, just let people get to humans if if the AI agent isn't resolving it.
- And even just thinking about your example there with, like, more emotional queries, and you're already doing something about those, but now you can use Monitor to see how much that's actually happening and see what else maybe you could do to kind of potentially improve the product.
- So that never happens in the first place.
- Okay.
- So we are a little bit tight on time.
- I have one more question for the two of you.
- What's next in your AI first strategy?
- So, Kat.
- Yes.
- So right now, it's on external facing surfaces and mostly helping my admins get to tickets if they need to.
- I'm really considering either having it be that first responder in my everyday tickets, see how it fares against my support engineers, as well as supporting customers in the product, be it an end user support tool or in Slack, where we are talking to our customers a lot as well.
- Nice.
- Cool.
- Radeon.
- I think for us, yeah, next step is Fin procedures.
- So we're going to start off, like we were discussing earlier, with administrative queries or something around tracking details just to get the confidence levels up.
- And then after that, we've got around nine or ten use cases, nonclinical, of course.
- But yes, we're going be rolling that out this year.
- So we are very excited for that.
- Yes.
- So you were speaking about doing those read only procedures first to build confidence and trust.
- And you needed to get some buy in from your engineering team as well, Yes, exactly.
- Yes.
- Do you want to tell us a little bit about that actually just before we start to close it out?
- Because I thought that was fascinating earlier that you and then many other customers that I've spoken with today were talking about internal stakeholders kind of being the next unlock rather than the product No, I think it's like with any company, we're a sort of fast growing scale up.
- It's where you stack rank the priorities.
- And I think, particularly as a health care company, some of those priorities are very much clinic related, so they do carry a higher risk.
- And I think, yes, just with our engineering team, I think for a lot of them, it's going to be new working with Intercom's interface.
- So it's just, yes, trying to warm them up to that.
- And we were talking about setting up a call as well.
- But like that's why we said we'd go with a sort of slightly easier use case first to get them sort of used to it.
- And I think then we can go for the slightly more complex ones.
- But I think, yes, by then, they would have they've had the experience with the intercom interface, so yes.
- Nice.
- And then for our own support team, we're actually at eighty three percent automation rate at the moment, which I'm very proud of.
- But I've been voluntold by our leadership team that we need to keep trying to get as far as at least ninety five percent over the course of the next few quarters.
- So it's all in on procedures and getting Fin to answer more and more complex multistep queries now.
- That's up next for me for the next couple of months.
- So that's all we have time for, folks.
- If anybody in the audience has questions, we're going to be walking around the room in Paris online.
- If you still have questions, you can ask them in the chat channel, and one of our employees will get back to you there.
- And for everyone in Paris, there is wine and cheese and charcuterie when we wrap up, so we'll see you all there in a few minutes.
- And thanks so much for joining us today.
Transcript: Winning with Agents as your Frontline - Fin x Kalshi
- Good evening, everyone.
- You're very welcome to our event here tonight, which is focused on winning with agents as your frontline.
- And we're going to talk a little bit about what we call the new customer experience blueprint.
- Delighted that we have Shannon McGeera here with us from Kalshi.
- And later on, I'm going to have a hopefully, a very insightful conversation around how Kalshi have adopted AI from a support perspective and what it has meant for their business.
- My name is Declan Ivory.
- I'm VP of Customer Support at Finn.
- So like everyone else in the room, I'm practitioner.
- I have to run a support or customer experience organization day in, day out.
- I have to worry about all of the same things that you worry about in supporting your customers.
- I do have the advantage that I get to use BIN and all the new products before most of you.
- That's a nice part of my role.
- I like to think we road test products, but generally they are pretty robust when we get them and we're always delighted to get early access to them.
- Just the structure, so I'm going to talk a little bit about how AI is reshaping customer experience.
- And we do talk about shaping customer experience more than just customer support, and I'll talk a little bit about what that means and how we look at it in terms of our strategy.
- Then we're going to have a really good fireside chat with Shannon just around launching an agent at the Super Bowl is how we've termed it.
- You'll understand the context of that labeling when we have the discussion.
- And then afterwards, we're going to have time for networking.
- There's lots of demo stations downstairs if you want to see some of the new features and capabilities of Finn and also enjoy some drinks and food as well.
- So what I want to cover today is kind of two things really.
- I call the first one the shift.
- It's really about what's changing customer service and customer experience.
- What are the dynamics in the industry?
- What are we seeing?
- And then as I talk a little bit about what we have termed the blueprint.
- And the blueprint came about because when we implemented Finn ourselves at Finn back in twenty twenty three, there were no blueprints, there were no guidelines, there were no kind of processes or procedures to follow.
- So it was really interesting journey that we were on and there were a lot of our customers on that same journey.
- And we've taken all that combined knowledge and experience and we've published a blueprint that we very much have as a living document and we will enhance it over time with new learnings and new experiences from both our perspective and our customers' perspective as well.
- So I'll talk a little bit about that framework.
- Good news is I'm not going go into the whole blueprint or we could be here till midnight and you'll miss the drinks and nibbles and the NBA game as well.
- So to set the scene a little bit, this is a quote from one of our founders, Des Trainer, and basically his perspective is AI will change almost everything we do in some way and some things we do in every way.
- And it was always kind of perceived on day one when Genesys AI became a thing that customer service will be one of the first roles that will be transformed using AI.
- So if we think about it, if this is one of the areas and I'll say it's more than just customer support, it's actually your customer experience.
- How is AI actually changing customer experience?
- And we've kind of taken a really strong position on this, recently since we've launched Finn as a customer agent.
- We really, really believe that perfect is now possible when it comes to customer experience.
- And I'll talk a little bit about what we mean by perfect, what does that mean from a customer perspective, and also what is enabling that, reality where we can actually think about delivering customer perfect customer experiences.
- So if you think about it from a customer perspective, there are certain things that a customer wants.
- And when you think about a customer, they want either question answered or they want a transaction completed for them.
- There's very little else they want when they're talking to a supplier or a vendor or a service provider.
- They want immediate.
- You know?
- So they want their question answered right here, right now or their transaction, completed right here now.
- And that's twenty four hours a day, seven days a week, three hundred and sixty five days a year.
- That's what we mean by perfect experience.
- It's got to be accurate.
- You know, it's got to, you know, be a really high quality resolution whether that's answering the question or completing the task.
- It's got to be really accurate and do what the customer wants.
- It's got to be personal.
- It's got to understand that customer's context and make it a very personal experience for them.
- It's got to be consistent.
- And this is really no matter what channel customer contacts you on, it should be the same experience.
- It should be consistent in terms of how you approach that issue for the customer.
- And it should be available everywhere.
- So this is across every single channel, every single touchpoint you have with that customer, it should be the same experience.
- So what makes it possible?
- Ultimately, AI is the enabler.
- But what about AI that makes it possible?
- And the first thing is the concept of a customer agent.
- And people talk about an intelligent front door to your business or, where you can realistically think about AI fronting all of the customer lifecycle journey and being the first thing that a customer encounters when they try to engage with your business.
- And they get value out of that engagement.
- But it's got to work seamlessly with humans.
- Like at the end of the day, no matter how much you drive, and we use the term automation, like, no matter how much you use or drive automation, there will always be work that will need to be done by humans.
- And that interaction between an AI agent and the human, that's got to be seamless.
- Like, that's got to be a really integrated experience for the customer.
- It's got to be continuously improving.
- No business stands still.
- There are still a lot of people who think that deploying an AI agent is kind of a one and done type activity.
- It's not.
- Like, if if there's a whole life cycle of managing AI and continuously improving AI and ensuring that you're constantly driving up better customer experiences.
- And it's got to be human controlled.
- Like, Ultimately, you have policies, procedures, governance that you want to have for your business, and you want to be able to dictate the experience that your customers have.
- So you've got to be able to control AI when it particularly, it is that intelligent front door or the customer agent for all of your customer contacts.
- So that's what we kind of mean by perfect is possible.
- And when we talk about FIN as a single customer agent, it was originally built for customer service and customer support, which in fairness for most organizations is the vast majority of the interactions that customers will have with that organization.
- But there are many other interactions that are part of the customer journey, all the way from the sales cycle through onboarding success, all the way through to traditional support and service activities.
- And we're now very much positioning Fin as a single agent for the customer journey, and or the full customer life cycle.
- And yes, there's FIN for service, but we've recently launched FIN for sales, FIN for e commerce, FIN for successes in the works and there are other variants of FIN that we're thinking about to ensure that we can manage the entire customer life cycle and the entire customer journey.
- And the interesting thing is many of our customers were already beginning to use for use cases well beyond support and service because they could see that Fin could answer questions, it could actually complete work.
- And it doesn't really matter what the role is.
- If it's adding value to the customer, why wouldn't you expose Fin to carry out that work?
- So our customers, to some extent, have almost dragged us here.
- Like, they they have begun to use Fin in ways that we never conceived, and now we're building in capabilities that just make it a lot easier to use Fin for those particular roles.
- So what do need to deliver this?
- And there are really three key things here.
- So one is the agent technology.
- It's yeah.
- That's pretty key.
- That's interface with your customers.
- That's got to be top class.
- But you have to have an operating system that is actually allowing you to control exactly how FIN works.
- So things like guidance, procedures that you've got to be able to control Fin in a way that it presents your business in the way you want it to, presents your brand and is adhering to all of the policies and procedures that you want for your customers.
- And then pairing it all are the models, though, you got to think about the AI models.
- And in recent times, we have begun to develop our own models, and I'll talk a little bit about them on the next slide.
- And that helps us power the agent experience, helps us power the operating system as well that it controls and manages in.
- So what kind of architecture is evolving?
- So at the experience level it's really the customer agent and that is irrespective of role, could be sales, could be commerce, service, success, onboarding, but it's got to be interacting or integrated with human support.
- So you've got to think about that interface and that seamless integration.
- Control points, as I mentioned, are critical.
- And this is around things like understanding customer context, being able to build procedures in a way that you can complete work for the customer, having the right guardrails in place that you know when to escalate to human, you know when you have to adhere to certain regulatory, or legal requirements, etcetera.
- All that control plane is now part of Fin and most recently, launched a thing called Operator, which is actually another AI agent that is managing Fin.
- So again, it's just allowing you to manage Fin in a much more natural way.
- You just use a natural language prompt and operator will build whether it's the guidelines or the procedures that are for you.
- And it will manage that environment to allow you to do data analytics on the Fin environment as well.
- So again, it's really investing AI at that control level.
- And then at the very base intelligence layer, it's how we've built models that will help with the reasoning.
- We've trained a model on all of the customer service, customer support transactions that we've undertaken over a number of years.
- And that model, which we call APEX, is now highly tuned to deliver a really good, performance from a large language model point of view.
- So how are companies progressing towards perfect?
- If that's our goal around perfection, how are organizations moving there?
- What's the current state of the market?
- And we recently published our twenty twenty six customer service transformation report.
- We're just trying again engage with I think it was two thousand five organizations that we engage with or individuals we engage with just to understand how they are adopting AI, what kind of things that they're seeing.
- The full report is available.
- Again, the good news is I'm not going to go into the full report because again, we could be here till midnight.
- And I know that wouldn't be a good experience for you guys, so we'll avoid that.
- So a couple of sound bites.
- The first one that AI adoption is almost the norm at this stage.
- So there's very few organizations that aren't thinking about adopting AI, but depth actually makes the difference.
- And when we surveyed our, when we surveyed for the, trend report, basically eighty two percent of support teams have invested in AI, but only ten percent have actually deployed it at depth or at a mature level.
- Now I think that has accelerated as the year has gone I think it's probably higher than ten percent now.
- But that was quite an interesting stat that despite the fact eighty three percent of people have come down the road of deploying AI agent, only ten percent feel that they've done it at a depth that is kind of matured deployment for them.
- And what does this mean then in terms of return on investment?
- Again, we're seeing a real difference between those who have invested at depth and at a mature level are actually seeing better return on investment.
- So eighty seven percent of those who have deployed at depth are saying that they're actually seeing their core metrics or return on investment improve versus sixty two percent overall the survey.
- And really what that means is that the the mantra of customer service being a cost center is radically changing out to customer service being a value driver for the business.
- And that's what's really happening for these organizations that are deploying AI adept.
- They're actually able to demonstrate that customer service, customer support is actually value adding for the business.
- And there are some interesting approaches that organizations are adopting to drive that, and I'll touch on a couple of those when we go through the blueprint.
- The other thing that has really kind of changed between a twenty twenty five report and twenty twenty six report is that now the bar has moved from like, does AI work in this environment to is it actually good?
- Is it actually delivering the right customer experience?
- And for fifty eight percent of the organizations that we surveyed, customer experience is now the main priority, and that's up from twenty eight percent the year before.
- The year before, people just really focus on will it actually work.
- And now they actually focus on will it actually deliver a really good customer experience and how can I ensure that?
- The other thing that we're seeing is kind of organizations beginning to think differently about how they're structured because of AI.
- So support work now stands well beyond the inbox.
- So again, this concept of having to manage the AI agent, it's not just using as I say our operator feature or other features to manage it.
- You're also relying your humans to help optimize how Fin or an AI agent is working.
- And forty percent of teams are reporting that support roles or service roles have actually become far more strategic because they're adding value beyond just being in the inbox answering customer questions, etcetera.
- And again, I'll touch a little bit on that when we talk about some of the economics of delivering support as part of the blueprint.
- And support is becoming the AI transformation engine for many businesses.
- And I think this is a really com compelling point.
- Like, fifty two percent of organizations say they are scaling AI beyond support in twenty twenty six.
- So, again, you it's really resonating with our strategy of Finn as a customer agent taking on multiple roles and giving a single point of consistency in terms of how you deal with your customers.
- And basically what that means from the organizations who are going down this road, for many of them, customer service is actually becoming a transformation engine.
- Very often it's the customer service or customer support leader who's taking on a broader business transformation role and using AI across the entire customer journey and beginning to be a center of excellence for the rest of the business around how to deploy AI.
- That's a really interesting dynamic and it's transforming their perception of support and service within many organizations and actually elevating the role that many leaders have from a customer service point of view.
- So when you you look at what's coming out of that, transformation report, there really is an inherent maturity model evolving.
- And it goes all the way from what we call the traditionalist, there's basically still manual support, there's little or no AI, all the way up to what we call pioneer where customer service is is rearchitected around AI.
- And in fact, like, we actually should relabel that as customer experience is actually gonna be rearchitected around AI.
- Most people are at level one, level two.
- There are a number of people operating at level three, very few at level four yet.
- So this is a kind of a a pretty good way of testing and understanding where are you on that journey, you know, where do you fall from the maturity level.
- So we're all the time trying to refine this maturity model so that people can understand exactly where they are.
- So how can you accelerate progress?
- And that's what I wanna kinda do for the rest of the, presentation today is talk a little bit about this blueprint document that I mentioned earlier.
- And the concept behind it was, you know, people need some guidance around how do you actually deploy AI in the best way possible, How can you make sure you're accelerating the returns you're getting from deploying AI for customer service or for any customer interaction?
- And with the Blueprint, we tried to encapsulate all of the learnings that we had ourselves in Finn in terms of deploying our own technology, plus the learnings from some of our key customers who adopted Fin early and that we partnered with quite actively in the early days around understanding the best ways of deploying Fin.
- And in those days, we learned as much from our customers who were early adopters as much as they learned from us.
- So it was a really good partnership.
- And we've encapsulated a lot of that in the blueprint document which we published last year.
- And we deliberately broken it into two phases, the launch of phase, so when you initially deploy an AI agent, maybe doing a proof of concept, and then what you need to do or think differently when you want to scale it out and what's the impact of scaling it.
- So today I'm going to kind of go briefly through the launch of phase, but I'm going to spend a little bit more time on the scale it phase because I think that's where you're getting from that.
- I'm using AI agents to I'm deploying it at a very deep level within my business.
- That's I think where the compounding return actually comes for organizations.
- So when you think about launching it, there's three key things like build a business case.
- Like at the end of the day, it's not AI for AI's sake.
- You have to have a business problem or business issue that you're trying to solve, and you need to be very clear what that is.
- Sorry.
- Let's go back there.
- You need to evaluate the agent.
- Know, so at at the end of the day, there's multiple options and technologies out there.
- You got to think critically about what is the best, option for you.
- And then you got to deploy and learn.
- And this is what I was saying earlier.
- It's never a one and done thing.
- It's iterative.
- You will learn from every deployment that you do, whether that's your initial deployment or deploying AI or Fin across different roles.
- And you need to take those learnings and and iterate on them.
- And this is why the whole concept of continuous improvement is really, really important.
- So what are some of key questions?
- And and the first one is what I was mentioning.
- Like, what outcomes are you trying to solve for?
- Like, what's the business issue that you are trying to solve with AI?
- I say AI for AI's sake, it's just not not worth it really.
- Build or buy is still a question.
- Like, you know, lots of organizations are building AI capability in their own businesses.
- And very often, they say, well, if I'm using AI within my own core product and service, why don't I just build a service agent?
- Now when you think about the complexities that I mentioned earlier around all the control plane that you need, like the underlying models, etcetera, very often, you know, a buy versus build becomes very obvious, but it's still something that, you know, organizations will want to assess and understand, does it make sense to to build or buy?
- How do you measure success?
- How do you know it's actually delivering well for your customers?
- And what capabilities do you need to actually make it work well?
- Because it's not, as say, a one and done.
- Even for a proof of concept, you need to think about, for example, have you done due diligence on content and knowledge?
- Have you made the highest quality content available to Fin before you implement it to give it the best chance of success?
- So again, it's just looking at what capabilities you need to prove AI for your business.
- And then how should you roll it out?
- How should you actually test it?
- Is it on a small set of customers?
- Is it for a particular product line?
- Is it for a particular brand?
- So these are the decisions that you you need to think about from a proof of concept perspective.
- There is an interesting one, evaluating an agent, because many organizations will look at the performance of the AI agent.
- And by performance, it generally means the resolution rate.
- And very often people are paranoid about the resolution rate and think, I'm going go for the agent that gives me the highest resolution rate possible.
- And they go through the proof of concept, vendors do all kind of things to ensure that they get a really high resolution rate at your proof of concept phase.
- But for me, resolution rate is only part of the story.
- Probably the most critical thing actually is conversation quality.
- So have you actually resolved the issue in a high quality way for your customer?
- And that quality bar is just as important as the performance bar.
- And in fact, I would say even if you are a couple of percentage points lower in resolution rate versus another technology, but your customer experience is a lot higher, that's a much better solution to run with for your business.
- So again, you got to look at the combination of performance of the agent and the conversation quality.
- Like, is it actually delivering a really good customer experience?
- And the other thing, does the agent come with an ecosystem that allows you to continuously improve how it is operating and running for your business.
- And that has to give you a level of self manageability of the agent.
- You shouldn't have to rely on a systems integrator or the provider to come back every time you want to do something different with the agents.
- You should be able to manage it yourself.
- And that whole ecosystem of continuous improvement is something that you need to think about in a proof of concept.
- And unfortunately, a lot of people are still focused very much on just the performance and resolution rate of the agent.
- They deploy post, proof of concept and then discover all of a sudden that these other two elements just aren't there.
- The conversation quality isn't as good as it needs to be.
- There's no ecosystem to manage the agent on an ongoing basis.
- So if you're evaluating an agent, these are the three things you need to think about.
- So that's the launch of FaZe, very quick.
- I'd say we could have gone a lot deeper on it.
- But that last slide for me is kind of the key one.
- If you're evaluating an agent, think about those three components.
- Now when you want to scale it, what are the things or the pillars that you need to focus on?
- And there are three areas, customer experience, your organization and system design, and economics.
- And in our Blueprint report, there's quite a lot of information around all these three areas.
- I'm going to try and summarize it tonight, but happy to go deeper in Q and A later on or in the networking afterwards.
- Or if you're really a glutton for punishment, you can, go online and read the report tonight, and go into more detail.
- But before I go into those three pillars, scale is a journey.
- And if you think about the evolution of AI agents and how they have performed, like informational queries was really the first thing that that everyone went after with an AI agent.
- It was kind of the mundane routine questions that came into your team on an ongoing basis that very often your human support team weren't adding a whole lot of value like they were given pretty, rote answers, like pretty standard answers.
- And it was kind of pretty mundane work for the team and for customers.
- It wasn't actually adding a lot of value, particularly if they had to wait in the queue to get the question answered.
- Phase two, then people began to use AI agents for guided troubleshooting and for more personalized interactions.
- It's not just general questions, actually personalizing the answer based on the context of the customer.
- It could be based on their billing plan.
- It could be based on the customer segment they're in.
- So a lot more intelligence around how you are responding to customers in that kind of phase two, of scaling.
- And phase three is where you're actually using AI to complete work for the customer.
- So it's where you've integrated AI into your core business and operating systems and you're actually able to complete transactions end to end for a customer.
- And I'd say underpinning all this through all those phases, you've got to have this concept of continuous improvement, and we call it a continuous improvement flywheel.
- You're constantly trying to understand and iterate on how you can drive better experience for customers based on understanding the experience that you're delivering at any point in time and how you need to iterate through the agents to improve it.
- And from a perspective, we call this the train, test, deploy, analyze flywheel.
- And you'll see that even within our product when you open up VIN, you you see the the drop down for train, test, deploy, and analyze.
- From a customer experience point of view, like this this for me when you scale is absolutely critical because it's great to shoot for a really high automation rate.
- But if it is at the expense of customer experience, then it's not the right thing to do as a business.
- Like, I am very convicted that the ultimate benefit of AI is that you can actually deliver better customer experiences.
- In parallel, you may be able to drive efficiencies and cost savings and reduce costs to serve, but ultimately, motivation should be around, I can actually deliver a better customer experience using AI.
- And you need to transform from the kind of reactive support model that so many of us have operated in for in some kind sometimes decades to one that's more focused on AI designed customer experience.
- So thinking about that entire customer journey and how through that journey, an AI agent can actually add value to the customer and deliver a better customer experience than traditionally we've been able to do through, reactive support mechanisms.
- And as I mentioned, like one of the key differences that has happened over the last couple of years that AI agents don't just answer questions any longer.
- They can take actions.
- They can use real time data.
- They can resolve interactions end to end for a customer.
- They can actually complete the work.
- But to think about how you interact with your customers and how you design your processes and workflows, you do need to have an AI first mindset.
- You have to think about how exactly will AI interact with the customer along the journey, what value does it give at that point, and what you need to do when, for example, AI can't complete the work, can't answer the question how do you do that handover or escalation to humans.
- So again, it does require this concept of an AI first mindset, and you've the opportunity to really rethink how you support your customers on their journey.
- And I'm not using support into traditional customer support role.
- It's really customers think of almost every interaction they have with the business as some form of support.
- They are getting some value either in a question answered or a piece of work, transacted.
- And for AI native customer experience design, a couple of things are important.
- Treat customer experience as a product.
- It's just as important as the capabilities within your core product or service that you're delivering to your customers.
- You've got to be very intentional about the customer experience you want to deliver.
- And now with AI agent technology, you have a lot of control over that experience.
- You have a lot of ways that you can actually influence that experience.
- So treating it like a product is pretty crucial.
- Lead with AI and back with humans.
- So again, in the early days, even of Finn, like when Finn completed and, you know, for example, couldn't actually answer the question, there was no context maintained between Fin and the handover to, in our case, workflows and and human support.
- And that was one of the first things when we were using Fin, we advocated for the product level.
- You got to maintain context.
- You've got to be able to understand exactly what was Fin trying to answer, when it failed and how can you use that knowledge then to make it a better customer experience.
- So in our case, we use that context to drive the conversation to a human who had the skill or expertise to answer the question for the customer.
- So it's kind of very granular skill based routing.
- So again, that's all built into product now within attributes.
- But think about that hand off to the humans.
- What does that look like?
- How seamless can you make it?
- How much context can you maintain?
- You got to design for productivity.
- So again, with AI, you've you you and the insights AI can provide, you can actually design productivity into your support organization or into your customer experience.
- You can actually either go out to the customer because you know something is happening that's not what you expect for that customer.
- Or even when they do come into you from a reactive point of view, you're adding more value beyond the issue that they present you with and you're thinking about the next blocker they're likely to hit.
- You're trying to think about the next best action you can take when you're engaged with that customer.
- You need to build trust.
- So you need to make sure that it is a value add experience for the customer and you're not eroding trust through deploying an AI agent and scaling it.
- You want to personalize everything.
- So you want to really think about how can I make this as personalized as possible, really understanding the customer's context and building that context into how Finn is handling the customer and is trying to either answer the question or complete a piece of work?
- Now how do you know you're delivering a great customer experience?
- And I'm gonna touch very briefly on the thing called the customer experience score.
- So historically, we've all relied on CSAT, NPS, customer effort score.
- Like, no matter what we look in, they are all flawed metrics other than they are direct feedback from a customer, which you should always value.
- So there is value in it, but they're flawed metrics in terms of trying to understand the totality of the experience that you're delivering to your customers and understanding each individual experience because you never get that feedback from any of those survey mechanisms.
- So when we deployed, FIN internally, first thing was worked with the product team to make sure we could issue a CSAT survey at the end of the FIN interaction.
- So at least we had some insight into the experience of the customer.
- But again, my own kind of concern was it's just not good enough.
- I I don't know, are we actually improving things by driving higher resolution rates, automation rates?
- And we conceived this idea of a customer experience score where we use a combination of, you know, AI inferred scores like the quality resolution.
- We had an inferred CSAT score and then more deterministic things like, did we hit our first response time, etcetera.
- So there's a whole lot of of attributes we put together, and we built this customer experience score internally.
- And that gave us huge insight then into how Finna was performing at a very granular level for every single conversation.
- And we could actually go back and close the loop with customers if we calculated a low CX score for the customer.
- So they may not have filled in a CSAT survey.
- They may not have kind of given you any feedback, but we determined that a particular customer didn't have a good experience.
- We can proactively go out to the customer and say, hey, we know we didn't deliver well for you here.
- Can you give us some more insights?
- Can we understand a little bit more detail around what that experience was like?
- And that's quite powerful if someone is proactively going out to you and saying, hey, we didn't deliver to our standards.
- I know you didn't complain, but we'd like to learn and understand and improve.
- So CX score has been kind of pretty fundamental for our own internal operation.
- And then the product team built it into the product itself.
- So it's now part of Fin.
- And we've built on a little bit more through a capability called monitors.
- So monitors is very focused on QA.
- However, if you think about the concept behind monitors, you can actually build your own customized CX score.
- You can choose the attributes that are important to you.
- You can choose the weightings of those attributes and you can generate a score for every single conversation that you have whether it's Finn or it's Finn handing over to a human.
- You can calculate a score.
- And you can change that scorecard based on customer segment, based on product, even based on time.
- So you might have a different scorecard for the initial launch of a product and then switch to a different scorecard when it's more kind of in life support for the product.
- So again, it's giving a lot of flexibility.
- So that concept of customer experience score is is really, really critical.
- And you can look at it through different lenses.
- So here it's looking at a total score across a whole bunch of conversations and it's breaking down reasons as to why it was rated positively, why it's rated negatively.
- So there's lots of insights you can gain and again within our standard reporting you can go down to topic, sub topic, you can roll it up into a customer view.
- There's lots of ways you can use the score to really understand are you delivering well for the customer?
- Are you delivering well for the product or service?
- Are you delivering well for the individual transaction?
- So it's a level of insight that certainly in my time in support, I have never had that in a support platform before.
- So it's a it's a really exciting change, but it underpins customer experience absolutely paramount here.
- If you're not improving customer experience with AI, then it's not worth automating that work.
- You've got to be able to improve the experience.
- The next layer is kind of what we call organizational system design.
- And this is really recognizing that many organizations have had to think differently about how they structure their teams because of AI.
- Like, AI genuinely does change how support operates.
- And there's a couple of drivers for this.
- Like, at the end of the day, you're now in a system where particularly using it at a mature level.
- And and what does mature level mean is typically you've automated more than eighty percent of your support operations.
- In our own internal case in Fin, we are now automating eighty four percent of every single customer support interaction.
- And when you're at that level, your humans are actually doing a lot of work to help you optimize that whole system of, you know, Fin, how it's interacting, the knowledge and content there is access to, how it's actually interfacing with your business systems.
- And you're actually relying on your humans a lot of time to give you the feedback around why Fin hasn't been able to do a piece of work or why it hasn't completed a transaction well for a customer.
- So really, like the yes.
- Switches are really, a lot of your teams are spending more and more of the time optimizing the whole ecosystem as much as they are on the queues.
- And when they are in the queues, you've actually freed up time on your team that they can actually spend a lot more effort with the customers.
- One, the work coming through is more complex anyway, so they need to spend more effort, But you're actually allowing them to take on some of that more proactive work that I mentioned earlier, where they're actually beginning to think beyond the issue that the customer presents with.
- And again, that breaks down this whole kind of I just got to get my queue to zero mentality.
- It's actually the mentality is how can I add as much value as possible to the customer in this interaction?
- So again, it's changed the whole mindset.
- Humans have to manage the exceptions, not volume.
- So again, you're looking to make sure that you're enabling your team to spend as much time as possible in those exceptions and add as much value.
- And and the form shifts performance shifts from productivity metrics to outcome metrics.
- And again, that's a major shift, like, because all the time in a traditional world where you're trying to work down queues, they're difficult to handle, Productivity is is key.
- Here's about outcomes.
- And often productivity levels actually decrease for humans because the work is more complex, more nuances coming through to them.
- You actually want average handle time to go up.
- You want things like first contact rate to humans to go down.
- Like, these are kind of anti patterns for a typical support organization, but they're actually what you want and need in this kind of new world where you have a different operating model.
- Really interesting quote here from Nick Clark from BCG.
- Like, just because the tech works doesn't mean the organization is ready.
- So you get the best technology platform in the world, but you need and what Nick calls it is transformation discipline.
- So you need to think about how do you need to operate differently because it's not just about deploying a technology like Fin.
- It's how do you need to organize and structure yourself to get the most out of it.
- This is a pretty complex slide, and and I'm not gonna spend a whole lot of time on it.
- But the concept here is that there are a couple of what I would call key responsibilities that are emerging that really unlock AI performance and experience.
- And there are discrete roles in some organizations depending on the size of the organization.
- In other organizations, there are fractional roles across maybe, one or more people.
- But someone has to take responsibility for these areas.
- And the first one is, we call AIOps lead, but this is really someone who owns Fin as an agent and is taking overall responsibility for ensuring that Fin is optimized as much as possible.
- So like, next slide, I'll talk about my own organization structure and and how we've handled this.
- You then have this concept of knowledge manager or knowledge management.
- In fact, our own knowledge manager, we're probably going to relabel our role knowledge strategist because the knowledge and content is so important from an AI agent point of view, that you really want to invest in that in the right way.
- And while people traditionally might have had a knowledge manager, the way that role needs to work is radically changed in an AI first environment.
- So again, someone has to take responsibility for that.
- Conversation design, so thinking about what is it like to interact with your agent, what is it like when you do the handovers, to to humans, etcetera.
- You got to design that in.
- And then the whole automation specialist, so particularly now when you're carrying out work on behalf of customers, you got to think about the data connectors you need to have.
- You got to think about the systems you need to be able to interface with, what fields do you need to actually bring into a procedure to make it work for a customer.
- These are all new skills and capabilities that you need.
- And if you invest in these areas in the right way, that's, I'll say, when you really unlock the power of AI.
- And many organizations are beginning to put these in place as I say, either through part time roles for people or in some kind sometimes they're dedicated roles.
- Our own structure in, support in Fin, the key thing is we established this kind of AI support team pretty early on in the life of deploying Fin.
- And things like conversation design, knowledge management, assistance analysts, all live in this space.
- And that kind of role of automation specialists, that's actually been wrapped into the conversation design role as well.
- So our conversation designers are doing a combination of the traditional conversation design work that we did initially with Finn and now more of the automation work when we think about procedures and data connectors, etcetera.
- But we also made changes on the human support side.
- So with new job families recognizing there's more complex work coming in.
- We need different hiring, profiles.
- We need different skills within the team.
- And then this middle layer of support operations and optimization, so things like support insights, workforce management, enablement, etcetera.
- Like, we have been trying to use a lot of the capabilities within Finn here.
- We've also complemented that with some tooling like using Cloud Code, for example.
- And now we're actually really embracing operator because it's actually transforming the way we do support operations.
- So every layer here has been impacted in some way by AI, brand new roles on this side, but every other role here has been changed in some ways around the expectation of that role.
- And when you think critically about this type of structure and you build for it, that's where you see the real payback on the investment in an AI agent.
- And the continuous improvement mindset that we're trying to drive in our organization is that the first time you answer a question for a customer should be the last.
- So all the time, if something gets escalated to human, we challenge the humans to think about what prevented Finn either answering that question or completing that task for the customer.
- And we actually encourage everyone to log these kind of opportunities as continuous improvement tickets, and we build up a whole flywheel of work that we then use to optimize how Fin operates.
- So getting that mindset across your team, again, it's a cultural change, but a really important one and one that pays huge dividends.
- And this aligns very much with the flywheel that I mentioned earlier that is within the Finn product itself.
- So training is around configuring, setting up the agent's behavior and tone to your patterns.
- Testing is ensuring you evaluate before you deploy.
- You can test with synthetic conversations, etcetera.
- There's a lot of things you can do there from a test point of view.
- Deploying, making sure that you're launching across your channels that you're trying to address and also the audiences so you can do audience targeting and maybe a subset of your customers.
- And then analyze.
- So all the time you're analyzing what has been the impact of that deployment and then you go back through this flywheel of training, testing, deploy, so this continuous improvement mindset.
- Finally, I'm talk a little bit about economics.
- And I think this is probably the area that has least evolved from a blueprint point of view.
- I think there's lots of learnings yet around exactly how organizations are thinking about the economics of deploying AI.
- And there's probably a lot of opportunity for us to build a lot more capability around enabling people to think about the economics and demonstrate return on investment.
- But I'll show a couple of ideas that are emerging in the space that I think are interesting.
- This first one is hopefully fairly straightforward for people.
- Before AI, there was almost a linear relationship between the growth of your business and the growth of your support organization.
- And that was very challenging for people because if you were in a high growth environment, you had to hire people at a really high rate.
- And guess what?
- Humans cost more year on year.
- Like, it's not like technology where technology costs typically go down year on year.
- Human costs don't go down.
- So this was a real challenge for many organizations in that many organizations chose not to invest as much in support as they might need to, and that always often led to poor customer experience.
- So that's one reason why we've had this kind of traditional view where a lot of support that customers experience was pretty poor.
- It was difficult for organizations to justify the investment because it was all human capital that was involved.
- After AI, where AI is absorbing an amount of the interactions that you have from the support point of view.
- For many organizations, it's breaking that linearity.
- So you may still see growth.
- You may still add people to your team, but you're adding them at a much lower rate than you did before.
- If your business isn't growing or is contracting, then this is an opportunity to really scale back where it might make sense if that was the business priority for you.
- Now, thankfully, most business are aspiring to grow, etcetera.
- What we're actually seeing is that most support organizations are not actually downsizing.
- They are probably stabilizing in some way.
- Like, they're not actually growing, a lot.
- But the work that's been done in those, support organizations by humans is a lot more valuable, a lot more interesting for people.
- Now what are some of the metrics that you think about?
- And, recently, I kind of realized there were kind of two key things that we almost organically began to focus on from a metrics point of view.
- And one is what we call the AI agent performance, which today we call automation rate.
- That's a combination of resolution rate and involvement rate.
- So what percentage of our conversation does AI touch and what percentage of those conversations are resolved or completed by AI?
- And in our case, as I mentioned, we're driving eighty four percent automation rate.
- And if you 'd asked me two years ago, would we be at eighty four percent automation?
- I would say, absolutely not.
- But the technology has improved so much.
- The abilities to manage it has improved so much that eighty four percent is is very possible.
- And in fact, we're in a we have a project called Path ninety five.
- So we're trying to drive a ninety five percent automation rate across our business.
- Now why isn't it one hundred percent?
- I still genuinely feel there will always be some transactions that should go to human and need to go to human.
- So I think ninety five percent, I don't know whether it's the right figure.
- Like, we might end up with ninety two percent or might end up with ninety six percent, but it's probably not one hundred percent.
- But you have to have some aspiration around what you're trying to drive from an automation point of view.
- And that's one of the key metrics we track every single week now.
- And customer experience is the other one.
- Like at the end of the day, it's not automation for automation's sake.
- You've got you've got to be convicted that you can deliver a better customer experience with AI.
- And there, we'll look at things like resolution status, customer sentiment, service quality, repeat contact rate, etcetera.
- So a whole lot of capability that we look at to really understand, are we delivering the right customer experience?
- And obviously, customer experience score is kind of the the high level metric that we use, and then we we break it down by various dimensions.
- So simply deflecting context context is not enough.
- In fact, I actually hate the term deflection, because at the end of the day, it it doesn't feel right from a customer point of view.
- Like, if you're a customer, you don't want to be deflected.
- You actually want your question to be answered or your issue to be resolved.
- So resolution is the thing that's really important here.
- And if you just deflect without a quality resolution, you actually erode trust.
- And it's a combination of AI performance and customer experience that ultimately delivers the right business outcome for you.
- You're trying to drive loyalty for your customers.
- You're trying to ensure that they do repeat business with you, that they are more valuable to you over a period of time and they're getting more value from your product or service.
- So what does this unlock?
- Like, there is cost efficiency.
- Like, the the cost to serve is absolutely reduced today, Al.
- There's no you know, you you can't look at it any other way.
- It it is a reduction in cost.
- So there is a cost efficiency.
- But there's also an unlock around what you know, term here as revenue influence.
- If you're giving customers the ability to solve their issues and get the questions answered faster, getting they are getting more value from your product.
- Their time to value is reduced.
- That's faster time to value and you're reducing churn over time.
- So you're getting more loyalty from your customers and that has a revenue impact.
- And that's one of things we're kind of trying to put our heads around at the moment, how can we actually calculate that revenue influence.
- And then there's this comp what we call compounding reinvestments, so saving investments back in or putting the savings to the back in from a reinvestment point of view.
- So in our case, we have this very deliberate approach around consultative support where we'll proactively engage customers and maybe encourage them to use a product or service at a deeper level or maybe even adopt a product that they haven't adopted before.
- And that is the kind of a new, approach for the team.
- But I actually love doing the consultant support piece, and it's actually adding a lot of value back to the business, and that's where we can move from a cost center to be a value driver.
- So just to conclude, scale is the next frontier for many businesses.
- I know there are still many who are at the proof of concept phase and but I think there's a chance to accelerate now through that and think about scale.
- And what you're building towards, I believe it's kind of eighty percent plus of inbound volume being resolved.
- I don't know where the ultimate line is for any given business.
- I'd say in our case, we've kind of set ninety five percent of the line.
- It may or may not be right.
- We're currently hitting eighty four percent.
- Human optimized systems, not queues, like that's their key role going forward.
- Your organization structures will evolve around AI and CX becomes measurable and iterative so you can improve it on a constant basis.
- And support drives business outcomes and business value for your organization.
- And support leaders have a chance to actually pioneer the new operating model because, as say, customer service, customer support was seen as one of the first areas or activities where AI would have an impact.
- And other parts of business are now looking to support organization around how did they deploy AI, what worked well, and that is then opening up opportunities for support leaders to actually drive a lot more value across the whole customer experience and it's redefining new careers for them.
- So the blueprint is available at the, web address there.
- We have launcher agent.
- We have scale your agent.
- We are constantly refining it.
- So recently, we launched the blueprint for Fin for sales because there's some specific nuances when you think about the sales journey.
- Equally, when we launch Fin for success, we will have particular nuances around the blueprint as well.
- So this is a living document.
- We evolve it all the time.
- We try and take as much input from our own experience and our customers to really make sure that it's relevant across the board.
- And now I'd like to invite Shannon, to join me because Shannon has a very compelling story around Finn.
- Shannon is from Kalshi.
- We've talked a little bit about who Kalshi are and why Finn was, picked by Kalshi, what problem it solves.
- Shannon?
- Thank you.
- Thank you for having me.
- Shannon, before we get into the nitty gritty, I'd love for you to share with people a little bit about your own, career background and how you ended up at Kalshi.
- For sure.
- So, Kalshi is a prediction market platform.
- We are a CFTC regulated exchange.
- And so, basically, you can buy or sell yesno contracts based on the outcome of an event.
- I went to college for meteorology, and so I had started trading on CaliShu back in twenty twenty one, twenty two on the high temperature markets.
- And so I did really well with that, you know, volume ramped up around the twenty twenty four election.
- And then I was recruited to come work in operations, and now here I am.
- So you're a customer turned supplier basically.
- A hundred percent.
- Yeah.
- Very good.
- And and that journey, did did you start off in support and, you know, added?
- I did.
- Yeah.
- So I originally was recruited because I I was very active in the Calc sheet Discord server.
- And so I communicated a lot with my fellow traders at the time.
- And now, they're just our users.
- Very good.
- I love that story of a customer actually then joining the service.
- Love that.
- Yeah.
- I think you've given us some background on Kalti, but can you maybe explain why customer support is especially complex in in your world?
- For sure.
- So we are actually the first CFTC regulated Okay.
- Exchange in the US.
- And so it's hard.
- There's a lot of, you know, compliance and regulatory concern.
- We have markets.
- These markets are live.
- The things that are happening are happening right then.
- And so when somebody reaches out, you know, they don't have an hour or twelve hours, two to three days to wait on a response from support, they need help right then.
- Okay.
- You know, whether they're placing their trade or trying to deposit, you know, things like that.
- So it's it's the immediacy of the need and then also the regulatory framework that's around how you're handling it?
- Exactly.
- Okay.
- Interesting.
- You described, you know, I think one of the most severe scaling problems I've ever encountered in in my career.
- And, you know, it was kind of some wild stats that you shared where you've gone from, and I hope they got the figures, like thirty thousand tickets in twenty twenty four Two hundred and fifty thousand in twenty twenty five, and an estimate of at least one million plus in twenty twenty six and maybe more.
- Yeah.
- I'm sure it'll be more.
- Okay.
- Yeah.
- Back in twenty twenty four, we had thirty thousand.
- Twenty or, yeah, twenty four, we had thirty thousand.
- Twenty five, we had a hundred and fifteen thousand.
- And by the end of the first week of February, we had already passed that Wow.
- Okay.
- Of this year.
- And and in in that time, you've kept the c team roughly the same size.
- So, like, at what stage did you realize that a traditional support model simply wasn't going to work at the kind of scale and growth that you were seeing?
- Yeah.
- At at Calci, we're we're a very small team in general, not just support, but for the whole company.
- I think there's probably about a hundred and fifty of us now in a twenty two billion dollar company that's Wow.
- Just kind of unheard of.
- And so our support team is a total of eight people, and the culture and tight knit friendships that we have are so important that I didn't wanna sacrifice that at all.
- Okay.
- Great.
- So, basically, it was more cultural driven than anything else.
- The fact that you didn't wanna scale the team For because it didn't align with the business or even the culture you were trying to imbue in the team.
- So you sought an alternative approach?
- Yeah.
- Exactly.
- Okay.
- And obviously, that involves adopting AI, right?
- You have talked about the fact that it's a regulated business.
- And, obviously, there are concerns maybe around trust and compliance.
- You know, so how did you address maybe some of the concerns in the wider business around using AI when it came to things like trust and compliance?
- There was a lot of testing.
- A lot of testing.
- You know, before we had came across FIM, we had tested several different platforms to try to find, you know, what was gonna work best for us.
- You know, is this something that we can build ourselves?
- No.
- Not with a hundred and fifty people.
- Good decision.
- Yeah.
- Right?
- Not with, you know, as small as our team is.
- And at that point, we had no room to breathe.
- And so building it ourselves was not an option, and so we had tested some, you know, other different options from, like, our previous platform, other platforms.
- And really through the testing of those, figured out what absolutely didn't work Okay.
- To guide us then once we did, you know, across something that we knew we would have a little bit more control over Yeah.
- Than we already had, you know, a lot of that that work already done for us.
- And obviously, you you did evaluate Fin alongside those other solutions and and you made a decision to go with Fin.
- Like, what convinced you that Fin was the right option?
- What what was the capability that really kind of stood out for you that allowed you to understand you you could deploy it with the level of assurance that you needed for the business?
- It it felt like it was in house.
- Like, it felt like we had the capabilities.
- We had the, you know, connections.
- We had the access to it to have it feel like it was fully in house even though it's completely native with Infant.
- You know, we had those customizable opportunities where we don't have to, you know, something breaks, we have to call somebody else Yeah.
- In the middle of the night to try to fix it.
- We could just do all of that ourselves.
- And so, it made it made the choice really easy.
- So, it's around the ability to, you know, self manage it and set it up in a way that made sense Right.
- In your business and that you represented your brand Exactly.
- Culture.
- Interesting.
- Now, I have to turn to the Super Bowl because The good stuff.
- The good stuff, yeah.
- So first of all, maybe explain to people why the Super Bowl is such an important event for Kalshi and why it's such an operational stress for your team.
- For sure.
- Yeah.
- So like I said, know, twenty twenty four, we had thirty thousand tickets for the entire year.
- Twenty twenty five, that the Super Bowl was our first in twenty twenty five, the Super Bowl was our first big sports event Yeah.
- You know, on Calci once we had started with sports.
- But we went really hard with sports this past year.
- And so then we knew that the Super Bowl was going to be absolutely massive.
- And it was I I think it's still our number one day as far as volume Yeah.
- Trading, deposit sign ups.
- Yeah.
- You know, it just kind of blew all of those records out of the water as well as support volume.
- And so, yeah, I we we knew it was coming, so we just had to prepare.
- So, again, if I understand the kind of the the sequence here, you you've gone out, you've evaluated a lot of agents, you realized they weren't necessarily going to deliver for you.
- You had this impending event called the Super Bowl coming up.
- And, was literally two weeks before Super Bowl that you had the first conversation with Intercom as it was at the time and and, you know, began to evaluate Finn.
- Is that is that correct?
- It was it was about three weeks Three.
- Okay.
- Before.
- So, at that time, I had, I don't know, sent a message online through, like, request more information, something like that.
- And then had one of the sales guys had reached out to me the next morning, and we set up a call that afternoon, and I let him know like, this is Calshy, we move quick.
- Like, we we need this built out, you know, completely rolled out.
- And we had it fully live, rolled out a hundred percent, just thin on top of our previous platform to just do the AI, you know, frontline type support within a week and a half.
- We had it rolled out a hundred percent.
- Wow.
- And I wanna I wanna say our resolution rate at that point was At the end of that first week, it was around fifty percent.
- Okay.
- Fifty to sixty percent.
- Wow.
- So, standing start, literally, you know, three We saw immediate results.
- Okay.
- Yeah.
- For sure.
- Yeah.
- That's an incredible journey to get you your first sales contact right through to the end production at fifty percent resolution rate.
- And, you know We move fast.
- Having confidence to actually use as your frontline for an event like the Super Bowl.
- Okay.
- I think that's one of the most impressive implementation stories that I've heard so far for Finn.
- So well done.
- Thank you.
- Thank you.
- Now, obviously, the the event itself was a stress test.
- Yes.
- You know, as you were saying, you you couldn't really predict exactly what was gonna happen, but you just knew it was gonna be a tsunami.
- Yes.
- And, we we knew it it would be huge.
- And, you know, even leading up to it, we had the entire Fin team, our entire implementation team, presales, post sales, everybody was like all hands on deck to make sure that, you know, everything would run smoothly that weekend.
- Yeah.
- Now, you know, volume hit.
- Things did explode as as they or the volume exploded as expected, but things didn't go quite as smoothly, as you needed to.
- So walk us through what happened and what that then kinda led you to kinda move on to the next step.
- Yeah.
- It was kind of a chain of events because on on the platform itself on Kalshi, of course, we saw record sign ups, record deposits.
- Huge day.
- It it was a really big day.
- But with that, you know, because we were having some delays with deposits, having some delays with people sign up verifications going through because, of course, people wanna trade.
- And so that caused more users to reach out to support.
- And so in that twenty four hour period, we had about eighty thousand conversations.
- Okay.
- There were six of us in support.
- And so we had Finn for a while, and then our Finn continued to work just as planned.
- Our previous platform had rate limited us.
- Okay.
- And so, essentially, that impacted Finn and impacted our ability to receive messages, send messages out.
- You know, the procedures and workflows that we had built out for Finn on top of our previous platform all stopped working.
- Okay.
- And so everything just went dark.
- We had no way to really you know, thank goodness for, like, Discord and, you know, social media where we're like, if you're reaching out through support, please bear with us.
- We're doing our best.
- You know, to kinda give them that heads up of, you know, what was going on.
- But, yeah, I it was a frustrating time for sure.
- I can imagine.
- Correct me if I'm wrong, but on the back of that, then you kinda realized that what you really needed was a very integrated system where you wouldn't have any kinda challenges around, you know, the interface between BIN as a front end and a back end help desk.
- And you made the decision that very few customer support leaders wanna make as a rip and replace of your help desk.
- Is that correct?
- Yes.
- I that was I was, of course, extremely angry.
- We couldn't, you know, talk with our users.
- You know, reassured them that, yes, we're working on it.
- We have our entire team on.
- We had the entire Finn team on helping us, you know, trying to get through this.
- I was even online that night.
- Yeah.
- You know, it really was like all hands on deck.
- They were throttling, you know, things on the Fin side to make sure that our rate limits weren't being impacted as much.
- You know, really doing the best that they could making sure that everything that had come through was then reprocessed afterwards.
- And and, yeah, I mean, like, in in that that evening, was like, nope.
- We can't ever have a situation like this again.
- Yeah.
- You know, and it really came down to the way that the Finn team was there for us.
- Very good.
- Yeah.
- This leads on to the second almost kind of unbelievable thing that you did.
- Like, quite apart from inventing Finn in a two week period, you then basically replaced your underlying help desk in a matter of weeks as well.
- Two weeks later.
- Two weeks later.
- Okay.
- Two weeks later, we were fully rolled out.
- Wow.
- Okay.
- Yeah.
- Again, that that that is hugely impressive.
- But what made that rollout successful?
- Because at one level, like, an AI agent, yeah, not that it's easy, but it's relatively straightforward compared to ripping out your help desk.
- Yes.
- So, what was really the determining factor for that to be successful for you?
- We had a really great team.
- Really good, really, really good team on the Fin side and the Calci side.
- We had some engineers that were helping us on our own side internally, making sure that, you know, the mobile SDKs are, you know, properly set up, everything.
- How's it gonna look on the web?
- How's it gonna look in the app?
- How's it going to perform?
- If we do this, then what happens?
- We were all just kind of working around the clock.
- You know?
- Like, because then I had to have that same conversation with, you know, the full team Yeah.
- When we're looking at moving the help desk.
- I was like, yeah, we move fast.
- You gotta keep up.
- And they they met us no problem.
- Pretty good.
- You know?
- Yeah.
- They sprinted with us.
- Mean, genuinely, those two implementations are probably two of fastest implementations I've come across.
- So, well done on that.
- Now, you mentioned obviously using Fin then for FAQs or easy questions initially.
- But you've moved beyond that now.
- It's handling operational workflows like refunds and payment, troubleshooting.
- Like, how do you how did you approach that expanding AI into this more kind of higher stake support areas?
- Yeah.
- So, whenever we first rolled everything out with I think once once we finally got everything migrated over to the full help desk, of course, we did like a full audit on, like, our help center, make sure that Finn has the best guidance possible.
- But I think it's kind of natural to, like, okay.
- Now, I can take a breath.
- Yeah.
- But, we didn't.
- It was Excited for more punishment.
- It well, you're right.
- I wanna say that our our resolution right at that point was like sixty percent, and so it's like, okay.
- Well, let's get to sixty five.
- And then so it's like, okay.
- We're not really making enough changes here to get to we're sitting at sixty two, sixty three.
- How do we get to sixty five?
- Well, what if we do this?
- And then, you know, we tweak around with some stuff and then it goes up to sixty six, sixty seven, and then it's like, okay, go for seventy.
- And then and we just keep raising that bar every every milestone.
- So, really is that iterative continuous improvement mindset.
- Like, jumped on board on that straight away.
- For sure.
- Okay.
- Yeah.
- Now, obviously, you have used it to you are handled more higher stake interactions, as you mentioned.
- But it's also certain areas where you're intentionally not using AI, maybe broad compliance investigations, institutional accounts.
- How are you thinking about those and where you draw the line around where you use AI where you don't?
- What's interesting is we're starting to use Okay.
- In those areas.
- You know, the more that we've built out with the procedures, the guidance, and watching how it performs and like how how close to how strict can it really be with here's the guidance, don't stray from that.
- And then it follows and then it's just building that trust to where then it's like, okay, well, let's see what happens if we do this on a, you know, maybe a ticket that has an account restriction.
- And how is it going to identify that and, you know, say this, not that Yeah.
- Type thing.
- And now, we have it fully rolled out with account restrictions and kinda looking at it for some of the more deeper things like institutional compliance.
- Those are typically more white glove experience anyway.
- But but, yeah, we're definitely open to it.
- So, it is around enabling Finn to have more context on that customer and then ensuring that you're using things like guidance and procedures to put the right guardrails in place.
- Exactly.
- Okay.
- Yeah.
- Very good.
- And again, talking about your legal and compliance team, they tried to break Finn before launch and Yes.
- Tell us more about that.
- We actually had the entire company try to break Finn.
- It was like, please throw because we we had done this with every every different AI tool that we had tried.
- And with one of them on another platform, you know, I think it was one of the guys on our engineering team was like, talk to me like I'm a pirate.
- And that it wasn't Finn.
- This was a different Yeah.
- It did it.
- And so we were like, well, we can't have that.
- We can't have something that is manipulatable Yeah.
- That can be, you know, prompt ejected.
- So, we were just like, okay.
- No.
- And then, they tried that with femme and couldn't.
- Like, it it already had those kind of like baseline guardrails that established trust from the beginning.
- Very good.
- Yeah.
- They it was like, oh, this thing's good.
- It's good.
- And so you already had the foundation to just start building from there.
- And so you didn't have to worry about, like, the basics of AI guidance.
- It's interesting because I shared a story with you that we actually had one customer who wanted Finn to talk like a pirate because of who they were and their branding.
- And, again, they could set Finn up to do that, but again, that was intentional.
- But right.
- And they were putting the guardrails around.
- Exactly.
- It was like, as I laugh when you mentioned about the buyer because they well, I know customer and that's exactly what they love.
- Yeah.
- That's right.
- So, like, when you think about, you know, the results and and, you know, the impact, like, how has the team changed over the result of this?
- Because you mentioned, like, it's eight people, but I think you're still roughly six people handling tickets.
- Is that correct?
- I would say there's probably about five of us Thank you.
- Handling tickets at any time.
- You know, one guy on our team, all he does is automations.
- All he does is work with Finn and, you know, we kinda call him, like, the the support infrastructure.
- That's all he does.
- He's a guru.
- He really he he is our thin guy for sure.
- So All those responsibilities I spoke about earlier, he does them all.
- He did like, I hope.
- But the thing that's really cool is, like, you know, our our quote unquote support team is very heavy like, we're very heavy in operational procedures.
- And so we're not just working in support, we're also working with our engineering teams, our product teams, making sure that the feedback that's being brought to us from our users is making it all the way to the top and making sure that, like, we always have that that insight with them.
- And even it's really cool because, like, even as small of a team as we are, we all wear so many hats.
- And so, you know, while one of our guys, he he does all the automations and tweaking the procedures and stuff, it's the rest of us, it's our jobs to make sure that he knows, like, hey, this ticket over here, like, maybe instead of saying this, it could do this.
- And we're constantly thinking with every ticket that we go in and solve, how can we automate this further?
- How how can we take this to the next step?
- Why am I answering the same question four times in a row?
- You know, things like that that it's like, okay, obviously there's a knowledge gap Somewhere.
- We try to be really good about going through, like, our, you know, recommendations, the topics.
- Yeah.
- We we we love all the reporting that Finn has for us, but we go through all of that and make sure that all of those thing we have a constant, you know, feedback loop where he's getting the feedback from us of what we're seeing in the tickets, and then he implements those, tests them, you know, rolls it out, analyze it, and then it just just repeat.
- So, you really have embraced that continuous improvement mindset across the and, you know, also being a voice of the customers to the rest of the business as Exactly.
- Interesting.
- Yeah.
- I want to move on to, you know, some lessons learned.
- We've lots of support leaders, and their worry is that AI will diminish the role of support.
- And, I think your experience seems to be the opposite.
- Like, I and I'd love for you to share how you've kind of articulated that when we've had discussions earlier in the week.
- Yeah.
- Bringing AI into a lot of people, myself included, before we went with AI, I mean, we sat there and said, we're never doing AI.
- Never.
- We are not bringing bots into the mix, like, we pride ourselves on being so human.
- Well, then the volume exploded.
- And then you're drowning in conversations that you cannot keep up with.
- It's taking three to four or five days to respond.
- And when you do, it's a macro that may or may not actually solve their problem because you honestly don't have the time, you know, to really help people in the way that they deserve.
- And so when we eventually it was like, okay, well, we're gonna need to either hire like two hundred people or we're gonna need to incorporate AI.
- We really made the most of that to make sure then what Finn was resolving for us was great.
- Those were, you know, like a lot of, like, frontline tier one style conversations and questions, but then the things that were coming to us, we had more time to actually spend in those tickets and making sure that we're building those relationships and the trust with our users to be more human.
- It it became more human than eight of us drowning in in tickets.
- And so and what I don't know.
- Maybe it's just me.
- We have fin tuned really, really well to have great conversations with people to the point where even though it is like it's named Yeah.
- An AI, it's very obviously an AI.
- I'm pretty sure it starts the conversation that way.
- People are shocked.
- That's just a natural that it's just it feels like a very natural conversation.
- And Finn can answer questions better than us sometimes, especially like at the speed and accuracy.
- Yeah.
- But I think what intrigued me when we discussed, like, you're almost talking about Finn enabling you to rehumanize the support experience.
- And I and not to mention of Finn itself, actually, adding to that through the quality of the Right.
- That's really interesting.
- Yeah.
- Because Yeah.
- I think people all the time think that, well, AI is gonna dehumanize support.
- Right.
- That it's gonna turn you into Yeah.
- I feel it robotic, whereas Yeah.
- I think you're saying combination of high performance, high quality of Finn as an agent, enabling your team to spend more time on the things that matter for your customers.
- Exactly.
- But across the board, it's actually rehumanizing.
- Yeah.
- A hundred percent.
- Yeah.
- I love I love that concept because it's it's so much of a Me too.
- An an anti pattern to what people expect.
- Yeah.
- If you do run the regulated business and, like, there are lots of people, whether it's fintech or other regulated businesses, and who are nervous about AI.
- What would your advice be to those organizations based on your own experience having, you know, gone through that environment and implemented an AI agent?
- It's worth testing.
- Okay.
- Even even if it isn't, it's and it will not be perfect the first fifteen times you do it.
- You know, I the one that that's one of our favorite things about Phen is that you can tweak it in just even the smallest of ways.
- But it's also reliable for us.
- You know, we can count on Phen answering the same question at least very similarly a hundred times.
- You know?
- Whereas if you go into Chad GPT or Claude or, you know, any other basically, like, you know, unsculpted with no guidance, you're gonna get a different answer every single time.
- And so So, really, you're you're saying that Tesla on the basis that it's it's very consistent in terms of how it performs.
- You've lots of control to actually manage it the way you need Exactly.
- And, you know, and and you have the ability to iterate very quickly as well.
- Right.
- Because you say, you know, technology is never right the first time anyway.
- Never right.
- No matter what the But, mean, even, like, within Phen itself, you can upload, you know, sample sample questions, sample conversations, then test with those, which we took full advantage of.
- Probably the first month after we rolled everything out because we had to know you have to know the baseline Yes.
- To know what you you need to be training and, you know, building out guidance for.
- Very good.
- And you yourself, like, when you think about what happened, like, what's the biggest lessons you learned that you wish you knew at the beginning?
- That's a good question.
- Maybe to try more, maybe not hesitate as much.
- Okay.
- Because look at where we are now.
- I mean, did move fast anyway.
- We did we did move fast anyway, I definitely in the big I think I I held off on AI for much longer than I should have.
- Okay.
- Because you don't wanna turn into, you know, just a a support system of nothing but, you know, robot.
- Nothing which we haven't built out really well.
- Somebody asks for a human, they get a human immediately.
- But, I mean, there's nothing worse than reaching out through support and you ask for a human and you can't.
- I know.
- You know, with other companies, you know, sometimes you just can't get to a human when you want And, we know that too, that there's always going to be a certain percentage of people who should be speaking to a human or that are going to bypass Finn no matter what, which is totally fine.
- And we're we're about to throw it open to the floor q and a, so that's my q now for there are roving mics come out.
- There will be roving mics, so if you put up your hand, you can get a mic.
- But I have one last question for Shannon just before we throw it open to the floor, and I I have to ask this one.
- Like, you've become a very vocal advocate for Vinn and Intercom.
- Like, what's the single biggest reason?
- The team.
- Okay.
- Just the level of support you've had.
- The level of support that we've had.
- Every time I ask a question, it doesn't matter if it's, you know, super high level or basic surface level question.
- It's always handled with care.
- It's always So the partnership is the The partnership is just like unlike anything that I've I've seen or experienced for sure.
- Shannon, thank you very much.
- I think your story is so compelling.
- You know, the deadlines you work to, the kind of, the constraints you were facing, think is a phenomenal example of how AI can really transform customer experience.
- And I love the idea of it actually rehumanizing the support experience.
- That is phenomenal.
- It's like giving us our lives back.
- So Shannon, thank you very much.
- And before we depart the stage, we will open it for some questions.
- We've few minutes before we can oh, there we go.
- Hi.
- Shannon, you said before that Finn felt like it was in house.
- Right?
- Can you give an example of of something that you guys were able to do yourself that previously you might have had to ask the vendor to do instead?
- Yeah.
- For sure.
- So we have a lot of different procedures and workflows built out.
- And so, you know, if somebody has maybe certain flags on their account, things like that, then they may go through a different workflow than somebody who wouldn't have those flags.
- That's something that we can just go in and tune.
- I mean, if I wanted to pull out my laptop and do it right now, I could.
- Meanwhile, other platforms may not make that as accessible and as user friendly to operate.
- So even if it is accessible, even if it is there, that doesn't mean it's easy to navigate.
- And it's always been really it's nice because we have somebody on our team who's dedicated to doing that, but it's easy enough that it doesn't take a software engineer to have those capabilities?
- It a little bit.
- I mean, but I think it it's still very natural.
- Yeah.
- It's not so technical that you need to, you know, have a computer science degree.
- You know?
- Hi.
- Good evening.
- Hi.
- Two questions.
- With regard to people who English is their second language and you deal with support, whether it's broken English or poor spelling and poor grammar, can you talk a little bit about how Finn helps you with that issue?
- Yeah.
- For sure.
- I don't even know the number of languages that Finn I think it's forty six, I think.
- I wanna say Yeah.
- It pivots like it's that thing.
- And I think something that's been released recently, within the past month or two, is it translating it back to English for us.
- And so we get to have a conversation in English with people who may be reaching out to us in Russian or Spanish, Italian, Vietnamese.
- It's yeah.
- It's really cool.
- It's really fun to also like train to have it like click the button, have it translate back and then watch the conversation.
- Great.
- Thank you.
- It's really cool.
- Second question.
- What's on your roadmap that you would like to see Finn do next?
- Or if not on your roadmap, what is on Finn's roadmap that you're really excited or anxious for?
- I don't Do you wanna go first then I can I can talk about what I'm excited about the roadmap?
- I think we have been really blessed to play around with a lot of the beta features that that Finn's coming out with.
- I'm not sure what's public and what's not.
- Just you can talk.
- Okay.
- So, one that they had actually given me access to today was the workforce management, and that's something I'm really excited to dig deeper into to see how it forecast for us specifically because we're in such a hyper growth phase right now, not just with the company, but also with support, the industry, everything surrounding prediction markets.
- And so, yeah, that's that's next on my road map.
- And and from a Fin perspective, like, I'm really excited about what we are calling Copilot two.
- So it's where we're gonna really rebuild Copilot, have it very much work alongside the operator feature that we launched a couple of weeks ago.
- And I think that's going to open up a whole another level of productivity that we can deliver for support agents, like for the actual human support team.
- So again, it's capability for humans to make their job a little bit better.
- So I'm excited about that from a support perspective.
- Yeah.
- I have a question.
- You said that you're on, like, ongoingly trying to improve in, seventy percent, eighty percent.
- How much of that is dedicated to improving the prompts, improving the parameters, or also now context engineering?
- I know there's a lot going on about prompt engineering is dead.
- Context engineering is the new way to go.
- I was just curious about how you're thinking about how you're, you know, dedicating your your engineering resources to that.
- Yeah.
- It's one of those things that it's become so second nature that we're doing it subconsciously.
- As as we're navigating every single ticket and conversation that comes in, it's second nature to just be like, okay, but what if we answered within in this way?
- Even as I'm responding back to this user and letting them know why their deposit may be delayed, it's like, well, what's unclear in the product?
- And then we spend a lot of time working with, our product team, engineering, and everything within Kalshi.
- And then we kinda coordinate with the improvements that they're intending to make and how is that going to affect the UI, what questions may there be there, and then prepare FIN accordingly.
- Again, just to kind of add a little bit of color, like we're trying to take the complexity out of managing FIN all the time.
- So, for example, guidance, it's very much a natural language expression of the guidance that you want.
- It doesn't have to be very structured from a kind of a technical point of view.
- Similar with setting up procedures, actually an AI agent will help you set up a procedure if you don't want to have to map it out yourself.
- So all the time, we're trying to give to self manageability.
- So you don't need people with necessarily deep technical skills.
- Now one area where it does come in a little bit is data connectors.
- Like you do want to work with whoever is managing your underlying operating business systems to make sure you're exposing the right fields, that you're defining the APIs or the MPC servers in right way.
- That's where the complexity comes in a little bit.
- But even that, it's not deep engineering skill that's required.
- You guys mentioned, like, how fan and AI can help humanize support more.
- But at the same time, there's been, like, an increasing amount of studies and pulling data showing that there's more negative sentiment surrounding AI even as maybe adoption's increasing.
- So I'm kind of been curious how you guys reconcile that.
- Not all of the people who are using Fin are gonna be interacting with a human and people might not necessarily like that, especially like over the next couple months.
- So I'm kinda curious about how you guys are thinking about that.
- Yeah.
- I mean, I don't think honestly, there's we had nowhere to go but up.
- I mean, like, we we our first reply times were three to four days before we really started integrating AI.
- There were eight of us, you know, the we could have expanded to a two hundred person team.
- With that comes a lot of, you know, bureaucracy.
- It there comes, you know, your sacrificing company culture to have a support team that's bigger than the overall company.
- Things like that that would have been very not calchy.
- And so we wanted to make sure that we were staying true to brand, not just in the way that we're interacting with people.
- But I mean, honestly, our CX scores are great with Venn.
- The people who interact with Venn are really happy.
- And so that's really not something that we've experienced with the reports that, you know, there may be more negative sentiment.
- Sure.
- There's always gonna be people who hate AI.
- Always.
- Just like there's always gonna be people who say live agent, live agent, live agent to go right past FEN anyways, and that's fine.
- I mean, is a question of sentiments do change.
- So we track that quite a lot through customer experience score and really understand, are we delivering well for customers?
- Are are they to bypass AI?
- Because that's obviously a signal that something has gone wrong.
- But we're not seeing that.
- Like it is so conversational, so natural.
- Like that's part of the humanization piece.
- It actually feels very human to for Fin to answer your question, and I think that builds trust with people.
- Hello.
- Hi.
- Something we're seeing is as AI takes on the the simpler cases, in a lot of cases is that the the complex ones go to the humans.
- And with that, it takes them longer to to get through them.
- Were you able to quantify the shift in complexity that was going to your human agents, and did that change the interaction expectations or their success metrics?
- A little bit.
- I think I I mean, I'm not somebody who monitors our metrics twenty four seven because whether you reply back in ten minutes or twelve minutes, did you answer the question?
- It's you know, that's the kind of thing for me.
- But it's kinda interesting, you know, because we have built internal systems to kind of I mean, one, Fin has AI summary that can summarize the entire conversation for you up to that point.
- And so we utilize that a lot.
- We also have different things that we've built internally to kind of pull from those data connectors that Fin already has done.
- And so, Fin has already hit how many transfers has this person done, what's their transfer status.
- And so, let's say they're you know, reaching out about a deposit issue.
- And so we'll go ahead and pull their last three deposits and just have those sitting there in a view, in a note within the conversation so that we don't have to rework ourselves because Fin's already done all of that for us.
- And so, honestly, it's made it a lot quicker just because, you know, then we can open the ticket.
- We have the AI summary.
- We have our internal notes from there stating, you know, this person did a fifty dollar deposit and it's still pending.
- And then we just know, okay, this will process in one to three days.
- You know?
- Yeah.
- I mean, we've seen something, you know, maybe a little bit more extreme in that, at eighty four percent automation rate, you're really getting really complex stuff coming through.
- And our average handle time was probably order of magnitude, maybe it's 2x or 2.
- 5x what it was.
- But that's okay, like because it is more complex.
- I actually want the team to spend more time with the customer and add more value.
- And we are changing metrics.
- Like, we're constantly looking at how we think about the metrics for people so that we're not penalizing them because of the complexity of the work coming through.
- And we're actually internally trying to build a complexity measure as well, which if we do, we'll encourage the product team to put into the product as well.
- Think we can probably take maybe two more questions.
- I think we're coming close to the end.
- Hi.
- Thank you for taking my question.
- I was just curious with the rise of agentic trading, how do you think customer support will evolve?
- Oh.
- I think it I mean, it only gets bigger.
- You know?
- It can only get bigger and better from here.
- Everything it's it's funny because everything just moves so fast to where it's like the whole world is moving at calci speed.
- And so it's not just us rolling things out in, you know, a week and a half, everything else is just is changing too.
- And so then oftentimes, we have to match that speed.
- And so we're we're just constantly adapting to everything around us.
- Hey.
- How are you?
- So you touched on going from hitting sixty percent resolution rate, getting to sixty five, seventy, and you touched on, like, constantly iterating.
- Could you go through, like, what are those tangible iterations that you made?
- Was it updating your help desk?
- Was it creating new procedures?
- Was it looking at recommendations in the platform?
- What were those actual changes that enabled you to get there?
- Yeah.
- It's been a little bit of everything.
- Every week, we just kind of I we have our team is very close.
- We work very closely.
- And so we're kind of constantly always, you know, what's new?
- What's going on?
- What's changing here?
- And so something that I think has been pretty revolutionary for us is the incident monitors.
- Yes.
- Yeah.
- Like, the real time incident reporting.
- And so we see when it I believe it's still in beta.
- Yeah.
- But Finn knows what's going on before we do sometimes.
- And so then we're able to act on that really quickly, throw, you know, throw together a snippet or some sort of guidance to give to Fin so that Fin can start responding to those questions.
- We kinda go from there.
- This is like real time thinking.
- Kinda go from there.
- Is this gonna be long term?
- Yes or no?
- Yes.
- We're gonna build out a help center article for it.
- No?
- Okay.
- Once this is done, you know, maybe we'll put this into, like, an ad hoc category that will resolve this an incident and then, you know, prepare for the next thing.
- But in the in the early days, it was really easy because we were so surface level at sixty percent.
- We hadn't we had some data connectors, but they were very simple.
- It was just very validating that this person is a user.
- And now we have it to where it's digging down deeper, seeing if they had a referral code that tracked when they clicked on a friend's link and then, you know, went through the entire process.
- And so as we can continue to build out these more complex things and then, you know, the Kalshi product team continues to roll out new features, it's just constant iteration.
- And so it's a lot of like, here, let's tweak this procedure a little bit because instead of it doing this, I'd rather it just go through the full cycle.
- And so then we're analyzing the tickets that are going through those procedures, how can we take those specifically one step further?
- And how can we take those to one hundred percent completion?
- Let's say somebody needs document verification completed, how can we add in an endpoint there to call our vendor to do the document verification?
- How can we have Finn validate that that document verification was complete and then report that back to the user that you're all set, you're all good to go now?
- And so what started off very simple of like, is this person a user or not, can now validate the entire KYC flow.
- And so, yeah, now I believe we're at seventy five.
- Okay.
- Nice.
- We're getting close to eighty.
- You're on our heels.
- Right right now, we're on a kinda like a sprint on like building a lot of stuff out.
- But then once all that stuff is built, I I think we'll be above eighty, percent resolution.
- I mean, if I was to answer your question, I'd say all of the above.
- Like, it is very much looking at every aspect of the performance of the AI agent.
- And that's why I mentioned kind of the whole continuous improvement environment that an AI agent comes with.
- You've got to be able to know those signals and be able to take action on them.
- That's almost as critical, I say, as the performance of the agent itself.
- And that's really, really important.
- And that real time issue detection is a really, really cool feature.
- It's looking at real time real time anomaly in what's happening and actually surfacing that.
- And that's kind of groundbreaking just in terms of insights into what's going on at a point in time.
- So all the way from real time to post conversation closing, analyzing, looking at recommendations, looking ways to build data connector, etcetera.
- It's kind of it's almost relentless but in a good way because it's driving improvement.
- Yes.
- We do have to end the official Q and A.
- Before we end this evening, you are all invited to remain with us for some drinks and food.
- We will be here to answer any questions that you didn't get answered on the floor.
- We're more than happy for you to come up and ask questions.
- We are about to show a QR code on the screen here.
- We'd love if you would scan the code and give us some feedback on this event.
- Like we're all the time trying to understand how we can improve these events from your perspective, how we can add more value.
- So please take the time to provide some feedback so we can tune as we go forward with future events.
- And finally, Shanna, I would like to really thank you for one of the most insightful stories I've heard around implementing AI.
- And it's just a phenomenal story.
- I'm in awe of what you guys have achieved in Kelsey.
- So thank you very much for sharing that with us tonight.
- Yes.
- Thank you for having Thank you all for joining us tonight because I know you valuable time.
- There's many, many calls on your time.
- You all could be doing other things, so thank you for spending time with us.
- Thank you.
Transcript: Transforming Careers with AI: Real Stories from Fin Customers
- On time.
- Hello, everybody.
- Thanks for joining our webinar today.
- Welcome.
- I am Franca Matinovic.
- I'm Director of Support here at Intercom.
- And I'm thrilled to be joined by some real life Intercom customers who have truly transformed their career progression in the age of AI.
- We're here today to hear their stories and how they worked through to transform their careers in the last year.
- Hopefully this can inspire all of you and and hopefully they can also give you some advice into how you could transform your roles.
- So we've all heard, I think, people say, oh, AI is taking our jobs.
- But what if we flip the record?
- What about completely transforming our jobs?
- So before we head into the fireside chat, I do want to share a couple of housekeeping items.
- That's just so we are all on the same page.
- The session is being recorded and we will share it afterwards.
- We also do encourage questions from the audience.
- So there's a QA box on the right hand side of your screen.
- And at the end of our conversation, we'll dedicate some time to address some of your questions.
- So please go ahead and drop your questions in there.
- And in the interim, feel free to share how you're finding everything in the chat.
- And now that we've done that, I am really excited to kick us off and let Eric and Danielle introduce themselves.
- So, Danielle, let's start with you.
- Okay.
- I am, Danielle Constantine.
- I'm from Calgary, Alberta, and I am the CEO of MyHSA, which is a company that kind of exists in the intersection of tech and benefits.
- And we basically we're we're at a scale up phase now, providing, solutions to employers who wanna give their employees really cool benefits in Canada.
- Nice.
- How about you, Eric?
- Hi, everyone.
- I'm Eric Brulette, based in Lincoln, Nebraska.
- I'm the VP of support and education at Bloomerang.
- And Bloomerang is a company that provides, fundraising software for nonprofits, to raise more funds and do more good in the world.
- And so my role entails working directly with our support team and then also our education team that is primarily focused on the documentation and the and the one to many training for of our customers.
- Very good.
- Okay.
- Because we're talking transformation, I think it's only right we start talking before the transformation happens.
- So I'm curious, like, looking back to before AI became a part of your role, how would you describe your day to day work?
- Maybe we can start with you on this one, Eric.
- Yeah.
- I think personally before AI, kind of your classic case that that you might have heard is that I was doing a lot of manual and mundane tasks, And I was just occupying a whole lot of time.
- And then once I discovered AI and and the possibilities of leveraging tech a technology, I was able to kind of offload a lot of that that work onto AI agents or put kind of the manual processes in in a systems and let it run.
- And then that kind of freed up the the time to really think more strategically and and things like that.
- So the day to day felt like just like living in spreadsheets or doing tasks and and things like that.
- Now it's been a a fundamental shift in terms of, it's more strategic thinking and more strategic projects, because the the AI components can be offloaded and and free up iSpace a little bit more.
- Absolutely.
- I feel like that's a common theme of some of the tasks that used to be so manual kind of disappearing from real life.
- How about you, Danielle?
- Yeah.
- That's that's very much the same for me, but I'll kind of I love talking about where it all started.
- So my HSA started in two thousand thirteen, and when I came on board, it was two thousand eighteen.
- And that was when AI was really just a buzzword.
- It wasn't really something that everyone was talking about, and I was explicitly instructed that my HSA would never ever adopt bots or any kind of automation because our value proposition was very much human engagement and human touch.
- So and that was our differentiator in our industry as well-being insurance and everything.
- So it was very, very, very set out for me that we would never really adopt any kind of automation.
- But I did have a really heavy workload at the time.
- I was the sole chat provider, and I was taking up to eight chats at a time.
- So I started to build out a department of other chat staff, but they were also being overrun, and we were scaling quickly.
- So I guess really how it came about was that I I actually went into ChatGPT, which was kind of new at the time in two two thousand twenty one, I think.
- And I asked ChatGPT to provide some solutions for my workload, and it said it actually provided Intercom.
- It said Intercom at the top, which was very interesting.
- So I started looking into it, and it yeah.
- It it honestly changed everything, and I know we'll get into that a bit more, but really what Eric was saying where it takes off a lot of the manual, repetitious, monotonous work and, made it able made us able to scale.
- That's amaze I mean, thank you, Chaji Bedi.
- I swear we didn't program that.
- That is fantastic.
- I'm glad you're here today, Danielle.
- Since we're talking about, like, AI transforming careers, was there ever for either of you, like, an moment when it came to AI and your role specifically?
- For me, I my moment was more so when I saw the results.
- Before that, it was more of a eek, freaking out moment because I thought it might be a good idea, but I wasn't really sure.
- And, I I brought it to my leaders at the company, and they were open to trying it.
- But my peers were a little bit more hesitant.
- And so I kind of I kind of wasn't sure what the the outcomes were going to be, but I thought it's something we've got to try.
- I think the world is moving in this direction.
- So I guess the moment for me was more so once we actually deployed it and saw it start to do things that we didn't think it was going to be able to do, and we learned how to guide it and work with the programming and machine learning that's in place and everything.
- That was when it became an moment.
- Very good.
- Yeah.
- I'm trying to think of mine, and I believe for me specifically was like when I saw it kind of not I'm not gonna say pushing back maybe on a customer is the right expression.
- It's like questioning.
- Question is like, are you sure?
- Like, or how did you do that?
- And I was like, oh, okay.
- Like, this is a new ballgame way back when, at the very, very start.
- So how about you, Eric?
- Do you have one?
- Yeah.
- Very, very similar to Danielle.
- Personally, I've always been interested in in tech and being on the forefront of cutting edge technology and and things like that and seeing how it can work into our teams, my workflows, my personal life, all of that types of stuff.
- So when ChatGePatil was originally rolled out, I signed up immediately, really not even knowing what it was, what it was capable of doing.
- And then intrinsically, I just leaned in and embraced the technology.
- Like Danielle said, I I knew that this was kind of the future of of where technology was going, and I needed to learn it myself so that I could apply it in in other areas.
- And I I say this a lot, especially to our support team that, like, chatbots and technology have have been around for decades.
- Right?
- Like, everyone knows that chatbots are are on websites and things like that.
- But the technology that AI AI has has provided for the industry has just accelerated the momentum and, really accelerated the results.
- And so my moment actually came on the demo with our sales rep for Intercom.
- Very similar to Danielle, I was doing a lot of research and understanding of what tools that could be applicable to our support team, and and Fin was was one of those.
- And then on the demo, seeing the sales rep take our help center, put it into Fin, have me ask the questions, and seeing the results come with no training or anything of that nature, I immediately, like, ended that call.
- Was like, we need this, and we need this tomorrow.
- So I think that was that was just cool just to see the the technology kind of wrap around our product and and what we how we serve our customers and how effective it was without guidance or or even any sort of tweaking that we had we had done yet.
- Fantastic.
- That's that's a great just really good to hear real life how that looked for you.
- And then I guess in the latest edition of Blueprint for AI agents, we talk in Intercom about these two kind of phases.
- First is launch it and then scale it phase of AI adoption.
- And what you were talking about there was basically the launch it phase when it's going from that buy in, that moment to set up and deployment.
- But scale it is actually where we see the real results, the real transformation, and for the business.
- And you've lived through both.
- So curious for for both of you, and and maybe we can start with you or Eric on this one.
- Like, what which one which stage was most challenging for you personally, and what did it teach you about your own growth?
- Both of the stages are, I think, equally challenging for different reasons.
- The the launch phase for me, the challenging aspect of it was getting buy in and commitment from other people across the organization.
- So thin or AI wasn't on the roadmap.
- It wasn't on people's radars.
- And so a lot of the work that I did in kind of the launch it phase was I needed to get executive buy in and kind of get everyone on on board with the importance of this, which I was able to do.
- And then once we we got the the approval, was it was all coordination of of cross functional leaders and and cross functional work.
- And we actually just took a really small team and and were really agile in terms of our our launch it phase.
- I I knew that we needed to show immediate, like, really immediate results.
- So we got implemented to launch very quickly and then kind of iterated on the launch phase.
- But I think the the key there was continuing to kind of sell the story to our executive team And also having a really small team that has really bought into what we're what we're doing was proof, like, crucial and critical for for our success.
- And then when you think about the scale it phase, we're we're fortunate that our resolution rate is, like, around sixty five, seventy percent.
- And so there there really weren't silver bullets that we could go implement and say, like, hey, if we added these five docs, we're gonna see a ten percent increase in in our resolution rate.
- We're we're in the we're in the phase right now of it's a lot of nickel and diming it.
- And so getting really deep into the data and the questions and how customers are asking questions versus how our documentation is laid out and and things like that, We're starting to see kind of those incremental improvements on our on our resolution rate, but the challenge there is you gotta go really deep.
- And you you have to kind of understand a lot of the a lot of the details and how things are, working with each other and the wording and things like that.
- So the scale of phase for us is all about small tweaks.
- Continual small tweaks over time will will lead to large outcomes.
- Very good.
- I'm like, from the people who are chiming in with us today, like, where would you position yourself?
- Like, are you in the launch phase?
- Are you thinking about it?
- Are you in the prelaunch phase?
- Or maybe you're in scale phase.
- Tell us in the chat.
- But, Danielle, how about you?
- Like, which which stage was kinda more challenging for you?
- Yeah.
- I'm excited to see those answers rolling in too.
- For me, the launch phase was the most challenging just because I was so scared.
- We actually launched on Halloween.
- So it was the scariest Halloween ever, and I was, like, ignoring trick or treaters as I was watching this AI handle questions in real time when trick or treaters were just, like, getting their own candy at the door.
- But that was the scary part for me just because I had put my neck out for this idea.
- I had told with quite amount of maybe assertion to my executives that this was going to be revolutionary for us.
- And so if it kind of like what Eric was saying, I was looking for results immediately.
- I was really hoping to see something good, and we did.
- It started out at, like, twenty percent resolution right off the bat.
- So we were really happy with that because we had just given it some of our PDFs and some some of our documents.
- And over time, scaling it, the scale it phase for me was so fun.
- Like, I became absolutely obsessed because our industry is complex.
- We're again, we're tech.
- We're insurance.
- We're not insurance.
- We're spending accounts.
- There's a lot of different things.
- There's government rules and all these things.
- And so, it's very complex, but I just kinda honed in on watching the chats.
- And anytime our AI couldn't handle or couldn't answer the chat and transferred it to a person, I'd immediately write an article to answer that question.
- And I just started cranking out articles.
- My support team probably can attest to the fact that I would write an article, copy and paste the URL, send it to them, and say, article.
- New article.
- I was sending, like, ten or twelve articles a day just obsessed with watching this AI take off.
- And, of course, there have always been the the great thing about intercom is there's always improvements.
- So there's been guidance that comes and everything.
- And then I become even more obsessed because it's not just the knowledge.
- It's all kinds of things, the tone, the voice, and everything.
- So scaling it was really, really fun.
- I I wouldn't say it was I mean, it was challenging, but it was intense and exciting.
- Launching was more challenging for me.
- But the whole thing actually was really fun.
- Like, if you just kinda lean into the experience, I think it's definitely something.
- And and the experience is so valuable as well, like, in your career, knowing how to do all that now.
- So I yeah.
- I love how real you're being.
- I think a lot of the people maybe outside of support who kinda struggle understanding what what support really does day to day, they just think it's like, what but what's the problem?
- And just turn it just turn it on.
- But there's a big, like, responsibility on making sure, like, we are the face of the like, our company to our customers.
- We need to make sure that what we're doing is right.
- And someone will maybe ask, well, but, like, isn't that the thing with humans as well?
- There's just something about, like, control and maybe, like, if a human makes one mistake, that's one mistake.
- But if, like, an AI makes a mistake, it feels bigger.
- And there's that fear at the beginning.
- But I think once you start seeing it in action and realizing how much it helps and how good it actually is, just all of it kind of evaporates.
- So I love that we're talking about the scale phase because we're you've Eric touched on, like, basically how how detailed it is these days.
- And that usually means and you, Danielle, you spoke about, like, writing all these articles.
- That then means that, like, probably you had to look at your organizations and your systems and rewire it, and the roles that, like, popped up around this AI.
- So how has this shift changed the kind of skills and responsibilities that really define success in your career?
- Maybe, Danelle, if you could start we'll could start with you on this one.
- Sure.
- So for me personally, of course, bringing this in meant that now it was my job to make sure it was doing well.
- So I became very much a knowledge keeper.
- And by becoming a knowledge keeper, I had to learn the answers to everything.
- So if I didn't know the answer to something, but I was teaching our AI, I would have to research and find the answer.
- And I actually, in the as a byproduct of training our AI, I somehow became an expert in our industry just because I was also learning everything as I was training the AI.
- So I was able to elevate my my expertise in our industry as well.
- And that goes for the staff that I had on support as well.
- At the time that we brought it in, I only had a few chat agents.
- One of them now is our head of experience, and I saw that she's tuned in right now, Paige.
- But she she has taken Intercom on as well and, had very much the same experience where she came on in a chat support, role.
- And then her role transformed because now she had to be the knowledge keeper.
- She had to guide the AI.
- There's so much experience that comes with that.
- There's so much learning that comes with that.
- And now now you're being seen as a leader in this space that is brand new, which is amazing.
- And, yeah, it really transforms roles because it I I would probably say this a few times, but I find that leaning into AI really enhances the human component.
- So it makes that side even more valuable.
- So if you are the one that's, you know, taking on not just the AI, but the stuff that AI can't handle, the more sensitive things, the more high stakes situations, projects, and things like that, improvements to the system, your your work has become more valuable.
- You're not being replaced.
- You're actually becoming more valuable because it's getting rid of all the destruction and all the noise, I find.
- That's amazing.
- And hello to Paige, the knowledge keeper.
- Eric, how about you?
- Yeah.
- I would I'd plus one everything that Danielle said.
- I'd I'd also lean in and and say, I think the commonality that you're hearing in a lot of our answers are embrace it and own it.
- And if you do that, if you have that mindset, it's it's gonna create this environment of people like, I also want to embrace it.
- I also want to own it.
- And they start to to kind of follow your lead there.
- So, I think for me personally, like, how this has shifted my my skills and responsibilities, I've led the entire project, from inception to to deployment, still really heavily involved in the in the scale phase as well.
- But it opened up a an opportunity for me to get more visibility across the entire company.
- I had to get executive buy in.
- I had to fight for what I thought was the the right thing for our team, for our customers.
- And transparently, the the project seen as a giant success at Blue Ring.
- It's it's one of the most effective cross functional projects that we've been able to implement at at our company in in in quite some time.
- And so as a result, that has given me more opportunities to be up in front of the company to speak.
- It's given me kind of more more of a voice and kind of drive towards kind of more strategic outcomes and and and things like that.
- So it's kind of opened up that door a little bit, and people have seen me personally in it in a little bit different light than they hadn't seen in the past where they knew, like, support was great.
- And and Eric's doing a great job of of being the support team, but he's capable of more.
- And so, as a result, I moved into a VP role.
- I took on three new teams.
- And the thin and and AI in general has really just given me that that opportunity to to be more strategic in my role, to own more beyond just support.
- And so I think those opportunities eventually would have come.
- Not saying they wouldn't, but I think with AI and in particular with with Dan, I think it just accelerated my growth and my accelerated my opportunities to get more exposure and to own more within within the company.
- It's fantastic.
- And, you know, it sounds like from both of your experiences, it turned out the fear of, like, A.
- I.
- Replacing the work actually was actually AI helps elevate the work.
- So what was the moment when you kind of made AI made you realize that and it's kind of allowed you to focus on higher impact opportunities?
- So we I think it's important to share.
- We implemented FIN back in October of last year.
- So we're still relatively new.
- But during that time, it's also twenty twenty six budget planning.
- Right?
- So putting together forecasts and things of that nature.
- So it was a little bit of a challenge for myself of I know this tool's tool's coming, and then I think I know the impact.
- And so when I was going through my my my budget and my planning, I was putting together the the forecast as I normally would.
- Right?
- If we didn't have the tool, we didn't have AI, what would the impact, what would the team growth need to look like to meet our our goals, our KPIs, and things like that.
- And then I overlaid my assumptions of what Fin could do.
- And if if you're anything like me, I color code everything with conditional formatting.
- I see a lot of red turn to green.
- And the red to green meant that our team in its current state is going to have additional capacity, which is amazing.
- Right?
- Because support agents, as we all know, are overwhelmed with the work.
- We never feel like we can get ahead, things of that nature.
- And so it was at that time when I was, like, visually seeing the impact that it could have.
- It's like, oh, man.
- We can now take really highly skilled, highly talented people on our team and put them into positions where we haven't been able to focus before.
- Focus on retention, focus on more proactive outreach with customers, and really start to drive towards, like, how can support continue to be a a driver towards retention of of our customers.
- And so to me, it wasn't it's not like a cost cutting exercise or anything like that.
- And it's really how do we reallocate the capacity that we have to drive more human led initiatives.
- And I think that that narrative and that story that I I told is very appealing to to everyone.
- It's appealing to the team because they see growth opportunities.
- It's a it's appealing to executives because they they see the budget and and all of those types of things.
- But I think the real impact is, like, what we can continue to do to serve our customers with humans who are highly knowledgeable and highly empathetic to be more proactive with our customers.
- I love that.
- And that really resonates with me because that was my focus this past year, was finding those opportunities to drive adoption of the product.
- And we started measuring everything in revenue as well.
- So now all of a sudden, the story of support is changing from the kind of cost, which is normally what support is perceived as as to like, oh, sounds like you could do more than that, which is has been so refreshing and just completely different narrative.
- And I love to hear that from you, Eric.
- How about you, Danielle?
- Very much the same as what Eric was saying, and I took some notes.
- The fact that you just launched Fin in October last year and you're already at a sixty five percent resolution is insane.
- So congratulations.
- Very, very cool.
- But I I did wanna say that from the outset because we we also weren't sure.
- But from the outset, as an ethical decision, we told our staff we are not going to replace you with AI.
- If this AI is so good that it handles even the whole entire job description of a chat agent, we will find something for you.
- So we went in with that energy.
- And as proof that it did not take jobs, I'm hiring right now for a CSR and for a client support coordinator.
- So if you look at my LinkedIn, you'll see I've got the hiring banner.
- We are scaling.
- And that's really the important part is that we we obviously didn't come into this relationship with Intercom looking for a cost cutting tool.
- We actually didn't.
- Truth be told.
- We looked in looking for a scalability tool because we were scaling faster than we could keep up with.
- I always make the joke that it felt like we were on a treadmill that was turned on way too fast, and we were, like, booking it and hanging on for dear life.
- We could not get off this hiring hamster wheel.
- And what we needed was a scaling tool.
- So we wanted to keep hiring and keep scaling, but what we wanted to do, to Eric's point, was reallocate human intelligence to something that that impacts the bigger picture more than taking the same chat over and over again.
- Is this eligible?
- Is this am I gonna get my claim reimbursed?
- How long for approval?
- Those kind of things.
- We don't need a really talented, skilled, person with really great character to answer the same question twenty five times a day.
- It's it's actually unfair to them.
- It's unfair to their intelligence.
- So what we were able to do is take them off of those tasks and even more complex inquiries down the line so that they can do projects.
- They can do outbound messaging.
- They can engage our Salesforce more because we have an external Salesforce.
- Again, kind of like Eric said, like, you start to see people with capacity, and that helps them kind of fall into their personal skill set, and then you can see that and reflect their role based on that.
- That's really what we've seen happening.
- So I I really don't think AI has replaced anyone's job here.
- I just think that it's reallocated the human intelligence to way bigger and better and cooler and less annoying things.
- Yeah.
- And it sounds like both of you kind of had that.
- While there was initial fears, sounds like you both had this vision of how this is gonna work for you.
- But to to kind of have that buy in yourself is one thing and to build that trust with your team and with your customers, I guess, is another one.
- And I'm curious, like, we was this something that was given top down to you?
- You were told, go on, figure this out.
- You have to incorporate AI, or did you sort of bring AI initiative to the top?
- And it seems like for both of you, there was more of you than there was top down, but I'm sure some of our folks are experiencing a different sort of directive.
- So I'm curious, like, if maybe Eric, we can start with you.
- Like, how did this happen for you?
- Yeah.
- This was not a top down directive.
- In twenty twenty five, the there was a lot more conversation at Bloomerang about AI and using AI in an intelligent fashions.
- But when I looked intrinsically, was like, I'm hearing this message.
- What does this mean for me a year, two years, three years from now?
- What does this mean for the team?
- And so I knew I'd I needed to get out ahead of it and and push AI up instead of waiting for it to be pushed down to me.
- So, yeah, I wasn't given the directive, but I think that that if if you think about it in your career, if you're not getting the directive, I think that this is a huge opportunity for you to manage up and coach up, and say that this is the opportunity that we need to go tackle.
- Because it it will produce really positive results, for your team.
- And as as the owner of the project, you'll be seen in a in a really positive light as well.
- Absolutely.
- That's brilliant.
- How about you, Danielle?
- One hundred thousand percent what Eric is saying.
- If you are not being told by your bosses to get AI going, tell your bosses it's time to get AI going because if you don't, you're gonna miss the train, and the train is at the station right now.
- We we got on the train early somehow.
- Again, I kinda look back, and I feel like it was not not to get all crunchy on you.
- But I feel like it was kind of kismet that it came into our sites at the exact time that it did because we were we were not going to be able to scale at the rate that we did if we did not lean into AI.
- And it transformed the way our company scaled and is looked at in our industry.
- And to answer your question again, I it it always brings me back to this kind of like, I almost disobeyed the instructions by saying, I know we already talked about this and that we're not gonna we're not gonna do bots or automation ever.
- Because, again, in our industry and insurance and things like that, the turnaround times are slow, wait times, like hold times are such a, you know, status quo, things like that.
- So we've had made it our value proposition to have a human available pretty much any time of day over our live chat.
- And so for me to then come in and be like, hey, I know we talked about this, but hear me out.
- I think we should get into AI.
- It was it was very much coming from me up.
- But, if there's anyone listening that is in leadership and is, here because someone brought the idea to you, I would just say be open to it.
- My my leaders were open to it, and we're open to the idea, and it completely transformed everything we've done ever since.
- Fantastic.
- Very good.
- So at Intercom, we really highlight in all of our resources, including blueprints that was shared earlier, the importance of treating AI as infrastructure that kind of continuously improves.
- How has this helped you grow into new roles or responsibilities you just didn't expect when you first started with this?
- And maybe, Eric, you can start here.
- Yeah.
- I've I've hit on a few of the the pieces already for me personally.
- I will say for our team, it's opened up more growth opportunities for for them.
- For context, we've promoted or or moved six people on our team in the first six weeks of the year to other roles and responsibilities, other teams, other other functions.
- You look back on twenty twenty five, we moved we promoted or moved six total people.
- And so I think that that alone is kind of showcasing the the impact that it, AI can have.
- We've grown our our documentation team as a direct result of this.
- We now have, like, a smaller team that's that's focused on the continual improvement and and things like that.
- So I would say the the roles and responsibilities are ever evolving, and it's evolving based on what we're seeing in the results from, from our work, or what what what we think AI could could do more for us.
- But I think that the roles and responsibilities and the shifting of of people into into those roles has been magic.
- I I think it's it's pretty pretty awesome to see the amount of growth that our team is is having.
- Yeah.
- I can testify that here as well.
- We've, you know, we started with what someone who one person who was managing the content, and then we have two.
- We had started with one person who was doing, like, conversation design and customer experience.
- Now we're kind of heading into two and nearly three space because, like, yeah, the amount of, like, little work that you know, the small incremental changes, the continuous improvement that can be done there is really, like, invaluable.
- All that, like, every single little tweak makes a difference.
- How about you, Danielle?
- Again, reflecting a lot of what Eric said, but I'll kind of speak internally.
- We we figured out that this is working really well, and we actually built our department our support department around the AI once it was built in there.
- So we have, you know, different tiers of employees.
- We've got, like, our frontline staff, our our coordinator, and then our our manager, and we saw that it worked.
- So then we we decided to build out that infrastructure departmentally, and and I'm still working on that, building it out in adjudication, building it out in sales, building it out anywhere we can in tech, see where we can adopt AI, and then build the department around that because that whole section of what used to be in the job description can be taken care of, and then you can build everyone else's skill set and roles to support that and handle the bigger stuff.
- And so that that's kind of how it has helped us at the infrastructure internally.
- And then externally, it's really cool because, now MyHSA is seen as kind of a thought leader in a very archaic industry.
- So we have had several different opportunities to talk about adopting AI in an industry that is known for being kind of a, like, ancient, like, paper claims and all these things, and now we can kind of spread the good news with everyone.
- So it's it's really been the infrastructure has been huge for our company and for our roles individually.
- Absolutely.
- And you are a thought leader.
- I love that, you know, people who come from support, they're like, what?
- We're thought leaders.
- No.
- We're not.
- You are a thought leader.
- It's such an intense thing, like, to like, I don't I I appreciate you saying that, but it feels weird when someone says that.
- But then when you think about, like, same with Eric and same with, I think, everyone at Intercom, you're pushing something that was on the horizon, and now it's very much here.
- That does that is the definition of a thought leader.
- So Exactly.
- Exactly.
- Very good.
- I'm gonna just remind folks who are listening to drop in some questions if they have any for us, and we're gonna I'm gonna ask one more question to our to Danielle and Eric, and then we will proceed to your questions.
- So okay.
- So for all the folks who are in the audience and they're curious, but they're also unsure about AI's impact on their own career.
- What's the one piece of advice or mindset shift you think it's essential to embrace AI as a catalyst for personal growth?
- Let's start with you, Danielle.
- My one piece of advice would just be that that, again, even if A.
- I.
- Manages to do an astronomical amount of your job, ninety percent, let's say ninety percent, that's probably not gonna happen.
- But even if it could do ninety percent of your job, the ten percent left that you do becomes exponentially more valuable, especially if you learn how to use the AI.
- You will think faster.
- You will execute things better.
- You will have better project outlines.
- You will have better outcomes.
- I I know I'm speaking anecdotally, but I really mean it.
- I really mean this.
- I I think that AI is here to enhance the human component.
- It really is.
- So, trust it, I guess.
- Again, plus one to to Danielle.
- I think the the embrace it and own it is really important.
- Because the more knowledge that you have on on AI, how it works, then that translates across so many different roles, responsibilities, next company that you're you're you're moving towards.
- It's really valuable knowledge to have right now.
- So really embrace it and and own it.
- And then also, don't don't start with a tool in mind.
- Start with a problem statement.
- What like, what problem are you trying to solve?
- And then how does the technology support solving that problem?
- And I think you'll if you continue to ask yourself or challenge yourself, like, how can we do this better?
- Or what what have we always wanted to do, but haven't been able to do it?
- You'll find yourself in situations where all we can, we can use this AI component or we can use AI for X, Y, and Z.
- And you'll start to see the results in your work.
- And as Danielle mentioned, that that time that you're freeing up for yourself is so incredibly valuable that then make sure you're capitalizing on it and thinking more strategically and acting on on more so that you're seeing, like, yes, AI is having you can see the tangible results, but you also can point out and say, like, these are the additional things that I'm able to take on and own as a result of AI.
- Okay.
- Great stuff.
- Thank you, guys.
- I really, really appreciate everything you've shared today.
- I think it's really refreshing to talk about, like, reality of someone's career and how they've approached it.
- So thanks to everyone who tuned in to listen.
- Hopefully, you got some inspiration.
- And after this, you're now kind of itching to get going and come and find us if you wanna chat more.
- But like like Eric said, think about the problem you're trying to solve and the solution is gonna present itself.
- So massive thank you to Eric, to Danielle, and everyone who was here with us today.
Transcript: Scaling CX in the AI era - Fin x Clay
- So first of all, I'd like to welcome everyone here this evening.
- Really appreciate that you have chosen to spend time with us this evening.
- Very conscious it's tech week here in New York, and that's one of the reasons why we're in town.
- But you do have a lot of options in terms of how you can spend your evening, but there's so many different events on.
- So we do appreciate you being here today.
- My name is Declan Ivory.
- I'm VP of customer support at Fin.
- I've been involved in customer support for Close On for decades, so I've seen a lot of changes, but like quite literally, I haven't seen anything like the changes that we have seen over the last few years and hopefully we'll touch on some of that as we go through the conversation this evening.
- A little bit about Fin in case you don't know who Fin is.
- So Fin has been around for fifteen years.
- We recently rebranded from Intercom to Fin and the main reason we did that, we had been building customer service solutions for our entire history, but over the last three years, we very much focused on building as a customer agent, initially focused on customer service and customer support, but more recently focused on the entire customer life cycle or the entire customer journey.
- So given the focus and the investment we had in Finn, we decided to rebrand the company as Finn.
- So that's who we are.
- I say we're Irish founders, we've been around for fifteen years.
- Customer service was a heritage, AI and Finn as a customer agent is where our future is.
- I'm delighted that we have George Stilty from Clay with us this evening because when you think about where AI is going, lots of people have dabbled with using AI for customer support.
- Some people have deployed it very deeply for customer support, but those that are really kind of forging ahead are thinking more critically around how can you use AI to actually build a connected customer experience.
- So looking at the entire customer journey and looking beyond the roles that we tend to think about when we talk about using an AI agent and more looking at it through the lens of the customer.
- What is the actual customer experience?
- And I'm delighted to say that Clay had been at the forefront of thinking in this way and have really been looking at how they can build a connected customer experience using AI.
- So George, I'd love if you maybe give a little bit of an introduction to yourself, like a little bit about how did you land at Clay?
- Like what's history?
- And also a little bit about Clay for those who may not have heard about Clay or may not know fully what Clay do, because it's a pretty cool company with some really cool technology.
- So I'd love for you to share that.
- Yeah, for sure.
- Thank you, Declan.
- So I'm George.
- I run the support team at Clay.
- And Clay, you can sort of think of it as your sort of all in one go to market platform for research and outreach for your customers.
- So you can tap into one of one hundred and fifty plus data providers, find your best customers, enrich them, and then sort of action on that data through emails or ads or CRM enrichment, whatever it might be.
- So I've been at Clay for a little over two years now.
- So it was March of twenty four.
- I actually started as a go to market engineer, but was only there for about a month before I took over the support team.
- And we were two or so people, and we are now a global team of thirty or so.
- We just opened up an office in San Francisco, and we just opened up an office in London.
- I also have someone out in Japan, so a pretty sprawling team now, which is very fun.
- And before Clay, I was at a small startup called Mutiny.
- And before that, I actually worked in the arts world.
- So I worked at Lincoln Center here in New York City and at a couple of agencies doing a mix of marketing and web analytics and that kind of stuff.
- So I'm super excited to be here.
- Thanks, George.
- I mean, it's great to get the insight on your history and what Clay is doing as a business.
- And obviously, as I said, it's a pretty cool business.
- Support is obviously critical for you guys.
- I'd love to talk a little bit about your journey from a support point of view and using AI.
- I know we've had lots of conversations about this over the last few years.
- We've never had it in front of a live audience quite this size before, so no pressure.
- But what was the main driver for Clay to really think critically around using AI for support as kind of the initial focus?
- Yeah, I think the initial focus sort of came it was a sort of a forcing function of the fact of how we sort of wanted to run our support team.
- So from very early on when took over the team, I had a conversation with our founders, Varun and Karim.
- And all the way from the top down, we always decided that we wanted support to be more than just a cost center.
- We didn't want it to be just this afterthought that just had to exist as a good enough function.
- We wanted it to be really good and full of really smart technical people, and we wanted to invest in it.
- And what sort of came very quickly after that is we realized that a lot of the stuff that we're seeing in the support queue is not that.
- It's things like I forgot my password, or I want a refund, or whatever it might be.
- And so from very early on, we decided that we needed AI to sort of help us build the team that we wanted so that the humans on our team can be more focused on the more strategic sort of engaging questions and almost acting as consultants, which I'm sure we'll talk a little bit about, rather than answering the forgot password.
- And so pretty early on in my time, we ended up engaging with Finn, and we've been using it ever since.
- I mean, you've also taken what I think is quite an unusual approach.
- You've made some decisions that probably at the time did seem unusual, now are beginning to kind of make more sense to people.
- And one was establishing this role of GTM engineer.
- And I think you guys have been credited with almost creating that role and other companies have emulated what you've done and also having this concept of a support engineering team within your CX function as well.
- Love to feel free to talk a little bit about that and what impact that has had on your whole AI strategy.
- Yeah, for sure.
- So I think they all sort of speak around this same sort of idea that we're getting at is that all of these roles are becoming a little bit more technical.
- So the go to market engineer is Clay's take on a salesperson.
- But they are a little bit more technical.
- They're sort of like a product minded account executive.
- You can think of them.
- And so they're not just selling you a product.
- They also know how do you build in clay and how to use clay and help our customers.
- And I think we have sort of embodied that across the organization.
- So we have our ops team, which is a central place for a lot of our operations that help us drive a lot of the plays that we want to do across our go to market team.
- We also have support engineers, which are support folks that are sitting on the support team, but they're sort of engaging with the engineering team on our bugs and helping fix those.
- And so it's sort of this being able to sort of bring a different level of technicality to all of these roles that we historically may not have been as technical.
- And that role is quite new.
- When you evolve it, it's become a little bit more commonplace.
- What would you say has been the biggest single impact of that role for Clay?
- Yeah, I think for the support engineer in general, mean, the biggest impact has been they've been sort of like an escalation point for the rest of the support team, which has been really helpful.
- I think Clay in general is a little bit more of a technical product than other products.
- And the support engineers sort of help us bridge that gap with the engineering base and be able to sort of have a really good relationship with the engineers to help us fix the things that are broken, but then also sort of be a conduit for product feedback or whatever it might be to build the next best thing.
- So helped build a really close relationship with EPD across the board.
- Love to tease through, like as you went on that journey of using AI to really, as you say, transform how you were delivering support and move from that cost center to a value center.
- Are there things about AI that surprise you either good or bad?
- Yeah.
- I think the thing that's still surprising me to this day is how you sort of we're going to talk a little bit about this.
- But what of surprised me when we first got into it was the differentiators are different than what I had originally thought.
- So before we sort of engaged with FIM, we were looking at a couple of different companies.
- And part of me thought one of them is just going to be that much better than the other one.
- And what we sort of realized is that the sort of AI models are all the same.
- And so that's actually not a differentiator in itself.
- What matters is the product behind what they're building or the content that we have, which at that point was not that good.
- And so it very quickly became clear to me that what you sort of originally thought was maybe this model is going be better than this model just at baseline was actually more around the sort of internal products, how you build these things, how you orchestrate these things and all of that.
- I think that's continuing to be a sort of surprise for me.
- Very good.
- I mean, it's interesting, our own journey at has kind of mirrored quite a lot of your own journey at Clay because again, we had this kind of real conviction we wanted to turn from being a cost center into being a value driver for the business.
- And that was one of the key focus points we had when we adopted Fin.
- And obviously we had the advantage of working for Fin.
- Adopting Fin was almost like a non negotiable.
- Had to do it but it was actually something that I was really excited to get behind because you know, as I mentioned, I have a long history in customer service and for many years, in fact, on a decade at this stage, I always had this conviction that AI would be able to transform how you deal with your customers and what the job of a support engineer is like as well will be transformed with AI.
- So having the opportunity to be an early adopter of FIM was really exciting from that point of view.
- So it was part around transformation to a value driver.
- It was also around the scaling similar challenge you had, like we were beginning to ship new product, new capability at a very fast pace as a business.
- We were adding a lot of customers and just being able to scale then in a much more manageable way because of Fin was a really, really big win for us.
- And then also transforming what we call our standard support policy.
- So we are quite constrained in what we could do for customers.
- We weren't offering twenty fourseven support prior to FIN.
- And all of sudden with FIN, we could build out that capability.
- And to the point you made, you're actually then using your humans in a much better way to interact with your customers and actually deliver a lot more value.
- And including in our case, do what we call now consultative support, which is actually being a lot more proactive with our customers and actually seeing how we can get them to use the product at a deeper level, is allowing them to get more value out of it.
- I think that's similar to the philosophy you have in play.
- Sure.
- Yeah, it's just a much better, it's more enjoyable for a team too, right?
- They're spending their time.
- We've actually seen like handle times go up and anecdotally everyone talking to us about how it's taking me longer to answer the questions that I have in the queue.
- And that's a positive signal for me because it means that they're not spending their time telling the fifteenth person what a credit is or whatever it might be.
- I remember, you know, we had a conversation quite a while back at the stage, you actually said, look, know, most organizations are trying to drive up first contact rate or first contact resolution.
- And you basically said, I want to drive it down.
- Right, right.
- Because anything that can be solved in one contact is probably something that AI should be handling.
- And so historically, we used to use that as a proxy for quality.
- And now we're flipping that on its head and saying, if it's solved in one response, then we probably want to figure out how to do that with AI.
- Yeah, and again, that really brought home to me that the metrics that you traditionally look at in this world, they really are turned on their head and you really need to think about them differently.
- It's fascinating even from that perspective.
- But obviously what we want to try and tease through this evening is like moving beyond support because I think this is where the organizations that are really getting the most value out of AI are really setting the pace.
- They're understanding that AI can be applied across the entire customer journey and like we use this term of connected customer experience.
- Some people talk about roles being blurred or roles being dissolved.
- Des Trainer, one of our own founders talks about AI being a convergent force across different roles.
- So I'd love for you to share a little bit about how that has kind of happened at Clay and I think you used the term qualify everywhere which is actually qualifying your customers at every single touch point.
- Love to hear more about that.
- Yeah, for sure.
- So qualify everywhere was sort of this idea that I threw into Slack one day that essentially, you know, when when I think about qualification or when I had thought about qualification, it it it always sort of, you know, was qualifying people on the form, on the website when they were sort of a new customer kind of thing.
- That was always sort of the use case of qualification.
- And what I sort of realized through our support work is that there's actually opportunities to qualify everybody, and there's opportunities to qualify them at different points within their sort of customer journey.
- And so what we started to do was open up we sort of used Clay to sort of qualify folks.
- They go through a number of different qualifications based on their ARR and how many people there are, or whatever it might be.
- And we were able to sort of expand that to different parts of our customer journey.
- So if someone's in the support queue and they ask us for a demo, we can do that same qualification.
- If somebody is talking to a growth strategist, we can understand that on a growth strategist is sort of like our CSM equivalent we can understand that on the support team and get them assigned to the right person very quickly.
- And so it's sort of this idea that qualification doesn't just have to be sort of in that beginning of the journey.
- It sort of happens everywhere.
- And we've gone so far now to actually have it hooked up to Finn.
- And so if somebody asks Finn for a demo, it does the same workflow of seeing if they're qualified, and if they are qualified, who they should be talking to and routing.
- And all of that sort of happens in one place but is sort of spread throughout a bunch of different parts of our customer journey.
- So I think that's just indicative of thinking much larger about the customer journey as opposed to just kind of boiling it down to individual pieces at different parts and sort of siloing those different pieces.
- And obviously as you go through that, I feel like you're looking at different tasks or roles and you're looking at how you can automate them through Fin.
- At what stage do you have kind of a decision point around is it better to basically have something as a deterministic workflow versus having it as an agent task?
- Have you thought about that?
- Yeah, it's a good question.
- We're doing a lot of work right now building out procedures and things like that with deterministic workflows.
- And I think that we're generally trying to give Finn as much capability as possible, obviously within reason.
- And so those procedures, I think, really come from looking at what our humans are doing.
- And if they are going through the same process of checking into this dashboard and then checking these logs and then coming to this conclusion, whatever it might be, we want to give Fin that same capability to be able to get a customer answer faster and take a lot of that work away.
- And so there's a lot of cases where we're doing that.
- There's also cases where we're just giving Fin access to customer data that maybe it didn't have access to.
- And that's just an ambient, making Fin better kind of thing and more personalized, is it understands how many credits a customer still has.
- It understands when they became a customer, whatever it might be, and can sort of help inform some of its answers at baseline as well.
- Very good.
- Think we talked about dissolving roles or walls falling down or convergence.
- And I think the place everyone goes immediately is support and success.
- I love your perspective on that.
- You see there's a clear delineation any longer between support and success?
- Or does it even matter whether there is a delineation or not?
- What's your thinking on this?
- Yeah, I think it's an interesting thing that I was literally just talking to my boss, our head of CX, Jess Bergson, today about what are the rules of engagement for different parts of the CX org, right?
- When think about traditionally support was maybe this more reactive sort of place where customers would go and you would answer their question, then you'd be done.
- And a CSM is sort of a relationship manager.
- You might have an expansion manager, whatever it might be.
- And I think that we're doing a lot of collapsing.
- I think the go to market engineer is an example of collapsing almost a sales engineer and an AE into one place.
- Our growth strategist at Clay is sort of a CSM, but also the onboarding manager and also the expansion and whatever it might be.
- And I think that between support and success, what we're finding is that as we're taking away the I always like to call it automating the ********.
- As we're taking away all that stuff and making it so that the questions that they are answering are more engaging and strategic and whatever it might be, that line between where support is able to go to and where success is really starts to blur.
- And I think that's ultimately a good thing for the customer.
- I think that when we can sort of provide that right up front, if their growth strategist is on a bunch of calls all day or whatever it might be, it allows us to sort of again, we're not thinking necessarily about what is this person's role versus this person's role in sort of passing the customer around.
- We're just able to sort of help them in the best way possible, which I think is the most important thing.
- You know, I love the concept that I've just kind of view that ultimately when a customer engages with any business, it's one or two things.
- They want a question answered or they want a task completed.
- And really they don't care who actually does that for them.
- And certainly the way we're building out our strategy now with Fin is like Fin is a customer agent.
- It will engage with a customer anywhere in that customer journey.
- To your point George, we'll have access to all of the data that's relevant for that customer.
- Effectively a very personalized experience can be crafted irrespective of where that customer is on the journey.
- In that world where the customer agent has that knowledge, content, personalized attributes, it should be able to answer a fair degree of the questions and complete a lot of the tasks when you think about using things like FIN procedures and data connectors on behalf of customer.
- And all of a sudden they don't have to navigate your organization around, well, is this a question for success or for the onboarding specialist or for the billing specialist or for the support org or for my AE?
- It's almost mind blowing how many people you might have to think about having to talk to as a customer.
- And you take away all that friction and it actually makes for much more seamless experience from a customer point of view.
- Yeah.
- I'm actually going to steal from my boss, what she said the other day is that where you silo those on your team if you were to create sort of an onboarding manager, where you gain efficiency in that from a company perspective, you're actually losing in customer experience.
- And so yes, it may be more helpful to have a very specific onboarding manager and then a very specific expansions manager.
- And they have these different scopes of work.
- They have this different remit.
- But as soon as you start to do that, you lose that efficiency in the way that the customer experience happens.
- I think that that's a really interesting way to sort of think about that trade off.
- Yeah, a really good way of thinking about it.
- Having a holistic view of the customer journey, I think is critical.
- I very much feel that, might be a bit controversial, like success only emerged as a discipline because the combination of account management and support could not deliver the value the customer wanted.
- They couldn't drive success for the customer.
- And ultimately we're all about driving success for the customer.
- That should be the mantra.
- So it's interesting in this world, I think people going forward, the humans in the organization will be valued for the skills and experience they bring to bear rather than the roles that they undertake on a traditional basis.
- And that means that organizations can think critically around how they organize for the future.
- You may actually choose to organize radically differently than you did in the past which I think is really exciting and I think it's going to open up some really new kind of organization structures that are going to be exciting going forward and quite innovative as well.
- Yeah, it's very exciting.
- Got to take us back now to because you've mentioned knowledge and content previously and we had a term here the knowledge problem.
- I like to think of it the knowledge opportunity.
- At the end of the day, like large language models are great but unless they know the specifics of your business and the specifics of your customer, they're never going to give a good quality answer.
- And we've all been focused on constraining an AI agent to only use the data that's relevant for your business and relevant to your customers.
- And that's fine if your organization is kind of expanding in a reasonably organic way but most organizations now are beginning to expand whether that's through customer acquisition or whether it's through shipping product at a rate that was never achievable before, again because of AI and all of a sudden the pace of change is radically different.
- And I think in your case, I think is it one hundred and fifty engineers pushing code and you have one person managing documentation and you have this concept of self healing knowledge.
- How are you thinking about that and how is that implemented in practice?
- Yeah, so we're, I think, battling this problem like a lot of other people are.
- How do we make sure?
- I think historically the knowledge base has always been this sort of static hub and you get the new hire to sort of read it over and update things when they come in.
- And maybe it's updated every three months or so.
- And I think what became obvious actually through our journey with Finn is that your knowledge base and your content is actually one of the most vital parts of your business at this point.
- And when I think about the product documentation, I think we're really lucky with product documentation because you have a really good source of truth in the code base.
- The code base is literally what is happening to the product.
- And so one of the things that we've been working on at Clay is to use that code base to validate the way that our docs are put together.
- And so we've set up a system using some Cloud agents and things like that where every time a support ticket is closing, we're running it through Cloud Code.
- Cloud Code has access to all of our knowledge.
- And it's updating our docs based on what customer or what Fin might have missed in that moment.
- And then we're validating that against the code base and having some confidence that that's correct because it's able to look at the code base and know that the limit was fifty and not one hundred or whatever it might be.
- And what that's allowing us to do is have this more rapid high velocity update to the way that we're doing we're updating content.
- Because like you said, if we have one hundred and fifty or two hundred engineers that are constantly pushing new features or updates to current features or whatever it might be, it's impossible to expect that you're staying up to date with that.
- And so in addition to our every intercom ticket comes through and we update it, we're also looking at, can we tap into whenever an engineer is pushing new code, can we have that run through our knowledge base and update things accordingly?
- Can we look at things that are happening in Slack and update that accordingly?
- All of these different inputs can feed into this agent and make almost a living, breathing doc as opposed to something that's static.
- So we've had this going for a few weeks now, and we've made like three hundred plus updates to our docs over two weeks, whereas we were at a cadence of maybe once a day our one person who's doing a heroic job, by the way.
- But the pace just has to be faster at this point and so we're really excited about sort of this self healing aspect.
- I think the good news, George, is that we're building capability into the product that will help with that.
- We announced Operator a few weeks back which has that philosophy of allowing you to work at speed, particularly identifying where you might need to make changes and actually generating those changes for you as well.
- So a lot coming in in that regard.
- And if you provide the right content and knowledge, obviously Finn or an AI agent will answer as many questions as possible or complete as many tasks if you're connecting to your business systems.
- But there are times when you just can't complete the task or can't answer the question.
- And how are you looking at that kind of escalation or handoff to human as a signal and how are you capturing that then in terms of driving continuous improvement?
- Yeah, it's a really good question.
- I think from the very beginning, whenever we implemented Fin, we always knew that one of the most important things to us is that we wanted to make that handoff process seamless.
- Nothing is more frustrating than wanting to talk to human and not being able to.
- And so we've always kept that as a very core part of the way that we utilize intercom or Fin.
- And so when we're thinking about how we're sort of using that as signal, there's some indication.
- You could almost go as far to say that any time Fin fails, there's something there for us to learn from.
- And maybe that's our knowledge is not up to date.
- Maybe it needs access to more systems.
- Maybe that's just good product feedback that we want to send to the product team.
- And so we're really starting to use Intercom as a signal.
- Someone on our team at DT, who's been running a lot of this Fin stuff, is actually piping all of our Intercom conversations into a clay table and then doing an analysis over them to understand was there good product feedback here, is there an opportunity to update our docs, whatever it might be, and sort of being able to, again, sort of rapidly use that customer support as a signal for every other part of the business.
- And obviously, using AI as a customer interface and then, you know, using your humans for the more value add work.
- How are you using AI to help those humans as well quite apart from, you know, Fin as an AI agent talking to your customers?
- Yeah.
- So I think that's one of the things that I've been looking really closely at recently is I think historically, I've always thought of AI and support as almost this dichotomy of either Fin is going to answer it or it's not.
- And we're working a lot on this part.
- How can we get Finn to answer it?
- I think there's a really big middle ground here of this co pilot space of how can we actually make the humans faster at the things that they're doing manually, right?
- And so we're starting to think about this copilot experience a little bit more.
- When I thought about Intercom's copilot, I was always sort of of the mind that that's sort of like a good thing for new folks, but eventually those sort of know things.
- And we're starting to think of it a little bit more broadly.
- Can you have this copilot hooked up to Snowflake and hooked up to the code base and hooked up to all of your logs?
- And you are almost in the same way that an engineer is just sort of asking the agent to do things for it.
- A support person can just ask the agent to do things for it.
- And so we're starting to explore things there.
- I think the really difficult part is, what is that interface?
- Do we just want everybody running Cloud Code?
- Do we want to use Intercom's Copilot?
- There's a lot of different ways to orchestrate that and set that up.
- But I think that the core idea is where can we hook AI into all of these systems and make it faster to get at the information that you want without having to log into AWS or whatever it might be.
- I love the concept you're describing, which is really kind of driving continuous improvement.
- When we went on the Finn journey ourselves in customer support, we really had this mantra, like the first time you answer a question ideally will be the last time, You're always challenging yourself like what prevented Finn from either answering the question or completing the task and just encouraging all of our support team to actually register, was it a content missing, was it a data connector missing and feed that into almost a continuous flywheel that we have around making improvements and making Fin better.
- And again, from a product perspective, part of that is built in now in terms of what we call recommendations within Fin.
- So again, our product team watched what our own team were doing around kind of driving teams to ruin and then decided to build it into the product.
- And we've also gone down the road of thinking like how can we improve the way that our customer support specialists or technical support specialists, technical support engineers work.
- And we've also embraced ClotCode quite actively and allowing people particularly for more complex troubleshooting where they might have to assemble a lot of information from different data sources.
- Clot is really good at that.
- And we're already getting to see some real productivity gains by exposing that to our team and allowing them to build skills that are allowing them to do deep troubleshooting.
- So again, the whole way we work has just fundamentally changed, but fundamentally it is this kind of concept of continuous improvement and always thinking about how you can tune and make things better, which ultimately is delivering a better customer experience.
- Yeah, for sure.
- I think the fun part about the Copilot experience too is that your barrier for what you can give it access to is much lower, right?
- I would never give Finn access to our code base because I don't want it to say something that's we're not released yet or whatever it might be.
- Whereas I would give any one of my sports specialists access to the code base.
- And having them as sort of a buffer to be able to filter the stuff that's coming through the AI, I think just allows us to hook it up to Stripe, hook it up to whatever you want because you have that sort of buffer period.
- So it's an exciting thing.
- Yeah.
- And we've even gone as far as allowing some of our engineers to actually fix bugs directly.
- We categorize them like we call them priority three bugs.
- But if the support engineer can solve it at the point of registration, why not?
- It doesn't have to go to the engineering team.
- Again, think differently about conversions and roles changing, even blurring between engineering and support.
- But I want to talk a little bit about the humans because we all talk about, we use the term automation, but it's really around how can you get Fin or an AI agent to complete work on your behalf.
- And if you have a conviction that you can deliver really good customer experience doing that, then you want to drive that automation rate as high as possible.
- And some people say, well, what's the goal, ultimate goal?
- Is one hundred percent automation the wrong goal?
- Is there a category of interaction where the human will always exist and always add value?
- Like, what's your kind of perspective on that?
- Yeah.
- I think, you know, we think about this a couple of different ways, Clay.
- I think, you know, generally speaking, we're trying to get to a place where we feel really good about the customer experience as a whole, right?
- And one of the things that we've talked a lot about is how does that how are our response times sort of an indicator of that?
- And one of the conversations that I've had with our head of CX is being very specific about the response times that we choose.
- Because I think that we could be thinking of like, how do we get that down to zero?
- Or how do we get that down to minute response times that is sort of an indication that like you're doing great?
- And what we found is actually that's not the model that we want to build, right?
- With minute response times, you're going to have a lot of folks that are sitting around because you have to have people ready at the wheel, sort of ready to chat with the next person.
- And we've sort of decided that actually like an hour or two is actually a response time that we feel more comfortable with because it allows us that flexibility to work with the customers a little bit more closely.
- And I think that's the same case with automation.
- At face value, you can think of, yes, one hundred percent automation rate is the goal.
- But being able to actually think critically about that, and it's like, Okay, yes, there's plenty of things that we want to automate, and we're not nearly where want to be with that.
- But getting to one hundred percent also isn't the ideal state.
- We want to figure out where that sort of happy medium is to encourage the things that we were talking about earlier of how can we make sure that our support specialists have more of an opportunity to engage with the customers and aren't penalized for that because they are not responding to enough conversations, whatever it might be, I think is the ultimate goal.
- And so we're really starting to think critically.
- And I think the tough part is it's always a moving target, right?
- We're constantly sort of driving that automation.
- We're constantly bringing on folks that are more technical and want to engage with customers a little bit more.
- And it's a little bit of a moving target.
- I think as you start to hit that plateau of automation, at some point you have to realize, Okay, there's actually this subset of questions, whether it's troubleshooting or strategic consulting or whatever it might be, are the things that you want to sort of keep with the humans?
- And then with those, how can you sort of speed them up on that and sort of hit them at both sides kind of thing?
- And as you've gone on this journey, like how are you thinking about the trigger for Finn to hand over to a human?
- Like what are some of the kind of rules of thumb you've developed around the guidance you need to give to Finn to do the right thing in terms of the handover?
- Yeah.
- I think that the handover, like I said earlier, it's it's really important that we keep that handover easy.
- Think we're starting to find, especially with things like procedures, even if you know you're not going to solve a full problem, being able to get some of that information up front can actually help a customer.
- And so we're thinking about ways that we can have intentional handoffs.
- If someone wants a refund, we maybe don't want Fin to go through and through with that in some cases.
- And so how can I actually gather the information to then speed up a support specialist?
- And so we're thinking about, you know, the handoff in very specific ways like that of how can we make sure that we're still providing that good customer experience, but then also making it, you know, as fast as possible as well.
- And, you know, you've talked about driving automation, you know, maybe not a hundred percent, but you know, I think you said there's things you still want to automate or, you know, have Finn involved in.
- What's kind of the what's still hard from your perspective?
- What what where is the focus at the moment and what hasn't moved as fast as you would expect or hope?
- Yeah, it's a good question.
- I think the things that are still hard are there's a couple of things.
- One, it's sort of deciding what to start automating.
- We know that there's a couple of repeat things that are coming through our queue, whether it's refunds or privacy and data deletion requests or whatever it might be.
- And it's sometimes difficult for us to prioritize those because they're all things that we want to engage with.
- I think the other part of it is, Okay, we know that we want to automate our privacy system.
- We now have to go and collaborate with lots of different people around the organization, whether it's the legal team or the marketing team or the engineering team to get access to the APIs or whatever it might be.
- And that process, I think, is still difficult.
- I think one of the things that I'm really excited about is we've just promoted somebody into the role who's got a little bit more of a computer science background.
- And she's sort of able to take a lot of this on as well.
- And I think that's back to exactly what you were saying of you start to have engineers actually sitting under the support org who are sort of in charge of getting us access to these things and building out the APIs and connecting the data and all that kind of stuff, again, that technical bar just becomes much higher.
- And being able to sort of have that all happening within the support org, I think, is really important.
- Yeah.
- I love the concept of being able to solve the problems that are preventing, in that case, getting data access and getting the right kind of information available to complete the task for the customer.
- Just going back to the automation piece, I have set a target for the team.
- We have this project called Path to ninety five.
- So I've said ninety five percent automation and we're currently at eighty four percent this So we have driven it quite hard already.
- But again, it's on the conviction that we can deliver a better customer experience.
- But the reason made it ninety five was similar to what you're saying.
- I still believe there are some activities that you will always want a human involved and you always want to get them involved as quickly as possible in those activities.
- Is ninety five percent the right figure?
- I don't know yet.
- I'm pretty impressed we've already gotten because we put this in place when we're at eighty percent and we've already gotten to eighty four percent pretty quickly and we're still delivering really good customer experience.
- You've got to be able to measure that customer experience and know that you are actually delivering well for the customers.
- But we did an interesting thing.
- We analyzed the gap between where we were and where we needed to be.
- Well, we actually did the analysis gap to one hundred percent and basically fifty percent could be solved by having data connectors and access to underlying business systems.
- There was still about eighteen percent that was down to content, which was kind of a little bit of a surprise, there's always kind of edge case conditions and content that this analysis surfaced for us.
- And then there was thirty two percent that I classify as a long tail of different edge case conditions, some of which may never be worth putting the effort into automating them or you don't want to automate them because really you want the human involved.
- So it's interesting to go through that exercise and stratify exactly where that gap was and what we might be able to focus on as low hanging fruit.
- Think one of the interesting things there too is that I think that the customer base is sort of changing in that respect too.
- I think that historically we used to always get the person who came into the queue, the first thing they would type is, I want not to talk to a human kind of thing.
- And as you sort of show the customer that there are actually more capabilities, that it actually has more information, that it can take more actions, you start to train that customer into, oh, actually, maybe I can do this without talking to a human.
- And one of the things that's really interesting is that when we think about the quality of our support, a really big indicator of that is obviously how quickly you get a response.
- It's not just how good that response was, it's how quickly you get that.
- And if you're getting that instantaneously, that's obviously going to be a better customer experience.
- And so I find myself doing that even in my day to day when I'm talking to an airline, I'm testing the capabilities of, can it actually book my seat?
- I don't know, maybe it can now.
- So I think that it's sort of this you get more returns as you make it even better.
- Yeah.
- I mean, I'd love if the airlines would take my international credit card.
- Yeah.
- They're not quite there yet.
- That's a that's a different a different challenge.
- Focusing a little bit on the human dimension, one thing that we have been very focused on is like how can we add more value when a human is involved with our customers.
- And I mentioned earlier, we have this concept of consultative support where you're giving your team the latitude and the autonomy to go beyond the issue that the customer is presented with and think about, well, how can I add most value to this customer using this part of the product more deeply or using in a way that they're going to avoid the next bottleneck?
- I think being able to think about that consultative approach again means that there are going to be logical handoff points where you want to hand off to your humans so they can bring that value to bear to your customers.
- Now, ultimately, we want to build some of that into Fin as well and get Fin to add that value as well.
- But certainly there's, as far as I'm concerned, a role for humans for the foreseeable future.
- They're adding huge value and all Fin is doing is actually allowing them the autonomy and the bandwidth to actually do that for customers, which is really exciting actually.
- It changes the role of support as well.
- Think people feel it's far more fulfilling role than it was in the past.
- Yeah.
- One of things that we're thinking about related to that is we've historically not given our tier one or enterprise customers access to Fin.
- We just want to it's a little bit more of a white glove experience, and we didn't have as much confidence in our Fin experience to sort of be able to help them.
- But one of the things that we're seeing is that there's plenty of things that are coming through our tier one queue that are very basic questions that Finn could be answering.
- And not just basic ones, but questions that Finn should be and can be answering.
- But the sort of result right now is that they might be waiting an hour or something for a response.
- And so one of the things that we're starting to do too is segment that question upfront and be able to say, if this is a how to question that is in our knowledge base or something along those lines, then send Fin after it and sort of be able to build in that experience a little bit more intentionally so that we can again, it's not necessarily a hand off at that point, but being able to sort of decide what Fin doing and what is Fin not doing and making sure that especially at that enterprise level, you're giving them access where it makes sense and then having them wait for a human when it doesn't.
- Yeah, we did something similar.
- We held off exposing Fin to our premier support customers, which is a kind of paid support layer.
- But then over time we have introduced it and given the customer the option to get Fin to answer the question if it makes sense.
- Very often it does.
- Now I hope you brought your crystal ball, George, because this is the party where we talk about where next.
- And if you look out and I hate to put timeframes, some people say two to three years, we've actually led on eighteen months and even that's ambitious.
- What's one thing that you think will look different in eighteen months' time when you think about this connected customer experience using AI to really manage the entire customer lifecycle and journey?
- Yeah, assuming we're all still here.
- I think one of the things that I'm starting to see that is going to become even more of a thing I'm seeing this specifically with the content is your customer is actually changing in what a customer is, right?
- And I think that the sort of very acute example of that is that we're starting to update our documentation.
- And right now, that whole self healing knowledge base I have is actually just sitting in a GitHub repo.
- And our branch team is really upset about it because it looks really bad and it's not a very pretty thing.
- But I'm actually not writing that content for human consumption necessarily other than through an agent.
- And so I think that that's going to continue to expand.
- And who you are supporting may be humans right now, but could be agents depending on what your product is.
- Your API could be changing the way that people are interacting with your product.
- The MCP is sort of an example of that.
- And I think that making sure that you're sort of building your support to do both of those things, think, is going to become probably more important as time goes on, as people start interacting through agents.
- Who that actual customer is going to change as well.
- So starting to see it now, I think it'll continue.
- I think one thing for me that I hope will be different in eighteen months is I think you will have an agent layer that has actually got really long term memory around the customer journey and you can set goals for that customer within the agent and it is constantly working to achieve those goals with the customer.
- I think that for me is where you're adding the ultimate value from an AI agent layer.
- I think the technology is evolving.
- It's our vision for Fin as a customer agent.
- We wanted to have that long term memory.
- We wanted to be able to be goal oriented in terms of the interactions with customers rather than being transaction oriented which is the way a lot of the Fintech interactions are today, they're transaction based.
- I really believe that we'll build out.
- But another thing that I think will be different and I'm not sure how we're going to actually manage this, your customer at the end of the day might be another agent and not a human.
- Again, I think we've all assumed for a long time that the person at the other end of this interaction is a human but over time the person or the entity at the other end will be an agent itself.
- Agent to agent communication and interaction I think is going be different in eighteen months and I'm not quite sure how that's going to net out yet but that's going to be an interesting place for us to be.
- For sure.
- An infinite loop of one agent trying to stay in contact with another agent.
- So yeah, that's going to be interesting.
- Now looking backwards a little bit and again for maybe people who aren't as far advanced on that journey with AI as maybe an organization like Clay, what do you wish someone had told you two years ago that would make life easier now?
- Yeah, I think probably to just start earlier.
- I think that, you know, what is it, the best time to start was like two months ago, and the second best time is right now.
- I think that, you know, Katie, who is one of the managers on my team, she really started to take this sort of Fin stuff over like three months ago, six months ago now, I guess.
- And we have made such huge gains over the past four or five months that we're super excited about.
- But we've been using Finn for a year and a half, right?
- And I think that all that stuff that we've been thinking about content and about how we think about these procedures and what we're giving it access to, I just wish we had started it earlier.
- Because I think that the more you do, the more you sort of figure out what works and what doesn't work.
- And I'm really sort of pushing my team sort of across the board, not just on Fin, but just keep doing things because you don't realize what is working and what is not working unless you're sort of in that iteration process.
- And I think it's so easy nowadays to sort of get in this paralyzed state of like, it's changing.
- Anything I build today is going to be irrelevant tomorrow, or whatever it might be.
- But the more you sort of just start something and go into Claude code and just try the thing that you've been thinking about, Because I think that that's just that rapid iteration is going be an adaptation is going be what sort of makes it good in six months from now.
- Yeah, I think when I look back and I think what would I like to have known two years ago, I'd love to have known how fast things were going to progress and knowing that I probably would have moved faster.
- Again, at the time I thought we were moving pretty fast but then when you kind of look back historically you think, yeah, we probably could have moved faster if we'd really understood how this technology was evolving, the capability was going to be available to us, the complexity that an AI agent could handle over time.
- So I think just weren't thinking big enough, think is probably the way I describe it, because it was a little bit of an unknown.
- I'd love to have known back then exactly how fast this technology was going to advance.
- George, we're about to kind of segue to Q and A with our kind of friends here tonight.
- So that's my cue to make sure that the roving mics are coming out to let people ask questions.
- Before we finish up, love for any questions from the floor for George or for myself or any comments you have so you put up your hand we have roving mics and hopefully someone will get a mic to you.
- I'm pretty fascinated by the constantly evolving documentation piece that you talked about.
- How much of that would you guess is maybe, like, the coverage of that?
- Like, do you feel, like, in general, like, all the updates that those hundred fifty engineers are pushing out, do you feel like it's doing a pretty good job of keeping most of that updated and maybe speaking to the accuracy piece too?
- I'd be curious.
- Yeah, it's a good question.
- I that I'd say it's doing an okay job.
- I think the biggest bottleneck right now is just me.
- We're doing a little bit automatically merging changes in if it has high confidence in that based on a couple of criteria versus manual review.
- Candidly, my actual product knowledge of Clay has diminished as I've been further away from the product.
- And so a lot of that stuff I'm looking at.
- And I'm like, sure, that sounds right.
- And it checked against code base, and so it's probably there.
- I think the reality is we're sort of, I think, pretty early on in the iterative process of it.
- And my hope is that it's just always becoming better and will eventually just be that coverage will be better and better.
- Will it ever have one hundred percent coverage?
- Probably not.
- Don't think what we're not doing is sort of taking our entire code base and giving it to an agent and saying, make sure everything's documented, because I don't think that would be very successful.
- So I think it's almost speaking towards what Declan said of just like, if something has been missed once, we don't want to miss it a second time kind of thing.
- And there's some hope that it will just continue to get better.
- Hi, my name is Zeeshan Jaffrey.
- I'm the head of GTM at a Y Combinator startup called Trident.
- And it's awesome to see that you have adopted agents in a huge way, both of these companies, Fin and Clay.
- I wanted to ask you, how do you address the new attack vector that AI agents introduced?
- I don't know if you saw, but Meta, they released their AI.
- Now they're giving out accounts all over the place by resetting one time passcodes through prompt injection attacks.
- So how are you addressing those security concerns?
- Yeah, it's a really good question.
- I think we're sort of in it right now.
- I think one of the things that we've been thinking about, our IT team specifically, is how can we sort of centralize agent access piece of one of the hardest parts for anyone at our company right now of building with AI is getting it access to Salesforce or Snowflake or Intercom or whatever it might be because those folks are maybe not as technical and don't want to hook up the MCP or whatever it might be.
- And we're starting to think about how can we sort of centralize that data access piece, not only to democratize it across everybody at the company, but also make sure it's done in a very secure way.
- Have we done a good job at that right now?
- Probably not.
- And I think a lot of people are just sort of using it.
- But I think it's definitely a risk that we're starting to think about a little bit more actively.
- Yeah, mean, it's like everything is, you you can never be one hundred percent sure that you're fully secure in any technology platform.
- Like, That's been the history of technology over the last forty years and why the whole cybersecurity industry has developed over time.
- We obviously built Fin.
- I would have to say the AI team and the security team at Fin are two of the most paranoid teams I've ever worked with around security and around trying to ensure the robustness of the agent environment, trying to anticipate what type of attack vectors are evolving, how we can build in capability within the agents to guard against that.
- So as has been the case for the last forty years I've been in tech, it's a battle.
- Can you outwit the people who are trying to be smarter and compromise systems and compromise new technology?
- So there's no guarantee but I think a level of paranoia is really critical.
- Particularly if you're using things like an agent for completing tasks, some people don't understand that you've already put guardrails in place for your humans who are accessing those systems.
- You need to make sure that at a minimum you're putting in the same guardrails for an agent and you probably want some more guardrails because the blast radius of an AI agent doing something would be slightly more than humans doing something but you've to just have the same level of paranoia.
- You have paranoia around your humans accessing your systems, just be as paranoid about an AI agent accessing your systems and make sure you're only exposing the attributes that actually are required to complete the task and that you're using whatever controls you would normally use anyway.
- So the good discipline that you've built up over the years should just apply for an AI agent as well, maybe with a little bit more paranoia just as I say because the blast radius is potentially a little bit more damaging.
- The guy with the hat.
- Hey, George.
- Thanks for putting this together.
- I'm a pretty heavy user of because I write a lot to the Clay support.
- What I want to say is I also see the journey and the development, which is really cool going in right direction.
- My question is, with all things AI, right, Finn was an impossible product before LLMs.
- What I'm wondering is, and I guess everybody runs into this with AI, where do you draw the line of like when the your customer doesn't put in good context or content as you referred to it, Fin will technically work, but it will not help.
- So where do you like say, hey guys, you're off to your own devices or is there a lot of enablement happening?
- How to structure content?
- Do you take, I guess Clay is probably more on the AI pill side of companies implementing that, but not everybody will be that.
- So where do you where do you how do you make your customers successful with AI where so much depends purely on the customer and not on the technical capabilities?
- Thanks.
- Yeah.
- I think that's a that's a great question, Andre.
- Thank you.
- I think he Slacked me the other day and said he's going to give me a really hard question.
- So here we are.
- I think the thing that I think about is a little bit of it is just happening naturally, sort of like what I said earlier, that the people are just sort of figuring out how to interact with agents just more generally.
- And that's not just in the tech world.
- That's just consumers in general.
- And so I think that that will just continue to happen more and more, just in the same way that most people figured out how to Google something before it was new.
- I think the other thing that I think a lot about with this is we always want to make sure that we're keeping that customer experience at the forefront.
- And one of the things that I and one of the conversations that I've had with my team before is it would be much easier if we asked everybody who was contacting to support to fill out a form of some predetermined things.
- What part of the product are you having an issue with?
- What's the specific error message that you're seeing?
- Whatever it might be.
- But that's just kind of a bad customer experience.
- Because every time you want to talk to somebody, you have to fill out that form.
- And so we've sort of taken on the hit there, the trade off that, yeah, we might not get the information that we want right up front, but we also don't want to make it we want to reduce the friction to contacting support and making sure that it's not difficult.
- And so the thing that we try to do is make sure that we're keeping that customer experience up front and, again, using procedures and AI to see if we can ask those questions with AI beforehand, like what is your table link or whatever it might be, so that we have them through the rest of the conversation.
- So yeah, it's a really good question and it is a struggle.
- Several dimensions to it.
- The first dimension is what George said, getting people to interact with an AI agent or Fin in the right way.
- So when we first launched Fin we discovered that customers were treating it like an old chatbot.
- So we try and give keywords or a very kind of succinct description of what they were doing which ultimately didn't allow the underlying large language model to function as well as it might do.
- It took actually longer to get the answer.
- So we just prompted our customers to actually describe the problem they were having and once we did that ultimately their problem was getting resolved faster and we actually saw a five percent increase in CSAT literally overnight when we changed the prompts.
- That's one way of getting people to prompt correctly.
- The other way is, again we've built in capability within FIN, it can ask clarifying questions so if the problem hasn't been presented in a way that's very clear to FIN, FIN will iterate a number of times asking clarifying questions.
- Obviously it will reach a stage where it will hand over to a human because it's just not getting to the root cause of the issue and that has definitely helped improve as well where Finn is asking the right questions at the right time and customers are beginning to explain a little bit more.
- Have a conviction and we're not quite here yet but even if the customer isn't explaining very well what the problem is, if you can understand exactly what they were doing on your platform prior to asking the question, you have insights into exactly where the problem was that they encountered and getting those type of signals from an underlying system like there are products beginning to emerge that will do that for you and if you can then build that context into FIN then Finn can use that information to infer exactly what the customer was trying to do even if the customer is not describing that in a very clear way for Finn.
- So I think the next evolution is really picking up those real time signals around exactly where the customer is interacting with your product and service, what they were trying to do when they initiated the support interaction and that's probably nine times out of ten the right area to focus on.
- I think just to add on to that, the other sort of interesting part is the other side of the conversation of once you've actually solved their question, can you infer from the product if you actually solved their question the right way.
- So in the case where somebody asks Finn a question, Finn responds, and then nothing else happens, it's really hard right now to actually know if we solved their question or if they just gave up.
- And being able to hook into the product that way helps on the other side of the conversation to determine success as well, which is kind of fun.
- Hi.
- So I guess one question around the go to market engineer discussion.
- So how are you thinking about hiring right now in terms of like finding people who are engineers or have engineering background who are also able to talk to customers like with empathy and vice versa, more people oriented people and how do you think about the composition of those teams and even like managing expectations as these roles collapse.
- Yeah.
- So that's generally it.
- Yeah.
- It's a great question.
- Specifically on support, we we actually haven't changed our our hiring process that much since the beginning because we've always sort of been of the mind that we wanted to hire really good people onto our support team.
- And and oftentimes, that meant really technical.
- And of course, we've always kept a very high bar for how well you can talk to a customer as well.
- And so I think the, you know, the thing that is changing probably the most is how we evaluate that.
- We knew we wanted to evaluate that, but we're changing the way we sort of yes, we want you to sort of navigate through a clay table, but how are you sort of thinking about troubleshooting within that sort of call that you're having?
- And what sort of things are you thinking about?
- Are you thinking almost like an engineer in the way that you think about, like, oh, I tried this, and it didn't work.
- So I'm going to try this next step, and then I'm going to debug from there, or whatever it might be.
- And I think that that's the kind of thing that we're starting to try to just do more of is make sure that we're keeping that bar very high for technicality and all that kinds of stuff.
- We have made some changes in how we hire.
- It was a time when we almost valued the relationship skills over the technical skills that we could bring people up to speed in the product pretty quickly.
- Now we're discovering that obviously the stuff coming through to our humans is far more complex, far more nuanced so we actually do need people who have strong technical background.
- We still need the relationship skills and you know that can sometimes be hard to get the two married but that's what we aspire to now from a hiring point of view.
- So we change our job profiles within the support organization, change our hiring process.
- Yeah.
- I'll also say actually we don't outsource any of our support either.
- So I think that's probably a big shift that's happening in industry is that there's some indication that things that you were historically outsourcing to are probably things that you can get AI to do now.
- And so it reduces that need.
- I think we have time for maybe one or two more questions.
- Yeah.
- I have a rather Oh.
- Is that Oh, sorry.
- Thank you.
- Oh.
- Alright.
- I guess you'll be next.
- Well, thank you.
- Sorry.
- I'm sorry for that.
- Thank you both.
- You're next.
- Probably a little bit more for Declan, although, George, you mentioned something as well.
- Declan, you're referencing that you're going from eighty four to ninety five percent automation.
- Right?
- And, George, you're referencing that I think, like, FCR, AHT as, like, the metrics of quality, right, may not hold anymore.
- I'm curious, Declan, how do you know like what's that ninety five percent is good, not just high?
- So from our point of view, like from very early on on this journey, I had this really strong conviction that I wanted to be able to measure every single customer interaction and assign what we call a customer experience score to it.
- I didn't want to rely just on CSAT or any other measure like NPS or customer effort score because they tend to be survey based.
- You don't get a huge response to them.
- They're not giving you a full lens on the experience you're delivering to your customers.
- So internally within the support team in Fin we built our internal version of customer experience score so for every single interaction we could determine how do we deliver to the standard that we have set for that particular interaction.
- And we then worked with the product team to build it into the product.
- So customer experience score is part of the standard Finn product now.
- Or that's right it's called pro it's kind of an add on for fin but it'll give you a customer experience score.
- And then we did a further iteration on that and it's a thing called monitors.
- Now monitors you can use it for quality assurance but you can also use it to develop your own custom customer experience score.
- Because we dictate customer experience score in the products of using our standard customer experience score.
- You can't change the attributes and you can't change the weighting.
- And that allows us to do benchmarking across customers, which some customers value that benchmark information.
- But with monitors you can actually build a custom customer experience score.
- You decide the attributes, you decide the weighting.
- You could have a different scorecard for different customer segments.
- You could have a different scorecard for different products.
- You might have a specialized scorecard for when you launch a product because you might want to go deeper in terms of that customer experience.
- So being able to measure every single customer interaction and design a set of attributes and weights that are important for you, that's how we know whether I'll use DevTrainer again like he said you know when you've gone too far and you know you will have begun to damage the customer experience.
- Once we see that we know we've gone too far Yeah.
- But we're paranoid about measuring it.
- Yeah.
- Yeah.
- The only thing I'll add is that with the customer experience score that Intercom has, they also give you these reasons, which have been really helpful for us.
- Because I think, you know, oftentimes if someone has a bad experience, it's actually not support's fault.
- It might be that, we didn't offer a product thing that they wanted or whatever it might be.
- And so being able to look at those reasons also helps us sort of delineate that feedback between what can we make better as opposed to what can we feed to the product team or something along those lines.
- And I think this will have to be our last question.
- Okay.
- Oh yeah, it's working now.
- I have a bit of a pointed how to question.
- So in my organisation, the customer team is sales, success and support, and that's across tiers, so tier one right the way through tier three.
- So our customers have one person that they know to email regardless of what user you are.
- Our challenge now is how do you start that triaging to Fin to provide that support and that experience when they are emailing directly?
- Is there a way that you've implemented it which is successful, starting off with tier one?
- How does that workflow or that that experience look from a customer perspective?
- Yeah, so I think I would say that the first thing you have to sort of figure out is what are the things that Finn is good at answering as opposed to what is, you know, that Finn is not good at answering.
- Right, so it may be, you know, if it's sort of this, tier one type thing where someone doesn't want to just has a how to kind of question, you can categorize that up front using I think they're Fin attributes You can categorize that up front and then determine what happens in your workflow based on what that sort of AI categorization was.
- So the very specific way that we do it is we built out a flow where if it is more of a how to question and again, it uses AI to sort of categorize it as how to or bug strategic, or whatever that might be.
- You can sort of write in prompts for each attribute.
- And based on what happens there, then you sort of either send Finn after it, or just send it straight to a human, or whatever it might be.
- But it really starts with identifying what those sort of classifiers are of what you feel confident Fin would be able to answer.
- Yeah and to add to that you can actually deploy Fin on email as well.
- So it doesn't require the messenger interface.
- You can deploy an email so you can actually do that characterization on the email and decide actually this is something can handle and Fin can do it or no this is something Fin can't handle I go straight to a human.
- Yeah, can make it very transparent.
- That's one thing about like an AI agent, should make it transparent, like a customer should know they're talking to an AI agent and not a human.
- We actually we took a while to add email into the mix because it's a little bit of a weirder experience.
- You sort of expect it over Messenger but not email.
- But we found it's actually, you know, it's been pretty positive for us because again, it gets them that answer much quicker than it would otherwise.
- We've got to come to the end of this part of the evening but we do invite you to stay a little bit longer with us for some drinks and networking.
- Prior to that though we're going put a QR code up here right behind me, right on queue.
- We would love if you would scan the code and give us some feedback on this event.
- We're all the time trying to iterate these events and understand how we can add most value to people who come and attend so your feedback is really important to us to make sure that we can the next time out improve on things and hopefully make whatever changes are necessary to make this a more fulfilling evening for each of you.
- Again, want to really thank you for taking the time to spend with us this evening.
- George, a huge thank you to you for the insights that you provided because I think Clay has been at the forefront of this kind of connected customer experience and scaling CX with AI.
- I love every conversation we have because I learned so much.
- You give me lots of good ideas, things that I can bring back to my organization as well.
- I very much feel we have a really strong partnership in that regard.
- So thank you for sharing this stage here tonight.
- We really appreciate the insights.
- And with that, I say enjoy the rest of the evening.
- We're around here for any questions that you have for the next while.
- There's some drinks and food, etcetera.
- So please do stay with us if you can.
- If there's a question you didn't get to ask, just find us.
- We'll answer it as best we can.
- With that, thank you very much.
- Thanks everybody.
- George, thank you.
- I appreciate it.
Transcript: Clear communication during an AI rollout
- Bringing AI into your support team changes the work.
- All this change can make people feel like they're losing control of their job.
- So don't just plan for one big announcement.
- You must communicate early and often on three things.
- What happens to the extra capacity if the AI frees up?
- How the role itself is going to change?
- And what's expected of them going forward?
- Even if you don't have all the answers yet, tell your team what you do and don't know.
- People would rather hear we're figuring it out than fill in the blanks themselves.
- I wrote a blog post all about this.
- You can read in the article section of the Blueprint Hub.
Transcript: Making your AI Agent sound like your brand
- The goal is to make AI agents sound more human rather than sound like AI.
- At the end of the day you want your AI agent to emulate your top performing reps.
- You should identify your top performing support reps and you should analyze ten conversations across a range of different topics and subjects.
- The top three things you're trying to pull out of the conversations are how are your support reps engaging and talking to your customers, how are they displaying empathy, and how are they dealing with difficult situations.
- It's probably the best definition of the brand that you're going to get and you can then use that to shape how your AI agent sounds and behaves.
Transcript: CS teams are product teams
- Some companies treat their customers like a nuisance, but your customer experience is actually just as important as your product and should be viewed as a product itself.
- With AI handling more and more customer conversations, the job of your customer support team is changing.
- They no longer just manage support tickets.
- They're designing an experience, understanding what good looks like, and driving real value for your business.
- The big shift is to stop optimizing for output and start thinking about outcomes.
- You need to treat your customer experience like a product.
Transcript: Talking to your CISO
- Want your CISO to approve your AI agent faster?
- Well, the biggest mistake that I've seen is bringing them in too late.
- So before you ask for sign off, try to explain three things.
- First, what the AI agent does.
- Second, what actions it may take.
- And third, what customer data it touches.
- This changes the conversation from can I trust this AI agent to how can I deploy this AI agent safely?
- And it's much more likely to land with a yes.
Transcript: Expanding CS
- We think about traditionally, support was maybe this more reactive place where customers would go and you would answer their question, then you'd be done.
- And a CSM is a relationship manager.
- You might have an expansion manager, whatever it might be.
- And I think that between support and success, what we're finding is that as we're sort of taking away the I always like to call it automating the ******** that line between where support is able to go to and where success is really starts to blur.
- We're not thinking necessarily about what is this person's role versus this person's role in sort of passing customer around.
- We're just able to sort of help them in the best way possible, which I think is the most important thing.
Transcript: Incorporating AI
- A lot of people, myself included, before we went with AI, mean, we sat there and said, we're never doing AI.
- Never.
- We are not bringing bots into the mix.
- Well, then the volume exploded.
- It's taking three to four or five days to respond.
- And when you do, it's a macro.
- May or may not actually solve their problem because you honestly don't have the time to really help people in the way that they deserve.
- We eventually it was like, okay.
- Well, we're gonna need to either hire, like, two hundred people or we're gonna need to incorporate AI.
- We really made the most of that.
- What Finn was resolving for us was great.
- Those were, you know, like, a lot of, like, frontline tier one style conversations and questions.
- But then the things that were coming to us, we had more time to actually spend in those tickets and making sure that we're building those relationships and the trust with our users to be more human.
- It it became more human.
Transcript: Certs and compliance
- Do not let AI certifications overwhelm you.
- When evaluating an AI agent, the security review is actually quite the same as any other security review that you would do for any other piece of software.
- It's about security, it's about privacy and how customer data is being handled.
- The AI part of the security review is actually quite small.
- It starts when you when you look at hallucinations, prompt injections, and how often the AI agent gets tested.
- So get your fundamentals right and then start looking at the AI risks later.
Transcript: Qualify leads everywhere
- What I sort of realized through our support work is that there's actually opportunities to qualify everybody at different points within their sort of customer journey, right?
- We sort of use Clay to sort of qualify folks, and we were able to sort of expand that to different parts of our customer journey.
- So if someone's in the support queue and they ask us for a demo, we can do that same qualification.
- If somebody is talking to a growth strategist, we can understand that on the support team and get them assigned to the right person very quickly.
- And so it's sort of this idea that qualification doesn't just have to be sort of in that beginning of the journey.
- It sort of happens everywhere.
- So if somebody asks Finn for a demo, it does the same workflow of seeing if they're qualified, and if they are qualified, who they should be talking to, and routing, and all of that.
- I think that's just indicative of thinking much larger about the customer journey as opposed to just kind of boiling it down to individual pieces at different parts and sort of siloing those different pieces.
Transcript: Exceeding expectations
- This is the first chatbot that gave me a correct answer.
- We're doomed.
- We didn't know how people would rate a bot.
- When we got last week, I kept it and I screenshot it, and I put it in every Slack channel I could put on.
- It was a very high paying client.
- They got their instant response, so the conversation got closed within three minutes.
- And the guy turned around and he goes, this is the first chatbot that gave me a correct answer.
- We're doomed.
- But it was the best we're doomed I could have had because when I can show that to our our VPs, they're kinda looking at, oh, it like even for our highest paying clients, this is they're happy.
- They're three minutes done.
Transcript: AI reactions
- How did your teams react?
- Claudia, how did the the wider support organization react when you were launching this?
- So actually, find more so that our support team were actually quite excited by Finn being able to take away those simple things and they didn't have to deal with them constantly optimizing or suggesting optimizations.
- And, yeah, I don't I don't think it was maybe as negative as it might be.
Transcript: AI roles in CS
- We have changed job descriptions, so we have our knowledge manager now and his work is is changing everything when we need to have it changed all of the time.
- We have Aloysius who's in the APAC time zone and she's doing all the workflows.
- That's a complete change for her.
- She was a support agent.
- Now she's rarely touching a ticket.
- So it is different, but they've bought into us because we were positive about it all of the time.
Events
Blueprint Roundtable: B2B support leaders who made AI work
Victoria Lowney of Inside Real Estate and Alex Rivas of HoneyBook rebuilt how their teams serve customers around AI. Join them and host Junan Pang for the decisions, trade-offs, and lessons behind that change.
October 28, 2026Virtual
RegisterStay up to date with us on LinkedIn
Follow us on LinkedIn