Launch it

AI fundamentals:
An introduction to AI Agents for customer service

What is an AI Agent for customer service?

A fully autonomous customer service agent that can manage all customer interactions from start to finish – without human intervention – using natural language, business context, and product knowledge. AI Agents can seek information, make decisions, and take action to comprehensively resolve even the most complex queries and requests.

Why are they different from chatbots?

Benefits of using an AI Agent

The key benefits of adopting an AI Agent revolve around customer experience and operational efficiency:

• 24/7 supportAI Agents are available around the clock, in any language, on any channel. This ensures your customers receive timely, conversational responses, regardless of where they are in the world.
• Fast resolutions at scaleCustomers don’t need to wait hours, days, or even weeks to get an answer or resolution to their question.
• Increased self-serviceAdding more ways for customers to get answers on their own at any time and on any channel boosts satisfaction, shortens time to resolution, and helps drive activation by making high-quality support instantly accessible.
• Enhanced agent efficiencyBy enabling the AI Agent to handle the bulk of questions, you can do more with less, and at a lower cost. Human agents can dedicate time to the highest value interactions, optimizing the system, and even shift focus to other areas like customer success.
• Competitive advantageAI Agents provide a differentiated support experience, so you can offer better support than your competitors and drive customer happiness, loyalty, and retention.

AI Agents create significant business value: reduced support costs, increased resolution speed, and improved customer experiences. They’re a strategic investment for modern support teams.

How do AI Agents work?

AI Agents:

• Deeply understand customer intentUsing large language models (LLMs) and advanced natural language understanding.
• Resolve queries and personalize interactionsUsing customer data, conversation history, and real-time context.
• Generate contextually accurate answersUsing a technique called retrieval-augmented generation (RAG), AI Agents pull relevant content from your support knowledge base and other defined sources to craft precise, brand-aligned responses.
• Know when to hand over to a humanWhen AI Agents aren’t able to resolve a query, they know the right time to get a human in the loop and can seamlessly hand conversations over to the right teammate.
• Continuously improve through feedbackThey create feedback loops, surfacing content gaps and failure points so teams can close them proactively. This reflects a closed-loop system where AI is both operational and diagnostic.

It’s important to note that not all AI Agents are built the same way. For example, Fin is built on a proprietary Fin AI Engine™, which is a patented AI architecture purpose-built for the scale and complexity of customer service. It includes:

• A custom retrieval-augmented generation (RAG) system, which understands the question, identifies, and ranks content by accuracy and relevance.
• Multi-stage validation, which checks every answer for quality, accuracy, and policy fit.
• Modular sub-agent architecture, which breaks each query into specialized tasks, each handled by a tailored LLM sub-agent to deliver the highest quality answers.

Getting started with AI

There isn’t a one-size-fits-all approach to AI Agent deployment. Factors like your industry, company size, and customer type will influence the route you take. But regardless, the first step is to set the AI Agent live.

Two possible approaches here are:

Step 1:
Build a business case
and quantify ROI

1. Identify high-value, quick-win use cases for AI

Start by aligning on goals for AI

• What problems are you trying to solve?
• How do those goals support broader company objectives – whether that’s scaling without additional headcount or improving the customer experience (or both)?
• How will you measure success? It could be resolution rate, CSAT, time saved, or other metrics that tie directly to your team’s objectives.

Defining this early helps ensure alignment and makes it easier to prove value later on.

Honeybook

The key to getting buy-in was to take a very clear and thoughtful approach, tying the AI tooling to specific team goals and challenges, and showing through initial testing how it could address them.

Analyze your existing conversation data to find high-value or quick-win use cases to automate

Ask:

• What questions do we see again and again?
• What % of support volume is repetitive or predictable?
• Where are the biggest delays happening that AI could speed up?
• What gets handed over to human agents that shouldn’t be?
• What questions could be easily answered with content you already have available?

Involve your team. Their insights can help pinpoint where improvements will have the biggest impact.

Look for workflows where AI can either fully automate resolution, or significantly reduce human effort

2. Prove the basic economics

What traditional support economics miss

Legacy models typically focus on a single metric: dividing total support cost by the number of conversations (or cases) handled. But this is an oversimplification. Here’s why:

• Conversation types varySome queries are simple, others complex and time-intensive.
• Human resolution rate isn’t 100%It’s easy to assume that human support resolves every customer issue, but just because a conversation was handled doesn’t mean it was fully resolved.
• Team costs go beyond salariesOverhead, tools, training, and attrition all add up.
• Automation value depends on qualityIf your AI Agent doesn’t resolve queries effectively, it doesn’t save money.

How to measure the real value of AI support

Operating in an AI-first world means you have to think about value differently:

The work you automate determines the value

Understand your fully loaded cost per conversation

You don’t need to model every line item to understand the economics. But you should recognize that your true human cost per conversation goes far beyond salaries:

• Benefits and taxes.
• Software and equipment.
• Management overhead.
• Coaching, training, and QA.
• Attrition, backfill, and onboarding.

Factor all of this in and you’ll get a more realistic cost baseline.

Note: Industry benchmarks put the average cost per conversation well north of $1, and often much higher for complex queries. That’s your breakeven point. Once your AI Agent starts resolving conversations below that threshold, it’s not just cost-neutral, it’s ROI-positive.

This is why performance is such an important component to understanding the value of an AI Agent. If it doesn’t resolve, it doesn’t return.

Look at outcomes, not just unit cost

For example, an AI Agent that costs $1 per resolution and resolves 50% of conversations is far more valuable than one that costs $0 per resolution but only resolves 30%, because unresolved conversations still require human follow-up, which costs time, money, and even customer satisfaction.

You also need to consider the total cost of ownership – onboarding, integration, training, and maintenance all affect the true ROI.

A simple formula to calculate cost savings

Once you’ve accounted for the variables above, you can use this high-level formula to estimate your potential savings:

Here’s an example of what this might look like:

This is just a starting point, but it gives you a clear way to model potential economic impact and begin forecasting the value of AI in your support organization.

If you want to get a sense of what this could look like specifically for you, use our ROI calculator.

Value beyond cost savings

• Scalability stallsYou can’t grow support at the pace of your business.
• Budgets inflateHeadcount costs rise linearly.
• Customer expectations outpace your teamCustomers expect instant, AI-enabled support – and churn if they don’t get it.
• Your agents burn outRepetitive tasks drive attrition.

When you adopt AI fully, however, the equation flips:

• You decouple support growth from headcount growth.
• You reinvest human time where it matters most.
• You redesign support as a strategic lever for growth.

• Driving deeper product adoption.
• Increasing retention and LTV.
• Turning support into a value driver.

This shifts the cost/value equation in a meaningful way:

• Higher cost per human conversation.
• Lower total cost per resolution overall.
• Much higher value per interaction – whether delivered by a human or an AI Agent.

3. Decide whether to build or buy an AI Agent

Intercom

I get the appeal of building your own system – it’s fun, it’s a great learning experience, and there’s something special about shipping your own AI code. But my advice? Channel that energy into your own product, not a non-strategic side quest.

You should only consider building if:

• You have a specialized AI team.
• You operate at massive scale.
• You have extremely niche support needs.

• You think it gives you full control.
• You believe it’ll be cheaper over time.
• You want something tailored to your use case.

But that control comes with a heavy operational burden. It means building and maintaining a full system. You’ll need:

• A retrieval-augmented generation (RAG) pipeline for high-quality responses.
• Query rewriting and content ranking to structure ambiguous customer questions.
• LLM orchestration logic across vendors and model versions.
• Confidence scoring, handoff frameworks, fallback logic, and human-in-the-loop guardrails.
• A process for feedback loops, versioning, regression testing, and ongoing performance improvement.
• A team of the right people who are keeping up with the technology, moving fast, and adapting.

The bar to beat best-in-class solutions is extremely high

• Faster time to deployment and ROI.
• Ongoing improvements built on usage data from multiple customers.
• Reduced risk around performance, security, and maintenance.
• The ability to focus your teams on higher-leverage work: improving content, training the AI Agent, and analyzing impact.

Anthropic

If you're debating whether to build your own AI solution or buy one, as a fast-growing company in a complex space, my advice would be to buy – and specifically, buy Fin.

Step 2:
Evaluate the AI Agent

Define your criteria for selecting
and evaluating an AI Agent

Entry criteria

1. Capabilities

• Task automation, not just answering FAQ-style questions.
• Personalization using customer data.
• Ability to guide and control the behavior of the AI Agent.
• Multi-language, multi-channel support.
• AI insights for real-time customer experience analysis.
• Seamless handovers to human agents.
• Transparency and control over the end-to-end experience.

Why it matters: Capabilities only create value if they lead to real outcomes. The most important measure of any AI Agent is its ability to fully resolve conversations – consistently, accurately, and at scale. That's the foundation for ROI, customer trust, and long-term success. Just as critical is transparency and control: teams need to understand how the AI Agent makes decisions, guide its behavior, and ensure it's operating in line with brand, policy, and quality standards. Without this, even advanced features can introduce risk and undermine trust.

2. Platform fit

Integrations and customization

• It connects with your helpdesk, knowledge base, CRM, and analytics tools.
• It's extensible via APIs, webhooks, or SDKs.
• It can reflect your business logic, data sources, and routing rules.

Security and compliance

• The vendor meets privacy and data protection obligations (e.g., GDPR, CCPA, HIPAA.).
• Certifications like SOC 2, ISO 27001, or ISO/IEC 42001 are in place.
• It can isolate or redact PII.
• It can support SSO, RBAC, and audit logs.

Why it matters: The AI Agent you choose needs to meet your company's standards. Platform fit ensures a safe, scalable deployment.

Evaluation criteria

1. Business performance

• Resolution rateConversations fully handled by the AI Agent with no human involvement.
• Deflection or containment rateConversations handled without reaching your team.
• Time savedHours of manual work offloaded from your support team.
• CSATIf you're testing in a live environment, track how comparable customer satisfaction with AI is to human-handled interactions.

When it comes to measuring AI's impact, we recommend:

1. Focusing less on deflection, and more on resolution

While helpful as an early signal, deflection is a limited, and potentially misleading, measure of success. What matters is whether the issue was resolved. With AI Agents like Fin now capable of resolving the majority of queries, we need to shift our focus from deflection to outcomes.

2. Measuring customer satisfaction across all conversations with AI

Across all Intercom customers who use CSAT, just 8% of conversations get scored, whereas AI can analyze 100% of customer conversations (involving both AI Agents and humans) in real time. If you're evaluating Fin, it assesses three critical dimensions of customer conversations:

• Resolution statusWas the issue actually resolved? And if there were multiple issues, were each of them resolved?
• Customer sentimentHow did the customer feel throughout the interaction?
• Service qualityWas the response clear, helpful, and efficient?

Why it matters: Business value comes from resolving the right volume, reliably, securely, and at scale.

2. Conversation quality

• AccuracyDoes it understand intent and deliver the right answer, every time?
• BehaviorDoes it know when to clarify or hand over to a human agent? Can it represent your brand's tone of voice and adhere to your company's policies?
• ExperienceDoes it create a smooth, fast, and satisfying experience for both customers and support teams?

Why it matters: Great AI Agents build trust by communicating clearly, responding appropriately and accurately, and staying on brand. Incorrect or vague answers, poor tone, or robotic behavior can undermine confidence.

How to run an evaluation

This four-step process helps you apply both evaluation lenses – business performance and conversation quality – in a structured, outcome-driven way.

Note: You don't need to run a multi-vendor evaluation to make a confident decision. In many cases, a single-threaded proof of concept (POC) with the strongest-fit solution is the fastest and clearest path forward. It gives you more control, lets you go deeper, and builds a stronger signal around how the AI Agent performs in your real-world environment.

Step 1: Define what success looks like

Note: We do not recommend tracking metrics in isolation. By combining both quantitative and qualitative metrics, you'll get a more complete view of the AI Agent's impact. This approach will also help ensure the evaluation isn't just about hitting numbers, but also about demonstrating real-world fit and usability, addressing both executive and customer concerns.

Clay

Keep what’s important to your business up front and center. For us, that was transparency and control over the customer experience. Focusing on the end goal helped us come to the right decision because we knew what was important to us

Step 2: Build a realistic test environment and train the AI Agents

Once you've defined what success looks like, you can begin testing.

1. Set up your test environment using real customer questions and your current knowledge base or help content

Source a range of customer conversations to test against:

• Complex queries that typically require multiple touchpoints from different team members.
• Vague queries that don't contain any "real" information and require further clarification to resolve.
• Edge cases that have been difficult for your human team to resolve.
• A few sensitive scenarios, such as billing disputes and cases where customers have become frustrated.
• Examples of queries in different languages, if you provide multilingual support.

Take this a step further and prepare variations of the same questions to test how the AI Agent handles different types of communication:

• Difficult questions that require information from multiple sources to answer.
• Different phrasings of the same question.
• Incomplete or fragmented queries.
• Questions with typos or grammatical errors.
• Conversations with various levels of formality.

2. Train the AI Agent

Prepare your knowledge base or help content

You need quality content for an AI Agent to deliver good results. Assess your knowledge base content for:

• CoverageMake sure you have adequate coverage for the testing cohort to give the AI Agent all the information it needs to address key questions and topics. For example, if you want to test whether the AI Agent can fully resolve queries on a specific topic, like your accounting product, or for a specific audience, like your Freemium users, it must have access to relevant content for both.
• AccuracyTo prevent the AI Agent from learning outdated information, make sure what you're exposing it to is accurate and up to date. For example, if your return policy has changed from 60 days to 30 days, update this.
• StructureThe more straightforward and comprehensive your articles are, the easier it will be for the AI Agent to consume them. Focus on simple language and an easy-to-scan structure.

You don't have to reformat or rewrite all your help content before running tests. This is just something to be aware of and potentially return to should content gaps emerge during the testing or you spot any glaring issues.

Configure the AI Agent's rules, tone, and behavior

Set these parameters in advance to ensure you're testing the AI Agent in a way that reflects your brand, policies, and expectations. This will give you a more accurate view of how it will perform in your real-world environment.

Step 3: Score performance and analyze results

Run your test conversations through the AI Agent and evaluate results through your two performance lenses:

• Business performanceHow well does it deliver results?
• Conversation qualityHow well does it communicate?

Business performance

Conversation quality

Important note: you should also evaluate the vendor, not just the AI Agent

Ask yourself: does this vendor feel like someone who will help us reinvent customer experience, or just someone selling software? Great AI Agents are backed by great partners. Look for vendors that are obsessed with support, transparent about how the technology works, and committed to co-building the future with you.

Clay

We're in a world where lots is changing and we're still learning about AI. The most important thing I would express to other CX leaders is that it's crucial to have the right partners alongside you to teach you what you need to know to be successful in this world.

Step 4: Decide whether the AI Agent is the right fit

• If business performance is low, you'll struggle to show ROI.
• If conversation quality is bad, customer trust may suffer.

Ultimately, the AI Agent you choose should be the one that fits your goals, supports your team, and will help you scale sustainably.

Deploy your AI Agent

Build a deployment plan

Deployment scope

Start by identifying the use cases you want your AI Agent to handle. Map out the ones you'll want to address now, next, and later. This gives you a roadmap to full automation, but also keeps things grounded by outlining the initial deployment targets – the conversations your AI will handle first. (You can use the use cases identified in your business case to get started, but now's the time to think beyond that.)

Rollout model

There are two possible approaches here:

• Fast-trackLaunch to a broad audience and iterate in production.
• Phased rolloutStart with specific topics, user segments or channels and expand in waves.

What you choose will be dependent on your company, industry, and customers.

Here's a way to think about this:

If you choose to phase, here's a model for expanding gradually:

• Phase 1High-volume topics, low-risk customer segments, specific channels.
• Phase 2Related topics, broader customer groups, new channels.
• Phase 3More complex scenarios, edge cases, all customers, all channels.

Prepare your team

A successful deployment requires clear accountability. Assign owners for:

• Content qualityEnsuring your AI Agent has the right information, in the right format.
• System configurationManaging integrations, workflows, and behavioral tuning.
• Performance trackingMonitoring early results and identifying opportunities to optimize.

Determine how you'll measure success

Prepare your AI Agent for production

• Content

  Knowledge your AI Agent has access to.

  Use this comprehensive Knowledge Management Guide if you're looking for detailed guidance.

• DataConnect relevant data sources, like your CRM or billing platforms, so your AI Agent can deliver contextual, customer-specific responses.
• BehaviorSet behavioral rules and determine its tone of voice. This ensures your AI Agent speaks with the same clarity and consistency as your best human agents.
• OrchestrationConfigure the high-level settings for the AI Agent – what languages it speaks, what channels it's available on, and how it handover to a human agent when required.

Launch to customers

• Is the AI Agent available in the right channels?
• Are customer segments clearly defined?
• Is the introduction message clear and on-brand?
• Are handover flows between humans and AI Agents easy and reliable?
• Are feedback loops (e.g. CSAT surveys) in place?

When you're happy with the experience, deploy your AI Agent and begin monitoring performance. If you're approaching the launch in phases, use this step to launch Phase 1, validate assumptions, learn fast, and build momentum.

Apply your success framework

Now that you're live, use the success metrics you previously defined to track real-world performance, for example:

• Resolution rateAre you hitting your benchmark for end-to-end automation?
• Customer satisfactionIs CSAT or your Customer Experience Score steady or improving?
• Efficiency gainsAre you saving time and reducing manual volume?

If the results are off-track, don't panic. Most early issues stem from content gaps, configuration issues, or data access. This becomes the next part of the process: moving from deployment to always-on monitoring and optimization. We'll break that down in the next chapter.