The four roles behind a high-performing AI Agent

1. AI operations lead

Owns AI performance, every day

Think of this as the new “air-traffic controller” for your AI Agent. The AI operations lead treats the AI as a living system that needs constant supervision, evaluation, and tuning to stay in shape. They’re responsible for what every leader ultimately cares about: quality, reliability, and ongoing improvement.

They see the whole picture: conversation quality, missing knowledge, flawed assumptions, unexpected failures, new opportunities for automation, and the subtle signals that the system is beginning to drift.

What this role does day to day

1. Reviews AI conversations and surfaces performance patterns

The AI ops lead monitors patterns in the AI Agent’s behaviour – the shift in tone that creeps in after a product launch; the sudden dip in resolution for a specific intent; the clusters of conversations that reveal a new customer behavior. They watch the system the way an air-traffic controller watches a radar, scanning for anomalies, trends, and early warnings.

Their job is to understand what the AI is doing right now, not what it was doing last week. AI performance plateaus without intentional ownership, and you can see this play out in teams that don’t have someone in this seat. What starts as a 2% dip becomes a 10% drop within days.

2. Prioritizes fixes and improvements

Once patterns emerge, they triage the required fixes like a product team handling bugs.

• Missing or incorrect content? They route it to the knowledge manager.
• Behavioral issues? They adjust guidance and guardrails.
• Action or system issues? They partner with the automation specialist.

They create the connective tissue that turns individual fixes into compounding improvements.

3. Defines and maintains AI guardrails

Leaders everywhere worry about AI doing things it shouldn’t. This role is the answer to that fear. The AI ops lead establishes the clarification logic, escalation rules, “never answer” policies, and safety boundaries that define what the AI can (and cannot) do. They protect customer trust by ensuring the AI always behaves within safe, predictable limits.

4. Aligns reporting with leadership

The AI ops lead also shares clear reporting on how the system is performing. They track resolution rate, CX Score, CSAT, automation coverage, and hours saved. They make the economic impact of AI visible, which is something our Blueprint research identified as an essential early step in every successful implementation.

Why this role exists now

AI systems are dynamic and they require constant tuning. A small dip in quality can quickly become a big operational issue. And no existing role – not support ops, not QA, not frontline managers – naturally owns this work. Someone has to. When they do, teams feel the benefit almost immediately.

2. Knowledge manager

Builds and maintains the structured knowledge AI depends on

So many leaders have said some version of the same thing: AI is only as good as the content you give it.

The knowledge manager (or AI knowledge manager, as we call this role in Fin) is responsible for that content. The role is quickly becoming more about knowledge strategy than just knowledge management – it has evolved into a blend of content designer, systems thinker, and information architect. Their job is to build and maintain the knowledge scaffolding that allows the AI to answer accurately, consistently, and safely.

What this role does day to day

1. Writes, maintains, and improves support knowledge – continuously

The knowledge manager updates articles after every product change, removes duplication, resolves contradictions, and eliminates the “knowledge debt” that silently erodes AI accuracy.

This is daily upkeep, shaped by how the AI performs. When the AI flags gaps through patterns or errors, the knowledge manager addresses them directly.

The goal is to design knowledge so well that the AI always has what it needs.

2. Structures knowledge for AI, not for browsing

Traditional help centers are built for humans skimming pages. AI needs something different: clean intent signals, crisp formatting, and clearly structured language. A knowledge manager spends as much time designing structure as they do writing content.

3. Works hand-in-hand with AI ops

Performance issues often stem from missing, outdated, or unclear knowledge. When the AI ops lead uncovers a pattern – recurring misunderstandings, escalating confusion, low-resolution categories – the knowledge manager resolves the underlying cause at the source.

4. Ensures accuracy and compliance at scale

The knowledge manager ensures every piece of content is correct, current, and compliant with policy and regulatory language, which becomes especially important as AI handles more sensitive or high-risk scenarios.

5. Develops a cross-functional knowledge strategy

This role is critical to creating strong cross-functional alignment around the content that will fuel your AI Agent’s success. Think of it as establishing a canonical version of truth of how you talk about any product or feature that can be used by engineering, product marketing, and go-to-market teams, as well as being used by support (AI and human). 

Why this role exists now

This is one of the highest-leverage roles in an AI-powered support org. Teams like Rocket Money and Anthropic are hiring knowledge managers because AI accuracy depends on the quality of knowledge feeding it. Without this role, resolution rate caps out early and never climbs.

3. Conversation designer

Designs how the AI speaks, clarifies, and interacts

AI is no longer a tool customers “use.” It’s a representative they interact with. Tone, clarity, pacing, and conversational structure matter more than ever, especially in voice. Every word shapes how customers perceive expertise, trustworthiness, and brand. The conversation designer (or AI conversation designer, as we call this role in Fin) ensures the AI feels human-friendly without pretending to be human, which is a balance that builds trust without misleading customers.

Our own support team learned this early: conversation design was one of the first roles we staffed when adopting Fin internally. It changed not only how we tuned AI, but how we understood the customer experience end to end.

What this role does day to day

1. Shapes the AI’s tone, voice, and communication style

A conversation designer ensures the AI’s language is consistent, clear, and human-like enough to feel trustworthy. They refine phrasing, tune politeness levels, adjust how the AI handles confusion, rethink how instructions are delivered, and shape the micro-interactions that define whether a customer feels cared for or dismissed. On channels like voice, this becomes even more important because natural cadence directly influences the customer experience.

Their work directly improves resolution, trust, and customer satisfaction.

2. Designs flows for high-value conversations

This role thinks deeply about the moves a conversation can make: how the AI clarifies intent, how it handles branching logic, how it communicates uncertainty, how it verifies information, when it escalates, how it hands off, and how it returns to the main thread without feeling mechanical. Conversation designer is one of the core roles that really treats customer experience as a product – they design the logic behind conversations in the same way a UX designer shapes product flows, but with language rather than screens.

3. Translates procedures and complex workflows into natural language and logic

As AI gains the ability to run structured procedures and actions, this role evolves into a kind of conversational system architect. They translate operational SOPs into natural-language instructions enriched with conditional logic, exceptions, fallback steps, and edge-case handling. This is where conversation design intersects with systems thinking; they ensure the AI can follow multi-step flows without creating confusion or risk.

They also repeatedly test these conversations. For example, at Fin, our conversation designer uses Simulations to run simulated conversations to see where the AI Agent gets confused, over-confident, or awkward, and refine flows until the interaction feels effortless end-to-end.

4. Ensures transitions to humans feel smooth and respectful

A conversation designer builds handoff sequences that feel seamless, providing the human agent with clear context, framing the situation with clarity, and maintaining continuity so the customer never feels dropped. These transitions carry significant emotional weight, and they must feel deliberate, not improvised.

They act as the customer’s advocate, noticing where a flow feels confusing or heavy, and redesigning it so customers never need to think about “how” to get help.

Why this role exists now

When AI is the primary interface your customers interact with, conversation design becomes a critical part of the customer experience. The way the AI speaks directly influences customer trust, brand perception, and operational outcomes. It’s a core competency for any AI-first support organization.

4. Support automation specialist

Builds the backend actions that allow AI to do real work

Where the conversation designer shapes the AI’s expression, the support automation specialist shapes its capability. This role turns AI from an answering machine into an outcome engine. They’re the bridge between the AI and the systems it must interact with, ensuring it can take action safely, reliably, and deterministically.

Support teams increasingly expect AI to do what a human would do: refund a charge, adjust a subscription, verify an identity, update an account setting, or pull relevant data. That requires a new technical role at the edge of support, ops, and engineering.

What this role does day to day

1. Creates and maintains backend workflows the AI executes

This includes building and maintaining:

• Fin Procedures with embedded steps.
• Action flows that call internal and external APIs.
• Automations that span billing systems, user identity layers, CRM objects, subscription entitlements, refund tools, and more.

They ensure the AI can act compliantly and predictably. In practice, they write the playbooks that turn intent into action.

2. Owns the integrations required for advanced automation

Many customer problems require data that lives elsewhere – billing platforms, internal databases, account systems of record. The automation specialist ensures the AI can retrieve, validate, and use that information safely. They create the interfaces that let the AI take the next step without asking a human to intervene.

3. Partners closely with product and engineering

Not every system is ready for AI-driven action. Some workflows require new endpoints, permission layers, safety gates, or deterministic fallbacks. This role works cross-functionally to get those changes built and deployed, ensuring the AI can operate safely across the full stack of internal tools.

4. Ensures reliability and safety at every step

When the AI takes real action, the automation specialist needs to implement guardrails, validation logic, exception handling, and safe execution paths.  
They confirm that:

• The AI has access to the correct data.
• The action matches policy.
• Edge cases are accounted for.
• Risky flows have deterministic constraints.
• Every action is auditable and reversible.

This is how the system maintains both trust and operational integrity.

Why this role exists now

Customers want more than answers, they want outcomes.

AI can now deliver those outcomes, but only with the right backend scaffolding. This role is how support teams modernize their operational architecture and unlock end-to-end automation.

How these roles work together: The new operating loop

These roles don’t operate as silos. They are interdependent parts of the same system.

• The AI ops lead identifies patterns and performance gaps.
• The knowledge manager resolves inaccuracies or missing content.
• The conversation designer improves the clarity, tone, and flow.
• The automation specialist expands the system’s ability to take action.

Each role feeds the others, and each improvement compounds the next.

This loop is the differentiator between teams that plateau early and teams that scale AI into a reliable, high-performing system.

How to get started (even if you can’t hire all four roles today)

Most teams don’t start out with a fully built version of this model. They phase into it gradually, assigning partial ownership first, then formalizing responsibilities, and finally hiring specialists once AI is handling enough volume to justify dedicated roles.

Phase 1: Assign ownership

Give each role’s core responsibilities to someone who can devote five to 10 hours weekly. Support ops, enablement, senior ICs, and technically inclined team members often take on this work in the early months.

Phase 2: Formalize the responsibilities

As AI resolves more queries, the need for maintaining and optimizing the system grows. What begins as “a few hours a week” quickly becomes core operational work – and needs to be recognized as such. Teams that formalize responsibilities avoid performance drift and knowledge debt.

Phase 3: Specialize and hire

Once the AI is handling 50–70% of incoming volume, these responsibilities naturally become full-time roles. At that point, investing in specialization is essential infrastructure for the next stage of scale.

The bottom line

AI changes the shape of your support team.

These four roles – AI operations lead, knowledge manager, conversation designer, and support automation specialist – form the backbone of the AI-first support organization. They bring order to the system, create stability in a constantly changing environment, and enable AI to deliver the outcomes leaders (and customers) expect.