Chapter 3What's next?
Implement a repeatable process for optimizing AI
The AI Agent has been trained, tested, deployed, and you've analyzed its initial results. It's handling real conversations, potentially taking actions, producing measurable outcomes, and generating a baseline of performance data.
The focus now shifts from launch to optimization.
The rollout framework becomes a repeatable process for optimizing the AI Agent, applied continuously in smaller, targeted iterations.
Think of it as a flywheel: Train, Test, Deploy, Analyze.
Each cycle compounds the next. Here's how the loop works:
1. Train
Strengthen knowledge, expand coverage, refine behavior
Training is the ongoing work of improving what your AI Agent knows, how it behaves, and the tasks it can handle.
This might include:
- Expanding topic and content coverage.
- Updating guidance to clarify tone, behavior, or handover rules.
- Connecting additional data sources for personalization.
- Defining or adjusting multi-step workflows for more complex scenarios that go beyond informational queries (e.g., cancellations or refunds).
The goal is to expand what your AI Agent can handle autonomously while delivering the same high-quality support as your human team.
With Fin, your team can manage training through a centralized system:
- ProceduresUse natural language combined with deterministic controls (when needed) to set Fin up to handle complex queries with multiple steps, business logic, third party systems, or cross-team approvals.
- GuidanceEnsures every response reflects your brand voice and policies. You can define tone, vocabulary, and style, and set rules for Fin to follow for specific brands, customer segments, or product lines.
- Data connectorsEnable Fin to securely retrieve data and take action in external systems like Shopify, Stripe, and Salesforce.
2. Test
Validate changes safely before they reach customers
Training is the ongoing work of improving what your AI Agent knows, how it behaves, and the tasks it can handle.
This might include:
- Identify mistakes before they impact customers.
- Validate new workflows or Procedures (if using Fin).
- Ensure tone, behavior, and policy alignment.
- Confirm you’re happy with how the AI Agent responds across channels and customer types.
Fin has built-in testing tools that let your team validate changes before they go live:
- SimulationsRun fully simulated customer conversations from start to finish to test how Fin will respond. You can see what Fin is doing, when it is reasoning, and where it passes or fails.
- PreviewChanges in real time, allowing you to see how Fin will respond, and refine answers until you’re ready to go live.
- Audience testingMakes it easy to test how Fin answers for various customer types across multiple brands by simulating different audiences or personas.
3. Deploy
Once updates are tested, roll them out in a controlled, measured way.
Deployment decisions typically include:
- Which channels (phone, email, chat, social, etc.).
- Which audiences (by geography, plan, or customer type).
Your team can configure how updates are deployed using Fin’s built-in controls:
- Channel configuratione.g. Fin over chat, email, phone (Fin Voice), social channels, or community platforms like Slack and Discord.
- Audience targetingDeploy updates selectively to specific customer groups (plan tier, region, lifecycle stage, etc.).
- Routing and channel-specific behaviorControl how Fin behaves by channel, including escalation rules and when humans step in.
4. Analyze
Post-launch, you need to keep track of your AI Agent’s performance so you can see what’s working well and what needs to be improved.
Focus on macro patterns, not individual queries. Look for:
- Topics with frequent handoffs or low resolution (often linked to content gaps).
- Follow-up questions that indicate lack of clarity.
- Segments or channels with lower satisfaction.
The goal is to identify improvements that will significantly increase resolution, reduce volume on the human side, or expand the AI Agent’s coverage.
Fin has built-in tools to surface these insights:
- Topics ExplorerAutomatically organizes conversations into topics and subtopics, showing what's driving volume and impacting quality.
- The Topics Trend ReportHighlights weekly changes (volume spikes, drops in resolution, emerging issues) so you can act before customer experience suffers.
- Customer Experience ScoreEvaluates and scores every interaction across sentiment, resolution, and service quality.
- The Optimize DashboardSurfaces AI-powered Suggestions for immediate improvements.
These insights flow directly back into the Train phase, completing the optimization loop.
Chapter 4Scale it
Go to the next chapter and learn how to redesign your customer experience, rewire your organization and systems, and rethink economics to sustain impact and achieve full transformation.