Use this article to test content, guidance, and procedure changes in one Salesforce test organization (an environment) by using a release, while your live organization keeps using Main Fin, the live version of Fin. It's for teammates who use Fin for Salesforce with more than one connected Salesforce organization and who can propose and publish changes.
How does testing Fin changes in one Salesforce environment work?
What is a Salesforce environment in Fin?
Fin for Salesforce can connect to more than one Salesforce organization. A workspace can hold a live organization and one or more test organizations. Each connected organization is an environment.
How is a release different from scoping a workflow to one environment?
You can already scope a workflow to one environment: go to Deploy, open a workflow such as Salesforce cases, and set Environment on the trigger. The default is "All environments".
That setting only controls which workflow runs. It doesn't change the content, guidance, or procedures Fin reads — Fin uses the same live configuration in every environment. A release changes the configuration itself. It pins a different version of each item in it, for one audience only. Items keep their identifier, so you don't need to copy them. One audience and one traffic split control the whole release, and you can roll it back.
Why does a release apply to only one Salesforce environment?
Fin writes the environment onto each conversation as a system-defined conversation attribute, which audience rules can read. An audience is a saved set of rules that selects which conversations a release applies to. A release rollout targets one audience, so the release applies only to conversations that match it.
How to test Fin changes in one Salesforce environment
Step 1 — Confirm the Salesforce test organization is connected
Go to Connect and find the organization under "Connect Fin to a test organization". Its status must be "Connected". If the status isn't "Connected", finish connecting the organization before you continue.
Step 2 — Create a release for your Fin changes
Open the release switcher at the top of the left sidebar, then click New release. You can also click Manage releases and then click New release.
In Name, enter a name that says what's changing, for example "Refund policy overhaul".
Optional: In Description, enter what the release changes and why.
Click Create release.
The name and description are for your team only. Fin doesn't use them. After you click Create release, the release opens in a draft state, where you can add changes.
Step 3 — Add your changes to the release
Inside the draft release, the Add changes to this release section lets you add, edit, or remove items and bundle several changes into one release.
Select the kind of change you want to make: Content, Guidance, Escalation guidance, or Procedures.
For content, click Public article, Internal article, or Snippet, write or edit the content, then click Save to release.
Repeat for every change you want to test in this environment.
Optional: Use the Preview panel on the right to ask Fin a question your customers might ask and check its response with the release's changes applied. Preview conversations aren't charged.
Content you create in a release shows the status "Publishes with the release" and is enabled for Service and Copilot. Hover over the status to see what it means: "Created in this release. When it's merged, it's published to the Help Center and enabled for Fin." Changes you save inside the release don't reach customers, and Main Fin stays unchanged, until you start a rollout in step 6.
Step 4 — Open your audiences
Go to Settings > Data > Audiences.
Step 5 — Create an audience for the environment
Create a new audience and give it a name that includes the environment name.
Click Add audience rule, then select Environment under "Conversation data".
Select the environment you want to test, then click Save.
Step 6 — Start a gradual rollout
Open your release, click Release at the top right, and select Roll out gradually.
Important: Don't select Publish to Fin Main — this applies the changes to Main Fin in every environment.
Step 7 — Select the audience and set the split to 100%
Optional: In Name, change the rollout name. Fin fills it in with the release name and today's date.
In Audience, select the audience you created for this environment.
In Traffic split, move the slider to 100%. The panel should show "100% Release" and "0% Fin Main".
Click Start gradual rollout.
Step 8 — Test your changes in the test and live organizations
Start a new conversation in your test organization. Fin answers using the release's configuration.
Start a conversation in your live organization. Fin answers using Main Fin.
Tip: When you're happy with the results, select Publish to Fin Main from the Release menu to apply the changes in every environment, or end the rollout to discard them. Learn more in How to roll out a release in Fin.
How to test a release in one Salesforce environment with Test in environment
Use the Test in environment option in the Release menu to test a release in one or more connected Salesforce test environments without creating an audience or setting a traffic split.
Open your release and click Release at the top right.
Select Test in environment. The Test in environment panel opens.
Optional: In Name, change the test name. Fin fills it in with the release name and today's date.
Under Environment, click Select test environments and choose the environment to test.
Optional: Under Results analysis, keep Identify conversations related to this release's changes for analysis selected to focus results on conversations where the release's changes apply. To narrow the analysis further, click Add Filter and choose a Fin attribute or a procedure.
Click Start test.
How to run an A/B test on a release
Use an A/B test on a release to compare Fin Main (version A) with the release (version B) on one goal, such as resolution rate. An A/B test runs the two versions of Fin side by side, splits conversations between them at random, and the version that does better on your goal wins.
Open your release and click Start A/B test beside the Release menu. The Create A/B test panel opens on the Goal tab.
In Name, keep or change the test name. Fin fills it in with the release name and today's date, and you'll find the test under this name in the Results tab.
Under What metric are you looking to improve?, choose the metric the test is judged on. The example shows Resolution rate. Click Next.
On the Setup tab, in Audience, choose who takes part.
Under Traffic split, move the slider to choose how many conversations get the release. The panel shows version A (Control) as a share of Fin Main and version B (Treatment) as a share of this release. For example, 10% shows "90% Fin Main" and "10% This release".
Optional: Under Results analysis, click Add Filter to also see results for only the conversations that match the filter. Results include all conversations by default, and you can follow both in the Results tab. Click Next.
On the Review tab, check the name, goal, audience, split, and what the results are measured on, then click Start A/B test. You can end the test at any time.
How does the traffic split affect how soon an A/B test gives a result?
The traffic split affects how soon an A/B test gives a significant result: the closer the split is to 50/50, the sooner you get one. A result is statistically significant when the gap between A and B is big enough that it isn't down to chance. The side with fewer conversations sets the pace, so 50/50 is the fastest and a split like 90/10 takes several times longer.
What happens after I start an A/B test?
The share of conversations you chose gets the release, and the rest stay on Fin Main.
You follow the goal metric, such as resolution rate, for A and B in the Results tab. Fin tells you when the difference is reliable.
If B wins, publish the release to Fin Main. If not, end the test and keep Fin Main.
How to see which environment a release is running in
After you start the rollout, the release switcher and the Releases page show the release's status. A release running in one environment shows a status such as "In drive-connect-6834-dev-ed", which is the name of the environment. A release that's rolling out to a share of conversations shows a percentage instead, such as 25%.
Need more help? Get support from our Community Forum
Find answers and get help from Intercom Support and Community Experts