Use this article to find, filter, and tag your Evals, create a new Eval, organize your Evals, import Simulations from a CSV (comma-separated values) file, add a Scorecard, and see how Fin is performing across your Evals.
What is the Evals page?
The Evals page lists every Eval in your workspace, so you can see how each group of tests performed and run them again. An Eval is a themed group of tests, and each test inside it is called a Simulation. To open the page, go to Test > Evals.
The Evals table has these columns:
Eval name: the name of the Eval, with a run button next to it.
Result: how the Eval performed on its latest run, for example 4/6 Passed.
Tags: any tags attached to the Eval.
Last run date: the date the Eval last ran.
Last run by: who last ran the Eval.
Last updated: when the Eval was last changed.
Created: when the Eval was created.
How to organize your Evals
To organize your Evals, group Simulations into Evals by whatever theme makes sense for your business. Many teams start with what isn't working yet, then build out from there. Common starting points include:
Known issues: conversations where Fin gave a wrong, incomplete, or unhelpful answer. Recreate each one as a Simulation, fix the cause, and run it again to confirm the fix.
Escalation scenarios: situations where Fin should (or shouldn't) hand off to a teammate.
Topic scenarios: one topic, like refund requests, tested across a range of variations. Your top 10 or 20 topics by conversation volume are a good place to start.
Behavioral scenarios: checks that Fin stays on tone and on brand across different situations, regardless of topic.
Over time, build one or a few Evals as a "golden set" that covers the conversations that matter most to you. Re-run the golden set regularly, for example weekly or whenever you change Fin's configuration, to make sure your most important scenarios still work.
How to find an Eval by name, result, or tag
To find an Eval by name, type in the Search evals box at the top of the Evals page. To narrow the list, click Add Filter and choose Result or Tag.
How to filter Evals by result
Go to Test > Evals.
Click Add Filter, then click Result.
Select one or more results: Not run, Running, Queued, Passed, Partially passed, Failed, Canceled, or Unprocessable.
How to create a new Eval
To create an Eval, choose whether to build it by hand or to import a list of customer scenarios from a CSV file.
Go to Test > Evals.
Click New eval.
Choose Create manually to build your Simulations yourself, or Upload a CSV to import a list of customer scenarios.
Give the Eval a clear, descriptive name of up to 250 characters, so you and your teammates can tell what it covers when you run it again later.
Each Eval can hold up to 50 Simulations. To import from a CSV file, see "How to import Simulations from a CSV file" below. To set up the Simulations inside an Eval, see the Set up Simulations for Evals article.
How to import Simulations from a CSV file
To import Simulations from a CSV file, upload it from the Evals page or from inside an Eval. Each row becomes one Simulation, and you review every column match before anything is saved. To import, you need Can view configuration and propose changes in a draft release or Can view configuration and publish changes directly to main (individually or in a release) under Fin AI Agent & Automation.
To import into a new Eval, go to Test > Evals, click New eval, and select Upload a CSV. To add Simulations to an Eval you already have, open it, select Import CSV from the options menu (...), and then choose that Eval under Import to.
What can a CSV file include for Simulations?
A CSV import can fill 5 Simulation fields. Only the opening message is required.
Opening message (required): the first thing the simulated customer says to Fin.
Follow-up instructions: what the customer knows, what they share with Fin if asked, and how they communicate.
Channel: the channel the simulated customer writes in.
Handoff: whether Fin should hand off the conversation: none, to a team, or to a workflow.
Expected answer: what Fin should say or do to pass. Each expected answer becomes a Fin reply check. To add several expected answers for one Simulation, put each one on its own line in the same cell.
To start from a ready-made file, click Download template in the Upload a CSV file dialog. The template has one column for each of the 5 fields. Column headers don't need to match the template exactly. Common names like question, prompt, persona, outcome, or ground truth are matched automatically. Comma-, semicolon-, and tab-separated files all work.
What are the CSV import limits?
Limit | Value |
Rows per file and per Eval | 50 |
File size | 10 MB |
Characters per expected answer | 1,000 |
Simulation fields to match | 5 |
How to upload a CSV file and match columns to Simulation fields
In the Upload a CSV file dialog, drag your CSV file in or click Choose file. The file is read in your browser and shown as a table, with its row and column count, before anything is saved. To use a different file, click Replace file.
Check the table, then click Continue.
In the Match the columns step, check the Simulation field matched to each column and change any that are wrong, or click Reset matches to start over.
Next to Import to, choose New eval and enter a name, or choose an existing Eval to add the Simulations to.
Each Simulation field is matched to one column, and columns you don't use are listed under Not used and ignored. If an expected answer is over 1,000 characters, you're asked what to do with it: Leave them out (the default) imports the row without that answer, Cut to 1,000 characters keeps the start of the answer, and Skip these rows leaves the whole row out.
How to fix Channel and Handoff values without editing your CSV file
If a Channel or Handoff value isn't one Fin recognizes, such as WhatsApp or Yes, a warning appears on that field.
Select the warning on the Channel or Handoff field.
Pick what each unrecognized value means.
Click Apply.
You can also choose Skip rows to leave those rows out of the import.
How to preview and import Simulations from a CSV file
Click Preview simulations to see each row as it will run, including the first message, follow-up instructions, Fin reply checks, and handoff.
Check the row count at the bottom of the dialog. It shows how many rows will import (for example, 9 of 10 rows) and what's left out.
Click Import.
After the import, a confirmation shows how many Simulations were created. Each row appears in the Eval as a Simulation named after its opening message, ready for you to click Run all. Imported Simulations use the Test customer and all brands, so open a Simulation to choose a specific customer. Tags are added to the Eval, not in the CSV file. When you import inside a Release, the Simulations are added to that Release.
When can't you import Simulations from a CSV file?
No column is matched to Opening message, or no row has one.
The new Eval has no name, or the name is already taken.
Your file has more than 50 rows.
If your rows would take an existing Eval over 50 Simulations, you see how many still fit and can select Import first [number] to import just those. If any rows are rejected when you import, nothing is saved. Each problem row is listed with the reason, and you can select Import without these rows to import the rest.
How to tag Evals and manage tags
Tags group related Evals so you can find them quickly as your suite grows. For example, tag by topic (Refunds), team or owner (Billing team), type of test (Regression, Escalation), or language (French). An Eval can have as many tags as you need.
How to create a tag for Evals
To see and create tags, click Tags at the top right of the Evals page. The Tags panel lists each tag and how many Evals use it, and you can find a tag with the Search tags box.
Go to Test > Evals.
Click Tags, then click New tag.
Under Name, enter a name for the tag, for example "Refunds".
Optionally, click Add evals and choose the Evals that should get the tag when you create it.
Click Create tag.
How to add tags to Evals
From the Evals list: select the checkboxes next to one or more Evals, click Tag above the list, and choose the tags to add. You can tag up to 100 Evals at once.
From inside an Eval: click + Tag under the Eval's name.
How to filter Evals by tag
Go to Test > Evals.
Click Add Filter, then click Tag.
Select one or more tags. Each tag shows how many Evals use it.
How to rename or delete tags for Evals
To rename or delete a tag, click Tags at the top of the Evals page, or select Manage tags in the tag picker. The panel shows how many Evals use each tag. Deleting a tag removes it from every Eval, including Evals in Releases.
Note: Tags apply to Evals, not individual Simulations. To group Simulations differently, split them into separate Evals. Eval tags are separate from the tags you use on conversations and contacts.
How to add a Scorecard to an Eval
A Scorecard adds quality scoring to an Eval, such as Clarification or Efficiency, on top of its pass/fail checks. To add one, open the Eval and click Add scorecard at the top right, next to New simulation. After you add it, a Quality evaluation section appears next to Simulation results. To create a Scorecard, add criteria, and set each criterion's score weighting and Critical setting, see the How to add a Scorecard to an Eval article.
How to see how Fin is performing across your Evals
To see how Fin performed across all of your Evals, look at the summary at the top of Test > Evals. It reflects the last time each Simulation was run, and it has two parts:
Simulation results: how many Simulations passed all of their Simulation checks and Handoff checks, for example 42/50 passed (84%). Simulations that errored count as failed, and Simulations that haven't run yet count toward the total.
Quality evaluation: for each Scorecard, the average review score and the average for each criterion, across all the Evals that use it. Use the arrows to move between Scorecards.
The summary updates with your search and filters, so filtering by a tag shows results for just those Evals. To compare earlier runs, open a Simulation and click the run history dropdown next to its result. It lists each run by date with its result.
Evals management FAQs
How many Simulations can an Eval hold?
How many Simulations can an Eval hold?
An Eval can hold up to 50 Simulations. To add more, delete existing Simulations or create a new Eval.
Can I create Simulations from a CSV file?
Can I create Simulations from a CSV file?
Yes, you can create Simulations from a CSV file. Click New eval, then choose Upload a CSV. To add imported Simulations to an Eval you already have, open it and select Import CSV from the options menu (...). Batch Test questions work too: export them to a CSV file, and each question becomes a Simulation's opening message.
Can I tag individual Simulations?
Can I tag individual Simulations?
You can't tag individual Simulations, because tags apply to Evals only. To group Simulations differently, split them into separate Evals.
Who can create, edit, and run Evals?
Who can create, edit, and run Evals?
Teammates with Can view configuration and propose changes in a draft release or Can view configuration and publish changes directly to main (individually or in a release) under Fin AI Agent & Automation can create, edit, tag, and run Evals. Teammates with Can view configuration can see Evals and their results, and can filter them by tag.
Need more help? Get support from our Community Forum
Find answers and get help from Intercom Support and Community Experts
