In order to optimize the accuracy of an answer that an LLM generates, the inputs the LLM receives must be refined for comprehension. The clearer and more understandable the query, the better the output.

Often in customer service, customers can write in to support without fully explaining or contextualizing their query. To solve this problem, the Fin AI Engine™ has been designed to refine the inputs that are sent to the LLM. This ensures that each customer message is optimized in terms of its meaning and context so that the LLM has the best possible chance of producing an accurate answer. In addition, the Fin AI Engine™ performs checks to see whether a Workflows automation or Custom Answer should be triggered based on the topic and context of a customer's query, as well as performing safety checks to filter out any questions that Fin shouldn't answer.

Once a query has been checked and optimized the next stage is to generate a response using the LLM. For this task, the Fin AI Engine™ has been designed to use a bespoke and enhanced retrieval augmented generation architecture (RAG for short).

RAG is a process that involves retrieving relevant information from a data source and combining it with a user’s prompt before passing it to an LLM. This additional context helps improve the model’s output by enhancing its base knowledge, and in doing so reducing the risk of inaccuracies like hallucinations. Intercom’s application of RAG is highly unique.

The AI group at Intercom has invested heavily in optimizing our unique application of RAG and continuously tests both the accuracy of the LLM and the individual steps of RAG to improve overall performance.

In the final step of the process, the Fin AI Engine™ performs checks to understand whether the output from the LLM meets the necessary response accuracy and safety standards. Many checks are performed, covering whether there is enough confidence in the response, the response is accurate enough, and whether the response is grounded enough in reality to address the question adequately.

To calibrate and enhance engine performance, the Fin AI Engine™ has advanced integrated tools that help optimize answer generation, efficiency, precision, and coverage.

Relying exclusively on the generative capabilities of an LLM to answer or solve a customer problem is not a reliable way to serve customers. Without the proper safeguards in place, LLMs can be open to manipulation or hallucinations, which could then impact your customers. To ensure safety and reliability, Intercom’s AI Engine™ has been designed with strict safety controls at each stage. If the necessary parameters for safety have not been met at any of the stages, Fin will let the customer know that it cannot answer the query and escalate to human support.

Intercom has implemented state-of-the-art security measures to protect Fin against a wide range of LLM threats, including those identified by the OWASP LLM Top 10. By consistently testing a variety of high-end LLMs, and deploying rigorous internal controls, security protocols, and safeguards, Fin is able to achieve the highest level of security and reliability while avoiding potential limitations and threats

That means you and your customers can always trust Fin’s answers as the safest, most accurate, and reliable of any AI agent. Learn more about Fin’s safety measures on trust.intercom.com.