Improving LLM accuracy with Entity RAG

Since Large Language Models (LLMs) exploded in popularity, largely led by OpenAI’s ChatGPT, enterprises have been exploring how to run LLMs on private, internal data sets that LLMs have never seen before and which must remain private.

Retrieval Augmented Generation (RAG) is the technique of augmenting LLMs with additional data from specific sources that you control. In this post we introduce EntityRAG, created by Tilores, as a significant improvement for the accuracy and reliability of LLMs, especially in regulated environments.

Typically, a RAG data source is based on a vector database. Vector search excels at retrieving data from unstructured data sources. For example, if you were building RAG for an insurance company, you might store all of your company’s standard insurance policy terms in a vector database. The LLM would therefore be able to retrieve similar knowledge and use the insurance company’s insurance policies as context via the vector database when provided with an appropriate prompt. 

However, what data source should a LLM use when the prompt is about a specific customer? In this case, a vector database would not be suitable for the following reasons:

To address this, we propose the use of entity resolution technology as a complementary data source to vector databases in RAG. We name this EntityRAG.

What is Entity Resolution?

Entity resolution is the process of deduplicating, linking, and making searchable, data records that relate to a real-world entity (i.e. person, company, product) that originate from one or more data sources. When used specifically for person data it is often referred to as “identity resolution”.

It relies on the use of fuzzy matching algorithms, such as Cosine similarity (which is often used in vector databases), to recognise when records differ slightly for specific attributes, but should be considered the same. 

Typically, entity resolution is used for financial crime detection, fraud prevention and as part of a customer data platform (CDP) to match customer profiles from different sources.

Importantly for regulated industries, where accuracy of data is critical, entity resolution systems can be fine-tuned to provide highly accurate, reliable and explainable results. 

What about Graph Databases?

Graph databases, such as Neo4j, have also been gaining popularity lately as a RAG source. Indeed graphs are exceptionally well-suited for organizing and depicting diverse and interlinked data in a structured way and can be complementary to vector databases in RAG.

However, they have certain limitations that make them less than ideal for certain LLM applications.

Lets looks at these issues:

By default, graph databases do not include fuzzy search. So if you search for “Phillip Mitchel” you would not retrieve the data for “Philipp Mitchell” due to the spelling mistake. You can overcome this by using a search engine, such as Elasticsearch, but then you are introducing another complex system that needs to be managed.

Transitive hops are a particular problem with graph databases. Because all data is, in theory, connected to all other data, retrieving data when the graph needs to traverse several nodes (data records) can be very slow. 

The accuracy of distinct entities is a particular problem in regulated industries. When you retrieve the data related to one customer, and it contains data from records that have been linked together, perhaps from disparate sources, you need to be certain that they belong together. 

While graph databases excel at representing complex relationships, they may face challenges in certain scenarios, particularly when dealing with customer data in regulated industries. The interconnected nature of graph structures can sometimes make it difficult to guarantee that retrieved data pertains only to a single customer, especially when dealing with large, complex datasets. 

EntityRAG for accuracy and reliability

To examine whether entity resolution in combination with a vector database would produce superior results to a vector database alone, we teamed up with enterprise LLM specialists Kern.ai to create a demonstration scenario in a travel insurance company example. Scroll to the bottom for the webinar video. 

In this demo, we tested three setups:

Baseline RAG

EnhancedRAG

EnhancedRAG + Tilores EntityRAG

In this case, the first name and last name were extracted from the customer message and used to search for the customer in Tilores.

The resulting response is far more specific and useful compared to the previous two results, as thanks to the EntityRAG the LLM knows which customer is relevant and which insurance policies they currently have. 

Conclusion

Imagine unlocking the full potential of your customer data effortlessly. Our collaboration with Kern.ai has demonstrated that an advanced entity resolution system like Tilores can significantly enhance the accuracy and relevance of large language model (LLM) responses. This innovation, embodied in EntityRAG, is set to revolutionize how regulated enterprises develop LLM applications, driving unparalleled accuracy and relevance.

But the benefits don’t stop there. Once your custom data is unified within Tilores, the possibilities are endless. EntityRAG is just one of many transformative applications. This unified, deduplicated data serves as a reliable source of customer truth, powering crucial functions like fraud prevention, KYC (Know Your Customer), and marketing operations. Best of all, you can achieve this without the daunting task of building an internal data infrastructure or the expense of migrating data into a centralized data warehouse.

Step into the future of data management with Tilores and experience seamless, efficient, and versatile solutions tailored to your enterprise's needs.

Want to try this yourself?

Sign up for a free Tilores account to unify your customer data for your own Entity RAG application. Our entity resolution experts are always happy to help you configure Tilores for the best results. 

Webinar video