A Practical Guide to AI Models and Their Role in MDM

When it comes to cleaning up large data sets, most master data management (MDM) solutions on the market follow the same predictable playbook: Create rules (generally from scratch), and layer on fuzzy matching logic. But this approach doesn’t sufficiently move the needle, often leaving more than 25% of the data—the more difficult and complex records—unresolved. As a result, humans must step in to curate these records manually, which is extremely time-consuming, expensive, and unscalable.  

Tamr takes a different approach to MDM, rethinking the challenge of delivering high-quality, trustworthy data from the ground up. Where other solutions rely on conventional approaches, Tamr embraces AI, machine learning, and agenticdata curation to tackle complex data mastering and data curation challenges. In this post, we’ll take a closer look at the different types of AI/ML, as well as what makes Tamr’s approach distinctly better than others in the market. 

What Are the Different Types of AI and Machine Learning?

AI/ML models come in several distinct forms, each built on different methods and designed to solve different types of data management problems.

1. Rules-Based Systems

Rules-based systems use a straightforward “if/then/else” structure to match and resolve business entities. And while rules-based systems seem quite simple on the surface, in reality, they become deceptively hard. Consider this example. 

A company writes a rule that states “If company is similar and address is similar, then do a match.” Easy enough, right? It is, if you stop there. But let’s keep going with the example because unpacking this simple idea begs the question: How similar is similar enough? 

For example, what if the two records have the same website, then how different can they be? What if the postal code is missing? Or if the two records are in an office complex that share the same address? Coding every scenario takes an unsustainable amount of effort, and you often end up coding conflicting rules. This makes support and extensibility nearly impossible. So, while you can get an easy start with a small number of rules, it quickly becomes unwieldy to maintain them or troubleshoot issues because you are coding the imperative logic as opposed to leveraging a declarative approach that lets you focus on outcomes. 

2. Machine Learning (ML)

ML algorithms are a type of AI that can be thought of as an optimized set of rules, typically aggregated together in an ensemble method to generate high-quality predictions across a wide range of input data scenarios. At their core, most ML models use some sort of linear classifier to match and resolve entities. However, unlike building a rules-based system, the whole point of the training phase of these models is to scan all the different scenarios in the data and, from that, determine which ones (i.e., which “rules”) are best-suited to the use case at hand. Further, while traditional rules-based systems typically require a significant amount of engineering time and effort to regularly add and update rules (and troubleshoot conflicts), ML algorithms are able to avoid this issue, which greatly minimizes the load on technical resources.  

That said, while ML models run extremely quickly and work well on a variety of common data problems, they can run into limitations as data problems become more complicated. 

3. Deep Learning

Deep learning models are more complex AI algorithms that typically take the linear classifiers of conventional ML and layer them on top of each other so that the output of one set of votes can be reasoned about and acted upon by a whole new population of classifiers. Deep learning models use complex techniques to accomplish these functions, and to perform well, typically require far more training data than conventional ML. However, the upside of deep learning is that in contrast to rules-based systems and conventional ML, deep learning models are very good at handling complex tasks and more elaborate hierarchical relationships.

While deep learning models run at a decent pace on traditional hardware, they are slower to scale as data volumes increase. Deep learning models are also generally more expensive and difficult to train. 

4. Large Language Models (LLMs)/Generative AI (GenAI)

LLMs are some of the newest AI models on the scene, offering an incredibly powerful approach to generating text, images, and code, and providing extremely confident responses to questions and prompts. However, LLMs are extremely costly, slow to run at scale, and can often be wrong, which makes it difficult to trust the outputs they produce. 

In addition, LLMs are often overkill when it comes to the majority of general data mastering tasks such as entity resolution. As such, trying to master your entire data set using GenAI would be incredibly expensive, slow, and error-prone. However, asking GenAI to help you resolve complex edge cases that remain after the general mastering process of the majority of an organization’s data would be beneficial and a more efficient way to use the technology. 

The Takeaway

So what does this all mean? Simply put, you want to use the right AI/ML model for the job. Rules-based systems can’t sufficiently scale at a reasonable cost, so using them to master all of your enterprise data is not ideal. Meanwhile, deep learning models and LLMs are very good at handling complex tasks, but using them to match and merge high volumes of records is overkill. Conventional ML models are best suited for high-volume entity resolution but can’t solve 100% of a company’s data mastering needs.

How Do Traditional Solutions Approach MDM? 

Traditional MDM solutions rely on rules. And while this approach was once considered innovative, today, it’s proving to be a liability. Writing, modifying, and maintaining rules is time-consuming for data professionals. As data evolves, rules must evolve, too. Not only does that put a significant strain on data teams, but it also prevents these solutions from scaling. And because traditional MDM solutions have limited flexibility and scalability, they often become a data silo themselves, exacerbating the very issue they are trying to solve. ‍

What Makes Tamr’s Approach to MDM Unique?

Tamr doesn’t do MDM like other vendors—and that’s what gives us an advantage. Instead of relying on outdated technology and an extensive use of rules for data mastering, Tamr’s AI-native approach embeds AI at the core of our solution and the data mastering process. Everything we deliver—from architecture to workflows to user interfaces—is all built around AI, an approach that is fundamentally different from what legacy MDM providers offer. 

With its power, intelligence, and efficiency, Tamr’s data mastering capabilities leverage specialized ML and deep learning techniques to take organizations 90% of the way toward achieving fully clean, trustworthy data in a cost-efficient, scalable manner—something rules-based systems can never achieve. Then, by incorporating select, organization-specific rules alongside ML, Tamr can account for idiosyncrasies specific to the organization, allowing it to resolve additional entities.

That leaves the “last mile”—the 5% or so of data that remains unresolved after the mastering process—that requires a higher level of knowledge, precision, and data preparation. And even though the last mile represents a small portion of the data, it still typically consumes a large portion of the data team’s time to resolve. But with the introduction of agentic data curation and our Curator Hub, Tamr can help there, too.

Agentic data curation is a new concept that has the potential to revolutionize data management by using LLM-based AI agents to automate more of the data curation process. These agents have the capacity to intelligently clean, curate, manage, and refine the last mile of enterprise data with minimal human intervention. By comparing outputs of entity matches and explaining the reasoning behind why records do—or do not—match, AI agents can provide the analysis and assistance humans need to address and resolve their trickiest edge cases. 

What are the Key Components of Tamr’s AI?

Tamr’s patented, AI-centric approach to data mastering and data curation combines machine learning and AI agents with human refinement and oversight to deliver value in days or weeks, not months or years. And because each AI/ML model has its own distinct benefits, Tamr’s AI-native approach embodies the motto “right tool for the right job.” Let’s take a look under the hood at Tamr’s AI capabilities and how they align to the different types of AI/ML models.

ML models are a fast and efficient way to tackle the core task of identifying if records are the same or not, offering scalability and high performance for entity resolution. Key capabilities include:

• Feature extraction and enrichment: Data science and data engineering techniques standardize the text that feeds the model, making it as easy as possible for the model to decide if records are the same or not. 
• Blocking and pre-grouping: Unique, patented capabilities allow Tamr to scale its machine learning and quickly determine which records can be the same—and which cannot.
• Pairwise classification and clustering: This machine learning approach compares two entities and predicts if they are the same or different based on similarities and differences in the records.

Deep learning models run in a SaaS-based cloud architecture that allows Tamr to offer exceptional performance and high-quality results in entity search. Semantic search, a data retrieval technique that leverages the meaning, intent, and context of a user’s query to find the most relevant results, is an example of Tamr’s use of deep learning models.

GenAI models provide powerful asynchronous processing that aids humans when dealing with sparse and/or uncertain data. Tamr’s agentic data curation is a new concept that uses generative AI and LLM-based AI agents to automate more of the data curation process by capturing and acting on the contextual insights needed to make confident curation decisions. 

All of these components work in concert to deliver efficient, accurate, adaptable, and intelligent data mastering. With Tamr, you gain:

• Efficiency: Tamr’s patented innovations in AI/ML mastering, data curation, insights, and feedback help companies master data quickly. 
• Accuracy: Tamr’s strong pre-trained models and deep industry expertise ensure that the problem is as easy as possible for the model to solve. 
• Intelligence: Tamr uses a blend of different AI/ML models to ensure the right model is solving each part of the MDM process. 

Tamr’s AI-Native Approach: The Only Way to Solve MDM

With 19+ patents to our name, Tamr uses AI in unique and proven ways to efficiently and scalably solve the MDM challenge. And this approach isn’t new to Tamr. Since its founding, the company has been dedicated to developing AI-based technology and committed to innovations that support the needs of global organizations with the most complex data environments. 

Tamr—with our AI-native approach—stands in sharp contrast to traditional MDM solution providers like Reltio and Informatica, delivering efficiency, accuracy, intelligence, and scalability that’s beyond compare. For more detail on how AI-native MDM compares to rules-based MDM, check out our infographic. 

And be sure to download our ebook, How Agentic Data Curation is Transforming Data Mastering: Perspectives from the Tamr Co-Founders, to learn more about the future of data management.