Every MDM program eventually lives or dies on matching. If the engine fails to link “Intl. Business Supplies GmbH” with “International Business Supplies,” you get duplicate suppliers, split spend and double payments. If it wrongly merges two patients or two customers with the same name, you get privacy incidents and broken trust. For two decades, matching meant carefully tuned rules and probabilistic scoring. In 2025 and 2026, vendors and data teams began adding embeddings and large language models to the mix, promising matches based on meaning rather than spelling.
The new techniques genuinely help with some of the hardest cases. They also introduce cost, opacity and new failure modes. This guide explains the main matching approaches, where each fits, how to combine them in a hybrid pipeline, and how to evaluate and govern the result so it stands up to auditors as well as business users.
Why Entity Resolution Is Back in the Spotlight
Two trends have pushed matching up the agenda. First, AI agents and assistants now act on customer, supplier and product records directly, so duplicate or wrongly merged entities cause immediate errors rather than slightly off reports. Second, data volumes and variety have grown: product descriptions in many languages, supplier names transliterated from different scripts, and customer interactions captured in free text.
Analysts reflect this. Gartner’s 2026 Magic Quadrant for MDM Solutions, as summarized on a market definition page, keeps profiling, matching, linking and merging as mandatory capabilities, and highlights vendors using generative AI and machine learning to automate stewardship tasks including entity resolution. Vendors have followed: Profisee’s 2026.R1 release introduced AI vector matching that uses embeddings to match records by meaning, giving the example of matching “H2O” with “Water” without synonym lists.
The Four Families of Matching Techniques
1. Deterministic Rules
Exact or normalized comparisons on strong identifiers: same tax ID, same LEI, same email after normalization, same GTIN. Rules are fast, cheap, transparent and easy to audit. They fail when identifiers are missing, mistyped or inconsistent, which is precisely when matching is hardest.
2. Probabilistic and Fuzzy Matching
Classic record linkage, in the tradition of the Fellegi–Sunter model, compares records field by field using string similarity, phonetic codes, date tolerances and address standardization, then weights each comparison by how informative agreement or disagreement is. Scores above an upper threshold auto-match, scores below a lower threshold are non-matches, and the band in between goes to stewards. This remains the workhorse of most MDM platforms because it is explainable and tunable.
3. Embedding-Based Similarity
Embedding models convert names, addresses or descriptions into vectors so that semantically similar text sits close together. This handles abbreviations, synonyms, word order and some multilingual variation that string metrics miss, and it is particularly useful for product data with free-text descriptions. Embeddings are also effective for candidate generation: finding the small set of plausible matches for each record using approximate nearest-neighbor search, instead of comparing everything with everything.
4. LLM Adjudication
A large language model is given two or more candidate records, plus context, and asked whether they refer to the same entity and why. LLMs can apply world knowledge, for example that a brand belongs to a parent company or that two addresses are the same building described differently, and they produce natural-language explanations. They are also slow and expensive relative to other methods, can be inconsistent across runs, and may be confidently wrong.
Use the cheapest technique that can decide each pair with confidence. Rules for the obvious, scoring for the measurable, embeddings to find candidates, and LLMs only for the hard residue, always with a human-auditable trail.
Decision Guide: Which Technique for Which Problem
The right choice depends on your data and the cost of errors. Use the questions below to decide.
- Do you have strong shared identifiers for most records? If yes, start with deterministic rules and use other techniques only for the remainder.
- Is the main variation spelling, formatting or typos? Probabilistic matching with good standardization is usually enough and easier to govern.
- Is the variation semantic or multilingual? Product descriptions, transliterated names and abbreviations benefit from embeddings, at least for candidate generation.
- Are hard cases rare but valuable? LLM adjudication can be worth its cost on the uncertain band, especially if it reduces steward workload.
- What is the cost of a false merge versus a missed match? In healthcare, financial services and anything involving payments, false merges are usually far worse. Keep auto-merge thresholds conservative and require deterministic evidence.
- Must decisions be reproducible for audit? If yes, the final decision should come from defined logic, with AI providing candidates, scores and explanations rather than the verdict.
That last point echoes Ataccama’s argument that AI should not independently decide that records match, because defined matching logic gives reproducible, auditable outcomes that financial reporting and regulation require.
A Hybrid Matching Architecture
In practice the strongest designs combine all four techniques in a pipeline where each stage narrows the problem for the next.
- Standardize. Clean and normalize names, addresses, phone numbers and identifiers; map codes to reference data such as ISO country codes. Good standardization improves every downstream technique.
- Block and generate candidates. Combine traditional blocking keys (postal code, name prefix) with embedding-based nearest-neighbor search so that each record is compared only with plausible candidates.
- Apply deterministic rules. Auto-link pairs that share strong identifiers and pass sanity checks, and auto-reject pairs with conflicting strong identifiers.
- Score the rest. Use probabilistic comparisons, optionally including embedding similarity as one feature among many, to produce a calibrated match score.
- Adjudicate the uncertain band. For pairs between thresholds, ask an LLM for a judgment and explanation, or route directly to stewards depending on risk.
- Steward and learn. Stewards review LLM-assisted proposals with the reasoning attached; their decisions become labeled data to tune thresholds, weights and prompts.
- Merge with survivorship. Once linked, apply explicit survivorship rules to build the golden record, independent of how the match was found.
Ataccama’s description of headless MDM shows the operational side of this: agents triage match proposals in bulk, separate clear cases from those needing a steward, explain why records were proposed as matches, and resolve only clear cases, with riskier and permanent changes requiring human approval.
Designing LLM Adjudication That Holds Up
If you use an LLM to judge uncertain pairs, treat it like any other production component: specify it, test it and constrain it. A few design practices make the difference between a useful assistant and an unpredictable one.
- Structured inputs. Present both records field by field with labels, standardized values and the source system, rather than pasting raw rows. Include relevant reference data, such as the meaning of codes.
- Structured outputs. Ask for a fixed schema: decision (match, no match, unsure), the attributes that support the decision, the attributes that conflict, and a short rationale. Reject responses that do not conform.
- Explicit decision rules in the prompt. State domain rules the model must respect, for example “different tax IDs in the same country mean different legal entities” or “a subsidiary is not the same entity as its parent.” This aligns the model with your survivorship and hierarchy policies.
- An “unsure” option. Forcing a binary answer inflates confident errors. Route “unsure” to stewards.
- Consistency checks. For high-impact pairs, ask twice with records in swapped order, or compare with a second model, and send disagreements to humans.
- Few-shot examples from your own data. A handful of labeled, domain-specific examples, including tricky non-matches, usually improves results more than generic instructions.
Cost control matters too. LLM adjudication should apply only to the uncertain band after cheaper stages have done their work. Cache decisions so the same pair is not re-adjudicated on every run, and batch requests where your platform allows.
Real-Time Matching for Agents and Applications
Batch matching consolidates historical records; real-time matching prevents new duplicates at the point of entry. When an AI agent or a web form creates a customer or supplier, it should call a “resolve or create” service that searches for existing candidates first. The same hybrid logic applies, but with latency limits: deterministic and probabilistic stages run inline, embedding search uses a pre-built index, and uncertain cases create a pending record or steward task rather than waiting for a slow model call. This is where MCP-based MDM tools become valuable, because agents can resolve entities through a governed interface instead of inventing new records.
Evaluating Matching Quality Properly
Matching projects often fail because they are judged by anecdotes. Build a measurement discipline before changing techniques.
- Create a labeled evaluation set of record pairs, including easy matches, hard matches, near-misses and known false merges, reviewed by business experts.
- Measure precision and recall separately. Precision tells you how many proposed matches are correct; recall tells you how many true matches you found. Report both, along with cluster-level metrics if you merge groups of records.
- Weight errors by business cost. A false merge of two high-value customers may matter more than ten missed duplicates of inactive suppliers.
- Track steward workload. The size of the uncertain band and the time spent per decision are as important as accuracy.
- Re-test on every change. New embedding models, prompt edits or threshold changes can shift results. Version models and prompts, and keep the evaluation set stable.
- Measure cost and latency. LLM calls per thousand records, and end-to-end time for batch and real-time matching.
Worked Example: Consolidating Supplier Records Across Three ERPs
A manufacturing group merges supplier masters from ERPs in Germany, Vietnam and the United States. Names appear in different languages and formats, tax IDs are missing for many smaller suppliers, and addresses use different conventions.
The team first standardizes names and addresses and maps countries to ISO codes. Deterministic rules link pairs sharing a valid VAT number or bank account plus country, resolving the bulk of large suppliers. For the rest, blocking by country plus embedding-based nearest neighbors on normalized names and addresses generates a short list of candidates per record. A probabilistic model scores those pairs using name similarity, address components, phone numbers and embedding similarity.
Pairs above the upper threshold are linked; pairs below the lower threshold are dismissed. The uncertain band, which contains many transliterated Vietnamese and German names, is sent to an LLM with both records and a structured prompt asking for a match judgment, the evidence, and any conflicting attributes. Stewards review the LLM’s proposals in the MDM interface, where the reasoning is attached, approving clear cases quickly and examining conflicts. Because bank details affect payments, no merge happens without either a deterministic identifier match or steward approval. Stewards’ decisions feed back into threshold tuning and the evaluation set, and the uncertain band shrinks over successive runs.
Two design choices in this example deserve emphasis. First, the team treated embedding similarity as one feature in the scoring model rather than a decision on its own, which kept the score calibrated and explainable: a steward can see that a pair scored highly because names were semantically close, the city matched and the phone numbers shared a prefix, rather than receiving an opaque similarity number. Second, the LLM never wrote to the master. Its output was a proposal with evidence, stored with the match link, so auditors can later see why two suppliers were consolidated and who approved it. When the group later adds a fourth ERP from an acquisition, the same pipeline runs with the existing evaluation set, and the team can show before go-live how precision and recall compare with previous runs.
Governance and Privacy Considerations
- Data minimization: send LLMs only the attributes needed to decide, mask sensitive identifiers where possible, and prefer models deployed in your own cloud tenancy for personal data.
- Explainability: store the evidence and reasoning for every automated or assisted match decision alongside the link.
- Reversibility: keep the ability to unmerge clusters and restore source cross-references cleanly.
- Model governance: version embedding models and prompts, document their intended use, and include them in change management.
- Bias checks: verify that matching quality does not degrade for names from particular languages or regions, a known weakness of tools tuned on English data.
Conclusion
Embeddings and LLMs are not a replacement for disciplined entity resolution; they are powerful additions to it. Embeddings widen the net for candidate generation and capture semantic similarity; LLMs help adjudicate the hardest cases and explain them in plain language. Rules and probabilistic scoring remain the backbone because they are fast, cheap and auditable. The winning pattern is a hybrid pipeline with clear thresholds, steward review of uncertain cases, a stable evaluation set and explicit survivorship. Build it that way, and better matching translates directly into cleaner golden records and safer AI agents.
How Hanoi Technology Can Help
Hanoi Technology designs and tunes hybrid entity resolution pipelines, combining rules, probabilistic scoring, embeddings and LLM adjudication with stewardship and evaluation built in.