Why Has My Data Not Matched?

The difference between data matching using rules-based and fuzzy matching, typically happens due to the variances in how these two approaches work.

1. Rules-Based Matching:

• What it is: Rules-based matching involves creating specific, predefined rules that outline how data should match. For example, exact matches, ranges, or specific conditions within your data.
• How it works: It is a process where you set exact conditions (e.g., “Name = ‘John Doe'” or “Date >= 2023-01-01”). If data does not meet these conditions exactly, it doesn’t match.
• Reason for data not matched: If the data has slight variations (like a typo, different formatting, or different wording), the match will fail. For example, “John Doe” might not match “Jon Doe” due to the different spelling of “John,” or “2023-01-01” might not match “01/01/2023” because the date formats differ.

2. Fuzzy Matching:

• What it is: Fuzzy matching is a technique used to find strings that are approximately equal to a given pattern, allowing for minor errors, typos, or variations in the data.
• How it works: It typically uses algorithms (like Levenshtein distance, Jaro-Winkler, or cosine similarity) to calculate how “close” two pieces of data are, even if they are not identical. The system tries to match data that are similar but not necessarily the same.
• Reason for data not matched: While fuzzy matching can find matches even when the data differs slightly, it might still fail in cases where the differences are too large. For example, “John Doe” might match “Jon Doe,” but it might not match “Jane Doe” because of a bigger difference in the names.

Why the Data Has Not Matched:

• Exact vs. Approximate: Rules-based matching only works when there is a perfect match (or matches that follow the exact rule), while fuzzy matching works with approximate matches.
• Data Quality: Data in the system may not be standardised or cleaned, which affects how both methods work. Variations in spelling, date formats, or even the use of abbreviations might cause mismatches in rules-based systems but can still be caught by fuzzy matching (depending on the threshold set).
• Matching Threshold: In fuzzy matching, the degree of similarity is important. If the threshold is set too high, then only very similar data will be matched, which could lead to missing matches. If the threshold is set too low, irrelevant matches might occur.

In summary, rules-based matching will always follow a strict set of conditions, while fuzzy matching allows for some flexibility, but the balance between these methods depends on the level of variation you’re willing to accept in your data and how you define “match.” On the other hand, mismatches happen when one technique is too rigid (rules-based) or when fuzzy matching is too lenient or set with an inappropriate similarity threshold.

Our advice for any organisation is that a decision-maker should not use fuzzy logic systems for mission-critical data, but it may be acceptable for some uses. However, if you want to have complete control of your address matching and be confident in the accuracy of your data and decision tree, you should only be using rules-based logic. At Hopewiser, we work closely with our partners and clients to determine their specific rules-based logic, what is, and what is not allowed to match, based on specific rules. This will give the very best outcome for the client.

We offer full Data Quality Services to help you maximise the potential of your data.

If you would like to learn more about how to assess your data then download our FREE Ultimate Data Guide for 2025.

For more information about what we can do for your data, click here to Contact Us.