Ultimate Guide: Mastering Month-End Closing With Microsoft Dynamics NAV 2017

Download

It is well-understood that CRM data is like gold.

We also know that both people and processes can lead to bad data – including duplicate data – in CRM. But what is so bad about bad data?

CRM Duplicate data prevents you from building a holistic view of customers. It can also make it nearly impossible to communicate with your customer base in a meaningful way. However, companies often don’t realize how big the problem is until it’s time to use, report on, or mine their data.  Imagine trying to pull contact information for a marketing campaign and having thousands of duplicate records, each with disparate pieces of information.

Using Microsoft Dynamics CRM duplicate detection rules and jobs, this post outlines the steps you can take to minimize the introduction of duplicate records in your system.

1. Get Clear About Your Definition of CRM Duplicates

When Microsoft Dynamics CRM duplicate detection rules are enabled the application provides users creating or editing records with a warning when a duplicate record is detected. However, the duplicate is not automatically resolved and the warning is not enforced. This mean that users need to decided how to proceed. Our dream CRM user would resolve the potential duplicate. However, most CRM users choose to ignore the warning and save the record anyways.

In short, users have the freedom and autonomy to decided whether the record is a true duplicate. However, they also have have the responsibility not to ignore and dismiss warnings for true duplicate records.

The definition of duplicate should be driven by how the data is being used later.

For instance, if Marketing will be targeting campaigns at Leads and Contacts then Lead records should never be created for existing Contacts.

It is surprising how often the definition of duplicate varies between individual users or different departments. Consequently, it is necessary to build consensus across the organization on what constitutes a duplicate.

  • Eg. If a user creates a brand new Lead in the system for a Contact that already exists, is this a duplicate?
  • Eg. If a user creates a subsidiary Account of another corporate Account and they both share the same website or head office address, is this a duplicate?

2. Review Pre-Configured CRM Duplicate Detection Rules

Microsoft Dynamics CRM includes a number of pre-configured Duplicate Detection rules. However, it’s important to review these rules because many of the pre-configured options don’t make sense for most organizations.

Dynamics CRM Duplicate Detection Rules

 

For instance, “Contacts with the same first name and last name” is a rule that should not be enabled in most systems. There are a lot of unique individuals with the same First Name and Last Name. It’s definitely possible to do business with 10 different people named John Smith! We do not want to desensitize users by configuring superfluous rules that will pop up every time they try and create or edit a record.

 

Like What You've Read? Subscribe to Our Newsletter!

 

3. Configure CRM Duplicate Detection Rules

Well-configured Dynamics CRM duplicate detection rules can be a powerful way to minimize bad data. However, you must fully understand how to build the rules. There are a couple less known facts to be aware of:

  • The criteria in a duplicate detection rule is applied as an ‘AND’ statement, not as an ‘OR’ statement.
    • example: Some system administrators will create one massive duplicate detection rule for Contacts with the same Name, Email, Business Phone, and Street Address. Rarely will this duplicate detection rule be applied in the system, as it relies on users entering the data across all these different fields in exactly the same way. It may also rely on optional fields that users may not be required to fill out.
  • When configuring duplicate detection rules, you must be strategic.  You can use the ‘AND’ statement to create powerful rules.
    • example: Remember earlier, when we mentioned that the ‘Contacts with the same first name and last name’ rule can lead to negative behaviour in end users? To avoid inundating users with useless duplicate warnings, we can easily tweak this rule to make it more useful. More specifically we can specify that, in addition to the first and last name, the Company Name must also match. With this rule in place a warning about a duplicate John Smith at ABC Company finally becomes useful for users.

CRM Duplicate Detection Criteria Eg

Publish your Duplicate Detection Rules prior to importing data

Duplicate Detection Rules are only applied when they are published. Unpublished rules remain in a draft state and are not considered in the system. Using the Duplicate Detection Settings in CRM you can specify when published Duplicate Detection Rules are applied. It is recommended to have Duplicate Detection Rules published prior to data imports in order to avoid introducing duplicates into your system en masse.

 

Enabling CRM duplicate detection - settings

 

TIP: Remember, duplicate detection rules in the system are not enforced.  Users may dismiss and ignore duplicate warnings. As part of a complete data quality management program it is important to proactively check for duplicate data in the system.  This means running a Duplicate Detection Job (See Step 5).

4. Train Users

Going through the exercise of defining, communicating, and configuring Dynamics CRM duplicate detection rules is not enough. Users must be trained on what to do when they receive a duplicate alert. Otherwise it’s an unwritten adventure with regards to how users will react.

The busy user may dismiss an alert and save the record anyways. The conscientious user may even try and help by deleting one of the duplicates. It is important to provide instructions on how to handle duplicate warnings in the system. More specifically, either merging existing duplicates or not creating a new duplicate record altogether.

During training users should understand the importance of merging records as opposed to deleting them. After all, deleting records in CRM leads to lost data and orphaned records.  Any sales activities tracked against a deleted Contact will potentially disappear. Any Opportunities once related to a deleted Contact are potentially orphaned. Bad processes will lead to bad data.

5. Set Up and Run CRM Duplicate Detection Jobs

As previously mentioned, minimizing the likelihood of duplicates in your system will require a bit of proactive effort. Luckily, duplicate detection jobs can be a big help.

 

CRM Duplicate Detection Job Settings

 

The process to set up a Duplicate Detection Job involves defining which entity and records to run the job on. The job will apply published Duplicate Detection Rules to the data set specified.

 

CRM Duplicates Webpage Dialog

 

A Duplicate Detection Job will not resolve duplicates.

Instead, it will highlight potential duplicate records that you may action by merging. Properly trained users will know what action to take when duplicates are detected.

 

View CRM Duplicates

 

6. Get into a Rhythm of Proactive Data Cleanup

Setting up Dynamics CRM Duplicate Detection Rules and running a Duplicate Detection Job is not a one-time event.  Organizations must be diligent in data quality management.  Proactively managing and addressing duplicate records helps to ensure that users are not left scrambling to make sense of data during critical times such as Year End Reporting or Marketing Campaigns.

Duplicate management should include scheduling regular Duplicate Detection Jobs.  It’s possible to schedule CRM Duplicate Detection Jobs to run automatically based on a specified interval and to receive an email alert after each job. The image below is an example of a scheduled duplicate detection job with email notification.

 

CRM Duplicate Detection Email Notification

 

Duplicate Detection Rules should be reviewed regularly after deployments to CRM. Many organizations do not realize that if they are publishing solutions to CRM which include metadata changes to an entity the Duplicate Detection Rules for that entity will be unpublished. This may not seem intuitive. For a deeper explanation check out Surviving CRM’s post.

A simple best practice is to add a Duplicate Detection Rule check to your organization’s post-deployment checklist.  This will ensure that Duplicate Detection Rules are reviewed after each deployment.

Is this where bad data ends?

Your CRM data is valuable and it’s worth protecting.  Using out of the box CRM functions and defining clear business processes will help your organization maintain quality data.

If you have any questions about this post or about Duplicate Data in your CRM, please don’t hesitate to reach out to us. Or, leave a comment below!

Comments

  1. Laura

    Thanks this is a really logical and well written article that has opened my eyes about the purpose and complexities of defining a duplicate!

  2. Brian Mahoney

    This is all great information and I appreciate it, but what if a contact is entered that is a duplicate of another contact, using the rules we catch the duplicate, but want to create an opportunity record now? How would I set that up?

  3. http://www.scribd.com

    I have been examining many of your articles and i must say nice stuff.
    I will definitely bookmark your site.

  4. TOM

    Hi! Can you help me with something, please? I will describe you the situation:

    One of my coleagues imported a list of contacts but unfortunately he wrote the first name in the place of last name and viceversa. Majority of the contacts were in the system but written right and now I have a lot of duplicates. Is there a way to find all the duplicates? Is there a way to create a rule for this kind of situations?

    Thank You!!

    1. Martha

      You could try creating a duplicate detection rule for Contacts, and search for records where last name equals first name AND first name equals last name?

      1. Mike

        How would you do that? When you are looking at just one Entity it seems to only match the field to the same field. I have a similar problem where I’m trying to compare Business Phone to Other Phone and don’t know how to match it.

      2. margie

        Similar issue here – I’ve created a field that specifies “Valid Duplicate” i.e., there are multiple people with the same Full Name and this is one of them. I’d like to create a duplicate rule that excludes records with this option field selected. No dice – I only have the “Exact Match” rule available.

  5. James Traub

    The criteria for a single rule is using an AND but what about multiple rules? Are multiple rules using an OR?

  6. Prkdynamic

    Can we stop triggering duplicate detection rules during update event. we need it to trigger only during create event. Please suggest.

Leave a Comment

Your email address will not be published. Required fields are marked *

*

Filter:

Archives

Register to recieve the latest ERP & CRM insights