As all financial organistions must demonstrate compliance with an ever growing number of rules and legislations, many vendors of customer screening systems will state that their products incorporate automated Customer Due Diligence (CDD) processes. After all, screening customers on a regular basis to see whether there are suspicious or sanctioned individuals and organisations in the data set, is an integral part of risk management.
The real challenge, however, lies in the quality of the screening process. This process must be aimed at cost efficiency, better customer experience and operational advantages. And that is only possible if you "understand" the data you are matching.

High precision matching (HPM) is a method in which probabilistic and deterministic methods are being deployed to achieve the best possible result. In other words, high precision matching uses fuzzy logic methods combined with knowledge on names, naming conventions, legal forms, companies, abbreviations, acronyms, cultural habits and the like.
It is, in fact, the ideal method to answer to the requirements of a really sophisticated screening process.
Let's have a look at some of those requirements:

• Being able to match against a large variety of external and internal lists, even if these lists include names of persons or organisations that are not in Latin script (think of Arabic, Mandarin Chinese, Hebrew, etc). HPM is applying solid transliteration capabilities in order to come up with highly reliable results. In addition, the results of all matching against all the lists are being consolidated in to one single view.
• Generate realistic matching scores: The data in a lot of lists is often misspelled, incomplete or sequenced incorrectly (Xao Yin Pin <--> Pin Yin Xao). Furthermore, aliases and nicknames are being used, as well as all kinds of different date notation. Keeping this in mind, a  matching score of 100% (as provided by many vendors) is not realistic at all. HPM generates an accuracy score that is congruent with the quality of the records that are being compared, without missing the actual match!
• Reducing the number of false positives: If the matching tool produces matches, that are in fact no-matches (no accurate hits against the different suspect lists), a lot of manual rework is involved to actually check these false positives. With HPM, a reduction of false positives up to 90% has been proven in the field.
• Not missing the real risk (no false negatives). If a real risk is missed and not flagged for further processing, organisations will have to suffer the consequences of such false negatives.

As I said, these are just some of the requirements for a sophisticated screening process. High precision matching is the main provision in achieving cost efficiency, better customer experience and operational advantages. To learn more, please read our white paper "Customer Due Diligence: automated screening with High Precision Matching" .
Enjoy!
​

In a recent publication of the Customer Experience Network in association with Forrester, some interesting research facts on trends, challenges and investments for the next 12 to 18 months are emerging.
CX Network conducted a survey and collected answers from more than 700 respondents. About 25% of respondents were solution providers, 65% were working in marketing and customer experience management and another 10% consisted of industry analysts,  bloggers and researchers.
There's is a lot of interesting stuff in the publication (presented as an infographic), but for this blog post I would like to focus your attention on the investment part. 
As the survey respondents were being asked to give their top-3 investment priorities, I was pleasantly surprised to see that the clear frontrunners in these predictions were "customer loyalty and retention", "customer centricity", "customer insights" and "data & analytics".

In the framework of our own Customer Reference program, I recently conducted a survey at our existing customers. In addition to questions such as their main purchasing driver to buy a Human Inference solution and whether they evaluated other vendors in the purchasing evaluation, I also asked them some quantitative questions with regard to to the impact of our solution: cost savings, revenue growth, impact on customer satisfaction , retention increase and data processing time.
Having plotted the survey outcome, together with marketing validation interviews, we found that for our customers the average increase in customer satisfaction & retention was 25-50%, where cost reduction and revenue growth showed an average increase of 5-10%.
Keeping in mind that the CX Network research also showed that the top investment challenge for 2016 is going to be able to demonstrate ROI.

Our customers actually proof what the respondents in the CX Network research predict.
This is nicely illustrated in the quote by Phillipa Snare, EMEA Global Business Marketing Director of Facebook: 
"Everyone talks a lot about customer data, loyalty and retention, and CRM systems; they talk about gathering and analysing the data but the most important thing is what you do with it afterwards. The focus seems to be in the wrong place and companies need to start talking: ’What are the experiences you want to deliver customers?”

​I believe you, Phillipa!

Last week, I read about the Google announcement on their Analytics 360 suite. Gartner states that it is Google's intention to compete in enterprise data-driven marketing and analytics. Interesting stuff, which once again shows data management is more dynamic than ever....

All this got me thinking about the latest developments within Human Inference. In our R&D department, people have been working on deploying machine learning for duplicate detection in high volume data files. In machine learning, algorithms are used, that can learn from and make predictions on data. The algorithms build a model from example input data, in order to make data-driven predictions or decisions.
Now this is quite a leap, but also the next logical step for Human Inference. Traditionally, we use large knowledge corpora with deterministic and probabilistic matching methods to achieve high precision matching results. We've done this for more than 30 years now and we've had great commercial success with that approach.
But, as the results at a customer site show, matching with our machine learning solution is extremely promising. After three days of training, the new Human Inference matching engine surpassed the results of 20 years of  "homegrown" matching efforts of that particular customer. 
So why not use both approaches, or even better.... combine the approaches into a very powerful matching engine and get the best of both worlds? 
Think of the surplus value of adding relevant knowledge to a versatile and very fast matching engine. Acronym recognition? Gender detection based on first names or salutations?  It is even possible to use machine learning to generate knowledge, which than can added to enhance the knowledge corpora. There are countless possibilities... 
The world of data management is changing rapidly and companies that want to keep up with the pace, must act now. Be ready, be smart!

.During the Gartner Enterprise Information and Master Data Management Summit this week in London, François Ruiter and myself presented the impact on implementing privacy rules and Customer Due Diligence (CDD) regulations in an MDM solution.
Here's a short summary on our approach to advance customer insights in MDM....
In a very well attended session we first explained that high precision matching is essential in achieving a single customer view. Really knowing your customer, however, reaches beyond the traditional data management methods. This means that the matching must truly deliver high quality results, be it within you 'owned' data environment or be it in matching with data outside that environment.
So we used privacy rules and CDD regulations to demonstrate how that is done. François Ruiter explained the new EU privacy rules and the concept of Customer Due Diligence. He than continued to show how the right to access to personal data is taken care of in the Neopost EDS DataHub. After that, François demo'ed a more tricky one: The right to have your data removed. How is the system able to prove that data has actually been removed?
Finally, we did a real time blacklist check for existing customers and for on-boarding a new customer.
As Ted Friedman explained in his opening keynote of this Gartner Summit: "Creating meaning  in your data starts with semantic reconciliation."This will lead to real trust in your data. I think he is absolutely right!
Please find our brochure on European privacy regulations in the DataHub here.

This year , the European Union has issued a new regulation on the protection of individuals with regard to processing and moving of personal data. This General Data Protection Regulation forces companies to consider the implementation of rules to comply with this new regulation. 
The Regulation uses the term "Privacy by Design" to illustrate that the privacy focus has to be taken into account throughout the entire lifespan of an information system. And here is a direct link to the Data Life Cycle: privacy rules must be considered in storage, modification and removal of personal data. 
Now think about an automated system that processes your customer data. What is the impact of such rules and regulations in the context of data management? 
There is a lot to consider: data storage, data access, rectification and removal of data, data security, audit trails, etc.
Let me give you an example: Article 17 of the Regulation states that a person concerned has the right that his or her data will be removed and that further deployment of the data will stop. Now if you are using an MDM solution, in which several source data records are accumulated in a golden record, you have to ask yourself how to remove the data records and the relations between the records. In addition, you will have to figure out how you will actually prove that  the record has been removed.....
During the coming Gartner Enterprise Information & Master Data Summit in London on 2 and 3 March, my colleague François Ruiter and I will present Know Your Customer - The next level, in which we will talk about and demo the implementation of privacy regulations in our MDM system, the DataHub.
I hope I will see you in London!

I saw an interesting tweet this week about Moore's Law on Big Data: "The amount of nonsense packed into the term Big Data doubles approximately every two years..."
In addition to the ironic reference of the growth of data in recent times, Moore quite funnily draws our attention to something which is quite significant for the "data business": We need a new acronym of buzz word every couple years: CRM, SaaS, ERP, CDI, MDM, Cloud.... are you still there?
As I have stated in a previous blog, I believe that the acronym is never the solution. And I agree with Moore's Law: there is a lot of nonsense going around as far as Big Data is concerned. Having had a lot of discussions on this subject with colleagues, customers, partners and competitors, I have come up with my own Law on Big Data:
​
"The only way to manage Big Data is through a sound and intelligent matching approach."

As organisations are trying to convert all kinds of data (owned, data, social media data, online data, etc.) into competitive advantages, they need a  matching approach for data that oftentimes lack metadata descriptions. Traditional matching methods will not do the trick: they are based on atomic string comparison functions (e.g. match-codes, phonetic comparison, Levenshtein distance and n-gram). The drawback of these functions is that they cannot distinguish between apples and oranges – you end up comparing family names with street names.
For REAL Big Data management you need an engine that will yield a high quality result for matching of records distributed over various heterogeneous data sources. That can only be achieved by combining a probabilistic and a deterministic approach. Please check my blog post on the the Gartner Summit this year to learn more about this approach.

During the Gartner Customer 360 Summit in San Diego in September, Gene Alvarez, managing VP of Gartner's CRM practise, shared some survey results in his presentation "Planning for the Customer Experience of 2020". 
He explained that the biggest challenge customer initiatives in CRM will have to face, are multiple sets of customer data and information existing in disparate systems.












Does this come as a surprise? To me, this feels like the "same old song".....

​For any CRM system in the world, the success of marketing, sales or services depends hugely on the quality of the customer data in that system. In addition, a CRM initiative must be embedded in a sound customer data management strategy. Gartner predicts that “Through 2017, CRM leaders who avoid MDM will derive erroneous results that annoy customers, resulting in a 25 percent reduction in potential revenue gains.”

So, if we look at these challenges, what kind of problems do actually occur in a day-to-day CRM practise? 
In her white paper on smart CRM, Esther Labrie shows some figures to demonstrate the effect of customer data being quite volatile and error-prone:

In the UK alone, between 5 and 8% of the population change address each year, with an additional 400,000 addresses becoming invalid as a result of a resident’s death. A large number of the 265,000 marriages and 115,000 divorces occurring in the UK annually result in a name change on the part of one of the spouses. Meanwhile, thousands of street names, postcodes and names of cities and towns are changed or modified each year, with the majority of the changes relating to the address.
In addition there are the data entry errors and typos made by call centre agents or customers using self-service data management. This may include misspelled names, missing house numbers or incorrect phone numbers. 
​
One can easily imagine the effects of bad data on business  in a single system. As the number of systems with incomplete, incorrect or duplicate data increases, creating real customer experience with a single customer view becomes harder and harder.

Everyone who wants to reap the true value of customer data, must make sure that customer data initiatives are part of a strategic plan. Managing customer data is not a simple task; it requires effort, budget and commitment across the entire organisation.
​Once again, check out the white paper....

This week I as having a discussion with some of my colleagues with regard to the next major release of our MDM solution. This discussion reminded me of blog post I had been working on, but somehow did not find the time to finish. So, being triggered by my colleagues (thank you, guys!), I decided to share my thoughts on one of the most important aspects of Master Data Management for customer data: the matching capabilities of the solutions. 
First things first. As you may (or may not) know, Gartner is using a quite comprehensive definition of MDM for customer data:
A combination of technology, processes and services to deliver an accurate, timely and complete view of the customer across multiple channels, lines of business, departments and divisions drawing customer data from multiple sources and systems.
I like the definition, but , I can also only reach one conclusion: If you really want to deliver this unique customer view across a multitude of channels and sources, you have to be sure that your matching engine is delivering the right data quality.
If we talk about matching, there are generally two approaches: deterministic and probabilistic matching.
Deterministic matching is usually knowledge- and rule based. For example, deterministic matching uses phonetic rule and algorithms for the recognition of acronyms, whereas
probabilistic matching is mostly using a more mathematical approach, in which all kinds of calculations and algorithms are used to determine the degree of similarity.
Look at the example “Jack London Ltd” and “Thompson London Ltd”: The pattern looks the same and a probabilistic method would probably recognise it like this. London is a city ands there is a high probability that this is true.
However, if we combine the two methods, we see that Jack is most a given name, which then changes the signification of the word London. It has become a surname, and we now see a different pattern.
To deliver the best required result, matching engines must combine both approaches. The better a matching is able to determine what is what in a particular context, the better the probability calculation of a certain match or a certain non-match.    
That is actually EXACTLY the way we humans would do this…    

As matching is used as sort of an umbrella term, we tend to think that matching is a "one size fits all-process". But look at e.g. comparing records, performing duplicate detection and merging data into a golden record... It appears that matching intelligently is not a simple task. Here's why:

• There are many different data formats, which sometimes calls for pre-processing to actually make formats compatible.
• Then there is the volatility of customer data. Customer data changes quite quickly. For example: In the UK, around 7% of the people move house every year. In Germany, this number is even higher: 10 %.
• Whenever data is being processed, human errors are being made. We all know it and it is a fact we should take into account when we process the data ourselves…
• The ever-growing internationalisation of business leads to more linguistic diversity and other cultural matching problems: A truck and a lorry are the same thing, but a football and a football are not…

• And of course there always are specific requirements when it comes to matching data. Think about, for example, the degree of precision when matching with sanction lists in an anti-money-laundering-context.
  

So, in my opinion, if you want to assess the quality of your MDM solution, you should start by assessing the matching capabilities of the underlying matching engine. Here are  a few simple guidelines:
•Test with a lot of of real data
•Profile data sources in order to assess accuracy and completeness of data
•Compare and benchmark matching results
•Combine deterministic and probabilistic approaches
•Consider the interface for data stewardship (doubtful matches)
•Consider current AND future requirements
•Think across the border 
If you want to learn more, please download our white paper on High Precision Matching. Enjoy the read!

Last month I came cross a discussion paper by Experian. It's their Global Data Quality Research 2015. For this paper they interviewed representatives of more than 1200 organisations in the UK, the US, Australia, France, Germany, Spain and the Netherlands. These interviews resulted in some interesting facts: 
- 91% of the organisations are using data and data quality to optimise their customer experience
- however: 63% lack a centralised approach to data quality 
- and 83% tell us that revenue is affected by inaccurate or incomplete data
I have seen a lot of research papers over the years that actually convey a similar message. Bad data impacts your business.
However, this paper highlights the growing trend that organisations are increasingly aware of the potential of their data and that they see data as a means to get closer to their customers.
While I was thinking about this trend and contemplating to use it in a blog post...;-), my marketing colleagues at Human Inference were working on a very interesting infographic called "Why Data Management should be on every marketeer's mind". The infographic illustrates that data management will become crucial for the success of marketing departments. Without solid customer data, marketeers cannot create appropriate content, they will continue to shout instead of listen, they are unable to connect the dots in the customer journey, and they cannot reap maximum benefits from their IT investments.
Reading along (it's only two pages, but very informative....), you will find that leading companies have a much deeper understanding of their customers. It is because they
recognise their customers better and (thus) require less already-known information to be handed over before a customer is being serviced.
And that is also the link to the trend stated in the discussion paper: Data is a very powerful means to get closer to your customers.....

Last week I hosted a workshop during the conference "Digital business & eID" in the Netherlands. The theme of the workshop was data quality and fraud prevention and three speakers and myself each held a short presentation, ending in a statement or question to initiate discussion. 
My discussion topic, for example, was: "Without a single customer view, digital business will fail to be successful......" As you can imagine, this led to some lively discussions with regard to the fact that an electronic identity will only be of use if there is a way to establish reference to the original customer data. How do you actually take are of unique identification within the concept of eID? 
Customer data is the new oil, everybody believes in the surplus value of sound data management; but how do you really achieve an actual advantage in digital business?

My co-presenters covered a range of topics  in the general theme of the workshop. Martijn de Boer of CDDN talked about a new approach to increase data quality. Erik van de Poel of BKR (the Dutch foundation for consumer credit checks) presented his organisation's policy for prevention of identity fraud. Helen van der Sluys of the Dutch Ministry of Internal Affairs showed the participants the consequences of of identity fraud and identity theft. She illustrated her story by showing this movie of an "amazing" mind reader in the heart of Brussels. Check it out ......