All this got me thinking about the latest developments within Human Inference. In our R&D department, people have been working on deploying machine learning for duplicate detection in high volume data files. In machine learning, algorithms are used, that can learn from and make predictions on data. The algorithms build a model from example input data, in order to make data-driven predictions or decisions.
Now this is quite a leap, but also the next logical step for Human Inference. Traditionally, we use large knowledge corpora with deterministic and probabilistic matching methods to achieve high precision matching results. We've done this for more than 30 years now and we've had great commercial success with that approach.
But, as the results at a customer site show, matching with our machine learning solution is extremely promising. After three days of training, the new Human Inference matching engine surpassed the results of 20 years of "homegrown" matching efforts of that particular customer.
So why not use both approaches, or even better.... combine the approaches into a very powerful matching engine and get the best of both worlds?
Think of the surplus value of adding relevant knowledge to a versatile and very fast matching engine. Acronym recognition? Gender detection based on first names or salutations? It is even possible to use machine learning to generate knowledge, which than can added to enhance the knowledge corpora. There are countless possibilities...
The world of data management is changing rapidly and companies that want to keep up with the pace, must act now. Be ready, be smart!