6 Considerations when Evaluating an Advanced Analytics Product
When you select any technology product, you need to do a thorough evaluation. Here are six important considerations for evaluating an advanced analytics product.
- By Lakshmi KP
- October 3, 2016
With the big data revolution happening around us, there are many advanced data analytics products available for both retail and enterprise consumption. There are products for fraud detection, for recommendations, for sentiment analysis, for game matchmaking, for preventive maintenance of equipment, and so on.
How can an enterprise select the right data product? As with selecting any technology product, you need to go through a thorough evaluation process -- looking at your business drivers, technology landscape, and organization culture. However, with an analytics product there are specific nuances that need to be considered.
Here are six important considerations for evaluating an advanced analytics product:
#1: Look Beyond the Magic
The world of data and machine learning is intriguing to most of us. There is great excitement when a machine thinks like a human or a superhuman. Yet this shouldn't prevent you from making rational decisions.
You'll need to ask a few questions. For example, will using this product to provide recommendations to your customers actually increase your revenues over time? Would it be better if you used people to complete a process such as categorization rather than automating it?
As with any other business decision, you need to look beyond the magic and hype to make sure you are getting the best value.
#2: Understand All Data Sources
Many advanced data analytics products rely on several reference data sources to build their underlying models, not just on the data that your systems provide. A payment fraud detection product could use German credit fraud data to build its models. A product recommendation engine could use the Google merchant catalog for reference.
You must understand which data sources the product has used. There may be cases where you don't trust the data source or where the data source is not adequate to meet the nuances of your business. There could also be legal or financial implications for your business in using certain data sources.
#3: See Output for Yourself
Given that there is usually a lot happening "behind the scenes" in an analytics product, you must have a high degree of trust and confidence in the team behind the product. However much you might respect the data science team behind the product, you cannot be sure of the product until you see it delivering results for you.
It may be worthwhile to see the product work in your business context. If possible, perform testing to see if the product will meet your business objectives. In some cases, you might also want to conduct tests of different product configurations to make sure you are tuning it right.
#4: Check for Self-Correction
Just like humans, many analytics solutions can learn and self-correct. A recommendations engine could learn from the customer purchase patterns. A fraud detection engine could learn from the payment settlement history.
You might want to check if the product has a provision for self-correction. You might also want to check what "self-correction" implies -- does it happen behind the scenes, does it require an explicit refresh of the data or the model, is there provision for human input to aid the machine with self-correction?
#5: Assess Impact of Data Volume
The product must be able to process increasing data volumes. Although this is true for any technology product, it is vital in advanced analytics because many such products are used with big data and in real-time response.
The rate of data volume growth could differ across businesses. If the product is dependent on external data sources that are growing in volume, you should consider those as well.
#6: Don't Forget Integration
Often we get so carried away with the data science behind a new product that we forget the context in which it needs to be used. Make sure the product comes with the necessary hooks -- APIs or services or data feeds -- to connect to your ecosystem. You do not want to face unexpected effort when integrating the product with your existing systems.