TDWI Upside - Where Data Means Business

6 Considerations when Evaluating an Advanced Analytics Product

When you select any technology product, you need to do a thorough evaluation. Here are six important considerations for evaluating an advanced analytics product.

With the big data revolution happening around us, there are many advanced data analytics products available for both retail and enterprise consumption. There are products for fraud detection, for recommendations, for sentiment analysis, for game matchmaking, for preventive maintenance of equipment, and so on.

How can an enterprise select the right data product? As with selecting any technology product, you need to go through a thorough evaluation process -- looking at your business drivers, technology landscape, and organization culture. However, with an analytics product there are specific nuances that need to be considered.

Here are six important considerations for evaluating an advanced analytics product:

#1: Look Beyond the Magic

The world of data and machine learning is intriguing to most of us. There is great excitement when a machine thinks like a human or a superhuman. Yet this shouldn't prevent you from making rational decisions.

You'll need to ask a few questions. For example, will using this product to provide recommendations to your customers actually increase your revenues over time? Would it be better if you used people to complete a process such as categorization rather than automating it?

As with any other business decision, you need to look beyond the magic and hype to make sure you are getting the best value.

#2: Understand All Data Sources

Many advanced data analytics products rely on several reference data sources to build their underlying models, not just on the data that your systems provide. A payment fraud detection product could use German credit fraud data to build its models. A product recommendation engine could use the Google merchant catalog for reference.

You must understand which data sources the product has used. There may be cases where you don't trust the data source or where the data source is not adequate to meet the nuances of your business. There could also be legal or financial implications for your business in using certain data sources.

#3: See Output for Yourself

Given that there is usually a lot happening "behind the scenes" in an analytics product, you must have a high degree of trust and confidence in the team behind the product. However much you might respect the data science team behind the product, you cannot be sure of the product until you see it delivering results for you.

It may be worthwhile to see the product work in your business context. If possible, perform testing to see if the product will meet your business objectives. In some cases, you might also want to conduct tests of different product configurations to make sure you are tuning it right.

#4: Check for Self-Correction

Just like humans, many analytics solutions can learn and self-correct. A recommendations engine could learn from the customer purchase patterns. A fraud detection engine could learn from the payment settlement history.

You might want to check if the product has a provision for self-correction. You might also want to check what "self-correction" implies -- does it happen behind the scenes, does it require an explicit refresh of the data or the model, is there provision for human input to aid the machine with self-correction?

#5: Assess Impact of Data Volume

The product must be able to process increasing data volumes. Although this is true for any technology product, it is vital in advanced analytics because many such products are used with big data and in real-time response.

The rate of data volume growth could differ across businesses. If the product is dependent on external data sources that are growing in volume, you should consider those as well.

#6: Don't Forget Integration

Often we get so carried away with the data science behind a new product that we forget the context in which it needs to be used. Make sure the product comes with the necessary hooks -- APIs or services or data feeds -- to connect to your ecosystem. You do not want to face unexpected effort when integrating the product with your existing systems.

About the Author

Lakshmi KP is head of business engineering at Object Edge, a digital solutions company that provides products, consulting, and implementation services for eCommerce. You can contact the author at lakshmi.kp@objectedge.com or at https://www.linkedin.com/in/lakshmikp/.

TDWI Membership

Accelerate Your Projects,
and Your Career

TDWI Members have access to exclusive research reports, publications, communities and training.

Individual, Student, & Team memberships available.