TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

Data Science for Everyone

Statistical models and machine learning algorithms are often mysterious and confusing for average business and data professionals. However, even the most math-adverse can reach some understanding.

• By William McKnight
• February 22, 2017

Analysts, leaders, and other business professionals often see statistical modeling and advanced analytics as confusing and complex. In fact, many will not even attempt to understand them. They see data science as something beyond their grasp -- a world for Ph.D.s or people with rarified abilities. This should not be the case.

I often remind people that you don't have to be a mechanic or an engineer to drive a car. In the same way, you don't have to be a statistician or a data scientist to use statistical modeling -- or at least to understand what a model is telling you about your data. Just as you don't have to know the torque ratio of your transmission gears to drive your car to the grocery store, you don't have to know the complex math and theories underlying every statistical algorithm.

How to Understand an Algorithm

There are three keys to understanding statistical modeling and machine learning in a practical manner:

• Know what a particular model (or test) tells you (and what it does not)
• Know when to use that particular test
• Find a practical example of how the technique can be used in real life

Let's take a popular statistical machine learning algorithm with a scary name -- binomial logistic regression -- and break it down so any business professional can understand what it does and how to apply it in the real world.

What It Tells Us

Although binomial logistic regression sounds complex, it really isn't once you break it down. It's in the "regression" family, which means that the end result is an equation that will offer a prediction. Think of a best-fit line or a linear regression where you have an equation: y=Ax + B. A regression will provide values for A and B, you provide the x, and voilà, you get a prediction, y. This is similar to the FORECAST function in Excel.

"Logistic" tells us the data follows a messy logarithmic curve, rather than a neat straight line -- which is often the case with real world data. Finally, the "binomial" is a fancy way of saying "binary," which means it is predicting a binary outcome -- 1 or 0, on or off, pass or fail, or any outcome that has only two possibilities.

When to Use It

As the name implies, you can use binomial logistic regression anytime you need to predict a yes/no outcome and you have several numeric factors that possibly contribute to or be used to predict the outcome. For example, if you wanted to predict the probability that employees leave an organization, their salaries and tenures (years or months of service) might be good factors to try in the model.

You teach the model by feeding it past cases along with the actual outcomes of those cases. The more data it has to learn from, the more accurate it can be.

The model's resulting regression equation gives you a percentage probability of the "yes" case happening based on the factors. It works for individual cases, or you can add up all probabilities and get an overall score for the event -- whether it's good (overall chance of success) or bad (overall risk).

A Practical Example

Imagine you want to detect potential fraudulent bank card transactions. The factors to indicate fraud could be the number of transactions in a 24-hour period, the dollar amount, and/or the geodesic distance from the card holder's physical address (how far away from home the transaction was made).

Higher values than normal could suggest fraud, and the binomial logistic regression could spit back a percentage chance of the transaction being fraudulent (a number from 0 to 1). You can set a threshold of, say, >0.7 to notify the cardholder or (if you are really confident in the model) a threshold of >0.9 to deny the transaction.

What I hope this example illustrates is that although statistical models and machine learning algorithms may have cryptic names and perplexing equations, a practical grasp of understanding is attainable by even the most math-adverse professionals. Understanding what the models are actually telling us, when they are used appropriately, and how they can be used in the real world will demystify them.

Armed with a reasonable amount of practical knowledge, statistical models can be understood by business professionals and used by most analysts. In this age of big data, we have the analytical tools to go beyond averages and percentages. Let's use them!

For Further Reading:

• Data Scientist Skills for Success
• Why Data Science Must (And Will) Be Automated
• Data Science Is More than Data: Cultivate Business Knowledge and Relationships