By using website you agree to our use of cookies as described in our cookie policy. Learn More


Predictive Modeling Bound with Analytical Data is a Concept Whose Time Has Come

The strategic use of data can provide strategic advantages for any business -- you simply have to create the plan and make the move. The ROI will be almost instantaneous.

By David S. Linthicum

According to GigaOM's Derrick Harris: "A pair of Indiana University researchers has found that a pair of predictive modeling techniques can make significantly better decisions about patients' treatments than can doctors acting alone. How much better? They claim a better than 50 percent reduction in costs and more than 40 percent better patient outcomes."

The concept behind the research is to provide insight into the use of data as diagnostic tools for physicians, and, more important, to determine their willingness to leverage this data. If our doctors can consider what's actually happening and likely to happen, this study shows that they should be able to make better decisions. I hope my doctor reads this study.

As Harris further reports: "In order to prove out their hypothesis, the researchers worked with 'clinical data, demographics and other information on over 6,700 patients who had major clinical depression diagnoses, of which about 65 to 70 percent had co-occurring chronic physical disorders like diabetes, hypertension and cardiovascular disease.'"

Leveraging the Markov decision processes, they built a model used to predict the probabilities of future events based on events that immediately preceded them. Moreover, they leveraged dynamic decision networks that can consider the specific features of those events to determine probabilities. In other words, the model looks at the current attributes of a patient, and using huge amounts of data, provides the likely diagnosis and the best treatment to drive the best possible outcome.

The results are impressive. The study found "via a simulation of 500 random cases that their model decreased the cost per unit of outcome change to $189 from the $497 without it, an improvement of 58.5 percent." In other words, they could simulate the delivery of better patient care using predictive modeling and analytical data, which also reduced the cost of delivering the care. This is converse thinking with many healthcare providers who believe that the only way to increase the quality of care is to spend more money.

Besides the cost, at least in this simulation, the use of this data provides better patient outcomes. "They found their original model improved patient outcomes by nearly 35 percent, but that tweaking a few parameters could bring that number to 41.9 percent." The ability to leverage this data, coupled with a predictive modeling engine trained to provide this type of diagnostics, results in fewer missed diagnoses and thus reduces the use of ineffective treatments.

This study validates what most of us in the world of BI already know: the use of well-defined and well-designed predictive models, along with valid and clean analytical data, allows us to make much better decisions. Moreover, this technology allows us to lower costs and improves the performance of the business dramatically.

Looking back at the study, I'm not sure that anybody is suggesting we interface with a computer instead of a doctor when we're attempting to correct a health problem. However, we are suggesting that using predictive modeling and massive amounts of health-related analytical data provides a tool for physicians to make better diagnoses, and provides the right treatments the first time. This lowers the cost of care and assists the healthcare industry in providing better care with fewer resources, which seems to be the desired direction.

There are larger lessons to be learned here related to the strategic uses of this technology in general. For instance, we can learn how enterprises can bind predictive models with core business processes (such as the ability to spot quality control issues) and take pre-determined corrective action. Another lesson is to understand how analytics can locate and deal with the likelihood of fraud occurring within a business unit and how we can learn the best actions to take.

What's changed recently to make this technology possible? The commoditization of data storage and processing, including the rise of technologies such as Hadoop that are able to process petabytes of structured and unstructured data in a very short period of time. Another factor is the rise of "rental" computing models, such as the ability to leverage public cloud-based resources for data processing, using only what we need when we need it. Furthermore, the public generally accepts that these types of systems will only enhance our ability to do our jobs, not jeopardize them.

Studies such as the one cited in this article will continue, and I suspect we'll see the same types of results. The strategic use of data is something that will begin to provide strategic advantages within all businesses and all vertical industries. Those businesses simply have to create the plan and make the move. The return on investment will be almost instantaneous.


  David S. Linthicum is a big data and cloud computing expert and consultant. He is the author or co-author of 13 books on computing, including Enterprise Application Integration (Addison Wesley). You can contact the author at

TDWI Membership

Get immediate access to training discounts, video library, BI Teams, Skills, Budget Report, and more

Individual, Student, and Team memberships available.