TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

Using Advanced Analytics for Societal Good

If data can help us solve problems and improve the quality of life, we must promote and publicize analytics efforts that help advance humanity.

• By Sri V. Raghavan
• October 20, 2017

The Data on Purpose conference at Stanford University earlier this year featured some of the best minds discussing the virtues of using data to further the causes of corporate social responsibility. After seeing the presentations, and given my job in technology, I became very interested in how using data and analytics can make a difference to society.

I admit I was behind the curve in recognizing the burgeoning area of applying analytics to data to deliver critical insights for the greater social good, thinking progress in the field was exclusively occurring in corporations. After much research, I can now report that systems driving efficiencies in the private sector can and are being deployed for policy implementation in the not-for-profit world.

For Further Reading: Nonprofits Have Huge Need, Little Understanding of Analytics Overcoming the Hurdles of Health Data Analytics How Analytics May Improve Healthcare Outcomes

Measuring Impact

Think of credit scoring algorithms, predictive models that provide consumer purchase likelihoods, or customer churn models that determine an individual's propensity to take her business elsewhere. The same principles that have been instrumental in the development of these algorithms have been effectively repurposed to address some vexing questions that plague us as a society.

Projects such as the Impact Genome Project were created with the sole purpose of using program-evaluation data to deliver a set of solid criteria to determine the effectiveness and the efficacy of social programs -- to the extent that some business vernacular has been co-opted to deliver outcome metrics such as "single unit of impact."

The Impact Genome Project has collected more than 75,000 data points and has identified at least 125 types of social outcomes, the idea being that once we know the preference for a particular social outcome and how that outcome is measured, evaluating programs becomes straightforward. The added effect from all of this is greater efficiency in administering a social program and maximally impacting a given population.

Prediction as Prevention

A good example of advanced analytics positioned to be a formidable ally in delivering an outcome-based evaluation of social programs for the greater good of humanity can be seen in disease prevention. The default protocol for disease prevention is to monitor patient progress after a set of symptoms manifests or upon the full onset of an affliction and then devise a therapeutic course of action to prevent its spread. Although this course often results in complete cure, the physical, psychological, and economic damage may already be significant. In many cases the cure is well beyond the reach of the treatment and sometimes the damages are long lasting.

Now enter the world of precision medicine. In this new era, a large volume of data is collected on widely diverse variables -- including genes, localized environment, individual lifestyle, pre-existing conditions, socioeconomic factors, and demographics -- to develop predictive models that pinpoint individual susceptibility to diseases.

Imagine knowing far in advance of the onset of a debilitating condition that you are likely to be struck by a certain illness. Such predictive models may give us enough information to take corrective action in our lifestyles to disrupt or reverse the progress of a disease or perhaps prepare for its arrival by taking the necessary precautions.

Now, further imagine this being applied to an entire population that is particularly vulnerable to certain kinds of afflictions. The more we deliver powerful predictive models that foreshadow the incidence of these ailments, the better off we will be in preparing ourselves to successfully combat endemic diseases when they actually arrive.

There are many examples of advanced analytics having a positive impact on other areas, such as anticipating and protecting against environmental disasters, developing lifesaving medical interventions, and creating social programs for women and children in less developed economies.

Alternate Data Sources

Although wanting to create good outcomes with data is a worthy goal, it is not easy. Sometimes crucial data is missing, and that can hamper our ability to assess available policy options. This is where alternate data sources can be used as effective proxies.

Harvard economist Senthil Mullainathan shows how several new techniques and unusual data can be used to accurately assess economic activity. For instance, satellite photos in Uganda can be used to estimate harvest sizes so earnings from agriculture for the year can be accurately forecast.

Repurposing Corporate Innovations

The era of big data has been upon us for a while. Many for-profit organizations created solutions, products, and platforms that are hyperversatile in terms of ingesting, curating, processing, transforming, and analyzing data while providing intuitive ways of visualizing and operationalizing insights, often in near real time. There are many examples in the commercial world where data is not only an asset but also a positive force that makes companies successful, consumers happier, processes more efficient, patients healthier, and products better.

In the nonprofit world, the magnitude of challenges is no less serious and may be more critical for human progress. It is a cause for warm comfort that technology companies have contributed handsomely towards delivering the expertise and solutions that make all our lives better. More needs to be done, but the culture of making decisions based on solid data analysis is here to stay and is becoming more embedded in the day-to-day operational and the longer-term strategic ethos across all organizations.

If we believe in the value of data to help solve problems and improve the quality of life, we must begin new conversations that spark progress and call attention to all efforts resulting in successful applications of analytics to help advance humanity.