Q&A: The Human Side of Big Data
Big data isn’t just about growing data volumes or integrating new data sources. Human beings need to interpret the results of big data and build trust in the data. We look at how big data begins and ends with people.
[Editor’s note: Krish Krishnan, CEO of Sixth Sense Advisors, Inc., and Len Silverston, president of Universal Data Models, LLC, are leading several conference sessions about big data, including The Human Side of Big Data, at the TDWI World Conference in Chicago (May 5-10).]
BI This Week: Why is the human side of big data important?
Krish Krishnan: Big data starts and ends with people. To effectively start using big data, people need to create algorithms that interpret the data. For example, let’s take clickstream data. If someone stays on a particular Web page on an Amazon.com site for 45 minutes, what does this mean? Are they interested in the product that was displayed? Did they go on break and leave their computer on that page? Were they bored and hence they stopped clicking? People interpret the results of big data, and thus the first step is to understand how we as humans are creating the algorithms that are necessary to do anything with big data.
Big data ends with people. Once data is analyzed, humans make decisions based upon this data. For example, someone may make a decision to buy something on Amazon.com or Netflix when the item is suggested using big data algorithms. However, people’s decisions are most often based upon emotions and not intellect, as authors Chip Heath and Dan Heat explain in Switch: How to Change Things When Change Is Hard (Crown Business, Feb 2010).
We can use big data to understand and influence human behaviors only if we truly understand the complex nature of not only the machine intelligence, but we need to understand human intelligence and behavior.
What factors are important in understanding and affecting human behavior as it relates to big data?
There are many factors in the human dynamics of big data. We have found that four are significant and impactful.
- Understanding motivations
- Understanding vision
- Developing trust
- Managing conflict
Let me briefly explore each of these.
How people behave is largely determined by their motivations. Leading psychologists claim that most people do not actually know what their own motivations are, so how can we truly understand these motivations? There are various motivational models that can be useful in understanding how people behave. For example, The “Sedonna Method” talks about four key motivations: security, control, differentiation, and approval. If we can better understand why people do the things they do, then we can utilize big data to create value. [Editor’s note: For more, see The Sedona Method: Your Key to Lasting Happiness, Success, Peace and Emotional Well-Being by Hale Dwoskin; Sedona Press, September, 2003.]
Individuals together create a vision -- for example, a vision for a big data effort or a vision for a customer’s organization. Knowing what the vision is for a person or group is also key to using big data effectively.
We may know the motivations and vision of a person or organization, but if there is not trust -- in the data, in the people, in the processes, and in the algorithms -- then it is difficult to create real value.
Conflict is an inherent part of our human interactions. What if people have a completely different opinion about what the unstructured data means? It is important to understand how to resolve conflicts effectively in big data.
How does your TDWI course, The Human Side of Big Data, advance our understanding of the topic?
Len Silverston: Perhaps the most critical success factor for big data is understanding human behavior. The TDWI course I’m teaching not only discusses key factors in human behavior, but it provides practical frameworks and tools for understanding and affecting the human behavior as it applies to big data. Most importantly, the course provides a way to practice these tools and frameworks via class exercises. The course is not focused on technologies surrounding big data. Instead, we are looking at fun aspects such as human and organizational behavior that defines the success and failure of a program.
By the way, the course is not only intended to provide useful tools and insights but also to be a lot of fun!
Krish Krishnan, an expert in the technical and strategy aspects of big data, is teaming up with Len Silverston, who has studied, taught, published, and lead seminars for the last 10 years on human behavior in data management, for their course (The Human Side of Big Data) that provides insights into both the technical and human aspects of big data.