Q&A: Patterns of Information Management
Take a holistic view of how information flows around your organization's information systems and enjoy faster decision-making. Authors Mandy Chessell and Harald Smith show us how.
- By James E. Powell
- August 27, 2013
The authors of Patterns of Information Management -- Mandy Chessell and Harald Smith -- recommend taking a holistic view of how information flows around your organization's information systems. Think of this information supply chain as you might a manufacturer's transformation of raw material into finish goods.
BI This Week: "Patterns of Information Management" seems like a very broad and ambitious scope for a book -- what motivated you to address this subject?
Mandy Chessell and Harald Smith: As we've worked with clients over the last 10-15 years, we've seen them wrestling with many challenges related to their information management. There were specialist books on topics such as data warehousing, master data management, and data quality, but nothing that took a holistic view of how information flowed around an organization's information systems.
In IBM, our marketing teams were using an information supply chain metaphor to describe the flow of information from operational systems, through the data warehouse and to business intelligence applications such as reporting and analytics. This is similar in many respects to the supply chain model for a manufacturing process's transformation of raw material into finished product. We wanted to put an architectural lens on this concept and so the information supply chain pattern became the starting point and centerpoint of our work. It described the end-to-end flow of information through different systems (called information nodes in the patterns).
Writing the information supply chain pattern forced us to ask detailed questions about the approach, such as: "Why is this the best way to design the integration?" and "What are the benefits and liabilities it brings?" We realized there were many design decisions that must be made in order to architect an information supply chain. For example:
- When do you access information in place or copy it to a new location?
- What type of server should the information reside in?
- When does copying of information occur and how is it triggered?
- How is the information reengineered, protected and monitored?
When the answer to these questions was "it depends," it signified the definition of another pattern group to explain the options and the pros and cons between them. We shared these patterns with a number of clients who found them extremely helpful and so what started as a short whitepaper quickly became a sizable body of work that was enough for a book.
What are some of the key challenges you see for information management today?
Almost regardless of industry, organizations rely on information to maintain and drive their business, but effective use and management of information requires a holistic approach that crosses multiple silos in the organization. In most cases, you cannot simply throw more technology at the problem -- it must also encompass the organization's attitude and behavior as well as its people and processes. Many organizations find it hard to collaborate across their internal silos.
Beyond the organizational considerations, the volume, velocity, and variety of information all impact the management of information. Our traditional approach -- where we have different technology to support structured data versus unstructured data; or operational data versus historic data -- is no longer effective. Many problems require the analysis and use of multiple types of information to solve the problem. Such forces significantly impact the design of an information landscape, often bringing in new technology and requiring greater information sharing across the organization's silos because the volume and variety makes it too expensive for each team to hold its own private copy.
Why did you find patterns useful in addressing these challenges?
Patterns are written in a plain, natural language. This makes them accessible to both business and technical people alike.
The patterns lay out choices and how to select them -- pros and cons -- with worked examples and references to known uses. This provides insight through common conditions that readers are likely to have experienced themselves. Linked together, the group of patterns forms a comprehensive description of a topic area.
Decision-making is faster because the icons and the pattern names bring clarity and precision to design discussions and specifications. The terminology is neutral to the organization and helps to keep a focus on the issues at hand rather than the existing system implementations.
Many of the design decisions related to information supply chains are concerned with emergent properties. These are the properties that appear when components are combined rather than being inherent in any one component. For example, when a staging area (a type of information node) is inserted in an information supply chain to act as an interim storage location, the latency or delay in information delivery increases but so does the resilience of the information supply chain. Each pattern describes combinations of components interacting in a particular configuration. It is well suited to explain all of the properties and resulting benefits and liabilities that each combination of components brings.
How do the patterns help with emerging topics in information management such as big data or cloud computing?
Our book describes the fundamental patterns of behavior that an architect can use in the design of a solution. They do not assume specific technology or performance characteristics. New technology offers new implementations of these patterns. The patterns provide a way to assess where the technology can be used and which boundaries it expands. For example, consider Hadoop as a new technology. It offers a different approach to storing and processing data to a data warehouse. However, issues of how it is loaded, organized, archived, and governed are still important and have not changed.
Are there key prerequisites in using these patterns?
There are no prerequisites to use the patterns, but as with any effort in information management, some starting questions are handy. These really aren't any different than those you would ask in any project or initiative, and could include:
- What goal or problem must be addressed?
- What criteria define a successful solution to the problem?
- Who are the stakeholders and the people that must support the solution?
- What processes must the solution support?
- What information do these processes need?
- Where is this information located?
- How is this information managed?
- How are the processes provisioned?
- How is the solution validated?
- How is the solution secured?
As you work through these and other related questions, you are establishing the problem statements and forces that come into play, whether for a full information supply chain, an information flow, or perhaps a simple information reengineering step to correct some data.
Can someone start at any point with the patterns or is there a particular process needed?
One of the interesting aspects of patterns is that you can actually start at any level, whether coming at problems from a high-level strategic view; a low-level, "deliver the data to system X" view; or anywhere in-between. Chapter 2 of the book covers some basic, familiar examples of how patterns and pattern groups work together, along with the summaries and common entry points for the pattern groups, and a package of icons.
Any tips for readers on how they can best take advantage of the patterns for their work?
The patterns could be used in the following ways:
- Focusing on a particular pattern group to make a specific design decision. For example, one of our clients used the information key patterns to determine how customer records in different systems should be identified to ensure the information about an individual can be synchronized.
- Use the pattern icons and pattern names in a drawing tool to show the behavior of either an existing or proposed capability.
- Define preferred implementation technology for each pattern in a pattern group as a way of setting architectural standards.