View online: tdwi.org/flashpoint
September 1, 2011
ANNOUNCEMENTS
Submissions for the next Business Intelligence Journal are due December 2. Submission guidelines
New TDWI Checklist Reports: Data Discovery and Unified Information Access and Integrating Data Governance into Your Operational Processes
CONTENTS
Data Modeling: Agile or Fragile
Staffing Operational Data Integration Practices
Mistake: Prematurely Launching a Council
See what's
current in TDWI Education, Research, Webinars,
and Marketplace
Data Modeling: Agile or Fragile
Len Silverston
Universal Data Models
Topic:
Data Modeling
How can we create truly “agile” data models instead of fragile ones?
What does “agile” mean, anyway? According to the Agile Manifesto, there are 12 principles for agile software development, with a focus on delivering quickly and providing adaptability for changing requirements. Merriam-Webster's definition of agile is “marked by ready ability to move with quick easy grace.”
If something agile is characterized by speed and adaptability, and something fragile is easily broken, we must develop data models quickly, with flexibility, and in a way that prevents breakdown over time.
Developing Data Models Quickly
In my experience, there are two important ways to develop models quickly: (1) having reusable data model templates and (2) understanding various choices and options up front. I have spent more than 20 years developing, publishing, and using what we call “Universal Data Models,” or data model templates. We can save a substantial amount of time by using these templates.
While template data models save time, we cannot completely circumvent data modeling; we must understand information requirements and skillfully apply and customize template data models when appropriate, and/or consciously choose alternative model structures.
One of the greatest pitfalls in data modeling is thinking that there is only one correct way to model something. Dr. Graeme Simsion points out in his book, Data Modeling Theory and Practice, that the same information requirements can be modeled in many different ways. Because of this, we can model more quickly if we are already aware of ways to model common constructs such as categorizations, hierarchies, roles, statuses, contact information, and so on. We refer to these common constructs and options for modeling as Universal Patterns and distinguish them from reusable or Universal Data Models, which are models that are put together (often using Universal Patterns).
Adaptability and Flexibility
Another goal of agile methodology is to accommodate changing requirements. If we do not readily allow for change, then our data model may be fragile.
For example, say we need to quickly provide a customer contact data model for an agile effort. We develop a data model that simply shows the e-mail address, phone number, and postal address for a customer entity (or class), which seems relatively straightforward. Then we realize that we are maintaining numerous additional entities (e.g., supplier, partner, employee) with many attributes for each type of contact information (such as phone number or e-mail address). We find that we have added attributes haphazardly, such as additional contact numbers for Facebook, Twitter, Skype, secondary e-mail, and so on. Furthermore, each piece of contact information may have other associated data, such as the purpose of the e-mail or a do-not-solicit indicator. Eventually, we end up with a fragile data model that we continually patch where there are many different semantics, rules, and edits.
How can we create more adaptability and flexibility? The more a model is generalized, the more adaptable and flexible it is. In the contact data model example, we could generalize the model and define entities for postal address, e-mail address, and phone number that flexibly maintain all different types of information--instead of storing contact information as separate attributes. We could also further generalize the models and maintain a “contact mechanism” entity that stores all types of contact information, thus providing flexibility to add any new types of contact mechanisms--even ones that are not yet invented!
Why not model with the highest level of generalization? There are trade-offs; the more generalized the model, the more abstract and difficult it is to understand. Thus, it is important to find a good middle path when evaluating the level of generalization.
Conclusion
To create an agile data model, rather than a fragile one, good practices include using template models, understanding alternatives for common data requirements in advance, and choosing an appropriate level of generalization.
Len Silverston is a best-selling author, consultant, speaker, and expert in data management, data modeling, and human dynamics. He is the author of The Data Model Resource Book series and is a winner of the prestigious DAMA International Professional Achievement Award. He is a TDWI instructor and will be co-teaching Mastering BI with Best-Practice Architectures and Data Models: From Hub and Spoke to Agile Development with Claudia Imhoff on November 1, 2011, at the TDWI World Conference in Orlando.
References
Silverston, Len [2001]. The Data Model Resource Book, Revised Edition, Volume 1: A Library of Universal Data Models for All Enterprises, Wiley Computer Publishing.
Silverston, Len [2001]. The Data Model Resource Book, Revised Edition, Volume 2: A Library of Universal Data Models for Specific Industries, Wiley Computer Publishing.
Silverston, Len and Paul Agnew [2009]. The Data Model Resource Book, Volume 3: Universal Patterns for Data Modeling, Wiley Computer Publishing.
Simsion, Graeme [2007]. Data Modeling Theory and Practice, Technics Publications, LLC.
Highlight of key findings from TDWI's wide variety of research
Staffing Operational Data Integration Practices
One of the most pressing issues in operational data integration today concerns how it’s staffed and which team manages the staff. As OpDI practices have grown in terms of workload, staff numbers, and sophistication of solutions, opinions have shifted about who should be doing the work and where they belong on the organizational chart. In an ideal world, OpDI would have its own personnel, organizational structure, tools, budget, and so on. Alas, the reality is pretty much the opposite in most organizations today. But there’s still hope. As technical personnel increase the sophistication of OpDI practices and as business people increasingly realize that OpDI is mission-critical, OpDI becomes more visible and therefore gets more resources.
To get a handle on the groups staffing OpDI today, TDWI Research asked: “For DI outside data warehousing, who does the work most often?” (See Figure 7.) Responses show that OpDI is performed by a range of consultants and organizational units.
(Click to enlarge)
Source: Operational Data Integration: A New Frontier for Data Management (TDWI Best Practices Report, Q2 2009). Click here to access the report.
FlashPoint Rx prescribes a "Mistake to Avoid" for business intelligence and data warehousing professionals.
Mistake:
Prematurely Launching a Council
By Jill Dyche and Kimberly Nevala
We see it all the time: An earnest visionary perceives the need for data governance. She decides that it’s time for formal consensus and enlists an executive to support her effort to convene a “council” of data stakeholders.
An e-mail invitation goes out for the council meeting. It’s a lunchtime event--“Please make your selection from the attached menu”--and nearly every invitee shows up. The visionary begins discussing how data is an asset and that the company needs to begin managing it as such. Heads nod. The visionary goes on to explain that the newly formed council should meet regularly and discuss the company’s prevailing data issues and address any open problems. The follow-up meeting is scheduled.
Fewer people show up to the next meeting. Someone complains that the company has never really defined the term “customer.” Someone else pipes up about bad data on the billing system. A sidebar conversation starts on CRM consolidation. The next meeting never happens.
In this all-too-common example, data governance isn’t overtly cancelled. It simply fizzles like a damp firecracker. The problem? See Mistake Two. Well-meaning people who see the need will gravitate toward the “who” conversation (Who should be on the council? Who will sponsor it?) before understanding the “what” and the “how.” Until a core team of stakeholders deliberately designs a data governance framework (including guiding principles, decision rights, and the appropriate governing bodies), no sanctioned, cross-functional council will have either the clarity or the mission to affect change. There is no free lunch.
Source: Ten Mistakes to Avoid When Launching a Data Governance Program (Q1 2008). Click here to access the publication.
