APPENDIX: Development Techniques for Creating Analytic Applications

04/04/05

Evaluation Criteria for Analytic Development Environments

We're still in the early stages of the ADE market, but TDWI has high hopes for ADEs. We think they represent the future of BI tools. So what should you look for in an ADE? We've collected a number of requirements. Truth be told, no ADE on the market today possesses all of these characteristics, but many are making significant progress toward supporting most of them.

1. Supports a broad range of applications. The tool should enable developers to build analytic applications across a range of business domains, including business and operational management, customer relationship management, human resources, and shipping and logistics.
2. Empowers Business Analysts. The tool should be easy enough for business users with technical skills to rapidly prototype and build analytic applications for their workgroups. This means that the tool should clearly separate front- and back-end development responsibilities, so that power users can focus on building business logic, and IT staff can create the data architecture and abstraction layer.
3. Supports Rapid Development. This feature has several elements:
   
   1. Graphical Workbench. Developers should be able to drag and drop components onto a screen that supports WYSIWYG (“what you see is what you get”) development. The interface should be familiar to developers, preferably using Windows-based toolbars, menus, and conventions.
   2. Point-and-Click Development. A key facet of a graphical workbench is that it eliminates the requirement for coding and scripting.
   3. Hidden Screens. At the same time, developers should have the ability to create “hidden” screens where they can collect data, apply calculations, and perform other functions in the background. This is critical when pulling data from multiple sources.
   4. Flexible Object Placement. The tool should let developers place components or objects anywhere on the screen and be able to resize, move, or link objects using point-and-click actions. Ideally, objects automatically resize as additional objects are added to the page.
   5. Dynamically Linked Objects. Developers should be able to synchronize two objects, such as a chart and a table, by dragging one object onto to the other, or some other point-and-click technique.
   6. Flexible Page Navigation. Tools should let developers create any style navigation paradigm or choose one from a template, such as a briefing book, tree hierarchy, tabbed worksheets, dashboards, or buttons.
   7. Automatic Data Connection. Once IT establishes connections to data sources, the tool should automatically connect to those sources when activated. In other words, developers shouldn't have to configure ODBC (open database connectivity) connections.
   8. Effective Debugging. The tool supports a debugging utility that lets developers test an application in flight without writing scripts or changing contexts. The debugger should also let developers test selected portions of the application and query sample data sets.
   9. Easy Deployment. Once developers finish building the application, they should be able to publish it to the production server in an automated, failsafe fashion.
   
   
4. Supports Robust Application Logic. This is the heart and soul of an ADE. ADEs expose their functionality in the form of components that developers can use to create application logic. These components should have several characteristics.
   
   1. Supports multiple types of components. ADEs should give developers a robust palette of components, including:
      
      1. GUI Objects. Checkboxes, listboxes, sliders, buttons, shapes, layouts, screens, navigation methods, etc.
      2. Data Manipulation Objects. Sort, edit, calculate, etc.
      3. Query Objects. Metrics, queries, filters, etc.
      4. Event Objects. Queries that trigger an action once the result set fulfills a certain condition.
      5. Visualization Components. Advanced visualization techniques, such as maps, pareto charts, and scatter diagrams. These programs often form the heart and soul of an ADE because they provide a visual way to perform complex calculations against data. The more visualization programs, the more robust analysis the tool supports.
         For example, ADVIZOR Solutions has 12 interactive visualization components, including highly sophisticated scatter plots, histograms, data constellations, and Paraboxes. ProClarity offers 21 visualization programs, including its patented decomposition tree program. Business Objects has 50 “analytic templates,” including speedometers, stoplights, time sliders, and pareto charts.
      6. Administrative Objects. Authentication, access control, usage monitoring, metadata design, server processes, etc.
   2. Reusable Components. The key to an ADE is that the components or services listed above be reusable. This means that when developers create new objects, they can be stored centrally in an object library or catalog so they can be reused in other applications or contexts. By implication, this means that developers can update the object once in the library or catalog and change all deployed instances of that object automatically. Reusable objects and services greatly speed application development and maintenance.
   3. Portable Components. The components should ideally be platform neutral; that is, they work on any operating system, with any database, and in any development environment, whether it’s .NET or Java. In the latter case, it helps if the ADE offers different versions for the different platforms or a method for translating them.
   4. Extensible Components. The components should be extensible so that developers can create new functionality by tweaking existing components. Ideally, modifications can be done in a common programming or scripting language, instead of a proprietary one.
5. Accesses Multiple Sources.
   
   1. Support for any source. Like a good BI tool, an ADE should be able to access any data source, including relational databases, OLAP databases, legacy and packaged applications, flat files (XML, HTML, CSV), Web screens (scrapers) and Web Services via standard database interfaces (ODBC, JDBC, XMLA) as well as native interfaces where required or where performance can be enhanced.
   2. Support for any schema. Similarly, the ADE should be able to access data in any schema, not just a star schema or third normal form schema.
   3. Distributed Query. The ADE should also be able to pull data from multiple sources and:
      
      1. Place the data in separate objects on the same screen, and
      2. Integrate the data into a single object (i.e., table or chart), including applying calculations on the fly to the joined data without using an external staging area, database, or data warehouse.
6. Guided Analysis and Action. The ADE should make it easy for developers to create context-sensitive recommendations about what reports users should view next or actions they should take (send an e-mail, call someone, set up a meeting) based on what they are currently looking at. This “guided analysis” can be based on an internal recommendation engine or defined by experienced analysts using a point-and-click interface.
7. Read/Write Access. The ADE should allow users to both query and write to analytic data sources to support “what-if” modeling, planning, and budgeting applications. It should also be able to submit transactions to OLTP applications, either through an application layer or directly to the underlying database.
8. Flexible Output and Publishing. The ADE should support multiple file formats, including HTML, Excel, .PDF, .RTF, and so on; and be able to send reports to a variety of devices, including e-mail, portals, printers, and handheld devices. It should make it easy for users to publish, schedule, save, and subscribe to custom reports or views in a secure, flexible fashion.
9. Team Development. A key differentiator between ADEs and BI tools is that ADEs support team development features. This includes check-in/check-out, version control, audit trails, a central component library, and the ability for multiple developers to work on different parts of the same application simultaneously. It also manages the process of deploying applications from design to test to production servers.
10. Integration Architecture. ADEs support a plug-and-play (or services-oriented) architecture that enables the tool to quickly integrate additional services or “engines,” including third-party programs. For example, an ADE might integrate a set engine to facilitate market segmentation analysis, a modeling engine to perform data mining and statistical analysis, or a rules engine to set conditional alerts and automated actions.
11. Flexible Platform Support. The ADE should be portable across multiple platforms and provide a range of deployment options so customers can balance the trade-offs of interactivity, performance, and scalability.
    
    1. Clients. Desktop (“fat”) clients, applets/plug-ins, and zero-client browser-based options running .NET or Java applications.
    2. Servers. Run on Windows, all flavors of Unix, and Linux with support for a range of standard .NET and Java Web application servers.
12. Scalability. Increasing the number of users, data volumes, and query complexity while maintaining or improving performance requires a multi-faceted approach. At a minimum, an ADE should support (1) multi-threaded processes, (2) 64-bit memory, (3) pipelining and parallel processing, (4) automated load balancing, and (5) cluster management.
13. Robust Administration. ADEs should provide an easy way for developers to access and manage runtime metadata, integrate with existing LDAP security repositories, manage users, monitor usage, and configure processing to optimize performance, among other things.

Evaluation Criteria for Dashboards and Scorecards

Many vendors now offer ADEs for building dashboards and scorecards. Some solutions are more canned than others, prescribing a certain look and feel from the get-go. However, others present developers with a flexible portal interface that they can populate and paint with substantial flexibility.

In addition to the criteria listed, dashboard/scorecard-specific ADEs should support:

1. Specialized Visualization Components. These include speedometers, gauges, dials, maps, and other icons that visually depict performance against plan.
2. Rules Engine. Dashboards evaluate performance against metrics using predefined goals. Dashboard developers define a range of acceptable performance goals across multiple time intervals. To support this, ADEs need a strong rules engine that lets developers define high and low markers of performance over time for each metric. Ideally, the rules engine should have a simplified Boolean interface so that end users, not developers, define and mange the rules.
3. Alerts and Agents. The rules engine (above) should also support event management. That is, it should let developers and end users define rules about when and how they should be notified if goals are exceeded for a given metric (i.e., alerts) as well as when and how to initiate automated actions based on those alerts (i.e., agents). Visual alerts should be accompanied by text that explains the problem, a report that users can click to see data, and a URL to initiate additional action, such as to refresh a report or display contact information for someone to call. The rules engine should accept events from third party systems as well.
4. Time-Series Analysis. Monitoring performance over time against goals involves time-series analysis, which is process intensive. To ensure rapid response times when users access, filter, or drill into time-series tables and charts, ADEs should store metrics, rules, and results in a local repository. This lets the ADE apply rules to historical, time-series data without repeatedly querying the source database and bogging down performance.
5. Drill to Detail. Dashboards should enable users to drill down three levels: from graphical performance icons, to metric tables/charts, to detailed transaction data.
6. Collaboration. The purpose of most dashboards is to foster better communication between managers and staff, not to punish staff for poor performance. The metrics themselves communicate what is important to top executives (i.e., strategy) and how to implement the strategy at each level. Conversely, the staff should understand what the numbers represent, both historical and future trends and events. A good dashboard, therefore, enables users to attach comments to metrics and reply to those comments in a threaded discussion. It should also let users create workflows of activity, such as distributing a report to a group of people for their review and comments.
7. Strategy Mapping. The Balanced Scorecard methodology requires organizations to map strategies and initiatives to metrics and map causal relationships among metrics. This ensures that you've created metrics that work synergistically to achieve strategic goals. Whether or not your organization uses the Balanced Scorecard methodology, an ADE should support the ability to map causal relationships among metrics and between metrics and strategy.
8. Customization and Personalization. Every dashboard should be customized to a workgroup or individual so they only need to monitor a small number of metrics that are relevant to their roles and tasks. Users then should be able to personalize their views, à la a portal interface, so they view metrics, documents, and other relevant information in a manner that suits their preferences.