Necessity, Not Luxury: Automating Testing and Quality Assurance for BI
It was hard enough to manage testing and quality assurance in the old days. In the context of big data, cloud, streaming data, and other still-emerging technologies, automated testing and quality assurance is a necessity, not a luxury.
ETL testing and data migration aren't sexy subjects. Neither is software quality assurance testing, regression testing, stress testing, or dozens of other, similar IT practice areas.
They aren't sexy, but they are important. They have to be done -- and done well. Technology that requires less funding and fewer resources to do them well is a very good thing.
Demand for Testing and QA Products
Several companies market software designed to automate ETL testing, data migration, business intelligence (BI) testing, and quality assurance.
Two of them -- Torana, maker of iCEDQ, and Proden Technologies, maker of the Accel suite of products -- were exhibiting at TDWI's recent San Diego conference.
Marketers have never quite known what to do with products for development testing, software quality assurance, or data migration. They don't have the obvious and immediate salience of data visualization technologies or massive in-memory data processing servers. Furthermore, the emphasis of technology marketing has bypassed BI and data warehousing. Practice areas such as ETL dev-testing don't just seem unsexy -- they seem irrelevant.
On the other hand, the people behind iCEDQ and Accel<>BI argue that their products are as vital and in demand as ever.
"It is a necessity. People might be [implementing] new technologies, but they aren't abandoning their data warehouses. Actually, these are growing too. Data volumes are increasing, they're [incorporating] new sources, so now [ETL] scripting, handling that huge amount of data, is an issue," says Shrikant Patil, vice president of products with iCEDQ.
Using Testing Software -- iCEDQ
The testing process is straightforward with Torana's solution, says Patil. Developers first define rules for data comparison or validation; iCEDQ includes dozens of canned rules, but developers can also create custom rules. Once the rules have been defined, iCEDQ loads a subset of the data into its in-memory rules engine.
From there, iCEDQ performs comparisons -- e.g., identifying rows that are present in the source but not the target, rows that are present in both the source and the target -- and evaluates transformations, calculations, etc. From there, it generates an exceptions report.
This isn't rocket science. Patil concedes as much. If something like iCEDQ didn't exist, he argues, it would have to be invented. In fact, Patil says, it has been invented, and reinvented again, tens of thousands of times by different companies, with vastly different priorities and resources.
ICEDQ centralizes, automates, manages, and simplifies the maintenance of tasks that developers, DBAs, and others would otherwise have to undertake manually and separately.
"Most of our customers still write SQL scripts to get the data in[to] Excel [to test it]. They do sample testing; they do spot checks; it's much more of a manual approach," he says. "Some customers will have these sophisticated systems in place, but they're custom-built, and they usually don't scale. For example, one of our investment customers used to do the same thing in-house but they could only test 500 rows at a time. Now they migrated to iCEDQ and they can test millions of rows."
Using Testing Software -- Accel
Torana's iCEDQ targets data integration or data migration testing. Proden's Accel product line focuses on a similar problem, albeit in a different context. Accel<>BI is designed to simplify QA testing for BI platforms.
It handles regression testing, stress testing, and the automation of almost all aspects of dev-testing. It includes a tool for model testing and validation (Model Validator) that can help identify issues (such as faulty star schema joins) that often aren't detected until deployment.
Proden's Accel<>QA, on the other hand, aims to be an end-to-end test automation tool for both BI and data warehouse environments, while Accel<>enV is a counterpart to Accel<>BI: a quality assurance testing facility just for data warehouse systems. Finally, Accel<>RA is a release automation tool for Oracle Business Intelligence Enterprise Edition and other Oracle products.
Time Savings and Compliance Testing
Like iCEDQ, Proden's Accel products merely centralize and rationalize tasks that are otherwise accomplished using some combination of scripts, custom-built tools, and manual interventions.
Both vendors claim to do this better -- more cleanly, manageably, scalably, and cost-effectively -- than individual organizations left to their own devices. "One of our customers, a trading customer migrating from DB2, reduced the time it takes to test data migration [efforts] by 70 percent. There's that aspect of it: the huge time savings," says Patil.
"For a lot of our customers, compliance testing has become part of the ETL process; iCEDQ helps simplify this, too," he claims. "They can say, 'If we develop an ETL process, a minimum of 5 to 10 rules should be [validated in] iCEDQ. This transforms the way you can do quality assurance and compliance testing in a corporate environment."
It was hard enough to manage this stuff in the old days, these vendors say. In the context of big data, cloud, streaming data, and other still-emerging technologies, automated testing and quality assurance is a necessity, not a luxury.