Alvion Technologies Speeds Query Times for Online Self-Service Business Model
Commentary by Bojan Belovic, Database Administrator, Alvion Technologies, Inc.
Alvion Technologies, Inc. needed to maintain optimal query response times for an online self-service business model while moving to the UNIX platform to support growth needs. The company turned to Sybase IQ technology to create a system that provides end users the ability to execute ad hoc queries in real time, which makes it feasible to mine for desired combinations of geographic, demographic, and other attributes. Now Alvion can reduce storage costs by minimizing high-speed disk space requirements, and run nightly single-machine data loads fast enough to keep runtimes within an off-hours window.
Managing Alvion Customers’ Big Data
Alvion Technologies provides outsourced data management services to some of the largest marketing data list owners in the world—including Axiom, Equifax, Experian, and Dunn & Bradstreet. Its customers’ data sets range up to 190 million records and 200 attributes. Combined, Alvion manages about a terabyte of data for customers as an outsourcing service. Individual customers submit anywhere from 5,000 specialized records of information to 120 gigabytes of data. Alvion then runs customer-specific data transformations and uploads them to its production servers for access by the end user—the customers of the original data owners. Data update cycles vary from daily to annually.
Within Alvion’s terabyte of data, roughly 550 gigabytes are held in Sybase IQ and can be queried via Web-based storefronts that Alvion hosts for the data owners. “We use IQ in production because its biggest strength, as far as we are concerned, are those really, really fast counts,” said Bojan Belovic, database administrator for Alvion.
End users of this data typically run queries and counts to determine optimal selection criteria for specific direct marketing programs. This means that a typical user session includes numerous queries, run with slight variations, to find the strongest candidates for final data selection. When users query over and over, faster is better—much better.
Alvion’s World-Class Data Warehouse Application Service
Alvion sits squarely in the crossroads of e-commerce and business intelligence, leveraging its exclusive technologies, extensive expertise with very large data warehousing (VLDW), and marketing know-how to provide customers with fast, reliable marketing data.
Several years ago, Alvion was making news with high-capacity, high-speed, and low-cost marketing list services, but that combination was tricky to maintain. To control costs, the company started out with Windows NT–based production servers. Soon, Alvion was in need of a quick save to make this infrastructure work at the high-performance levels required for its online self-service business model.
That’s where Sybase IQ originally came into the picture. In fact, the Sybase IQ pilot installation was up and running in three days, with benefits immediately apparent. On a random set of previously submitted queries, Sybase IQ reduced large-list response times by a factor of six. One notable query, previously taking 53 minutes, now completed in three seconds. Customers were happy. Life was good.
“The response time is absolutely the number-one reason we are using Sybase IQ. We are talking about an order of magnitude difference,” Belovic said.
Upping the Ante
Alvion boasts 12,000 registered users. With more than 2,000 ad hoc queries daily, Sybase IQ average response times were still under 20 seconds. For its growth in data volume, Alvion was suitably prepared with Sybase IQ’s unique data compression. With Sybase IQ, disk space savings add value, especially considering the cost of the large, high-speed drives required for high-use VLDWs. Plus, the extra drives that were typically used to tune up database performance were unnecessary.
Today, Alvion has 50,000 registered users. Daily query loads have risen by 275 percent to 7,500, with one-third being very complex, demanding even more system resources. As big data grew bigger, Alvion’s commitment to market-leading response times forced an infrastructure build-out.
The Challenge: Alvion’s Success
Growth in Alvion’s business means four things:
- More data hosted from new customers
- Growth in existing customers’ record counts
- Increasing user sophistication and query complexity
- Fourfold end-user growth
Previously, to keep pace with growth and maintain performance, Alvion added Windows “reader” servers running Sybase IQ—a process Sybase calls “multiplexing.” Still, with the required data model, this left one soft spot. Despite handling the big data and the input/output, the “writer” server was subject to Windows’s two-gigabyte user space ceiling (inherent in its 32-bit design), which would eventually become a bottleneck. Alvion needed to continue providing fast service and accommodate tremendous growth.
Deploying the Sybase and Sun Reference Architecture
Although the NT platform had performed well, the data load runtime now forced Alvion’s hand. Nightly updates, necessarily confined to the single-writer server, used to complete easily. But now that critical data load stretched beyond 12 hours, motivating a hardware reexamination.
Consultation with Sybase field engineers and product experts turned up a recent joint effort by Sybase and Sun Microsystems that addressed hardware and software architecture for data warehouse applications handling raw data sets up to 100 terabytes.
It’s the Infrastructure!
Alvion selected Sun hardware for maximum performance while allowing headroom for growth with its present footprint. The implementation includes a four-CPU Sun Fire™ V880 with eight gigabytes of RAM, a two-CPU Sun Fire™ V480 with four gigabytes, and two four-CPU Sun Fire™ V440s, each with eight gigabytes.
To accommodate current loads, Alvion now runs half of production on Windows and half on Sun, all happily accessing the same Sun Fiber Channel–based SAN. The new Sun machines easily handle increased customer query loads. But most important, the Windows-imposed writer server bottleneck is gone. That function now runs on the big V880, leveraging its eight gigabytes of RAM and finishing in half the time.
“We looked at a couple of other vendors, but Sybase definitely stood out as the best.” —Bojan Belovic, Database Administrator, Alvion Technologies, Inc.
Belovic, Alvion’s database administrator, sums up the basis for Alvion’s Sun move: “If our data sets were smaller, with growth merely in queries loads, the Windows environment might have been appropriate. We’ve scaled Sybase IQ simply by adding servers. However, we needed to manage the larger data sets as well as increased query complexity. That required a hardware platform change. Fortunately for us, Sybase IQ is well suited for the Solaris environment, so the software platform remained unchanged.”
What’s Ahead
Until now, Alvion divided its data warehouse tasks between queries and order fulfillment. With the new Sun hardware in place, Alvion is looking to consolidate all operations there. To do this, the company will add the other half of its customers’ terabyte of data to the Sun-Sybase IQ servers. Amazingly, Sybase IQ provides such effective compression that the new data will not impact the system’s performance or query speed. However, this change will dramatically improve Alvion’s competitive advantage by shortening data fulfillment times while maintaining the trademark query speeds so appreciated by its customers.
Belovic believes that Sybase IQ is up to that task, too.
“Our fast query times are what define us in this industry. We are building in more complexity with Sybase IQ as the backbone while maintaining lightning-fast response times. That’s going to take us a long way.”
This article originally appeared in the issue of .