Case Study: Analytic Software Tracks Billions of Mobile Ad Responses
For big data challenges, look no further than the world of mobile advertising, where the need for nearly instant tracking of responses from millions of consumers to ads popping up on their mobile devices means a big need for near-real-time analytics.
- By Linda L. Briggs
- September 25, 2012
It's worlds beyond anything the characters in the popular TV drama "Mad Men" might imagine -- nearly instant tracking of millions of consumer responses to ads popping up on mobile devices, giving advertisers an ad hoc opportunity to target ads based on responses.
With the number of mobile devices predicted to exceed the world's population by the end of 2012, the world of mobile advertising is a perfect -- if highly demanding -- area for analytics. Making sense of the massive data generated by tracking mobile advertising responses is a perfect use for analytics, providing the software can work quickly enough with truly big data sets.
The fast-growing mobile ad company InMobi, one of the world's largest mobile ad networks, captures many billions of ad impressions monthly. Mobile advertisers then use that data to shape and optimize their ad campaigns for the best return on investment.
Begun in the Asia-Pacific region in 2007, InMobi has grown to become a leader in the mobile technology space. It launched in the U.S. and Europe in 2010; its global network grew to handling nearly 100 billion ad impressions monthly. A private company, InMobi is underwritten by well-known venture capital firms including Kleiner Perkins Caufield & Byers; the company founders come from tech leaders such as Google, Amazon, and AT&T.
InMobi's analytics software captures data about actions taken by consumers on their mobile devices, tracking the click-through itself as well as the country code, time of day, the handset used, the network carrier, and additional behaviors and demographics on the user -- as many as 100 dimensions in all. The data can be combined with the history and profile of a user. That enables InMobi customers -- advertisers -- to determine the most effective ad to serve up to a particular customer at any one time. Just as important, fast feedback about user reactions enables advertisers to quickly alter that decision based on current data.
For example, an advertiser might want to assess how many mobile phone users in North America are using which Apple operating system and the rate at which users are upgrading their phones. Using the feedback in a big-picture manner, advertisers can compare the performance of each mobile ad to their campaign objectives, giving them information to help direct spending for maximum return.
InMobi's numbers reflect the data management challenge: The company says it can reach 485 million consumers in more than 165 countries monthly. According to InMobi's vice president of technology, Mohit Saxena, the company has grown "tremendously" and now handles over 100 billion ad impressions monthly.
The company had a scalable, expandable Hadoop-based database platform. The problem: the system wasn't user-friendly. Business users "wanted to slice and dice all the intelligence that we gather through our platform in a fraction of a second," then use the data for measuring a variety of business metrics, Saxena explained. With its legacy relational database system, he said, both storage and performance had become problems. Running reports could take several minutes, for example, and the existing database solution wasn't flexible enough to support customers' varied demands for analysis.
Each time a user wanted to query the data or measure a different metric or dimension, data summaries (and more) had to be created. Given InMobi's huge data needs and rapid growth, the once-useful DBMS had become a bottleneck.
InMobi wanted to move to a more optimal solution that could perform fast analysis across huge volumes of advertising data. The company turned to Infobright's columnar-based database, deploying the Infobright Enterprise Edition (IEE) both for its flexibility in creating dimensions and records, and its high data compression ratio to help deal with data storage issues.
InMobi considered other solutions, including building something themselves. "We are a technology company, after all," Saxena said, and building the software in-house was feasible. However, after careful consideration, the rapidly growing company chose buy over build. "Time is very important to us," Saxena said. "Even three months is a very long time. If something is available and can save months, and it has all the features and functionality that you need, and the price is right, there's no point in building it yourself. ... Infobright was a pretty good match for us."
InMobi uses Infobright in conjunction with Hadoop. About 60 to 70 terabytes of raw data logs stream into the Hadoop cluster each month, of which three to four terabytes of summary information are loaded into the Infobright system for real-time analysis. The speed improvement -- critical for allowing InMobi business users to make on-the-fly adjustments to their ad campaigns -- has been substantial. Queries that used to take 30 minutes to several hours in Hadoop are now completed in under 10 seconds.
However, the Hadoop data can't be exposed to business users in its raw format -- it must be formatted and then delivered, which is where Infobright excels. Information is extracted from Hadoop and loaded into Infobright, which runs on top of Hadoop as the company's internal data store. The system pulls information from Hadoop and pulls it into Infobright, making for a seamless transition for users and a response time of seconds. "Anyone can go back several weeks, even several months," Saxena said, and run reports against the data.
InMobi keeps data in Infobright going back at least several months -- "as long as possible" based on space constraints, Saxena said -- allowing users to run historical queries that return snapshots of the data in just seconds. Storing six months of Hadoop data, an optimal amount, adds up to several hundred terabytes, Saxena estimated, which becomes substantially less with Infobright's deep compression ratio.
The manner in which InMobi has implemented Infobright makes the system scalable, Saxena said, and allows new dimensions to be added without writing any new software. Although many of today's largest technology companies have moved to the now-well-proven columnar storage database architecture, Saxena said, Infobright has added a polished front end that makes the technology accessible to smaller companies as well.
As mobile devices continue to grow in popularity worldwide, the massive quantities of data they produce will continue to provide a compelling area for BI and analytics growth. As InMobi shows, the right software can help to make big, fast, and frequent amounts of data manageable.