As Analytic Search Heats Up, Information Builders Touts WebFOCUS Magnify
Information Builders makes use of key open source search technologies -- namely, Lucene and Solr -- to power its own search offering, WebFOCUS Magnify.
- By Stephen Swoyer
- September 17, 2013
Information Builders Inc. (IBI) has a pedigree that's even lengthier than that of SAS Institute Inc. IBI turned 38 this year; SAS, 37. This arguably makes IBI the Brahmin of BI. With founder and CEO Gerry Cohen still at the helm, IBI seems to be thriving.
Its vitality could be seen as a function of its resourcefulness: consider the extent to which IBI has made pragmatic use of open source software (OSS) technologies to power or to build out its WebFOCUS analytic stack. Almost half a decade ago, IBI delivered WebFOCUS RStat, a data mining and statistical analysis product based on R, the powerful (but difficult-to-use) OSS programming environment for statistical analysis.
IBI also tapped the Apache Lucene and Solr projects to power its WebFOCUS Magnify, a product that first debuted in 2007. With the analytic search market starting to heat up, Information Builders now positions Magnify as a one-stop offering, integrating BI, performance management, data integration, and (thanks to RStat) text-analytic search capabilities, too.
Back in 2006, IBI unveiled its first-ever BI search product: WebFOCUS Intelligent Search. That product was tightly yoked to the Google Search Appliance. A year later, IBI delivered Magnify, a mostly new offering based on the Apache Lucene search library. "We started with Lucene, [and] evolved [Magnify] from there," says Dan Grady, social analytics and enterprise search manager for Information Builders.
Pieces of the former WebFOCUS Intelligent Search were incorporated into Magnify, Grady explains. So, too, were bits from the Solr, an OSS enterprise search platform that uses Lucene. "We do the faceted-based search [in Solr] and when you build out those facets ... we can use business intelligence to drive what the facets are," Grady explains.
Faceting is a non-taxonomic scheme for classifying information in multiple dimensions; facets are derived from explicit fields (e.g., "author," "tags," or "format") or are products of textual analysis -- typically derived via named-entity recognition, or NER. (For this reason, Magnify also makes use of algorithms or functions from WebFOCUS RStat.)
"Let's say we built a search-based app around CRM; the likelihood of 'close,' for example. That would [involve the] use [of] an RStat or an R-based algorithm," Grady continues. "Creating a search-based app now is almost like building a report. We build a report and the columns and the things that you need have just become a facet in the search. It's actually loaded into Magnify for you."
IBI's approach with respect to analytic search is somewhat similar to that of the former Endeca, which was acquired by Oracle almost two years ago. It doesn't position Magnify as a platform play -- i.e., as its own app, surfaced via its own user interface, with its own user experience (a la Neutrino BI, for example) -- but as a technology or service embedded in the context of operational apps.
The emphasis, then, is on function- or purpose-specific apps. In this regard, users could consume the results of a Magnify search without actually leaving their bread-and-butter applications. This is consistent with Information Builders' approach to reporting, performance management, predictive analytics, and other BI-esque technologies.
"It isn't this grandiose, all-inclusive search platform. It is more [a way to build] purpose-driven apps. The difference between Magnify and Endeca is that [with respect to] Endeca, they didn't grow up as a business intelligence company. We did. We have that background, so all of the data that you needed for the [Endeca] app had to be loaded into the search index, which drove up the cost of the hardware and storage," Grady argues. "We only feed into the search index what people will search on. This [permits us to] take advantage of what both [search and BI] technologies are good for, and allows us a lot of flexibility."
Grady and Information Builders position analytic search as a tool for information gathering and discovery. It's a complement to, not a replacement for, existing technologies; at its best, search helps to buttress or to enhance analytic discovery, Grady insists.
"Search is good for finding things -- for example, a person, a place, a thing. Once you find any one of those things, all of the questions you have around it are going to be 'anything-related' questions -- the kind [of questions] you have to drill down into and analyze. For example, how many [of this widget] am I going to sell in the future. If they're a student -- and we have a several [academic] institutions using [Magnify] for this purpose -- how many courses are they going to take this semester? How many [courses] and in what categories [do they need to take] to graduate? These are all analytic questions."
On the other hand, it is possible to build a rich analytic search platform using free OSS tools. Savvy organizations can use a combination of Solr, the Apache unstructured information management architecture (UIMA) project, R, and other technologies to build and deploy an analytic search platform that includes natural language search (NLS) capabilities. In fact, a tutorial at this year's O'Reilly Open Source Convention (OSCon) covered precisely this topic.
Grady concedes as much, but cites IBI's experience with RStat as an illustrative counterexample. To this day, he claims, R is prohibitively difficult to use; the OSS R workbenches for Windows and Mac are crude, at least compared to what IBI delivers with its visual RStat tool. (The R workbench for Linux is a command-line interface.) In this respect, he argues, WebFOCUS Magnify marries the back-end capabilities of Lucene and Solr (along with NER via RStat) with a visual, self-service development environment (WebFOCUS Developer Studio) and an ability to consume data from more than 400 different sources. (This feature comes by way of IBI's iWay adapter portfolio, as well as its enterprise service bus.)
For existing WebFOCUS customers, Grady argues, it's a no-brainer. "If you look at search and the way that search applications work, what they're really useful for is [to] help you navigate through categories of results. You can do a search for something and start navigating through the results [on the basis of] 'price range,' or [filtering on the basis of] other categories. It's a way to filter or to fine-tune for [the purpose of] analysis."