The Future of Search
Enterprise search in 2020 will have evolved from being your personal assistant to being your personal advisor.
By Bob Potter and Gary Mumford, Rocket Software
Search is everywhere. We search the Web 20 times a day, search for content in our professional and personal software applications a dozen times a day, and search our laptop computers and company's databases and content systems a half dozen times daily. We expect search to work quickly and accurately, and we get frustrated when we get answers such as, "No results found for your inquiry." We expect our search inquiries to work even when we misspell words, are ambiguous, or not really sure exactly what we're looking for. Search should read our minds and give us the information we need before we type in the words and select that ubiquitous magnifying glass icon. That's the bar, and it gets higher every year. It's not a pipe dream.
Let's separate enterprise search from Web search for the sake of argument. Web search has the benefit of billions of dollars of investment from Google, Microsoft, and others, and has come a long, long way since the days of Netscape and Yahoo! Those search technologies index everything published on the Web and are absolutely amazing in terms of breadth…but very weak in terms of depth.
Enterprise search is the organized retrieval of structured and unstructured data within an organization. It has much deeper demands in terms of security and handling key words, phrases, and paragraphs within documents and records. In addition, ranking algorithms used in web search fall apart in most enterprises due to the lack of incoming or outgoing links to file or database content. Enterprise search needs often dictate that all search results, rather than just the most relevant, be listed.
Open source search solutions such as Apache Solr/Lucene exploded onto the scene about a decade ago and appealed to independent software vendors and large enterprises with hardcore development staffs. These search technologies were embeddable, extensible, and free of license charges. Companies that use open source search are developing applications for internal and commercial use. IT technicians and business analysts don't have a clue how to use these technologies: the target user is a hardcore developer who commands premium wages.
There are market drivers stoking the fires of heated demand for better and more advanced search products. The first is big data. Forget about volume, variety, and velocity. The first "v" (volume) is overwhelming by itself. By 2020, there will be more digital data than all the grains of sand on the earth. We're buried in data and it is growing exponentially. Data lives everywhere -- in your databases, in the cloud, in your appliances, and now in your wearables. With 100 billion connected devices in the world, mobility is driving the amount of data stored and managed through the roof and we want to find what we're looking for instantly.
The second market driver is cloud infrastructure. Because computing has become a utility, the cost of scale-out data processing is plummeting. Search is becoming a service. IBM in particular, with Watson Analytics and natural language processing, is offering search and analytics as a paid service.
The third driver is analytics. Visual data discovery is easy and affordable but users don't want to live in their data silos. They want to find and retrieve data that is locked away in other sources beyond the SQL query and combine that data with their structured data for analysis.
People are looking for insight and answers, not lists of search results based on keywords. They want to analyze text, speech and video and preempt a crisis or exploit a business opportunity. People want to be informed of important information before events take place. They want a personal assistant when they are looking for information
How Enterprise Search Will Evolve
Enterprise search in 2020 will work much differently than it does today. Apple's Siri, IBM's Watson, and Microsoft's Cortana have shown the world how enterprise search and text analytics can combine to serve as a personal assistant. Enterprise search will continue to evolve from being your personal assistant to being your personal advisor.
Decades of advances in computational linguistics are on the verge of being unleashed in commercial form thanks to the computational power and scalability provided by the cloud. Scholarly papers have been published on the ability to determine age, sex, demographic, education level, and even the mood of the writer.
Enterprise Search in 2020 will take into account your location, your age, your education level, your profession, who you're e-mailing, and the document you're writing. It will search, retrieve, and suggest information to guide you along the way. Remember the animated paperclip from Microsoft Office 97? Think of that paperclip actually having a brain and being so useful that you are willing to sacrifice anonymity so you can benefit from its utility.
Writing a letter to someone special? Enterprise Search in 2020 will tell you when your tone is off. It will recognize similar tones from the last time you wrote that person and were met with a less-than-ideal response. It will also suggest terms based on other letters that other people have written that resulted in a more favorable response. It's like a life coach without the spray tan and the nice suit.
Writing a sales pitch for a client? Enterprise Search in 2020 will present you with useful data insights based on what you've written and suggest changes to your vocabulary so you speak the same language as your client, so you are more likely to drive your point home.
While there are undoubtedly privacy issues to work out, Enterprise Search will transform from being your personal assistant to being your personal advisor, a voice of reason that guides you to make better decisions, prevents you from making mistakes, and automatically presents you with relevant information, all before you even ask a question.
Bob Potter is senior vice president and general manager of Rocket Software's business information/analytics business unit. He has spent 33 years in the software industry with start-ups, mid-size, and large public companies with a focus on BI and data analytics. You can contact him at [email protected].
Gary Mumford, managing director, enterprise search and text analytics joined Rocket software in 2009. He brings over 15 years of software engineering and product management expertise gained throughout his career at companies such as Microsoft, Fast Search & Transfer, and NextPage. His specialties include Web-based enterprise search, text analytics, content management, and scale-free information retrieval systems. Gary holds a B.S. degree in computer science from Utah Valley University, graduating Cum Laude. You can contact the author at [email protected].