Big Data and the Midmarket: Identifying and Overcoming the Top Challenges
We examine three key obstacles that stand between midmarket companies and successful big data initiatives.
By Joanna Schloss, Business Intelligence and Analytics Evangelist, Quest Software
Long thought of as relevant only to enterprise organizations, big data is on the move. Midmarket organizations in significant numbers are disregarding common wisdom and seizing for themselves the opportunities big data initiatives offer. Not only are these midsized organizations aggressively embracing and investing in big data projects, but most are reaping significant benefits when they do, particularly when it comes to faster decision making and improved product quality. But the road to big data success is not without challenges.
According to the findings of a recent survey by Competitive Edge Research, three primary obstacles stand between midmarket companies and successful big data initiatives: lack of organizational collaboration, lack of necessary skills, and lack of proper tooling. As more midmarket companies gear up to dive into the data analytics game, understanding and overcoming these challenges will likely stand as the difference between a project's ultimate success or failure. Let's examine each of these challenges and how to best address them before they derail your big data efforts.
Lack of IT/Line-of-business Cooperation
A bell that rung repeatedly for years, the need for close collaboration between IT and line-of-business rings especially true when it comes to midmarket big data projects. Line-of-business teams excel at many things, chief among them innovation and the ability to creatively solve pressing business challenges. What lines of business are not so great at, however, is securing, managing, and scaling IT systems. The notion that any given big data or data analytics technology enables lines of business to bypass IT and go it alone is thus inherently flawed.
Moreover, for data analysis projects to be successful, the right people need access to the right data at the right time, and that's something that requires alignment and cooperation between the two groups. Creating this alignment is not easy, but here are some things that can start moving the needle in the right direction:
- Senior executives must have a common vision of what the organization is trying to achieve. Their vision needs to account for the shared interests of both IT and the line of business, and they need to make it clear that they won't tolerate anything short of productive collaboration.
- IT needs to come to the table with a plan that doesn't compromise the innovation line-of-business needs. Simply saying no to a given request drives LOB to find workarounds, which just creates more silos and more work for IT when the inevitable problems start.
- parties need to clear the air and eliminate any pre-existing tensions. Projects may not have gone perfectly in the past, but big data is about looking forward, not back. Put key stakeholders together to air out any difficulties, and move forward with a clean slate. Build a plan that meets the needs of both groups and execute on it.
- The organization must commit to making collaboration a requirement and back up that commitment with the solutions to empower it. Embrace data agnostically and put into place solutions that empower teams to connect to all data types, in all locations.
Lack of Skills
There's no getting around it – there just aren't enough data scientists available to meet the needs of every organization interested in data analytics. Until educational institutions build up their data science programs, the problem is going to continue, and most companies don't have time to wait. Even when qualified data scientists can be found, the cost to employ them is often prohibitive, even for large enterprises.
Rather than wait for the elusive data scientist, midmarket companies should instead invest in developing those skills from within. Don't think about developing a single data scientist. Think instead about developing data science teams. This will reduce dependence on any one individual and reduce exposure to risk should that person ever leave the organization. It'll also enable companies to get more brainpower involved with data analysis projects.
Start by providing on-the-job training for team members already within your organization, including business analysts and DBAs. This training should be a priority, so don't skimp either on the cost of the training or any associated staff time. The long-term benefits will far outweigh the upfront investment.
Lack of Tools
There are two issues that impact tooling for big data projects in midmarket organizations: budget and data complexity. Midmarket companies can't afford to make multimillion-dollar investments in massive analytics platforms. Instead, these companies need to focus on investing in solutions with a midmarket design point -- that is, tools that are cost-effective and easy enough to use today but scalable enough so they won't require a rip and replace when the business grows. Fortunately, analytics is one of the hottest markets in all of IT, leading to the almost daily delivery of new tools and technologies from start-ups and large vendors alike.
Finding tools that help manage growing data complexity is equally important. Midmarket organizations struggle to manage the wide variety of data types (text, video, social, etc.) that now make up their data environment. Big data really means all data, so it's important for midmarket companies to look for solutions that are data type-agnostic and allow them to connect to, integrate, and analyze data regardless of its type and location.
Benefits
As everyone is now beginning to understand, the potential benefits of a data-driven approach to business are not simply the domain of enterprise organizations; midmarket companies have just as much to gain from harnessing data to improve their bottom line with better decision-making. Getting there won't be easy, but the road is far smoother for those organizations that foster cooperation between the business and IT, and ensure that teams have the right skills and the right tools for the job.
Joanna Schloss is a subject matter expert in the Dell Center of Excellence specializing in data and information management. Her areas of expertise include big data analytics, business intelligence, business analytics, and data warehousing. With a blend of experience in both startup and G500 environments, Joanna has successfully launched a myriad of products, from business-focused analytic applications to data warehousing tools such as Business Objects Data Services. Within the Dell center of excellence, she helps clients deal with the challenges of managing multiple data platforms, applications systems, and analytic environments. You can contact the author at [email protected].